2 packs: 15% off with code BUY2. 3 or more: 20% off with code BUY3.

Free Anthropic CCAR-P Practice Questions

37 free Anthropic CCAR-P practice questions from the CertStash pack of 148, each with the correct answer and a full explanation. The rule is the same for every exam: ten free on the product page, 37 free here, all 148 in the pack, and each step includes the one before it.

Question 1

A managed agent deployment for claims triage has grown from 6 tools to 34 tools over 18 months as product teams added capabilities. Triage accuracy has declined from 91 percent to 78 percent, and average tool selection latency has increased by 2.3 seconds. A junior engineer has proposed adding a tool router agent in front of the current agent to filter the tool list per request.

Which two findings should you present to justify capability decomposition before adding the router? (Choose two.)

Each correct answer presents part of the solution.

  1. Tool descriptions overlap across multiple claim categories within the agent's tool set.
  2. Several tools have not been invoked across the most recent 90 days of traffic.
  3. The 34 tools serve four distinct claim-workflow domains within the triage scope.
  4. The router pattern is well documented across publicly available agent literature.
  5. The proposed router introduces an additional model call on every incoming request.
Show answer and explanation

Correct answer: A, C. Tool descriptions overlap across multiple claim categories within the agent's tool set; The 34 tools serve four distinct claim-workflow domains within the triage scope.

Capability decomposition is justified when the single agent's tool set is the root cause of the decline. Overlapping tool descriptions across claim categories make tool selection ambiguous, which directly explains falling accuracy and slower selection, and a router would still have to choose among the same confusable tools. The fact that the 34 tools serve four distinct claim-workflow domains shows natural boundaries along which the agent can be split into focused agents, each with a small, clearly described tool set. Together these findings show the problem is structural, so decomposing by domain addresses it rather than layering another component on top.

Why the other options are wrong

  • B. Unused tools support pruning the tool list, not splitting the agent along capability boundaries.
  • D. How widely a pattern is documented says nothing about whether this agent's tool set needs decomposition.
  • E. The extra model call is a cost of the router, not evidence that the capabilities should be decomposed.

Question 2

You are comparing patterns for a batch document-classification job that follows fixed steps: extract metadata, classify, summarize, and persist.

Which pattern is the best fit and why?

  1. A workflow pattern, because agentic patterns do not support tool invocations and therefore cannot execute the persist step that writes results to the downstream store.
  2. An agentic pattern, because the open-ended planning capability of agents produces more consistent structured outputs than a fixed workflow graph on classification tasks.
  3. An agentic pattern, because agents are categorically more accurate than workflows and therefore always preferable regardless of step predictability.
  4. A workflow pattern, because the steps are well-defined and predictable per-request token cost is preferred over flexibility.
Show answer and explanation

Correct answer: D. A workflow pattern, because the steps are well-defined and predictable per-request token cost is preferred over flexibility.

When the steps are known in advance and always run in the same order, a workflow that orchestrates model calls and code through predefined paths is the right fit. Extract, classify, summarize and persist form a fixed sequence, so a workflow delivers predictable behavior, predictable per-request token cost and simpler debugging. Agentic patterns earn their extra cost and variability only when the number or order of steps cannot be predicted and the model must decide its own path, which is not the case for this batch classification job.

Why the other options are wrong

  • A. Agents can invoke tools, so the persist step is not a reason to avoid an agentic pattern.
  • B. Open-ended planning adds variability and does not make structured outputs more consistent than a fixed workflow.
  • C. Agents are not categorically more accurate, and step predictability is exactly what should drive the choice.

Question 3

A platform team operates a self-hosted multi-agent system on Kubernetes that orchestrates seven specialized agents for invoice processing. The team spends approximately 40 percent of engineering capacity on infrastructure maintenance, message bus reliability, and agent state recovery. The CFO has asked you to evaluate moving to managed agent infrastructure to reclaim engineering capacity. The security officer requires that all customer financial data remain within an approved network boundary.

Which factor should most heavily influence your recommendation?

  1. Whether the current seven agents map cleanly to the patterns supported by managed agents.
  2. Whether managed agents reduce per-invoice token costs across the existing processing volume.
  3. Whether managed agents support the current bus topology used by the platform team.
  4. Whether managed agent data handling satisfies the network boundary required by security.
Show answer and explanation

Correct answer: D. Whether managed agent data handling satisfies the network boundary required by security.

A security requirement that customer financial data stay within an approved network boundary is a hard constraint, so it must be resolved before any efficiency benefit is weighed. If the managed agent service cannot handle data in a way that satisfies that boundary, the migration cannot proceed regardless of how much engineering capacity it would reclaim. Pattern fit, token cost and messaging topology are all secondary considerations that matter only once the data handling requirement is confirmed.

Why the other options are wrong

  • A. Pattern fit is a design detail that can be adapted, while the data boundary is non-negotiable.
  • B. Token cost is not the CFO's stated goal and cannot override a security requirement.
  • C. A managed platform replaces the self-run message bus, so matching the current topology is not the deciding factor.

Question 4

You are selecting a pattern for a compliance Q&A assistant that must answer policy questions with citations to the authoritative internal source set. Latency, cost, and audit predictability are prioritized.

Which pattern is the best fit?

  1. Augmented LLM with retrieval-augmented generation over the indexed authoritative corpus and citation rendering on each response.
  2. Multi-agent orchestration with a planner, researcher, and writer agent for every query.
  3. A pure agent loop with open web-browsing tools to surface the most current policy information, without constraining retrieval to the authoritative internal corpus.
  4. A static prompt with the entire policy corpus concatenated into every request.
Show answer and explanation

Correct answer: A. Augmented LLM with retrieval-augmented generation over the indexed authoritative corpus and citation rendering on each response.

An augmented LLM that retrieves from the indexed authoritative corpus and renders citations on each answer meets every stated priority. Retrieval keeps responses grounded in the approved policy set and makes each answer traceable to its source for audit. A single retrieval step followed by one model call keeps latency and cost low and behaves predictably, which suits a compliance Q&A assistant far better than heavier orchestration or unconstrained search.

Why the other options are wrong

  • B. Running a planner, researcher and writer on every query adds latency and cost with no benefit for direct policy lookups.
  • C. Open web browsing pulls in non-authoritative content and breaks both citation integrity and audit predictability.
  • D. Sending the whole corpus on every request inflates cost and latency and weakens precise citation.

Question 5

You are a solution architect designing a Claude-based assistant with access to 60 internal tools across multiple business domains. Loading every tool definition on every request increases token usage and time to first response.

Which design pattern best addresses this issue without sacrificing capability breadth?

  1. Apply progressive tool discovery so a curated initial subset is exposed and additional tools are loaded on demand based on the task.
  2. Use a separate model call to summarize all 60 tool definitions before each user turn.
  3. Increase the maximum context length and load all 60 tool definitions on every request, accepting the higher token cost and latency as necessary for full capability.
  4. Hard-code a fixed set of five tools per request to reduce token usage, regardless of whether those tools are relevant to the current task.
Show answer and explanation

Correct answer: A. Apply progressive tool discovery so a curated initial subset is exposed and additional tools are loaded on demand based on the task.

Progressive tool discovery exposes a curated initial set of tools and loads further definitions on demand when the task needs them. This keeps every request lean, cutting token usage and time to first response, while all 60 tools remain reachable, so capability breadth is preserved. Anthropic's tool search approach follows the same idea: the model searches for and loads relevant tool definitions instead of receiving every definition up front, which also improves tool selection accuracy when the catalog is large.

Why the other options are wrong

  • B. An extra summarization call on every turn adds latency and cost rather than removing it.
  • C. Loading all 60 definitions on every request keeps the exact token and latency problem the design must solve.
  • D. A fixed set of five tools regardless of relevance sacrifices the capability breadth the design must keep.

Question 6

You are reviewing a customer-support agent's configuration. Each candidate tool falls into one of four categories: (1) required to complete defined tasks; (2) frequently used and reduces hand-offs; (3) occasionally useful for unrelated work; (4) speculative future utility.

Which categories should typically remain in the agent configuration?

  1. Only category 3, because occasionally useful tools for unrelated work provide broader coverage and should take priority over tools required for the agent's defined tasks.
  2. Only category 4, because speculative future-utility tools provide the most flexibility and should be configured even when no defined task currently requires them.
  3. Categories 1 and 2 only, because they map to defined tasks and the agent's regular hand-offs.
  4. All four categories, because broader tool access is categorically better for agent performance regardless of whether the tools map to defined tasks or regular hand-offs.
Show answer and explanation

Correct answer: C. Categories 1 and 2 only, because they map to defined tasks and the agent's regular hand-offs.

An agent's tool set should contain what it needs for its defined job and nothing more. Tools required to complete defined tasks are essential, and tools that are frequently used and reduce hand-offs directly support the agent's regular work, so both categories stay. Tools that are only occasionally useful for unrelated work or have speculative future utility widen the attack surface, add tokens to every request and give the model more options to confuse, so least privilege means removing them until a defined task needs them.

Why the other options are wrong

  • A. Tools for unrelated work fall outside the agent's scope and must never outrank tools its defined tasks require.
  • B. Speculative tools serve no current task and only add risk and selection noise.
  • D. Broader tool access degrades selection accuracy and violates least privilege.

Question 7

You are reviewing a peer's draft system prompt that contains contradictory instructions: one section says to never speculate beyond the supplied source, and another says to confidently fill in any gaps.

Which response is most appropriate?

  1. Add a priority instruction directing the model to evaluate all instructions and apply whichever appears most contextually appropriate on each request.
  2. Remove or rewrite the gap-filling instruction so the prompt consistently constrains the model to source-supported content.
  3. Increase temperature so output randomness masks the contradiction.
  4. Keep both instructions and rely on the model to decide which one to follow on each request.
Show answer and explanation

Correct answer: B. Remove or rewrite the gap-filling instruction so the prompt consistently constrains the model to source-supported content.

Contradictory instructions make model behavior unpredictable, because the model has to guess which directive wins on each request. The fix is to resolve the conflict in the prompt itself by removing or rewriting the gap-filling instruction so every section consistently keeps the model to source-supported content. Clear, consistent instructions are a core prompt engineering practice, and constraining the model to its supplied sources also reduces hallucination, which matches the intent of the never-speculate rule.

Why the other options are wrong

  • A. Letting the model pick whichever instruction seems appropriate leaves the contradiction in place and keeps behavior inconsistent.
  • C. Raising temperature adds randomness and does nothing to resolve conflicting instructions.
  • D. Keeping both instructions guarantees inconsistent behavior across requests.

Question 8

A document analysis service processes legal filings averaging 80,000 tokens each. Each filing is queried by attorneys an average of 14 times during a case. The current architecture sends the full filing on every query. The CFO has asked you to reduce per-query costs while preserving response quality. The security officer requires that filing contents not be stored outside Fabrikam's tenancy.

Which optimization approach should you recommend?

  1. Summarize each filing once at intake and run all subsequent queries against the summary.
  2. Cache the filing as the prompt prefix for reuse across the 14 queries per case.
  3. Index filings in a vector store and retrieve only the relevant passages per query.
  4. Move the workload to a smaller Claude model to reduce the per-token cost paid.
Show answer and explanation

Correct answer: B. Cache the filing as the prompt prefix for reuse across the 14 queries per case.

Each 80,000-token filing is sent unchanged about 14 times, which is the ideal case for prompt caching. Placing the filing as the cached prompt prefix means later queries that arrive within the cache lifetime (5 minutes by default, refreshed on each use, or 1 hour with the longer TTL) read it from the cache at a fraction of the normal input token price and with lower latency, while the model still sees the full filing, so response quality is preserved. No separate summary database or vector index is introduced, so no second copy of the filing contents is created.

Why the other options are wrong

  • A. Querying a summary loses detail attorneys need, so response quality is not preserved.
  • C. A vector store adds a new copy of filing contents and retrieves only fragments, risking both the storage requirement and answer quality.
  • D. A smaller model trades away response quality, which the CFO asked to preserve.

Question 9

You are designing the prompt for a ticket-triage classifier with thirty well-defined categories and clear category descriptions in the prompt.

Which technique is most appropriate as the starting point?

  1. a prompt that demands the model invent new categories when the supplied ones do not fit
  2. a chain-of-thought prompt for a routing decision that does not require multi-step reasoning
  3. a zero-shot prompt that specifies the categories with their descriptions and the required output format
  4. a prompt with no category descriptions, expecting the model to infer the category set
Show answer and explanation

Correct answer: C. a zero-shot prompt that specifies the categories with their descriptions and the required output format

With thirty well-defined categories and clear descriptions already in the prompt, a zero-shot prompt that lists the categories, their descriptions and the required output format is the right starting point. It gives the model everything it needs for a single-step routing decision while keeping the prompt short, fast and inexpensive. Examples or reasoning steps can be added later only if evaluation shows specific categories being confused.

Why the other options are wrong

  • A. Inventing new categories breaks the fixed taxonomy that downstream routing depends on.
  • B. Chain-of-thought adds tokens and latency to a routing decision that needs no multi-step reasoning.
  • D. Omitting the descriptions forces the model to guess the category set and makes results inconsistent.

Question 10

You are designing a content moderation classifier that processes high volumes of user-generated comments under a tight per-message latency budget using well-defined classification labels.

Which model selection best aligns with the workload?

  1. Opus, because every moderation decision requires maximum reasoning depth regardless of classification complexity.
  2. Haiku, because its latency and cost profile align with high-volume classification workloads that require limited reasoning depth.
  3. Sonnet, because larger general-purpose models are preferred even when workload latency requirements are strict.
  4. Sonnet with extended thinking enabled, because deeper reasoning should be applied to every moderation request to improve edge-case handling.
Show answer and explanation

Correct answer: B. Haiku, because its latency and cost profile align with high-volume classification workloads that require limited reasoning depth.

High-volume moderation with well-defined labels and a tight per-message latency budget is a classification workload that needs speed and low cost far more than deep reasoning. Haiku is Anthropic's fastest and most cost-efficient model tier and is positioned for high-volume, latency-sensitive tasks such as classification, extraction and routing, which covers moderation labels. Choosing it meets the latency budget and keeps per-message cost low at scale, with larger models reserved for the rare cases that genuinely need escalation.

Why the other options are wrong

  • A. Opus adds latency and cost for reasoning depth that well-defined labels do not require.
  • C. A larger model works against the strict latency requirement without improving simple label assignment.
  • D. Extended thinking on every request adds latency and tokens that a tight per-message budget cannot absorb.

Question 11

You are configuring tool permissions for a Claude-based assistant. The assistant's defined responsibilities require read access to a knowledge base and write access to a draft queue, and nothing else.

Which scoping design best applies least privilege?

  1. Allow access to all tools with read permissions globally and restrict write permissions to the draft-queue tool, without scoping to only the specific tools required for defined tasks.
  2. Per-role allow-list of exactly the read-knowledge-base and write-draft-queue tools, enforced at the orchestration layer.
  3. Allow access to every tool in the catalog and rely on the model to decline tools it should not use.
  4. Disable all tools entirely to achieve a minimal permission surface, accepting that the assistant can no longer perform the read-knowledge-base or write-draft-queue tasks it was designed for.
Show answer and explanation

Correct answer: B. Per-role allow-list of exactly the read-knowledge-base and write-draft-queue tools, enforced at the orchestration layer.

Least privilege grants exactly the access a role needs and nothing more. A per-role allow-list containing only the read-knowledge-base and write-draft-queue tools matches the assistant's defined responsibilities precisely, and enforcing it at the orchestration layer makes the restriction deterministic: tools outside the list are never offered to the model and cannot be called, regardless of what the prompt or injected content asks for. The assistant remains fully capable of its intended work with the smallest possible permission surface.

Why the other options are wrong

  • A. Granting read access to every tool exposes far more than the knowledge base the role requires.
  • C. Relying on the model to decline tools it should not use is not an enforced control.
  • D. Disabling all tools prevents the assistant from performing its defined tasks, so it fails the requirement.

Question 12

A Claude architect is designing a HIPAA-compliant pipeline that processes patient records.

Which two design decisions directly support HIPAA compliance requirements? (Choose two.)

  1. Setting max_tokens to a low value to minimize the volume of text generated per request.
  2. Selecting the highest-capability Claude model to maximize diagnostic accuracy.
  3. Enforcing role-based access controls so that PHI is retrievable only by authorized personnel.
  4. Ensuring patient data is never included in training feedback loops sent to the model provider without a BAA in place.
  5. Using streaming responses to reduce perceived latency for clinical users.
Show answer and explanation

Correct answer: C, D. Enforcing role-based access controls so that PHI is retrievable only by authorized personnel; Ensuring patient data is never included in training feedback loops sent to the model provider without a BAA in place.

HIPAA requires that protected health information be accessed only by authorized personnel and shared with service providers only under appropriate safeguards. Enforcing role-based access controls so PHI is retrievable only by authorized users implements the minimum-necessary and access-control expectations directly in the pipeline. Ensuring patient data never flows to the model provider for training or feedback without a Business Associate Agreement in place keeps every disclosure of PHI within a contractually governed relationship. Both decisions address how PHI is accessed and disclosed, which is the core of HIPAA compliance.

Why the other options are wrong

  • A. A low max_tokens setting limits response length and has no bearing on how PHI is protected or disclosed.
  • B. Model capability affects quality, not the access and disclosure safeguards HIPAA requires.
  • E. Streaming improves perceived latency and does not provide any privacy or security safeguard.

Question 13

After a prompt template update, several previously passing test cases now produce unexpected outputs.

Which test type is specifically designed to detect this category of failure?

  1. Integration tests that validate cross-component pipeline behavior.
  2. Adversarial tests that probe for prompt-injection vulnerabilities.
  3. Regression tests scored against a stable reference set of known-good behavior.
  4. Smoke tests that confirm high-level system availability after the change.
Show answer and explanation

Correct answer: C. Regression tests scored against a stable reference set of known-good behavior.

Cases that used to pass and now fail after a change are the defining symptom of a regression. Regression tests score the system against a stable reference set of known-good behavior, so any prompt template update that alters previously correct outputs is flagged immediately. Running this suite on every template change is what catches unintended side effects before they ship.

Why the other options are wrong

  • A. Integration tests verify how components work together, not whether known-good outputs changed after an edit.
  • B. Adversarial tests probe resistance to attacks such as prompt injection, not unintended changes to normal outputs.
  • D. Smoke tests confirm the system is up and core paths run, which is too shallow to catch changed outputs.

Question 14

You are building an ethics review checklist for deployments supported by artificial intelligence (AI).

Which two checks belong on the list? (Choose two.)

Each correct answer presents a complete solution.

  1. Confirm that vendor licensing terms permit the planned production use of the model.
  2. Verify that outputs do not rely on generalizations about people that the underlying data does not support.
  3. Confirm that high-impact decisions retain human accountability rather than being attributed to the model.
  4. Restrict ethics review to outputs that exceed a defined model-confidence threshold.
  5. Confirm that latency and throughput targets are met across supported user populations.
Show answer and explanation

Correct answer: B, C. Verify that outputs do not rely on generalizations about people that the underlying data does not support; Confirm that high-impact decisions retain human accountability rather than being attributed to the model.

An ethics review focuses on how an AI deployment affects people. Verifying that outputs do not rely on generalizations about people that the underlying data does not support guards against stereotyping and unfair treatment. Confirming that high-impact decisions keep human accountability, rather than attributing them to the model, ensures that a responsible person owns the outcome and can be held to account. Both checks address ethical risk to affected individuals, whereas licensing, confidence gating and performance targets belong to legal, technical or operational reviews.

Why the other options are wrong

  • A. Confirming licensing terms is a legal and procurement check, not an ethics check.
  • D. Limiting review to high-confidence outputs leaves confidently harmful outputs and other cases unexamined.
  • E. Latency and throughput targets are performance criteria, not ethical considerations.

Question 15

You are investigating an MCP server that fails on first launch but succeeds on subsequent runs. The system permission dialogs appeared on first launch.

Which response is most appropriate?

  1. Recognize the first-run permission grant as the cause, document the expected behavior in onboarding guidance, and confirm that subsequent runs succeed.
  2. Reinstall the operating system to clear all permission state without first confirming whether the one-time permission grant caused the failure.
  3. Disable operating-system permission dialogs entirely, accept the resulting security implications, and proceed without confirming whether the failure recurs.
  4. Treat the first-run failure as a permanent fault, replace the MCP server, and do not verify whether subsequent runs succeed.
Show answer and explanation

Correct answer: A. Recognize the first-run permission grant as the cause, document the expected behavior in onboarding guidance, and confirm that subsequent runs succeed.

When an MCP server fails only on first launch, the system permission dialogs that appeared at that moment are the evident cause: the operating system blocked the server's access until the user granted it, and later runs succeed because the grant persists. The proportionate response is to recognize this one-time behavior, record it in onboarding guidance so users know to approve the prompts, and confirm that subsequent runs succeed. This resolves the issue without weakening security controls or replacing working components.

Why the other options are wrong

  • B. Reinstalling the operating system is drastic and unjustified when the one-time permission grant explains the failure.
  • C. Disabling permission dialogs removes an important security control to solve a problem that only needs a one-time approval.
  • D. Replacing the server discards a component that works once permissions are granted, and skipping verification leaves the diagnosis unconfirmed.

Question 16

A pilot AI assistant for procurement specialists shows 89 percent first-response acceptance, but follow-up surveys reveal that specialists frequently override the assistant's vendor recommendations after consulting unrelated criteria the assistant did not consider. The pilot owner wants to ship the assistant as is, given the strong acceptance rate.

Which two discernment-competency observations should you raise BEFORE approving the launch? (Choose two.)

Each correct answer presents part of the solution.

  1. The acceptance rate alone proves readiness for general production use.
  2. The survey response rate may not be statistically representative of all specialists.
  3. The unconsidered criteria represent a scope gap in the assistant's input space.
  4. Acceptance does not establish whether recommendations remain correct after specialist review.
  5. The pilot duration was probably too short to demonstrate reliability across the full year.
Show answer and explanation

Correct answer: C, D. The unconsidered criteria represent a scope gap in the assistant's input space; Acceptance does not establish whether recommendations remain correct after specialist review.

A high first-response acceptance rate measures whether specialists initially accept the output, not whether the recommendation holds up. The survey shows specialists often override vendor recommendations after review, so acceptance does not establish that recommendations remain correct after specialist scrutiny. The overrides also rest on criteria the assistant never considered, which is a scope gap in its input space: the assistant is missing information that the decision genuinely depends on. Both issues need resolution before launch.

Why the other options are wrong

  • A. This restates the pilot owner's flawed conclusion rather than raising a concern about it.
  • B. Survey representativeness is speculative and does not address the concrete override pattern the survey revealed.
  • E. Pilot duration is a speculative concern that the evidence in the scenario does not raise.

Question 17

You are integrating human review into a high-volume classification pipeline where reviewing every output is infeasible.

Which sampling strategy best balances throughput with quality oversight?

  1. No sampling, relying entirely on user complaints to reveal quality and safety problems after they affect users.
  2. Risk-stratified sampling that reviews all low-confidence and high-impact outputs and a smaller random sample of high-confidence routine outputs.
  3. Inverse sampling that reviews only high-confidence routine outputs and skips low-confidence and high-impact outputs.
  4. Universal review of every output regardless of confidence or throughput impact.
Show answer and explanation

Correct answer: B. Risk-stratified sampling that reviews all low-confidence and high-impact outputs and a smaller random sample of high-confidence routine outputs.

When reviewing every output is infeasible, review effort should follow risk. Risk-stratified sampling reviews all low-confidence and high-impact outputs, where errors are most likely or most costly, and adds a smaller random sample of high-confidence routine outputs to detect blind spots and calibration drift. This keeps throughput high while giving oversight where it has the greatest effect.

Why the other options are wrong

  • A. Relying on user complaints means problems are found only after they have affected users.
  • C. Inverse sampling skips exactly the outputs most likely to be wrong or harmful.
  • D. Universal review is infeasible in a high-volume pipeline, as the scenario states.

Question 18

You are responding to an adversarial input pattern in which users include text claiming admin authority and instructing the model to bypass safety restrictions.

Which combination of controls most effectively mitigates this attack pattern?

  1. Trusting that the model will intrinsically recognize and reject all bypass attempts without prompt-level instructions, runtime classifiers, scoped permissions, or audit logging.
  2. Prompt-level instructions that treat user content as untrusted data, runtime classifiers that detect override attempts, scoped tool permissions that cannot be elevated by user content, and audit logging of attempts.
  3. Removing all safety restrictions and guardrails to eliminate the attack surface that bypass attempts target, accepting that this makes the assistant unrestricted for all inputs.
  4. Granting users any privilege level they assert in their message content, on the assumption that cooperative behavior requires honoring self-declared authority without independent verification.
Show answer and explanation

Correct answer: B. Prompt-level instructions that treat user content as untrusted data, runtime classifiers that detect override attempts, scoped tool permissions that cannot be elevated by user content, and audit logging of attempts.

Users who claim admin authority inside their messages are attempting a social-engineering style prompt injection, so defense must work at several layers. Prompt-level instructions tell the model to treat user content as untrusted data rather than as a source of privileges, runtime classifiers detect override attempts independently of the model, scoped tool permissions ensure that nothing in user text can elevate what the system is allowed to do, and audit logging records every attempt for investigation and tuning. This combination provides defense in depth, so no single bypass defeats the controls.

Why the other options are wrong

  • A. Relying solely on the model's intrinsic judgment leaves no independent control when an attempt succeeds.
  • C. Removing all guardrails abandons safety entirely instead of mitigating the attack.
  • D. Honoring self-declared authority hands attackers exactly the privilege escalation they are seeking.

Question 19

A research summarization assistant has been deployed for six months. A user has flagged that a generated summary contained a fabricated citation. The product team has asked whether the incident requires architectural action or whether it is an isolated case.

Which two Diligence-competency actions should you take? (Choose two.)

Each correct answer presents part of the solution.

  1. Sample recent summaries to estimate the fabrication frequency across the population.
  2. Disable the assistant immediately for all current users without any prior diagnostic analysis.
  3. Review whether citation-grounding controls exist anywhere in the current generation pipeline.
  4. Communicate to users that the assistant does not fabricate output as a matter of design.
  5. Treat the incident as an anecdotal isolated case and take no further investigative action.
Show answer and explanation

Correct answer: A, C. Sample recent summaries to estimate the fabrication frequency across the population; Review whether citation-grounding controls exist anywhere in the current generation pipeline.

Diligence means treating one report as evidence to investigate rather than either dismissing it or overreacting. Sampling recent summaries estimates how often fabricated citations occur across the whole population, which tells the team whether this is a rare event or a systemic pattern. Reviewing whether citation-grounding controls exist in the generation pipeline, such as constraining claims to retrieved sources and verifying each citation against them, shows whether the architecture itself allows fabrication. Together these answer whether architectural action is needed and point to what that action should be.

Why the other options are wrong

  • B. Disabling the assistant for everyone before any diagnosis is a disproportionate reaction that skips the investigation diligence requires.
  • D. Telling users the assistant does not fabricate output is false and contradicts the reported incident.
  • E. Dismissing the report as anecdotal without investigation is the opposite of diligence.

Question 20

You are preparing a HIPAA-eligible deployment for a healthcare customer.

Which configuration supports HIPAA compliance using Anthropic-offered tools?

  1. Claude Free with no contractual addendum, since consumer products meet HIPAA requirements out of the box.
  2. Claude Enterprise with a signed Business Associate Agreement, Zero Data Retention enabled, and audit logging configured for compliance tracking.
  3. Disabling all audit logging so that no PHI is recorded in any log store, on the assumption that the absence of logs satisfies HIPAA requirements without a signed BAA.
  4. An ad-hoc personal Claude account used by individual clinicians for PHI-related tasks, with no Business Associate Agreement, no Zero Data Retention, and no audit logging configured.
Show answer and explanation

Correct answer: B. Claude Enterprise with a signed Business Associate Agreement, Zero Data Retention enabled, and audit logging configured for compliance tracking.

Handling protected health information requires a signed Business Associate Agreement with the vendor, so the deployment must run on a commercial offering covered by that agreement, such as Claude Enterprise. Zero Data Retention, where Anthropic authorizes it for the covered use, further limits what the provider retains, and audit logging gives the customer the access records HIPAA expects for tracking who used the system and how. Together the contractual, retention and accountability controls form a defensible HIPAA-eligible configuration.

Why the other options are wrong

  • A. Consumer products with no Business Associate Agreement are not suitable for processing PHI.
  • C. Removing audit logging eliminates required accountability records, and no configuration replaces the need for a signed Business Associate Agreement.
  • D. Personal accounts with no Business Associate Agreement, retention controls or logging expose PHI with no compliance basis.

Question 21

You are an architect supporting the iteration phase of a deployed Claude-based system.

Which activity most directly fits this phase?

  1. Rebuild the entire system architecture from scratch at the start of every iteration cycle regardless of what production telemetry, evaluations, and stakeholder feedback indicate is needed.
  2. Stop measuring production outcomes once the deployment has successfully launched, treating the go-live milestone as the end of the evaluation and iteration cycle.
  3. Discard the evaluation framework and reference set once the deployment is in production, accepting that future iterations will have no structured basis for measuring the impact of changes.
  4. Review production telemetry, sampled output evaluations, and stakeholder feedback to identify the highest-impact change for the next cycle, then plan the change against the evaluation framework.
Show answer and explanation

Correct answer: D. Review production telemetry, sampled output evaluations, and stakeholder feedback to identify the highest-impact change for the next cycle, then plan the change against the evaluation framework.

The iteration phase is a feedback loop that uses evidence from the running system to decide what to improve next. Reviewing production telemetry, sampled output evaluations, and stakeholder feedback identifies the highest-impact change, and planning that change against the existing evaluation framework ensures its effect can be measured and regressions caught. This keeps improvement continuous, targeted, and grounded in data.

Why the other options are wrong

  • A. Rebuilding from scratch every cycle ignores the evidence and wastes the working system.
  • B. Stopping measurement after launch removes the information that iteration depends on.
  • C. Discarding the evaluation framework leaves no structured basis for judging whether changes help or harm.

Question 22

You are defining where human review must remain in a planned automated pipeline.

Which placement reflects sound human-in-the-loop design?

  1. Place a human reviewer only after the irreversible action has already executed, making the review a post-hoc audit rather than a meaningful pre-action check or approval gate.
  2. Place a human reviewer between the model's output and any high-impact, irreversible action, with explicit criteria for what the reviewer must check before approval.
  3. Place a human reviewer in the loop for a randomly selected sample of requests, without defining criteria for what the reviewer should check or which output categories require mandatory review.
  4. Remove all human review steps from the pipeline to maximize throughput, accepting that high-impact and irreversible actions will be taken without any human approval or oversight.
Show answer and explanation

Correct answer: B. Place a human reviewer between the model's output and any high-impact, irreversible action, with explicit criteria for what the reviewer must check before approval.

Human-in-the-loop review is only meaningful when it can still prevent harm. Placing the reviewer between the model's output and any high-impact, irreversible action makes the review a real approval gate, because nothing irreversible happens until a person signs off. Giving the reviewer explicit criteria for what to check keeps approvals consistent and substantive rather than a rubber stamp. This design concentrates scarce human attention where errors would be most costly and hardest to undo.

Why the other options are wrong

  • A. Review after an irreversible action can only record the damage, not prevent it.
  • C. Random sampling without criteria can let high-impact actions through unreviewed and gives reviewers nothing consistent to check.
  • D. Removing all review leaves high-impact, irreversible actions with no human oversight at all.

Question 23

You are compiling guardrail tactics for a customer-facing assistant.

Which two tactics belong on the guardrail list? (Choose two.)

Each correct answer presents a complete solution.

  1. Embed approved override phrases that let trusted users relax guardrails on demand.
  2. Lower sampling temperature globally to reduce the chance of off-policy completions.
  3. Validate the model output against a structured schema before downstream actions are taken.
  4. Layer prompt-level guardrails with runtime content checks rather than relying on either alone.
  5. Rely on a single hardened system prompt that enumerates every disallowed behavior.
Show answer and explanation

Correct answer: C, D. Validate the model output against a structured schema before downstream actions are taken; Layer prompt-level guardrails with runtime content checks rather than relying on either alone.

Effective guardrails are layered and enforced outside the model as well as inside it. Validating the model output against a structured schema before any downstream action runs ensures malformed or out-of-policy results are caught deterministically instead of being acted on. Layering prompt-level guardrails with runtime content checks gives defense in depth, because instructions shape the model's behavior while independent runtime checks catch whatever the prompt fails to prevent. Together these tactics provide controls that do not depend on any single point of failure.

Why the other options are wrong

  • A. Override phrases embedded in the prompt create a bypass that attackers can discover and exploit.
  • B. Lowering temperature changes output variability and is not a policy control that prevents off-policy behavior.
  • E. A single system prompt is one point of failure and cannot enumerate every disallowed behavior.

Question 24

You are defining transparency practices for a customer-facing assistant whose responses are materially shaped by AI.

Which transparency practice most directly supports responsible deployment?

  1. Misrepresent the AI's role in producing responses in order to make the assistant feel more trustworthy or more human, undermining informed user consent and organizational transparency.
  2. Refuse to answer any user question about how the responses were produced or whether AI was involved, treating the AI's role as confidential operational information.
  3. Disclose AI involvement to end users in line with the organization's transparency policy and provide a documented path to reach a human when needed.
  4. Disclose AI involvement only to internal staff and operators while withholding that information from the end users whose interactions are materially shaped by the AI system.
Show answer and explanation

Correct answer: C. Disclose AI involvement to end users in line with the organization's transparency policy and provide a documented path to reach a human when needed.

Responsible deployment means people know when AI materially shapes the responses they receive and can reach a person when the situation calls for it. Disclosing AI involvement to end users in line with the organization's transparency policy supports informed consent and trust, and a documented path to a human gives users recourse for cases the assistant handles poorly or that need judgment. This combination is honest with users while keeping the assistant useful.

Why the other options are wrong

  • A. Misrepresenting the AI's role deceives users and undermines informed consent.
  • B. Refusing to say whether AI was involved denies users information they need to judge the responses.
  • D. Disclosing only to internal staff withholds the information from the end users who are actually affected.

Question 25

You are selecting a model for a new production workload and must complete the upstream steps before testing candidate models empirically.

Which two steps must be completed BEFORE running a representative sample on a candidate model? (Choose two.)

Each correct answer presents part of the solution.

  1. Select the lightest model that consistently meets the quality bar across the sample.
  2. Retest the model choice when a new model version becomes available.
  3. Choose a candidate model based on the requirements profile and known capabilities.
  4. Define the quality bar, latency tolerance, and expected volume for the workload.
  5. Sign off the production rollout plan with the platform and security teams.
Show answer and explanation

Correct answer: C, D. Choose a candidate model based on the requirements profile and known capabilities; Define the quality bar, latency tolerance, and expected volume for the workload.

Empirical testing needs a target and a candidate before it can begin. The team must first define the quality bar, latency tolerance, and expected volume, because these requirements are what the sample results are judged against. With that requirements profile in hand, the team chooses a candidate model based on the profile and the known capabilities of the available models. Only then does running a representative sample produce meaningful evidence, after which the team can pick the lightest model that meets the bar and plan for future retesting.

Why the other options are wrong

  • A. Selecting the lightest model that consistently meets the bar depends on sample results, so it comes after testing.
  • B. Retesting when a new model version arrives is an ongoing step that follows the initial selection.
  • E. Signing off a production rollout plan happens after a model has been selected and validated.

25 questions in. Want the rest? The first half of the pack, free by email · The full pack

Question 26

You are reviewing an integration specification for security gaps.

Which two findings constitute valid security gaps in the specification? (Choose two.)

Each correct answer presents a complete solution.

  1. Tool calls execute server-side under a least-privilege service principal scoped to the requested action.
  2. Service credentials are placed in the prompt context, where they can leak into logs and traces.
  3. Role-based access control is enforced only at the response-rendering layer after the model accesses restricted data.
  4. Per-user OAuth tokens are exchanged with scope-restricted permissions and refreshed within the active session.
  5. Tool inputs and outputs are encrypted in transit using transport-layer security between services.
Show answer and explanation

Correct answer: B, C. Service credentials are placed in the prompt context, where they can leak into logs and traces; Role-based access control is enforced only at the response-rendering layer after the model accesses restricted data.

Two findings describe real weaknesses. Placing service credentials in the prompt context exposes secrets to anything that records model inputs, such as logs, traces and evaluation datasets, and lets a manipulated model repeat them in output; credentials belong in the tool execution layer, never in the context window. Enforcing role-based access control only when the response is rendered means the model has already read restricted data, so that data can surface through summaries, tool calls or prompt injection before the rendering filter runs. Access decisions must be made before data reaches the model.

Why the other options are wrong

  • A. Server-side execution under a least-privilege service principal scoped to the requested action is a sound control, not a gap.
  • D. Per-user, scope-restricted OAuth tokens refreshed within the session are a recommended practice, not a weakness.
  • E. Encrypting tool traffic in transit with transport-layer security is a baseline protection, not a gap.

Question 27

When communicating an architectural decision to a security and compliance reviewer, which content set is most aligned with that audience's primary concerns?

  1. Feature delivery schedule, roadmap dependencies, and scope boundaries for the product manager.
  2. Implementation timeline, component interfaces, and deployment sequence for the engineering team.
  3. Threat model, control mappings, residual-risk acceptance criteria, and audit traceability.
  4. Decision rationale, business outcomes, and high-level risk summary for executive review.
Show answer and explanation

Correct answer: C. Threat model, control mappings, residual-risk acceptance criteria, and audit traceability.

A security and compliance reviewer needs to judge whether the design's risks are understood, controlled and accountable. A threat model shows what could go wrong and how, control mappings show which safeguards address each threat and which requirements they satisfy, residual-risk acceptance criteria show what risk remains and who has agreed to carry it, and audit traceability shows that actions and decisions can be reconstructed later. Tailoring the content to these concerns lets the reviewer reach a decision efficiently.

Why the other options are wrong

  • A. Delivery schedules and roadmap dependencies address a product manager's planning concerns, not security review.
  • B. Implementation timelines and component interfaces serve the engineering team, not a compliance assessment.
  • D. Business outcomes and a high-level risk summary suit executives and lack the detail a security reviewer needs.

Question 28

An engineer inadvertently commits an API key to the repository by placing it in .claude/settings.json.

Which configuration design principle was violated?

  1. Project-scope configuration should always override user-scope configuration.
  2. Managed configuration must be applied before project-scope configuration is read.
  3. Credentials must be resolved at runtime from a secret store and never stored in version-controlled files.
  4. MCP server definitions must reside in environment variables to prevent scope conflicts.
Show answer and explanation

Correct answer: C. Credentials must be resolved at runtime from a secret store and never stored in version-controlled files.

The project file .claude/settings.json is shared configuration intended to be checked into source control, so anything placed in it is exposed to everyone with access to the repository and stays in its history. Credentials such as API keys must never be stored in version-controlled files. They should be resolved at runtime from a secret store or environment, so the key can be rotated and access controlled without leaking it through the codebase. Committing the key to the shared project settings violates exactly this principle.

Why the other options are wrong

  • A. Scope precedence between project and user settings has nothing to do with where a secret is stored.
  • B. The order in which managed and project configuration is applied does not cause a credential to be committed.
  • D. There is no principle requiring MCP server definitions to live in environment variables, and the issue here is a stored credential.

Question 29

You are rolling out monitoring for a Claude-based deployment and must complete the specification steps before instrumenting the deployment.

Which two steps must be completed BEFORE instrumenting the deployment to emit metrics and traces? (Choose two.)

Each correct answer presents part of the solution.

  1. Define the metrics and the slices the deployment will monitor across normal and adversarial traffic.
  2. Tune the alert thresholds based on observed normal-state distributions to reduce false positives.
  3. Define the service-level objectives (SLOs) and the error budgets the deployment will be held to.
  4. Build the dashboards that surface metrics across slices at the cadence the team operates on.
  5. Document the dashboards, alerts, and runbooks for the on-call rotation that will respond.
Show answer and explanation

Correct answer: A, C. Define the metrics and the slices the deployment will monitor across normal and adversarial traffic; Define the service-level objectives (SLOs) and the error budgets the deployment will be held to.

Instrumentation must know what to emit and what it will be measured against. Defining the metrics and the slices to monitor across normal and adversarial traffic specifies exactly which signals the deployment has to produce. Defining the service-level objectives and error budgets sets the targets those signals are evaluated against. Once both are specified, the team can instrument the deployment, then build dashboards, tune alert thresholds on observed data, and document runbooks.

Why the other options are wrong

  • B. Tuning alert thresholds needs observed normal-state data, which exists only after instrumentation.
  • D. Dashboards display emitted metrics, so they are built after the deployment is instrumented.
  • E. Documenting dashboards, alerts, and runbooks depends on those artifacts already existing.

Question 30

An operations engineer reports that a Claude-based pipeline began returning malformed JSON responses after a scheduled maintenance window, causing downstream processing failures.

Which two investigative steps most directly isolate the root cause? (Choose two.)

  1. Clear the prompt cache and resubmit all pending requests to eliminate stale cached prefixes.
  2. Compare the current system prompt and output schema configuration against the last known good version from before the maintenance window.
  3. Switch to a different Claude model tier to rule out provider-side changes as a contributing factor.
  4. Increase the max_tokens limit to determine whether output truncation is causing incomplete JSON structures.
  5. Replay a set of premaintenance requests against the current configuration and inspect the raw model output before downstream parsing.
Show answer and explanation

Correct answer: B, E. Compare the current system prompt and output schema configuration against the last known good version from before the maintenance window; Replay a set of premaintenance requests against the current configuration and inspect the raw model output before downstream parsing.

The malformed JSON began right after a maintenance window, so the fastest way to isolate the cause is to find what changed and confirm where the output breaks. Comparing the current system prompt and output schema configuration against the last known good version from before the window exposes any configuration drift introduced during maintenance. Replaying pre-maintenance requests against the current configuration and inspecting the raw model output before parsing shows whether the model itself now returns malformed JSON or whether the failure is introduced downstream. Together these steps narrow the fault to a specific change and layer.

Why the other options are wrong

  • A. Clearing the cache and resubmitting requests changes state without identifying what broke.
  • C. Switching model tiers introduces a new variable instead of isolating the change made during maintenance.
  • D. Raising max_tokens tests one guess about truncation rather than isolating what changed in the window.

Question 31

You are listing characteristics of robust guardrail design for an enterprise deployment.

Which two characteristics belong on the list? (Choose two.)

Each correct answer presents a complete solution.

  1. Centralized log retention for guardrail violations with quarterly review by the security team.
  2. Per-role tool allow-lists enforced at the orchestration layer before any tool call executes.
  3. User feedback channels that route reported guardrail failures into the product backlog for triage.
  4. Periodic refresh of the system prompt wording to keep refusal language current and clear.
  5. Adversarial-input coverage in the evaluation set with regression tracking on guardrail performance.
Show answer and explanation

Correct answer: B, E. Per-role tool allow-lists enforced at the orchestration layer before any tool call executes; Adversarial-input coverage in the evaluation set with regression tracking on guardrail performance.

Robust guardrails are enforced deterministically and verified continuously. Per-role tool allow-lists enforced at the orchestration layer before any tool call executes apply least privilege outside the model, so a manipulated model still cannot invoke tools its role does not permit. Adversarial-input coverage in the evaluation set, with regression tracking on guardrail performance, proves that the guardrails withstand attack patterns and detects any weakening when prompts, models or tools change. Together they combine preventive enforcement with measurable assurance.

Why the other options are wrong

  • A. Quarterly review of retained logs is a slow after-the-fact activity and does not itself make the guardrails robust.
  • C. Routing user reports to a backlog is a feedback process, not a characteristic of the guardrail design.
  • D. Refreshing refusal wording is cosmetic and does not strengthen enforcement or verification.

Question 32

A customer support team has proposed delegating customer refund decisions to a Claude-driven workflow with no human review for refunds under 50 USD. The team's reasoning is that small refunds are low-risk and human review would erase the efficiency gain.

Which Delegation-competency principle should guide your response?

  1. Delegation should always include human review on every decision the workflow produces.
  2. Delegation scope should reflect the type of risk involved, not the transaction size alone.
  3. Delegation scope should be set primarily by maximizing efficiency gains across the workflow.
  4. Delegation should be avoided entirely wherever financial transactions occur in the workflow.
Show answer and explanation

Correct answer: B. Delegation scope should reflect the type of risk involved, not the transaction size alone.

How much to delegate to Claude should be decided by the kind of risk a decision carries, not by the dollar value of a single transaction. Refunds under 50 USD can still involve fraud patterns, policy exceptions, repeated abuse by the same account or customer-relationship harm, and those risks scale with volume even when each amount is small. The right response is to analyze which refund situations are low risk in type, delegate those, and route the rest to human review, rather than accepting a threshold that treats amount as the only measure of risk.

Why the other options are wrong

  • A. Requiring human review on every decision ignores that some decisions are genuinely low risk and wastes the delegation benefit.
  • C. Setting scope primarily to maximize efficiency ignores the risk the delegated decisions carry.
  • D. Avoiding delegation wherever money is involved is an absolute rule that discards safe, well-scoped automation.

Question 33

A Claude-based research assistant begins producing responses that confidently contradict its retrieved source documents despite no change to the retrieval pipeline.

Which two diagnostic actions most directly identify the root cause of this behavior? (Choose two.)

  1. Increase the context-window size to allow more retrieved chunks per query.
  2. Switch the retrieval index to a denser embedding model to improve chunk-relevance scores.
  3. Determine whether the failure reproduces on the previous model version to test for a model mismatch.
  4. Reduce the temperature setting to lower response variance across all query types.
  5. Inspect the system-prompt grounding instructions to determine whether citation constraints remain intact.
Show answer and explanation

Correct answer: C, E. Determine whether the failure reproduces on the previous model version to test for a model mismatch; Inspect the system-prompt grounding instructions to determine whether citation constraints remain intact.

Because the retrieval pipeline did not change, the root cause lies in what drives the model's use of the retrieved content. Checking whether the failure reproduces on the previous model version isolates whether a model version change introduced the behavior. Inspecting the system prompt's grounding instructions shows whether the constraints that require answers to follow and cite the retrieved documents are still present and intact. Together these two checks locate the cause instead of masking the symptom.

Why the other options are wrong

  • A. Adding more retrieved chunks changes the input but does not diagnose why the model contradicts the sources it already has.
  • B. Changing the embedding model alters a retrieval pipeline that is not implicated in the failure.
  • D. Lowering temperature may reduce variance but does not identify the cause of ungrounded answers.

Question 34

A Claude architect is implementing safety controls for a customer-facing advice assistant that must never provide regulated investment recommendations.

Which two guardrail implementations most directly enforce this constraint? (Choose two.)

  1. Increase response temperature to introduce variability that reduces the likelihood of specific recommendations.
  2. Add an output classifier that detects and blocks responses containing regulated investment-recommendation language.
  3. Limit session length to reduce the volume of queries processed per user per day.
  4. Log all user queries to a SIEM for post-hoc compliance review.
  5. Define explicit out-of-scope categories in the system prompt with fixed refusal phrasing for investment advice requests.
Show answer and explanation

Correct answer: B, E. Add an output classifier that detects and blocks responses containing regulated investment-recommendation language; Define explicit out-of-scope categories in the system prompt with fixed refusal phrasing for investment advice requests.

Two layered controls directly enforce the constraint. Defining explicit out-of-scope categories in the system prompt, with fixed refusal phrasing for investment advice requests, steers Claude to recognize these requests and decline them consistently. An output classifier that detects and blocks regulated investment-recommendation language acts as an independent check on every response, catching anything the prompt-level instruction misses before it reaches the customer. Prompt-level scoping plus output screening is the standard defense-in-depth pattern for a prohibited content category.

Why the other options are wrong

  • A. Raising temperature adds randomness and does not reliably prevent investment recommendations.
  • C. Limiting session length reduces volume but does nothing to stop prohibited advice in any given response.
  • D. Logging queries to a SIEM supports later review but does not prevent the advice from being given.

Question 35

You are designing a human-in-the-loop validation workflow.

Which two design elements belong in the workflow? (Choose two.)

Each correct answer presents a complete solution.

  1. capture of reviewer decisions as labeled data that feeds the evaluation set over time
  2. mandatory reviewer agreement with the model output to maintain consistent reviewer-model alignment
  3. aggregate reviewer throughput metrics published weekly to leadership for headcount planning
  4. uniform sampling of all model outputs to give every request an equal chance of review
  5. risk-stratified sampling that concentrates reviewer attention on low-confidence and high-impact outputs
Show answer and explanation

Correct answer: A, E. capture of reviewer decisions as labeled data that feeds the evaluation set over time; risk-stratified sampling that concentrates reviewer attention on low-confidence and high-impact outputs

An effective human-in-the-loop workflow spends limited reviewer time where it matters and turns that time into lasting value. Risk-stratified sampling concentrates review on low-confidence and high-impact outputs, where errors are most likely and most costly. Capturing reviewer decisions as labeled data then feeds the evaluation set over time, so the system's tests grow to reflect real failure cases and each review improves future measurement as well as the immediate output.

Why the other options are wrong

  • B. Requiring reviewers to agree with the model defeats the purpose of independent validation.
  • C. Throughput reporting for headcount planning is a management metric, not a validation design element.
  • D. Uniform sampling spreads review evenly and under-covers the risky outputs that most need attention.

Question 36

You are building an evaluation pipeline for a Claude-based deployment and must complete the specification steps before running the deployment against the dataset.

Which two steps must be completed BEFORE running the deployment against the evaluation dataset? (Choose two.)

Each correct answer presents part of the solution.

  1. Publish the aggregated metrics to a dashboard and gate releases on threshold checks.
  2. Curate and label the evaluation dataset to match the defined slices.
  3. Review failure cases with subject matter experts to refine the scoring rubric.
  4. Define the metrics and slices the framework will report across representative, edge, and adversarial cases.
  5. Score the deployment outputs against the reference labels and aggregate the metrics.
Show answer and explanation

Correct answer: B, D. Curate and label the evaluation dataset to match the defined slices; Define the metrics and slices the framework will report across representative, edge, and adversarial cases.

An evaluation run is only meaningful when you already know what you are measuring and have the data to measure it. Defining the metrics and slices first, across representative, edge and adversarial cases, establishes what the framework will report and how results will be broken down. Curating and labeling the evaluation dataset to match those slices then supplies the inputs and reference labels the run will be scored against. Both specification steps must be in place before the deployment is run, because scoring, review and release gating all depend on them.

Why the other options are wrong

  • A. Publishing metrics and gating releases requires results that only exist after the run is scored.
  • C. Reviewing failure cases requires outputs from a completed run, so it follows execution.
  • E. Scoring outputs against reference labels happens after the deployment has produced those outputs.

Question 37

An engineering organization is adopting Claude Code across 200 developers. A team lead has proposed that AI-generated pull requests bypass the standard code review process for changes under 50 lines, on the grounds that small changes are low-risk and review capacity is constrained.

Which two Diligence-competency objections should you raise? (Choose two.)

Each correct answer presents part of the solution.

  1. Removing review eliminates the verification mechanism required for AI-generated output across the workflow.
  2. Code-review capacity should be expanded to handle every code change submitted across the team.
  3. The threshold should be increased to 200 lines to capture more changes.
  4. AI-generated pull requests should be rejected by default across the organization.
  5. Line count is a poor proxy for the actual risk introduced by a code change.
Show answer and explanation

Correct answer: A, E. Removing review eliminates the verification mechanism required for AI-generated output across the workflow; Line count is a poor proxy for the actual risk introduced by a code change.

Diligence means taking responsibility for verifying AI-generated work before it is relied upon. Letting AI-generated pull requests skip review removes the verification step that catches errors, security flaws and unintended behavior in that output, so the organization would be shipping code nobody has checked. Using line count as the exemption criterion is also flawed, because a change of a few lines can alter authentication, permissions or data handling and carry far more risk than a large cosmetic change. Both objections show why the proposal weakens accountability for AI-assisted code.

Why the other options are wrong

  • B. Expanding review capacity for every change is a resourcing preference, not an objection to removing verification of AI output.
  • C. Raising the threshold to 200 lines would exempt even more unreviewed code and keeps the flawed line-count criterion.
  • D. Rejecting all AI-generated pull requests is an overcorrection that discards the productivity the adoption is meant to deliver.

That was 37 of 148.

The full Anthropic CCAR-P pack has all 148 questions, each with the answer, the explanation and why the other options are wrong, plus a questions-only copy for timed runs. US$39, paid once, with free monthly updates and a pass-or-your-money-back guarantee.

Get the full pack

Not ready yet? Get the first 74 free as one PDF by email. Get the Half Pack →