System Prompts with Explicit Criteria

Official Exam Guide Objectives

Task 4.1: Design prompts with explicit criteria to improve precision and reduce false positives

Knowledge of

• The importance of explicit criteria over vague instructions (e.g., "flag comments only when claimed behavior contradicts actual code behavior" vs "check that comments are accurate")
• How general instructions like "be conservative" or "only report high-confidence findings" fail to improve precision compared to specific categorical criteria
• The impact of false positive rates on developer trust: high false positive categories undermine confidence in accurate categories

Skills in

• Writing specific review criteria that define which issues to report (bugs, security) versus skip (minor style, local patterns) rather than relying on confidence-based filtering
• Temporarily disabling high false-positive categories to restore developer trust while improving prompts for those categories
• Defining explicit severity criteria with concrete code examples for each severity level to achieve consistent classification

What You Need to Know

Vague instruction is the most expensive habit in production prompt engineering. "Be conservative." "Only report high-confidence findings." "Use your best judgement." Each reads like sound direction and none of them gives the model a boundary it can apply, which is precisely why the exam offers them as answers.

What works instead is explicit categorical criteria: a statement of what gets reported and what does not. Set the two side by side for a CI/CD code review pipeline:

Wrong approach:

Review this code. Be conservative. Only report high-confidence findings.

Correct approach:

Flag comments only when claimed behaviour contradicts actual code behaviour.
Report bugs and security vulnerabilities.
Skip minor style preferences and local patterns.

Nothing in the first version can be acted on. "Conservative" resolves differently depending on what the model assumes the reviewer cares about, and "high-confidence" asks for a threshold against a scale the model has no way to calibrate. The second names categories to report — bugs, security — categories to leave alone — style, local patterns — and one precise trigger for comment findings, namely a contradiction between what a comment claims and what the code does.

The False Positive Trust Problem

Noise in one category does not stay in that category, and this is the part the exam presses on. Where "documentation mismatch" findings are wrong 40% of the time, developers stop reading the security findings too — even at 98% accuracy. Trust attaches to the output as a whole, so the weakest category sets the credibility of the strongest.

The remedy runs against instinct: temporarily disable the high false-positive categories while their prompts are reworked. Confidence in the categories that already work returns as soon as the noise stops. The broken category is then improved against concrete code examples and switched back on once its precision justifies it.

Nothing is being given up permanently. The trade is category coverage now against the credibility of everything else, and coverage is the cheaper thing to lose.

Severity Calibration with Code Examples

Severity levels need concrete code examples, because prose descriptions leave the boundary to interpretation. Compare:

Prose description (insufficient):

Critical: Issues that could cause system failures or data loss
Minor: Issues that affect code readability but not functionality

Code example approach (correct):

Critical — Unsanitised user input in SQL query:
  query = f"SELECT * FROM users WHERE id = {user_input}"

Minor — Inconsistent variable naming:
  userName vs user_name in the same module

"Could cause system failures" is a judgement the model has to make afresh on every finding, and it will not make it identically twice. A worked example fixes the boundary at a specific pattern instead, and classification stops drifting between invocations because there is nothing left to weigh.

Why Confidence-Based Filtering Fails

"Only report high-confidence findings" appears repeatedly as a distractor, and it is tempting because it sounds like disciplined engineering: filter on confidence, keep the strong signals. The problem is that a model's self-reported confidence is poorly calibrated — it will report certainty on findings that are wrong and hedge on findings that are correct, so filtering on that number discards good findings and keeps bad ones.

Confidence does have a proper job. Routing low-confidence output to human review is genuinely useful, and Task Statement 4.6 covers it. What it cannot do is establish what qualifies as a finding, because that judgement has to exist before there is anything to be confident about.

The ordering is what the exam wants: explicit criteria first, confidence-based routing second. Applying the second without the first filters an undefined set.

Deep Dive

Where the system prompt actually goes

The Messages API has no "system" role. Criteria you write for a reviewer live in the top-level system parameter of the request, alongside model, messages and max_tokens — not as the first entry in the messages array. The docs are explicit: "if you want to include a system prompt, you can use the top-level system parameter — there is no "system" role for input messages in the Messages API."

await client.messages.create({
  model: "claude-sonnet-5",
  max_tokens: 4096,
  system: REVIEW_CRITERIA, // ← explicit categorical criteria live here
  messages: [{ role: "user", content: diff }]
});

client.messages.create(
    model="claude-sonnet-5",
    max_tokens=4096,
    system=REVIEW_CRITERIA,          # ← explicit categorical criteria live here
    messages=[{"role": "user", "content": diff}],
)

Two related mechanics matter for a review pipeline. First, consecutive same-role messages are combined into a single turn rather than rejected, so appending criteria as an extra user message does not produce the separation you might expect. Second, max_tokens is a ceiling, not a target — "our models may stop before reaching this maximum" — so a short finding list is not evidence that your criteria suppressed output.

Sourceplatform.claude.com › messagesfetched 2026-07-30

The "right altitude" — the two failure modes your criteria sit between

Anthropic's context-engineering guidance names the exact failure modes the exam dramatises. At one extreme, "engineers hardcoding complex, brittle logic in their prompts to elicit exact agentic behavior. This approach creates fragility and increases maintenance complexity over time." At the other, "engineers sometimes provide vague, high-level guidance that fails to give the LLM concrete signals for desired outputs" — that is "be conservative" exactly. The target is "the Goldilocks zone between two common failure modes … specific enough to guide behavior effectively, yet flexible enough to provide the model with strong heuristics", and the stated goal is "the minimal set of information that fully outlines your expected behavior."

This gives you a principled reason why the exam's correct answer is never "add more rules": Anthropic explicitly does not recommend "stuffing a laundry list of edge cases into a prompt in an attempt to articulate every possible rule."

Sourceanthropic.com › effective-context-engineering-for-ai-agentsfetched 2026-07-30

Heuristics plus guardrails, not rigid rules

The multi-agent research write-up reaches the same conclusion from production: the prompting strategy "focuses on instilling good heuristics rather than rigid rules", paired with "explicit guardrails to prevent agents from spiraling out of control". The same post reports that early in development "a prompt tweak might boost success rates from 30% to 80%" — which is why criteria refinement, not model swapping or temperature fiddling, is the highest-leverage first move on a noisy review pipeline.

Anthropic also found that vague delegation is what produces junk: "Without detailed task descriptions, agents duplicate work, leave gaps, or fail to find necessary information." A category with a vague criterion behaves the same way — it produces overlapping, low-value findings.

Sourceanthropic.com › multi-agent-research-systemfetched 2026-07-30

Temperature is not a precision control

A recurring distractor is "lower the temperature to reduce false positives". temperature is documented as "Amount of randomness injected into the response", defaults to 1.0, and ranges 0.0–1.0; the guidance is to use values "closer to 0.0 for analytical / multiple choice, and closer to 1.0 for creative and generative tasks". Crucially, "even with temperature of 0.0, the results will not be fully deterministic." Lower temperature can tighten variance; it cannot invent a decision boundary the prompt never defined.

Sourceplatform.claude.com › messagesfetched 2026-07-30

Express categorical criteria as machine-checkable labels

Where criteria are categorical, the prompting best-practices page says to stop asking for prose: "For classification tasks, use either tools with an enum field containing your valid labels or structured outputs." A severity criterion expressed as enum: ["critical", "major", "minor"] on an extraction tool is enforced by the schema, whereas the same criterion written in prose is enforced only by the model's interpretation. Task 4.3 covers the mechanism; the point for 4.1 is that explicit criteria and enforceable enums are the same design decision at two layers.

Sourceplatform.claude.com › claude-prompting-best-practicesfetched 2026-07-30

Measure criteria changes on a small eval set

Anthropic's documented practice is to start small: "We started with a set of about 20 queries representing real usage patterns … clearly see the impact of changes", and to grade with "an LLM judge that evaluated each output against criteria in a rubric: factual accuracy, citation accuracy, completeness, source quality, and tool efficiency". A "single LLM call with single prompt outputting scores 0.0-1.0 and pass-fail grade was most consistent." Manual review still matters — "People testing agents find edge cases that evals miss."

The Agent SDK guidance ranks feedback types for you: "The best form of feedback is providing clearly defined rules for an output, then explaining which rules failed and why." That is the same instruction shape as good review criteria — say which rule fired and why — and it is stronger than an LLM judging free-form quality, which the same page calls "generally not a very robust method".

Sourcesanthropic.com › multi-agent-research-systemclaude.com › building-agents-with-the-claude-agent-sdkfetched 2026-07-30

Quick Reference

Exam Traps

Practice Scenario

Build Exercise

Build an Explicit Criteria Code Review Prompt

Difficulty: Intermediate (2/4)

45 minutes

1. Write a system prompt with vague instructions (be conservative, only flag important issues) and test it against 5 code snippets containing known bugs, security issues, and style nitpicks

Why: Establishing a baseline with vague instructions demonstrates the false positive problem the exam tests. You need empirical evidence that phrases like be conservative give the model no actionable decision boundary.

You should see: Inconsistent classification across the 5 snippets: some style nitpicks flagged as critical, some genuine bugs missed or marked minor, and different results if you run the same snippets twice.

2. Rewrite the prompt with explicit categorical criteria: define exactly which issues to report (bugs, security vulnerabilities) and which to skip (style preferences, local patterns)

Why: Explicit categorical criteria are the correct approach tested on the exam. This step demonstrates that concrete categories eliminate the ambiguity that causes false positives.

You should see: The rewritten prompt has clear categories: report bugs and security vulnerabilities, skip style preferences and local patterns, flag comments only when claimed behaviour contradicts actual code behaviour.

3. Add concrete code examples for each severity level — critical, major, minor — showing actual code patterns, not prose descriptions

Why: The exam specifically tests that code examples outperform prose descriptions for severity calibration. Prose like issues that could cause system failures forces the model to interpret, while code examples remove ambiguity entirely.

You should see: Your prompt now contains at least one code snippet per severity level, each showing the actual pattern that defines that severity, not a prose description of what that severity means.

4. Compare false positive rates between the two versions on the same test set and document which approach produces more consistent classification

Why: Quantifying the improvement validates the explicit criteria approach and builds the evaluation skill the exam expects. You should be able to articulate why one approach outperforms the other with data, not intuition.

You should see: A clear reduction in false positives with the explicit criteria version. The vague prompt should produce 30-50% inconsistency while the explicit criteria version should be below 15%. Classification should be stable across repeated runs.

5. Temporarily disable any category with above 25% false positive rate and document the criteria refinements needed before re-enabling

Why: The trust recovery strategy is a key exam concept: high false positive rates in one category destroy developer trust in ALL categories. Disabling problematic categories restores system-wide trust while you iterate on their criteria.

You should see: A document listing which categories exceed the 25% threshold, what specific criteria refinements are needed (e.g., add code examples for edge cases), and a re-enablement plan with target false positive rates.

Sources

• Claude Certified Architect Foundations Exam Guide — Task Statement 4.1 — Anthropic
• Prompt Engineering Overview — Anthropic
• Building with Claude API (Skilljar) — Anthropic

Appendix A — Build Exercise Step Hints

Progressive hints revealed by the "Stuck? Get a nudge" control on each step.

Step 1. Write a system prompt with vague instructions (be conservative, only flag important issues) and test it against 5 code snippets containing known bugs, security issues, and style nitpicks

Why: Establishing a baseline with vague instructions demonstrates the false positive problem the exam tests. You need empirical evidence that phrases like be conservative give the model no actionable decision boundary.

You should see: Inconsistent classification across the 5 snippets: some style nitpicks flagged as critical, some genuine bugs missed or marked minor, and different results if you run the same snippets twice.

const systemPrompt = `Review this code. Be conservative. Only report high-confidence findings.`;

const testSnippets = [
  { code: "query = f\"SELECT * FROM users WHERE id = {user_input}\"", expectedSeverity: "critical" },
  { code: "const unused_var = 42;", expectedSeverity: "minor" },
  { code: "let userName = getUser(); let user_name = userName;", expectedSeverity: "style" }
];

system_prompt = "Review this code. Be conservative. Only report high-confidence findings."

test_snippets = [
    {"code": 'query = f"SELECT * FROM users WHERE id = {user_input}"', "expected_severity": "critical"},
    {"code": "const unused_var = 42;", "expected_severity": "minor"},
    {"code": "let userName = getUser(); let user_name = userName;", "expected_severity": "style"},
]

Step 2. Rewrite the prompt with explicit categorical criteria: define exactly which issues to report (bugs, security vulnerabilities) and which to skip (style preferences, local patterns)

Why: Explicit categorical criteria are the correct approach tested on the exam. This step demonstrates that concrete categories eliminate the ambiguity that causes false positives.

You should see: The rewritten prompt has clear categories: report bugs and security vulnerabilities, skip style preferences and local patterns, flag comments only when claimed behaviour contradicts actual code behaviour.

const systemPrompt = `Flag comments only when claimed behaviour contradicts actual code behaviour.
Report: bugs, security vulnerabilities, logic errors.
Skip: minor style preferences, local naming patterns, formatting choices.
Do not flag unused imports unless they shadow a used import.`;

system_prompt = """Flag comments only when claimed behaviour contradicts actual code behaviour.
Report: bugs, security vulnerabilities, logic errors.
Skip: minor style preferences, local naming patterns, formatting choices.
Do not flag unused imports unless they shadow a used import."""

Step 3. Add concrete code examples for each severity level — critical, major, minor — showing actual code patterns, not prose descriptions

Why: The exam specifically tests that code examples outperform prose descriptions for severity calibration. Prose like issues that could cause system failures forces the model to interpret, while code examples remove ambiguity entirely.

You should see: Your prompt now contains at least one code snippet per severity level, each showing the actual pattern that defines that severity, not a prose description of what that severity means.

Add to your prompt:

Critical — Unsanitised user input in SQL query:
  query = f"SELECT * FROM users WHERE id = {user_input}"

Major — Missing null check before property access:
  const name = response.data.user.name; // no null guard

Minor — Inconsistent variable naming:
  userName vs user_name in the same module

Step 4. Compare false positive rates between the two versions on the same test set and document which approach produces more consistent classification

Why: Quantifying the improvement validates the explicit criteria approach and builds the evaluation skill the exam expects. You should be able to articulate why one approach outperforms the other with data, not intuition.

You should see: A clear reduction in false positives with the explicit criteria version. The vague prompt should produce 30-50% inconsistency while the explicit criteria version should be below 15%. Classification should be stable across repeated runs.

const results = { vague: { tp: 0, fp: 0, fn: 0 }, explicit: { tp: 0, fp: 0, fn: 0 } };
// Run both prompts against each snippet
for (const snippet of testSnippets) {
  const vagueResult = await classify(vaguePrompt, snippet);
  const explicitResult = await classify(explicitPrompt, snippet);
  // Compare against expected severity
  updateMetrics(results.vague, vagueResult, snippet.expectedSeverity);
  updateMetrics(results.explicit, explicitResult, snippet.expectedSeverity);
}

results = {"vague": {"tp": 0, "fp": 0, "fn": 0}, "explicit": {"tp": 0, "fp": 0, "fn": 0}}

# Run both prompts against each snippet
for snippet in test_snippets:
    vague_result = classify(vague_prompt, snippet)
    explicit_result = classify(explicit_prompt, snippet)
    # Compare against expected severity
    update_metrics(results["vague"], vague_result, snippet["expected_severity"])
    update_metrics(results["explicit"], explicit_result, snippet["expected_severity"])

Step 5. Temporarily disable any category with above 25% false positive rate and document the criteria refinements needed before re-enabling

Why: The trust recovery strategy is a key exam concept: high false positive rates in one category destroy developer trust in ALL categories. Disabling problematic categories restores system-wide trust while you iterate on their criteria.

You should see: A document listing which categories exceed the 25% threshold, what specific criteria refinements are needed (e.g., add code examples for edge cases), and a re-enablement plan with target false positive rates.

const categoryMetrics = {
  documentation_mismatch: { fp_rate: 0.40, action: "disable" },
  security_vulnerability: { fp_rate: 0.05, action: "keep" },
  logic_error: { fp_rate: 0.12, action: "keep" }
};

// Refinement plan for disabled categories
const refinements = {
  documentation_mismatch: {
    issue: "Flagging outdated comments as mismatches when code is correct",
    fix: "Add examples distinguishing stale comments from genuine contradictions",
    targetFpRate: 0.15
  }
};

category_metrics = {
    "documentation_mismatch": {"fp_rate": 0.40, "action": "disable"},
    "security_vulnerability": {"fp_rate": 0.05, "action": "keep"},
    "logic_error": {"fp_rate": 0.12, "action": "keep"},
}

# Refinement plan for disabled categories
refinements = {
    "documentation_mismatch": {
        "issue": "Flagging outdated comments as mismatches when code is correct",
        "fix": "Add examples distinguishing stale comments from genuine contradictions",
        "target_fp_rate": 0.15,
    }
}

Appendix B — Interactive Study Prompts

Two prompts to paste into Claude. B1 drills the judgement the exam actually measures; B3 reviews the prompts you wrote for the Build Exercise above. The exam simulator between them is the interactive quiz on this page.

B1. Concept Check — Discrimination Drill

B2. Exam Simulator

B3. Build Coach — Prompt Review

The Build Exercise and its hint ladder are already on this page. This prompt is for the one thing the page cannot do: review the criteria you actually wrote.