Context Management & Reliability flashcards

Every card in this deck

The whole deck as a list, for scanning or printing.

What context window size do Claude Fable 5, Opus 5, and Sonnet 5 ship by default?

1,000,000 tokens — the default, no beta header required, billed at standard pricing.

What context window size does Claude Haiku 4.5 have?

200,000 tokens.

What counts toward the context window on every request?

The system prompt, every message (incl. tool results, images, documents), tool definitions, and the model's output for the turn including extended thinking.

Do cached prompt prefixes still count toward the context window?

Yes — prompt caching changes what you pay for those tokens, not whether they count toward the window.

What happens if the input ALONE exceeds the model's context window?

A 400 invalid_request_error ('prompt is too long') on every model.

On Claude 4.5+ models, what happens if input + max_tokens exceeds the window?

The request is accepted; if generation reaches the limit, it stops with stop_reason 'model_context_window_exceeded'.

What API endpoint lets you estimate token usage before sending a request?

The token counting API — POST /v1/messages/count_tokens (free, separate rate limit from message creation).

What is 'context rot'?

Recall accuracy degrades as the number of tokens in context increases — more context is not automatically better.

What is the 'attention budget' concept, and what causes it to deplete?

LLMs have a limited attention budget; every added token depletes it — driven by n² pairwise token relationships stretching thin at long context lengths.

Which Claude models have automatic 'context awareness'?

Sonnet 5, Sonnet 4.6, Sonnet 4.5, and Haiku 4.5 — they track their remaining token budget via API-injected tags, with nothing to enable.

What is 'compaction' (Anthropic's official term)?

Taking a conversation nearing the context window limit, summarising its contents, and reinitiating a new context window with the summary.

What is the documented risk of compaction?

Overly aggressive compaction can lose subtle but critical context whose importance only becomes apparent later.

What is the safest, lightest-touch form of compaction?

Tool result clearing — dropping old tool outputs while keeping the rest of conversation history.

What does a persistent 'case facts' block protect, and why does it sit outside summarised history?

Transactional facts (amounts, dates, order numbers, statuses) — because progressive summarisation destroys exactly this kind of numerical precision.

What is the structural (not prompt-based) fix for the lost-in-the-middle effect?

Place key findings summaries at the beginning of aggregated inputs, with explicit section headers throughout.

Name the three valid escalation triggers for a support agent.

Explicit customer request for a human · policy exception/gap · inability to make meaningful progress.

Name the two unreliable escalation triggers.

Sentiment-based escalation (frustration detection) and self-reported model confidence scores.

A customer explicitly says 'I want a human.' What should the agent do?

Escalate immediately — no investigation, no 'let me try first.' This is an absolute rule with no exceptions.

Distinguish a policy gap from a policy violation for escalation purposes.

A gap means policy is silent on the situation (escalate); a violation has a documented 'no' answer (does not require escalation).

A customer is frustrated but the issue is straightforward. What's the correct move?

Acknowledge the frustration and offer the resolution directly — do not escalate. Escalate only if they then reiterate wanting a human.

A tool returns multiple matching customer records. What should the agent do?

Ask for additional identifiers (email, phone, order number) — never select based on heuristics like most-recent or most-active.

What Agent SDK mechanism guarantees an escalation check runs on every tool call, deterministically?

A PreToolUse hook — hooks are used to 'require human approval for sensitive actions' rather than relying on the model to remember.

A PreToolUse hook should route a sensitive action to a human rather than kill it outright. Which permissionDecision value does that, and what do the other three do?

ask — it surfaces the call for human approval. deny blocks it, allow lets it through, defer ends the query so you can resume it later.

Why is canUseTool the wrong place to put a mandatory escalation check?

It only fires when the permission flow falls through to a prompt — any call auto-approved by an allow rule, acceptEdits, or bypassPermissions skips it entirely. Use a PreToolUse hook, which runs before every other step.

What is the hook decision priority order?

deny > defer > ask > allow. A deny blocks the operation even in bypassPermissions mode.

What must a structured handoff to a human agent include?

Customer ID, root cause analysis, and recommended action (plus amount, where relevant) — since the human lacks the conversation transcript.

What's the proportionate first response to poor escalation calibration, before adding classifiers or sentiment models?

Add explicit escalation criteria with few-shot examples to the system prompt.

Name the four elements of structured error context a subagent should return on failure.

Failure type · what was attempted · partial results gathered · alternative approaches.

Name the four failure type categories.

Transient (retry candidate) · validation (bad input) · business (rule violation) · permission (access denied).

Distinguish an access failure from a valid empty result.

Access failure: the tool couldn't reach the source (timeout/connection error) — consider retry. Valid empty result: the query executed and found nothing — this IS the answer, no retry.

How does the Messages API signal a client tool execution failure?

Return the error message as tool_result content along with 'is_error': true.

Do you need to handle is_error results for server tools (web_search, code_execution, etc.)?

No — server tools run on Anthropic's infrastructure and Claude handles their errors transparently.

When a tool_result reports an invalid/missing-parameter call, what does Claude do by default?

Retries 2-3 times with corrections before apologising to the user.

What is the 'silent suppression' anti-pattern, and why is it the worst one?

Returning empty results marked as success after a failure (e.g. a timeout) — it's invisible, so the coordinator never retries or tries alternatives.

What is the 'workflow termination' anti-pattern?

Killing the entire pipeline because one subagent failed, discarding results other subagents completed successfully.

What should subagents do with transient failures before propagating to the coordinator?

Attempt local recovery (retry logic, fallback sources); only propagate errors they cannot resolve, including what was attempted and any partial results.

What are coverage annotations in synthesis output for?

Explicitly noting which topic areas are well-supported vs which have gaps due to unavailable sources, instead of silently omitting the gap.

Why does Anthropic say agent errors are especially dangerous if unhandled?

Because agents are stateful — minor failures can be catastrophic, and 'we can't just restart from the beginning.'

What is 'resume-from-failure' / durable execution?

Building systems that resume from where the agent was when the error occurred, instead of restarting from scratch.

What did Anthropic observe about a single failed step in production agent systems?

One step failing can send agents onto an entirely different trajectory — the 'prototype-to-production gap' is wider than anticipated.

What does 'context degradation' look like in an extended codebase-exploration session?

The model starts referencing 'typical patterns' instead of the specific classes, methods, and file paths it discovered earlier.

Is context degradation a token-limit problem?

No — it's the model losing grip on specific findings as verbose output accumulates. A larger window still fills with verbose output.

What is Anthropic's official term for 'scratchpad files'?

Structured note-taking, or agentic memory — writing notes persisted outside the context window that the agent reads back later.

What kind of work does structured note-taking excel at, per Anthropic's guidance on choosing among the three long-horizon techniques?

Iterative development with clear milestones.

What is the PRIMARY benefit of subagent delegation for codebase exploration?

Context isolation — keeping the main agent's context clean while subagents handle verbose exploration. Parallelisation is secondary.

How much does a subagent return to the coordinator relative to what it explored?

It may use tens of thousands of tokens exploring but returns only a condensed, distilled summary.

What does Claude Code's /clear command do?

Starts fresh with an empty context; the previous conversation is saved and resumable via /resume.

What does /compact [instructions] do, and give an example.

Replaces history with a summary, optionally focused on what you specify — e.g. '/compact Focus on the API changes'.

When is /compact best used — proactively or only at the context limit?

Proactively, during extended sessions, to protect context quality — not only as a last resort.

What creates a Claude Code checkpoint, and what does /rewind restore?

Every user prompt creates a checkpoint; /rewind can restore code and conversation, conversation only, or code only.

What is the blind spot of Claude Code's checkpointing?

It does not track files modified by Bash commands (rm, mv, cp, etc.) — only direct edits through Claude's file editing tools.

What content does a subagent's context window start with when spawned?

Fresh, with no parent conversation — the only content passed in is the Agent tool's prompt string.

Why is summary injection needed between exploration phases?

Because Phase 2 subagents don't automatically inherit Phase 1 findings; without an injected summary they'd duplicate Phase 1 exploration (cold start problem).

What does a crash-recovery state manifest contain?

Explored paths, key findings, current phase, and next steps — loaded by the coordinator on resume and injected into agent prompts.

Why is the 'aggregate metrics trap' dangerous — e.g. a system reporting 97% overall accuracy?

The aggregate can hide 40-60% error rates on specific document types, because high-volume, high-accuracy segments dominate the weighted average.

What's the correct validation sequence before automating extractions?

Measure accuracy by type+field → calibrate confidence with validation sets → set thresholds → stratified sampling → only then reduce human review.

Should stratified sampling include high-confidence extractions?

Yes — high-confidence items are automated and unreviewed; they're the blind spot where novel error patterns go undetected without sampling.

What does calibrating a confidence score require?

A labelled validation set (ground truth) — comparing the model's reported confidence to actual accuracy per field/document type.

What LLM-as-judge scoring format did Anthropic find most consistent for scalable evaluation?

A single LLM call, single prompt, outputting a 0.0-1.0 score plus a pass-fail grade.

Name the five dimensions in Anthropic's research-agent evaluation rubric.

Factual accuracy, citation accuracy, completeness, source quality, tool efficiency.

Does automated evaluation (evals, LLM judges) make manual human testing unnecessary?

No — people testing agents find edge cases that evals miss; manual testing remains essential even with automation in place.

What is a detected_pattern field used for?

Tagging what specifically triggered a finding, enabling systematic analysis of dismissal/error patterns rather than reviewing anomalies one by one.

What does 'calibrated review routing' mean when a model self-reports confidence?

The self-reported score is checked against a labelled validation set before being used as a routing threshold — raw confidence alone is not a routing signal.

What is the documented fix for false-positive rates eroding developer trust?

Explicit severity criteria with concrete examples for each level — not a single blanket confidence threshold.

How should limited reviewer capacity be allocated?

Route the highest-uncertainty items first — never spread capacity evenly across all extractions.

Name the five fields of a structured claim-source mapping.

Claim, source URL, document name, relevant excerpt, publication date.

At which step of a multi-agent pipeline does attribution most commonly get lost?

Step 3 — synthesis, where an agent compresses and paraphrases findings without explicitly preserving claim-source mappings.

What is the CitationAgent pattern?

A dedicated pipeline stage that runs after synthesis, whose sole job is locating specific citation points for the report's claims.

What are 'source quality heuristics' and why were they added?

Prompt-level guardrails against picking SEO content farms over authoritative sources — early agents consistently favoured the former until this was fixed.

What is the artifact/filesystem output pattern, and what does it prevent?

Subagent outputs bypass the main coordinator for certain results, improving fidelity and cutting token overhead — it prevents information loss from repeated copying through multi-stage processing.

Two credible sources report different statistics for the same measure. What's the correct handling?

Annotate with both values and full source attribution — never arbitrarily select one, average them, or pick the more recent/authoritative source.

Source A (2023) reports 8% growth; Source B (2024) reports 12% growth. Is this a contradiction?

Not necessarily — different publication dates can explain different numbers as a trend, not a contradiction. Publication dates must be preserved to interpret this correctly.

What does a conflict_detected boolean field enable in a structured schema?

Distinguishing a field that is genuinely in conflict (two credible but differing values) from one that is simply null/absent.

How should financial data, news, and technical findings each be rendered in a synthesis report?

Financial data as tables, news as prose, technical findings as structured lists — never flatten everything into one uniform format.

What structural practice preserves attribution when passing context between agents?

Using structured data formats that separate content from metadata (source URLs, document names, page numbers) rather than blending them into prose.