Context Management & Reliability cheat sheet

5.1 — Context window management

• Counts toward the window: system prompt + every message (incl. tool results/images/docs) + tool definitions + output (incl. thinking). Cached tokens still count — caching changes cost, not window usage.
• Overflow: input alone > window → 400 invalid_request_error. Input + max_tokens > window (Claude 4.5+) → request accepted, stops with stop_reason: "model_context_window_exceeded". Use the token counting API (/v1/messages/count_tokens, free) to stay under limits.
• Context rot: accuracy/recall degrades as tokens grow — driven by a limited "attention budget" and n² pairwise attention stretching thin. Curate, don't just expand.
• Context awareness (Sonnet 5/4.6/4.5, Haiku 4.5): models auto-track remaining token budget via API-injected tags — nothing to enable.
• Compaction = summarise a conversation nearing the limit, reinitiate with the summary. Risk: over-aggressive compaction loses subtle context that matters later. Lightest-touch form = tool result clearing.
• Progressive summarisation trap: destroys amounts/dates/IDs. Fix = persistent case facts block, outside summarised history, never summarised.
• Lost in the middle: models handle beginning/end reliably, miss the middle. Fix is structural — key findings summary FIRST, then detailed sections with headers. Not a prompt reminder.
• Tool result trimming: strip 40+ field lookups to the 5 relevant fields before they enter context — do it in a PostToolUse hook or the tool itself.
• API is stateless — every request needs full conversation history; selective truncation breaks coherence.

5.2 — Escalation & ambiguity resolution

Three valid triggers: explicit human request (escalate immediately, zero investigation) · policy gap (policy silent — not the same as a policy violation, which has a documented "no") · inability to make progress (after a genuine attempt).

Two unreliable triggers: sentiment/frustration detection (doesn't correlate with complexity) · self-reported confidence (poorly calibrated — confident on hard cases, hedges on easy ones).

Frustration nuance:

• Ambiguous customer match (multiple records): ask for additional identifiers (email/phone/order number). Never select by recency/activity heuristic — privacy risk.
• Proportionate first fix: explicit escalation criteria + few-shot examples in the system prompt, before classifiers or sentiment models.
• Deterministic enforcement: PreToolUse hook > prompt instruction, when escalation/compliance must be guaranteed. permissionDecision: allow/deny/ask/defer. canUseTool fires only when the flow falls through to a prompt — auto-approved calls skip it.
• Hook priority: deny > defer > ask > allow; a deny blocks even under bypassPermissions.
• Structured handoff must include: customer ID, root cause analysis, recommended action (+ amount) — the human has no transcript access.

5.3 — Error propagation in multi-agent systems

Four structured error elements: failure type · what was attempted · partial results · alternative approaches.

Four failure types: transient (retry) · validation (fix input) · business (escalate) · permission (auth change needed).

• Access failure (timeout/connection error) → consider retry. Valid empty result (query ran, found nothing) → this IS the answer, no retry. Conflating these = wasted retries or missed recovery.
• is_error: true in tool_result content = client-tool failure signal. Server tools (web_search, code_execution, etc.) — Anthropic handles their errors; you don't need is_error for them.
• Invalid/malformed tool call → Claude auto-retries 2-3 times with corrections before apologising.
• Anti-pattern 1 (worst): silent suppression — empty results marked success; invisible, coordinator never retries.
• Anti-pattern 2: workflow termination — killing the whole pipeline on one failure, discarding completed work.
• Correct pattern: subagents attempt local recovery first; propagate only unresolved errors with what-was-attempted + partial results.
• Coverage annotations: synthesis explicitly flags gaps ("limited due to X") rather than silently omitting them.
• Agents are stateful — errors compound; you can't just restart from scratch. Fix = resume-from-failure / durable execution. One bad step can send an agent onto an unpredictable trajectory (the "prototype-to-production gap").

5.4 — Codebase exploration & context degradation

• Context degradation: model starts saying "typical pattern" instead of citing the specific class/file it found earlier. Not a token-limit problem — a bigger window doesn't fix it.
• Scratchpad files = structured note-taking / "agentic memory" (Anthropic's term). Best fit: iterative work with clear milestones. Deliberate from the start, not a rescue move.
• Subagent delegation: primary value is context isolation, not parallelisation. Subagent may burn 10k+ tokens exploring, returns only a condensed summary. Fresh context at spawn — only the Agent tool's prompt string passes in, no auto-inherited parent history.
• Summary injection between phases: inject Phase 1's findings into Phase 2 subagent prompts to avoid cold-start duplication.
• /clear — empty context, previous conversation saved, resumable via /resume.
• /compact [instructions] — replaces history with a (optionally focused) summary, e.g. /compact Focus on the API changes. Use proactively, not just at the limit.
• Checkpoints: auto-created per prompt; /rewind restores code/conversation/both — "local undo," not git. Blind spot: does not track Bash-made changes (rm/mv/cp), only direct file-tool edits.
• Crash recovery manifest: explored paths + key findings + phase + next steps, loaded by the coordinator on resume.

5.5 — Human review & confidence calibration

• Aggregate metrics trap: 97% overall can hide 40-60%+ error rates on specific document types (handwritten receipts, international formats). Always validate by document type AND field.
• Validation sequence: measure by type+field → calibrate confidence (labelled validation sets) → set thresholds → stratified sampling → only then reduce review.
• Stratified sampling must include high-confidence extractions — that's the automated blind spot where novel error patterns go undetected.
• Raw confidence ≠ calibrated confidence. 0.90 on dates might mean 94% accuracy; 0.90 on amounts might mean 82%. Calibration requires ground-truth comparison.
• LLM-as-judge template: one call, one prompt, 0.0-1.0 score + pass/fail grade — most consistent at scale.
• Evaluation rubric dimensions (transferable to validation design): factual accuracy, citation accuracy, completeness, source quality, tool efficiency.
• Human testing stays essential even with automated evals — it catches edge cases evals miss.
• detected_pattern field: tags what triggered a finding, enabling systematic dismissal-pattern analysis instead of one-off review.
• Reviewer capacity: route highest-uncertainty items first, dynamically — never spread capacity evenly.

5.6 — Information provenance & multi-source synthesis

Five claim-source mapping fields: claim · source URL · document name · relevant excerpt · publication date.

• Attribution most often dies at synthesis (step 3) — compression/paraphrasing drops the mapping unless explicitly preserved.
• CitationAgent pattern: a dedicated post-synthesis stage whose only job is locating citations — routes around the synthesis attribution-loss risk.
• Source quality heuristics: guard against SEO content farms outranking authoritative sources — correct claim-source mapping to a bad source is still misleading.
• Artifact/filesystem output pattern: large subagent outputs bypass the coordinator, preserving fidelity and cutting token overhead from repeated copying.
• Conflicting sources: annotate BOTH values with attribution — never arbitrarily pick, average, or prefer "more recent."
• Temporal awareness: different publication dates can explain different numbers as a trend, not a contradiction — publication/collection dates are required in structured outputs.
• conflict_detected boolean: distinguishes "genuinely in conflict" from "simply absent/null."
• Content-appropriate rendering: financial data → tables · news → prose · technical findings → structured lists. Never flatten to one format.
• Preserve attribution across agent hand-offs by keeping content and metadata (source URL, doc name, page) as separate structured fields, not blended prose.