Domain 5 · 15% of the exam · ~9 of 60 questions
Context Management & Reliability
Context window management, escalation and ambiguity, error propagation, codebase exploration, human review calibration, and provenance.
What this domain covers
The smallest domain, and the one most concerned with what happens when things degrade rather than when they work. Context degradation is not the same problem as running out of tokens, and the exam tests that distinction directly. The rest of the domain is about judgement under uncertainty: when an agent should stop and ask a human, how confident a confidence score really is, and how to keep a claim attached to the source it came from once several sources disagree.
What the exam asks here
Drawn from the in-scope topic list in the official exam guide.
- Context window management: token budgets, progressive summarisation, lost-in-the-middle effects, and position-aware input ordering
- Trimming verbose tool output and extracting structured facts instead of carrying raw text forward
- Escalation and ambiguity: explicit criteria, honouring stated customer preferences, and identifying policy gaps
- Error propagation across multi-agent systems, and recovering locally before surfacing a failure upward
- Human review workflows: confidence calibration against labelled sets, stratified sampling, and accuracy by field and document type
- Information provenance: claim-to-source mappings, temporal data, conflict annotation and coverage-gap reporting
The 6 task statements in this domain
- 5.1Context Window ManagementManage conversation context to preserve critical information across long interactions
- 5.2Escalation & Ambiguity ResolutionDesign effective escalation and ambiguity resolution patterns
- 5.3Error Propagation in Multi-Agent SystemsImplement error propagation strategies across multi-agent systems
- 5.4Codebase Exploration & Context DegradationManage context effectively in large codebase exploration
- 5.5Human Review & Confidence CalibrationDesign human review workflows and confidence calibration
- 5.6Information Provenance & Multi-Source SynthesisPreserve information provenance and handle uncertainty in multi-source synthesis
Common questions about Domain 5
How much of the CCAR-F exam is Context Management & Reliability?
15% — the lightest domain, about 9 of the 60 items. It is small enough to leave until last and important enough not to skip, because its questions are the ones that punish guessing.
Is context degradation the same as running out of context window?
No, and the exam tests the difference directly. Running out of window is a capacity problem with a mechanical fix. Degradation happens well inside the limit: as a session grows, earlier instructions compete with accumulated tool output and the model starts to lose the thread — the lost-in-the-middle effect. Trimming verbose output and extracting structured facts addresses it; raising the limit does not.
When should an agent escalate to a human?
Against criteria decided in advance, not on a feeling of uncertainty. Genuine policy gaps, explicit customer preferences that conflict with the default path, and irreversible actions are the standard triggers. An agent that escalates whenever it is unsure escalates constantly, which is its own failure mode.
What makes a confidence score trustworthy?
Calibration against a labelled validation set, at field level rather than per document. A model that reports 0.9 should be right about nine times in ten; without measuring that, the number is a preference, not a probability. Stratified sampling is how you measure error rates across document types instead of averaging them into invisibility.