Claude Certified Developer – Foundations: The Complete Study Guide

A comprehensive, exam-aligned study guide preparing developers for the Anthropic Claude Certified Developer – Foundations (CCDV-F) exam across all eight content domains.

Table of Contents

• Chapter 1: Introduction: The Claude Platform and the CCDV-F Exam
• Chapter 2: LLM Fundamentals for Claude Developers
• Chapter 3: Claude Model Selection and Tradeoffs
• Chapter 4: Cost, Tokens, Caching, and Batch Processing
• Chapter 5: Claude API Mechanics
• Chapter 6: Software Engineering Foundations for Claude Integration
• Chapter 7: Designing Claude Applications
• Chapter 8: Configuration Management and Claude Code
• Chapter 9: Prompt Engineering
• Chapter 10: Context Engineering, Output Handling, and Debugging
• Chapter 11: Agent Architecture, Patterns, and Frameworks
• Chapter 12: Building Agents with Claude: SDK, Loops, and Hooks
• Chapter 13: Tools and MCP Servers
• Chapter 14: Security, Safety, and Guardrails

Chapter 1: Introduction: The Claude Platform and the CCDV-F Exam

Welcome to your preparation for the Claude Certified Developer – Foundations exam. Before you write a single practice prompt or spin up your first agent, it pays to understand what you are being tested on, why the exam is structured the way it is, and how the technology under the hood actually fits together. This chapter is your map. By the end of it you will know exactly what the credential proves, how the exam is scored, how its content is weighted across eight domains, and where each piece of the Claude platform lives in the bigger picture.

Think of this chapter as the “table of contents for your effort.” Every hour you spend studying should be aimed at a target, and this chapter defines the targets.

Learning Objectives

• Describe the purpose, audience, and value of the Claude Certified Developer – Foundations credential, and identify the profile of the Minimally Qualified Candidate.
• Explain the exam format, the criterion-referenced scaled-scoring model, and the eight weighted content domains defined in the exam blueprint.
• Map the Minimally Qualified Candidate’s capabilities to a personalized, weight-driven study plan.
• Recognize the major components of the Claude platform — the API, SDKs, Claude Code, the Agent SDK, MCP, and the Opus/Sonnet/Haiku model families — and explain how they combine to form production applications.

The Certification and Its Value

Purpose of the CCDV-F Credential

The Claude Certified Developer – Foundations certification — exam code CCDV-F — validates that an individual can build, integrate, and ship production-grade applications, agents, and workflows using Anthropic’s Claude platform at a foundational level [Source: source-material.txt]. The word “foundational” is deliberate: this is not a proof that you are a world-class AI researcher, but a proof that you can competently and independently own or significantly contribute to real, shipped Claude-powered systems.

The credential exists to provide an independent assessment of the knowledge, skills, and abilities required to build Claude-based applications competently. In practical terms, earning it sends a clear signal to employers, clients, and teammates: this person can translate a business requirement into a working system that uses Claude — through API integration, agent and tool construction, prompt and context engineering, evaluation, security, and model selection [Source: source-material.txt].

A helpful real-world analogy: think of a private pilot’s license. It does not certify that you can fly a fighter jet, but it does certify that you can safely and independently operate an aircraft under normal conditions. The CCDV-F is the “private pilot’s license” of Claude development — it says you can safely take a Claude application from the runway of an idea to the cruising altitude of production.

Because the underlying technology evolves rapidly, the credential is time-limited: it is valid for 12 months from the date it is awarded [Source: source-material.txt]. On-time renewal is a free, non-proctored assessment on the Anthropic Partner Academy — the official training and enablement channel where the exam is registered and where preparation resources are offered. If your credential lapses, you must retake the full exam at full fee to regain certified status.

Intended Audience and the Minimally Qualified Candidate Profile

The exam is targeted at what standardized-testing professionals call the Minimally Qualified Candidate (MQC) — the theoretical person who has just barely enough skill to be considered competent. Every question on the exam is calibrated to this person. If you know more than the MQC, you should pass comfortably; if you know less, the exam is designed to detect that.

The MQC for CCDV-F is described as “a hands-on technical individual who builds, integrates, and ships Claude-powered applications, agents, and workflows [and who] bridge[s] Claude’s capabilities and production-ready applications, translating technical requirements into working applications” [Source: source-material.txt]. Concretely, the audience is AI and machine learning engineers, technical leads, and senior software engineers operating at the intersection of business requirements and technical implementation [Source: source-material.txt].

Just as important is who the certification is not for. It is explicitly not intended for non-technical or casual users of Claude applications, for individuals without hands-on software development experience, or for roles limited to prompt writing alone without broader application-development responsibility [Source: source-material.txt]. If your entire relationship with Claude is typing questions into a chat box, this exam is not aimed at you.

Recommended Experience and Prerequisites

There are no mandatory prerequisites or courses required to sit this exam — the credential is awarded based on exam performance alone [Source: source-material.txt]. That said, Anthropic publishes a recommended experience profile, and it is worth measuring yourself against it honestly before you schedule.

[Source: source-material.txt]

The MQC “possesses strong foundational knowledge and applied skills in software development,” including the ability to build agents and workflows using the Claude Agent SDK and agentic frameworks, integrate Claude through the API and client SDKs, operate Claude Code for codebase modernization, write effective prompts and apply context engineering, design and run evaluations (“evals”), and build custom tools and MCP servers. Crucially, the MQC understands tradeoffs — in model selection (cost, latency, capability) and in tool type (built-in tools, custom tools, Skills, MCPs) — and can apply appropriate patterns to meet technical requirements [Source: source-material.txt].

Notice how often the word tradeoff appears. That is a strong hint about the exam’s cognitive level: it rarely asks “what is X?” and far more often asks “given these constraints, which of these four reasonable-sounding approaches is best?”

Key Takeaway: The CCDV-F credential is an independent, foundational proof that you can build and ship production Claude applications, aimed at working engineers with roughly 1–5 years of software experience and at least six months on Claude. There are no hard prerequisites — but the exam is calibrated to a Minimally Qualified Candidate who understands real-world tradeoffs, not just definitions.

Exam Format and Scoring

The Structure: 53 Items in 120 Minutes

The mechanics of the exam are straightforward, and knowing them removes test-day surprises.

[Source: source-material.txt]

Two item formats appear. Multiple-choice items ask you to select a single best answer. Multiple-response items ask you to select more than one correct answer — and, helpfully, each item tells you exactly how many responses to select, so you are never guessing at the count [Source: source-material.txt].

Do the arithmetic on pacing: 120 minutes across 53 items is roughly 2 minutes and 15 seconds per item. That is generous for a definition question and tight for a dense scenario. The practical strategy is to answer the ones you know quickly, bank the extra time, and spend it on the multi-part scenario questions that dominate the heavily weighted domains.

Scaled Scoring and the 720 Cut Score

Your result is reported as a scaled score on a range of 100 to 1,000, and you must reach 720 to pass [Source: source-material.txt]. A scaled score is not the same as “percent correct.” A raw score (say, “you got 41 of 53 right”) is converted through a statistical process into a point on the 100–1,000 scale. This conversion lets Anthropic keep the difficulty of passing constant even when different candidates receive slightly different question sets — an easier form requires a few more correct answers to hit 720, and a harder form requires a few fewer.

A common misconception is that 720 out of 1,000 means “you need 72% correct.” That is not what a scaled score means. The raw percentage of questions you must answer correctly to reach a scaled 720 is set by the standard-setting study, not by simple division. Do not walk into the exam assuming you can miss 28% of questions.

Criterion-Referenced Assessment and Domain-Level Reporting

The CCDV-F is a criterion-referenced assessment. This is one of the most important concepts in this chapter, so let us define it precisely: in a criterion-referenced exam, each candidate is measured against a fixed performance standard — not against other candidates [Source: source-material.txt]. You pass by demonstrating the knowledge and skills defined in the blueprint, not by outperforming some percentage of your peers.

Contrast this with a norm-referenced exam (the opposite model), such as a test “graded on a curve,” where only the top X% pass regardless of absolute skill. The table below makes the distinction concrete:

The passing standard itself was established through a formal standard-setting study, in which trained subject-matter experts judged the level of performance expected of a minimally qualified candidate [Source: source-material.txt]. This is why the MQC profile matters so much: real experts sat in a room and decided exactly how well the MQC should perform, and the 720 cut score encodes their judgment.

Your score report shows two things: your overall pass/fail status with the scaled score, and the percentage of items you answered correctly within each content domain. Critically, those per-domain percentages are provided to help you understand your performance — they are not used to determine pass/fail. Only your total scaled score decides that [Source: source-material.txt]. In other words, you cannot “fail” a single domain; you can only fail the exam as a whole. This has a strategic implication: because everything rolls up into one scaled score, it is fine to be merely adequate in a tiny domain if you are strong in the big ones.

Key Takeaway: The exam is 53 items in 120 minutes, scored on a 100–1,000 scale with a 720 cut score. It is criterion-referenced — you are measured against a fixed competency standard (the MQC), never against other candidates — and while your score report breaks out per-domain percentages, only the single total scaled score determines pass or fail.

The Eight-Domain Blueprint

Domain Weights and Relative Importance

The exam’s content blueprint is the single most valuable study document you will ever see for this credential, because it tells you exactly how the questions are distributed. The blueprint defines eight content domains, and each domain’s domain weight is the approximate proportion of scored items drawn from that domain [Source: source-material.txt]. These weights are not arbitrary — they reflect the relative importance of each domain to competent performance, as determined through a job task analysis and content-validation surveys.

[Source: source-material.txt] (item counts are approximate, derived from applying each weight to 53 items.)

The distribution is dramatically uneven, and that is the whole point. Applications and Integration alone accounts for one-third of the exam (33.1%) — more than the bottom five domains combined. The top three domains — Applications and Integration (33.1%), Model Selection and Optimization (16.8%), and Agents and Workflows (14.7%) — together make up 64.6% of the exam. Meanwhile Claude Code (3.1%) and Eval, Testing, and Debugging (2.6%) together account for under 6%.

Figure 1.1: The eight exam domains grouped by weight tier

graph TD
    Exam["CCDV-F Exam: 53 items"] --> High["High-weight tier (64.6%)"]
    Exam --> Mid["Mid-weight tier (29.7%)"]
    Exam --> Low["Low-weight tier (5.7%)"]
    High --> AppInt["Applications and Integration (33.1%)"]
    High --> ModelSel["Model Selection and Optimization (16.8%)"]
    High --> Agents["Agents and Workflows (14.7%)"]
    Mid --> Prompt["Prompt and Context Engineering (11.0%)"]
    Mid --> Tools["Tools and MCPs (10.6%)"]
    Mid --> Security["Security and Safety (8.1%)"]
    Low --> Code["Claude Code (3.1%)"]
    Low --> Eval["Eval, Testing, and Debugging (2.6%)"]

A visual note: this data would be well served by a horizontal bar chart or a treemap showing each domain’s share of the 53 items. That visual is added in a later pipeline stage — do not add it here.

How Domains Map to Real Developer Competencies

Each domain is not an abstract category; it maps directly to a concrete thing developers do on the job. The following table connects each domain to the real competency it measures, drawn from the blueprint’s detailed objectives.

[Source: source-material.txt]

Building an Efficient Study Strategy from the Blueprint

Here is where the blueprint becomes a study tool rather than a reference document. The naive approach is to give every domain equal study time. The blueprint tells us that is a mistake. A far better strategy is to weight your study time roughly in proportion to the domain weights, while accounting for your existing strengths.

Consider a worked example. Suppose you have 40 hours to study and you allocate them purely by domain weight:

The lesson is stark: nearly 26 of your 40 hours should go to just the top three domains, and you can responsibly spend barely an hour each on Claude Code and Eval/Testing/Debugging. That does not mean ignore the small domains — a couple of easy points still count toward your 720 — but it does mean that mastering the mechanics of the Claude API, model selection, and agent construction is where the exam is won or lost.

A refinement: adjust these baseline hours up or down based on an honest self-assessment against the MQC profile. If you already build production REST integrations daily, you can trim your Applications and Integration hours and redirect them to a domain you are weaker in, such as MCP server authoring. The blueprint gives you the ideal allocation; your self-assessment personalizes it. This mapping — MQC capabilities to blueprint weights to your own gaps — is the recommended foundation for your entire study plan.

Key Takeaway: The eight-domain blueprint is your study roadmap: Applications and Integration (33.1%), Model Selection and Optimization (16.8%), and Agents and Workflows (14.7%) together make up nearly two-thirds of the exam. Allocate study time in proportion to the weights, then personalize that allocation against your own gaps versus the MQC profile.

The Claude Platform Landscape

Having mapped the exam, we now zoom out to the technology it tests. The Claude platform is Anthropic’s developer ecosystem for building AI applications. Its core building blocks are the Claude API, the official SDKs, Claude Code, the Claude Agent SDK, and the Model Context Protocol (MCP) [Source: https://www.anthropic.com/learn/build-with-claude]. This section gives you a “10,000-foot view” of each; later chapters dive deep.

The Core Components at a Glance

Let us briefly unpack the two that most often confuse newcomers.

The Claude Agent SDK gives developers programmatic access to the same tools, agent loop, and context management that power Claude Code, in both Python and TypeScript. It runs the agent loop inside your own process. It was formerly called the “Claude Code SDK” and was renamed to reflect a broader vision — the harness that powers Claude Code can power many other kinds of agents [Source: https://code.claude.com/docs/en/agent-sdk/overview]. A common development path is to prototype locally with the Agent SDK and then move to managed/production deployment [Source: https://www.anthropic.com/engineering/building-agents-with-the-claude-agent-sdk].

The Model Context Protocol (MCP) is an open standard, open-sourced by Anthropic, that connects AI agents to external tools and data sources through a single protocol rather than fragmented, custom integrations [Source: https://www.anthropic.com/news/model-context-protocol]. With MCP, an agent can query databases and integrate with services like Slack, GitHub, Git, Postgres, Google Drive, and Puppeteer without writing bespoke tool implementations for each one [Source: https://platform.claude.com/docs/en/agent-sdk/mcp]. A useful analogy: MCP is to AI tools what USB-C is to devices — one standard plug that replaces a drawer full of proprietary adapters.

Figure 1.3: MCP as one universal protocol between an agent and many external systems

graph LR
    Agent["Claude agent"] --> MCP["MCP: one universal protocol"]
    MCP --> Slack["Slack"]
    MCP --> GitHub["GitHub / Git"]
    MCP --> Postgres["Postgres database"]
    MCP --> Drive["Google Drive"]
    MCP --> Puppeteer["Puppeteer"]

The Claude Model Families: Opus, Sonnet, and Haiku

The platform offers three headline model tiers, which trade off intelligence, speed, and cost. A memorable framing from the field: “Haiku is the sprinter, Sonnet is the steady builder, and Opus is the careful reviewer” [Source: https://tech-insider.org/claude-opus-vs-sonnet-vs-haiku-2026/].

[Source: https://www.remoteopenclaw.com/blog/best-claude-models-2026] [Source: https://tech-insider.org/claude-opus-vs-sonnet-vs-haiku-2026/]

Note the roughly 5x price gap between Opus and Haiku on output tokens — that gap is precisely why model selection is worth 16.8% of the exam. Choosing Opus when Haiku would do wastes money; choosing Haiku when the task demands Opus wastes quality.

How the Pieces Fit Together for Production Applications

The models and components are not meant to be used in isolation. The most sophisticated production systems in 2026 use all three model tiers together: Haiku serves as the router that classifies incoming requests and handles the simple ones directly; Sonnet processes the bulk of medium-complexity work such as code generation, document analysis, and data extraction; and Opus handles the 10–15% of requests that require deep reasoning or complex multi-step problem solving [Source: https://www.remoteopenclaw.com/blog/best-claude-models-2026].

Figure 1.2: Multi-tier model routing in a production application

flowchart TD
    Request["Incoming request"] --> Haiku["Haiku: classify and route"]
    Haiku --> Simple{"Complexity?"}
    Simple -->|"Simple: handle directly"| HaikuOut["Haiku response"]
    Simple -->|"Medium: bulk work"| Sonnet["Sonnet: code, docs, extraction"]
    Simple -->|"Hard: deep reasoning (10-15%)"| Opus["Opus: multi-step analysis"]
    HaikuOut --> Response["Final response"]
    Sonnet --> Response
    Opus --> Response

Picture a typical production application as a stack, from the raw model up to the finished agent:

1. Model tier (Opus / Sonnet / Haiku) — the reasoning engine, selected per task.
2. Claude API — the channel that carries messages to and from the chosen model.
3. Official SDK (Python/TypeScript) — wraps that channel, handling auth, streaming, token management, and errors [Source: https://www.anthropic.com/learn/build-with-claude].
4. Agent SDK — adds the agent loop, tool orchestration, and context management on top.
5. MCP servers — plug in external tools and data (databases, Slack, GitHub) via one protocol [Source: https://platform.claude.com/docs/en/agent-sdk/mcp].
6. Claude Code — for the coding-agent use case, ties it all together at the CLI, configured through CLAUDE.md and settings.json.

Every one of these layers maps directly onto an exam domain — the SDK and API to Applications and Integration, the model tiers to Model Selection, the Agent SDK to Agents and Workflows, MCP to Tools and MCPs, and Claude Code to its own domain. Understanding the stack, in other words, is not separate from passing the exam; it is the exam.

A visual note: an architecture layer diagram showing the model at the bottom and Claude Code / the finished agent at the top would clarify this stack. It is added in a later stage — do not add it here.

Key Takeaway: The Claude platform is a layered ecosystem: the API and SDKs provide the channel and conveniences, the Agent SDK supplies the reusable agent loop that also powers Claude Code, and MCP is the “USB-C” standard connecting agents to external tools. Production systems combine Haiku (routing), Sonnet (bulk work), and Opus (hard reasoning) — and each layer maps onto an exam domain.

Chapter Summary

The Claude Certified Developer – Foundations (CCDV-F) credential is an independent, foundational validation that you can build, integrate, and ship production-grade Claude applications, agents, and workflows. It is aimed at working engineers — typically with one to five years of software experience and at least six months on Claude or a comparable LLM — and it is calibrated to a Minimally Qualified Candidate who understands real-world tradeoffs across model selection and tool type. There are no mandatory prerequisites; the credential is earned on exam performance alone, and it stays valid for twelve months, after which a free on-time renewal (or, if lapsed, the full exam) keeps you current.

Mechanically, the exam is 53 items in 120 minutes, mixing multiple-choice and multiple-response formats, scored on a 100–1,000 scale with a 720 cut score for a $125 fee. It is criterion-referenced: you are measured against a fixed standard set by subject-matter experts, never ranked against other candidates. Your score report breaks performance out by domain, but only the single total scaled score decides pass or fail. The content blueprint distributes those items across eight weighted domains, and the distribution is deliberately lopsided — Applications and Integration (33.1%), Model Selection and Optimization (16.8%), and Agents and Workflows (14.7%) together make up nearly two-thirds of the exam. The most efficient study plan allocates time roughly in proportion to those weights, then personalizes that allocation against your own gaps versus the MQC profile.

Underlying every question is the Claude platform itself: the API and official Python/TypeScript SDKs that carry and simplify model calls; the Agent SDK that supplies the reusable agent loop also powering Claude Code; MCP, the open “USB-C” standard for connecting agents to external tools and data; and the three model tiers — Haiku for speed and volume, Sonnet for balanced everyday work, and Opus for the hardest reasoning. These layers are not trivia; each maps directly onto an exam domain, which means that genuinely understanding how the platform fits together is the surest path to a passing scaled score. With this map in hand, you are ready to begin the deep work — starting in the next chapter.

Key Terms

Chapter 2: LLM Fundamentals for Claude Developers

Learning Objectives

• Explain how large language models generate text through next-token prediction and autoregressive decoding.
• Define tokens, context windows, sampling, temperature, and non-determinism, and describe how each behaves on Claude models.
• Distinguish zero-shot, single-shot, and multi-shot prompting, and describe Claude’s fast, extended, and adaptive thinking modes.

Before you can build reliable applications on Claude, you need a working mental model of what the model is actually doing when it “responds.” This chapter builds that model from the ground up: how text is generated one unit at a time, what a token really is, why the same prompt can produce different answers, how much text Claude can hold in mind at once, and how to steer its behavior through thinking modes and examples. These fundamentals underpin every design decision you will make as a Claude developer — from estimating cost, to fitting a document into context, to deciding whether a task needs deep reasoning or a fast answer.

How LLMs Work

Next-token generation and autoregressive decoding

Claude is built on a large language model (LLM) — in Anthropic’s words, an “AI language model with many parameters that are capable of performing a variety of surprisingly useful tasks,” trained on vast amounts of text data [Source: https://platform.claude.com/docs/en/about-claude/glossary]. The single most important idea for a developer to internalize is how that model produces text. Claude’s underlying model is autoregressive: during pretraining (its initial training on a large unlabeled corpus of text), the model learns “to predict the next word, given the previous context of text in the document” [Source: https://platform.claude.com/docs/en/about-claude/glossary].

That is the entire engine of generation. The model repeatedly predicts the most likely next unit of text given everything that came before it, appends that prediction to the running context, and then predicts again — one unit at a time, looping until it decides to stop. “Autoregressive” simply means regressing on itself: each new piece of output becomes part of the input for the next step.

Figure 2.1: The autoregressive next-token generation loop

flowchart TD
    A["Input context: prompt + tokens generated so far"] --> B["Model computes probability distribution over next token"]
    B --> C["Sample one token from the distribution (top-p / nucleus)"]
    C --> D["Append sampled token to the running context"]
    D --> E{"Stop token or max_tokens reached?"}
    E -->|"No"| A
    E -->|"Yes"| F["Decode token stream back into text output"]

Analogy — the world’s most well-read autocomplete. Think of the predictive text on your phone’s keyboard. As you type “I’ll meet you at the…”, it suggests “office,” “airport,” or “station.” Claude works on the same principle, but at a vastly larger scale: instead of guessing from your recent messages, it draws on patterns learned from an enormous body of text, and instead of suggesting one word for you to accept, it strings its own predictions together continuously to form paragraphs, code, and arguments. The keyboard predicts and stops; Claude predicts, commits, and feeds its own prediction back in to predict the next unit — that feedback loop is what “autoregressive decoding” describes.

Concretely, per Anthropic’s guidance, when Claude generates text it “calculates probabilities for each possible next word, then randomly chooses a sample from this probability distribution” [Source: https://platform.claude.com/docs/en/about-claude/glossary]. To avoid nonsensical outputs, it uses top-p sampling (also called nucleus sampling): only words whose cumulative probability reaches a threshold — typically 0.99 or 0.999 — are considered as candidates. In other words, the model ranks possible next tokens by probability, keeps just enough of the top candidates to cover ~99% of the likelihood mass, and samples from that “nucleus,” discarding the long tail of implausible options.

A crucial nuance: a raw pretrained model is “not inherently good at answering questions or following instructions” [Source: https://platform.claude.com/docs/en/about-claude/glossary]. A model that has only learned to continue text will happily continue a question with more questions rather than answering it. Claude is therefore not a bare language model — it has been fine-tuned and trained with RLHF (Reinforcement Learning from Human Feedback) to behave as a helpful assistant. In RLHF, humans rank two or more example outputs, and the reinforcement learning process encourages the model to prefer outputs similar to the higher-ranked ones [Source: https://platform.claude.com/docs/en/about-claude/glossary]. Anthropic frames Claude’s goals as the “HHH” triad: helpful, honest, and harmless. So when you send a prompt, you are interacting with a next-token predictor that has been carefully shaped, on top of raw prediction, to follow instructions and behave well.

Tokens and tokenization

The “units” the model predicts are not always whole words. They are tokens — “the smallest individual units of a language model,” which “can correspond to words, subwords, characters, or even bytes (in the case of Unicode)” [Source: https://platform.claude.com/docs/en/about-claude/glossary]. Before the model processes any text, that text is encoded into a series of tokens; the model reasons entirely in tokens, and its output is a stream of tokens decoded back into text for you.

For Claude, a token approximately represents 3.5 English characters, though the exact number varies by language [Source: https://platform.claude.com/docs/en/about-claude/glossary]. Tokenization is a balancing act: larger tokens improve data efficiency during inference and pretraining and are used when possible, while smaller tokens let the model handle uncommon or never-before-seen words. The tokenization method a model uses affects its performance, vocabulary size, and ability to handle out-of-vocabulary words [Source: https://platform.claude.com/docs/en/about-claude/glossary].

Figure 2.2: The tokenization pipeline — text in, text out

flowchart LR
    A["Raw text: 'I'll meet you at the office'"] --> B["Encode: split into tokens (words, subwords, characters, bytes)"]
    B --> C["Token IDs fed to the model as its working units"]
    C --> D["Model reasons and generates output tokens"]
    D --> E["Decode: token stream back into readable text"]

Analogy — LEGO bricks for language. A tokenizer is like a set of LEGO bricks. Common words such as “the” or “and” are single large bricks — efficient and reused constantly. A rare or invented word like “supercalifragilistic” gets built from several smaller bricks (subwords or even individual characters), so the model can still represent it even if it never saw that exact word during training. This is why token counts don’t map cleanly to word counts: a short common word may be one token, while a long unusual word may be four or five.

A change that directly affects cost estimation: Claude Opus 4.7 and later Opus models, Claude Fable 5, Claude Mythos 5, Claude Mythos Preview, and Claude Sonnet 5 use a newer tokenizer that produces approximately 30% more tokens than earlier models for the same input text [Source: https://platform.claude.com/docs/en/build-with-claude/token-counting]. The exact increase depends on content and workload shape. The practical warning for developers is explicit: do not reuse token counts measured on older models when estimating cost or context-window fit. Recount your prompts against the specific model you plan to use, because usage and billing on these newer models reflect the new tokenizer’s counts.

To make this recounting easy, Anthropic provides a token-counting API. The endpoint POST /v1/messages/count_tokens lets you determine the number of tokens in a message before you send it, which supports rate-limit and cost management, model-routing decisions, and fitting prompts to a target length [Source: https://platform.claude.com/docs/en/build-with-claude/token-counting]. It accepts the same structured inputs as message creation — system prompts, tools, images, and PDFs — and returns the total input tokens, e.g. { "input_tokens": 14 }.

The table below summarizes the key properties of the token-counting endpoint.

Non-determinism and why outputs vary

Because generation samples from a probability distribution rather than always taking the single most likely token, running the same prompt twice can produce different text. You might expect that setting temperature to 0 — which pushes the model toward always choosing the most probable token — would give you identical output every time. It does not. Anthropic states plainly that even with temperature set to 0, “the results will not be fully deterministic and identical inputs may produce different outputs across API calls” [Source: https://platform.claude.com/docs/en/about-claude/glossary]. This holds both for Anthropic’s first-party inference service and for inference through third-party cloud providers.

There is an additional source of variation on modern models: with adaptive thinking (covered later in this chapter), the amount of thinking Claude allocates is determined dynamically per request, so thinking-token usage also varies from call to call [Source: https://platform.claude.com/docs/en/build-with-claude/context-windows]. The developer takeaway is a design principle, not a bug report: do not assume byte-identical outputs from identical inputs. Build systems that tolerate wording variation — validate structure and meaning rather than exact-string matching, and never make correctness depend on a response being reproduced verbatim.

Key Takeaway: Claude generates text autoregressively, predicting one token at a time by sampling from a probability distribution and feeding each prediction back into its context. Tokens are subword units of roughly 3.5 English characters, and the newer tokenizer on Opus 4.7+, Sonnet 5, and Fable/Mythos 5 emits ~30% more tokens — so always recount with the token-counting API for the exact model you will use. Because generation samples probabilistically, outputs are non-deterministic even at temperature 0; design for variation rather than assuming identical responses.

Context Windows and Sampling

Context window size and limits

The context window is “all the text a language model can reference when generating a response, including the response itself” — a “working memory” that is distinct from the vast training corpus baked into the model’s parameters [Source: https://platform.claude.com/docs/en/build-with-claude/context-windows]. Everything in the request competes for space in this window: the system prompt, every message in the messages array (including tool results, images, and documents), your tool definitions, and the output Claude generates for the turn — including its extended thinking. Each response reports how much was consumed in its usage field. When prompt caching is in play, the input count splits across input_tokens, cache_read_input_tokens, and cache_creation_input_tokens, and all three count toward the window; caching changes what you pay, not whether tokens occupy the window [Source: https://platform.claude.com/docs/en/build-with-claude/context-windows].

Analogy — a whiteboard, not a library. The model’s training is like a career’s worth of knowledge in your head; the context window is the whiteboard in front of you during a single meeting. The whiteboard holds only what’s written on it right now — the agenda, the notes, the answer you’re drafting — and it has a fixed size. When it fills up, something has to give.

Window sizes differ by model:

For every model with a 1M-token window, 1M is the default — no beta header is required, and long-context requests are billed at standard pricing [Source: https://platform.claude.com/docs/en/build-with-claude/context-windows]. This is a change from an earlier era when 1M was a gated beta; some older docs referenced a context-1m beta header, but current 1M models do not require one. Additionally, Claude Fable 5 and Claude Mythos 5 have a 1M window and can generate up to 128K output tokens (max_tokens) in a single request. A single request can include up to 600 images or PDF pages (100 for 200K-window models), and request-size limits may be reached before the token limit.

A larger window is not automatically better. Anthropic warns of context rot: as token count grows, “accuracy and recall degrade” [Source: https://platform.claude.com/docs/en/build-with-claude/context-windows]. Curating what goes into context matters as much as how much space is available — a focused 20K-token prompt often outperforms a bloated 500K-token one. (A diagram contrasting a lean, well-organized context against a padded one would reinforce this point well.)

What happens when you overflow the window? The behavior depends on the model:

Modern Sonnet and Haiku models help you manage this. Claude Sonnet 5, Sonnet 4.6, Sonnet 4.5, and Haiku 4.5 track their remaining context window (their “token budget”) automatically: the API injects a total budget into the system prompt, such as <budget:token_budget>200000</budget:token_budget>, and after each tool call injects an update like <system_warning>Token usage: 35000/200000; 165000 remaining</system_warning> [Source: https://platform.claude.com/docs/en/build-with-claude/context-windows]. This is automatic — developers never send these tags themselves. For conversations that need to run past the limit, server-side compaction (beta; header compact-2026-01-12) automatically summarizes earlier conversation on the server, and context editing offers tool-result clearing and thinking-block clearing for finer control.

Temperature, top-p, and sampling parameters

Temperature “controls the randomness of a model’s predictions during text generation” [Source: https://platform.claude.com/docs/en/about-claude/glossary]. Higher temperatures yield more creative, diverse outputs — varied phrasing, and in fiction, varied answers; lower temperatures yield more conservative, deterministic outputs that stick to the most probable phrasing. Adjusting temperature lets you encourage exploration of rare or surprising word choices rather than only the single most likely prediction. Two related sampling parameters are top_p and top_k.

Analogy — the creativity dial. Recall that at each step the model has a ranked list of candidate next tokens with probabilities. Temperature is a dial on how much weight it gives to lower-ranked candidates. Turn it down toward 0 and the model almost always picks the top candidate — safe, predictable, repetitive. Turn it up and the odds of picking a surprising alternative rise — more creative, but also more prone to wandering. top_p (nucleus sampling) and top_k are complementary: they limit which candidates are eligible at all (by cumulative probability, or by fixed count), before temperature reshapes the odds among them.

Here is the shift every Claude developer must know. Claude Fable 5, Claude Mythos 5, Claude Mythos Preview, Claude Opus 4.8, Claude Opus 4.7, and Claude Sonnet 5 reject non-default temperature, top_p, and top_k values with a 400 error — on every request, regardless of whether thinking is active [Source: https://platform.claude.com/docs/en/build-with-claude/adaptive-thinking]. On these newest models, you should omit these parameters entirely and instead guide behavior through prompting and the effort parameter (discussed in the next section). This is a deliberate architectural move away from manual sampling-parameter tuning and toward adaptive thinking as the primary control surface. If you port older code that hard-codes temperature: 0.7 onto Sonnet 5, it will fail — a common and easily avoided migration mistake.

Deterministic vs. probabilistic behavior

It is tempting to equate “temperature 0” with “deterministic,” but as established in the previous section, that equation is false. Sampling parameters shape the shape of the probability distribution — how peaked or how spread out it is — but they do not eliminate the underlying non-determinism of the inference system [Source: https://platform.claude.com/docs/en/about-claude/glossary]. Lowering temperature makes outputs more consistent and conservative, which is often what you want for classification, extraction, or format-constrained tasks; raising it makes outputs more varied and exploratory, which suits brainstorming and creative writing. But “more consistent” is not “guaranteed identical.” On the newest models, where these knobs are off the table entirely, the right lever for consistency is a clear, well-structured prompt with examples — the topic that closes this chapter.

Key Takeaway: The context window is Claude’s finite working memory holding the system prompt, all messages and tool data, and the generated output including thinking — everything counts, and accuracy degrades as it fills (context rot), so curation matters. 1M-token windows are the no-beta-header default on Opus 4.6+, Sonnet 5/4.6, and Fable/Mythos 5, while older models offer 200K. On the newest models, temperature, top_p, and top_k are rejected outright — behavior is steered through prompting and the effort parameter instead of sampling knobs.

Model Thinking Options

Fast mode and extended thinking

By default, Claude answers directly: it reads your prompt and begins producing the response. For hard problems, though, jumping straight to an answer can hurt quality — the same way a person who blurts out the first thing that comes to mind may get a tricky math problem wrong. Extended thinking addresses this by giving Claude enhanced reasoning for complex tasks: it generates internal “thinking” content — a private scratchpad of reasoning — before committing to a final answer [Source: https://platform.claude.com/docs/en/build-with-claude/adaptive-thinking].

Historically, extended thinking was configured manually: you enabled it with thinking: {type: "enabled", budget_tokens: N}, where budget_tokens is a subset of max_tokens, is billed as output tokens, counts toward rate limits, and counts toward the context window [Source: https://platform.claude.com/docs/en/build-with-claude/adaptive-thinking]. Manual thinking gives precise control over how many tokens Claude spends thinking, but it has been superseded on newer models. It is deprecated on Opus 4.6 and Sonnet 4.6, and rejected with a 400 error on Fable 5, Mythos 5, Sonnet 5, Opus 4.8, and Opus 4.7. Only older models such as Sonnet 4.5 and Opus 4.5 support only manual type: "enabled" with budget_tokens.

Adaptive thinking and effort levels

The modern replacement for a fixed thinking budget is adaptive thinking, enabled with thinking: {type: "adaptive"} (no beta header required). Rather than you guessing how much reasoning a request needs, “Claude evaluates the complexity of each request and determines whether and how much to use extended thinking” [Source: https://platform.claude.com/docs/en/build-with-claude/adaptive-thinking]. At the default effort level (high), Claude almost always thinks; at lower effort levels it may skip thinking entirely for simpler problems. Adaptive thinking is the recommended mode on Opus 4.8, Opus 4.7, Opus 4.6, Sonnet 5, and Sonnet 4.6, and the only thinking mode on Fable 5 and Mythos 5.

Analogy — an experienced triage nurse. Manual budget_tokens is like telling every patient “spend exactly ten minutes with the doctor,” regardless of whether they have a paper cut or a broken leg. Adaptive thinking is a triage nurse who sizes up each case and routes the paper cut through quickly while giving the broken leg the time it needs. This is why adaptive thinking “can drive better performance than extended thinking with a fixed budget_tokens for many workloads, especially workloads that mix trivial and complex requests, and long-horizon agentic workflows” [Source: https://platform.claude.com/docs/en/build-with-claude/adaptive-thinking]. It also automatically enables interleaved thinking — Claude can think between tool calls — which makes it especially effective for multi-step tool use, complex coding, and long-horizon agent loops.

Figure 2.3: Adaptive thinking — how Claude decides whether to reason

flowchart TD
    A["Request arrives with thinking: {type: 'adaptive'}"] --> B["Claude evaluates request complexity"]
    B --> C{"Complex or multi-step task?"}
    C -->|"Yes"| D["Allocate extended thinking (private scratchpad)"]
    C -->|"No, and effort is lower"| E["Skip thinking, respond directly"]
    D --> F{"Tool calls involved?"}
    F -->|"Yes"| G["Interleaved thinking between tool calls"]
    F -->|"No"| H["Produce final answer"]
    G --> H
    E --> H

The per-model defaults contain nuances worth memorizing, because “thinking on” is not universal:

Adaptive thinking’s triggering is promptable. System-prompt guidance such as “Extended thinking adds latency and should only be used when it will meaningfully improve answer quality… When in doubt, respond directly” reduces thinking, while “This task involves multi-step reasoning. Think carefully before responding.” increases it [Source: https://platform.claude.com/docs/en/build-with-claude/adaptive-thinking]. Per-message steering works too (“Please think hard before responding.” / “Answer directly without deliberating.”).

Whether thinking is shown is controlled separately by the display field: "summarized" returns a readable summary of the reasoning, while "omitted" returns thinking blocks with an empty thinking field (faster time-to-first-text-token when streaming). On Fable 5, Mythos 5, Sonnet 5, Opus 4.8, Opus 4.7, and Mythos Preview the default is "omitted" — you must explicitly set display: "summarized" to see the reasoning text; on Opus 4.6 and Sonnet 4.6 the default is "summarized" [Source: https://platform.claude.com/docs/en/build-with-claude/adaptive-thinking]. Critically, display affects only visibility, not billing: thinking happens and is billed identically under every setting, and usage.output_tokens_details.thinking_tokens reports the raw internal reasoning tokens you are charged for.

Running alongside thinking mode is the effort parameter, set via output_config: {effort: "..."}. It controls “how eager Claude is about spending tokens when responding,” trading response thoroughness against token efficiency [Source: https://platform.claude.com/docs/en/build-with-claude/effort]. It is available on all supported models with no beta header, and — importantly — it affects all tokens in the response: text, tool calls and function arguments, and extended thinking. That breadth gives it two advantages over thinking configuration alone: it doesn’t require thinking to be enabled, and it influences the number of tool calls, not just reasoning length. effort: "high" is the default and produces exactly the same behavior as omitting the parameter.

Availability varies at the top end: max is available on Fable 5, Mythos 5, Opus 4.8, Mythos Preview, Opus 4.7, Opus 4.6, Sonnet 5, and Sonnet 4.6; xhigh on Fable 5, Mythos 5, Opus 4.8, Opus 4.7, and Sonnet 5. Model-specific starting points: for Opus 4.7 and 4.8, start with xhigh for coding/agentic work, use high as the minimum for intelligence-sensitive workloads, reserve max for genuinely frontier problems, and set a large max_tokens (starting around 64K) at xhigh/max. For Sonnet 4.6, medium is the recommended default. For Fable 5, start with high [Source: https://platform.claude.com/docs/en/build-with-claude/effort]. (Note: Claude Code’s “ultracode” mode is not an extra API effort level — it pairs xhigh effort with multi-agent permissions.)

When thinking improves reasoning quality

Thinking is a tool, not a default virtue — it adds latency and token cost, so it should earn its keep. The guidance is straightforward. Use adaptive thinking for agentic behavior: multi-step tool use, complex coding, and long-horizon loops, where reasoning between steps materially improves outcomes. Use higher effort (high, xhigh, max) for complex reasoning and coding where quality outweighs speed and cost. Use lower effort (low, medium) for simple, high-volume, or latency-sensitive tasks such as classification and lookups [Source: https://platform.claude.com/docs/en/build-with-claude/effort].

Think of max_tokens and effort as two different kinds of control: max_tokens is the hard cap on total output (thinking plus text), while effort is the soft dial on how eagerly Claude spends within that cap. Effort is a behavioral signal, not a strict token budget — at lower effort Claude still thinks on sufficiently difficult problems, just less. A practical debugging rule: if you see stop_reason: "max_tokens", either raise max_tokens or lower effort [Source: https://platform.claude.com/docs/en/build-with-claude/effort]. For the best experience, combine effort with adaptive thinking.

Key Takeaway: Extended thinking lets Claude reason on a private scratchpad before answering; modern models replace fixed budget_tokens with adaptive thinking ({type: "adaptive"}), which decides per request how much to think and auto-enables interleaved thinking for agentic work. The effort parameter (low → medium → high-default → xhigh → max) governs how eagerly Claude spends tokens across text, tool calls, and thinking, and high equals omitting it. Reserve high effort and thinking for genuinely complex, multi-step, or agentic tasks; use low effort for simple, high-volume, latency-sensitive work.

Fundamental Prompting Techniques

Since sampling knobs are disappearing on the newest models, prompting is your primary steering wheel. The three foundational techniques — zero-shot, single-shot, and multi-shot — differ only in how many worked examples you place in the prompt, but that choice has an outsized effect on accuracy and consistency.

Zero-shot prompting

In zero-shot prompting you provide no examples. The prompt includes the role, task description, and expected response format, but no demonstrations of input-to-output [Source: https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/multishot-prompting]. Success depends entirely on clear, direct, detailed instructions — you are relying on the model’s general capability to infer what “good” looks like from your description alone.

Worked example (zero-shot classification):

You are a support-ticket classifier. Classify each ticket into exactly one
category: BILLING, TECHNICAL, or ACCOUNT. Respond with only the category name.

Ticket: "My invoice shows a charge I don't recognize from last month."

There are no examples here — just a role, a task, a fixed set of labels, and a format constraint. Zero-shot is fast to write and cheap in tokens, and for straightforward, unambiguous tasks it is often all you need.

Single-shot and multi-shot (few-shot) prompting

Single-shot prompting adds exactly one example demonstrating the desired input→output mapping. Multi-shot (also called few-shot) prompting provides multiple examples. Anthropic is emphatic about their value: “Examples are one of the most reliable ways to steer Claude’s output format, tone, and structure,” and “a few well-crafted examples… can dramatically improve accuracy and consistency” [Source: https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/multishot-prompting].

Analogy — showing versus telling. Zero-shot is handing a new hire a written policy and hoping they interpret it as you intended. Few-shot is showing them three completed examples of the work and saying “do it like these.” Most people — and most models — generalize far more reliably from concrete demonstrations than from abstract instructions alone.

Worked example (multi-shot classification): extending the ticket classifier above with demonstrations, wrapped in tags:

You are a support-ticket classifier. Classify each ticket into exactly one
category: BILLING, TECHNICAL, or ACCOUNT. Respond with only the category name.

<examples>
<example>
Ticket: "I was double-charged for my subscription this month."
Category: BILLING
</example>
<example>
Ticket: "The app crashes every time I upload a photo."
Category: TECHNICAL
</example>
<example>
Ticket: "I need to change the email address on my profile."
Category: ACCOUNT
</example>
</examples>

Ticket: "My invoice shows a charge I don't recognize from last month."

The examples do two jobs at once: they pin down the exact output format (a bare category name) and they resolve ambiguity the instructions alone might leave open.

Anthropic’s best practices for examples are concrete:

[Source: https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/multishot-prompting]

A useful sanity check from Anthropic’s general prompting guidance: “Show your prompt to a colleague with minimal context… If they’d be confused, Claude will be too” [Source: https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/claude-prompting-best-practices]. Providing the context and motivation behind your instructions helps Claude generalize and deliver more targeted responses.

Choosing the right technique

The decision is a trade-off between effort, token cost, and reliability. The following matrix summarizes when each technique fits.

A sound default workflow: start with a clear zero-shot prompt. If the output is inconsistent, wanders in format, or misreads edge cases, add examples — moving to single-shot and then to a diverse multi-shot set of 3–5. Because examples are the most reliable lever for format and consistency, and because sampling parameters are unavailable on the newest models, few-shot prompting is often the first tool to reach for when tightening reliability.

Key Takeaway: Zero-shot prompts give no examples and rely on clear instructions; single-shot adds one demonstration; multi-shot (few-shot) adds several — ideally 3–5 diverse, relevant examples wrapped in <example>/<examples> tags. Examples are the most reliable way to steer Claude’s format, tone, and structure, improving both accuracy and consistency. Start zero-shot for simple tasks and add examples as reliability demands, especially since sampling-parameter tuning is off the table on the newest models.

Chapter Summary

This chapter built the mental model every Claude developer needs. Claude generates text autoregressively — predicting one token at a time by sampling from a probability distribution and feeding each prediction back into its context, then shaped by fine-tuning and RLHF to be helpful, honest, and harmless. Tokens are subword units of roughly 3.5 English characters, and because the newer tokenizer on Opus 4.7+, Sonnet 5, and Fable/Mythos 5 emits ~30% more tokens for the same text, you should always recount with the free token-counting API for the exact model you target.

Because generation samples probabilistically, outputs are non-deterministic even at temperature 0 — a design constraint, not a bug. The context window is Claude’s finite working memory (system prompt + messages + tool data + generated output including thinking), and accuracy degrades as it fills (context rot); 1M-token windows are the no-beta-header default on the latest models, while older ones offer 200K. On the newest models, temperature, top_p, and top_k are rejected outright — you steer behavior through prompting and the effort parameter instead.

For reasoning depth, adaptive thinking lets Claude decide per request how much to think and auto-enables interleaved thinking for agentic work, while the effort dial (low → medium → high-default → xhigh → max) governs token eagerness across text, tool calls, and thinking. Finally, prompting is your primary steering wheel: move along the zero-shot → single-shot → multi-shot spectrum, reaching for 3–5 diverse examples in <example> tags whenever you need to tighten format, accuracy, and consistency. Together these fundamentals inform every downstream decision — cost estimation, context management, model selection, and prompt design — that you will make throughout the rest of this guide.

Key Terms

Chapter 3: Claude Model Selection and Tradeoffs

Choosing a Claude model is one of the most consequential decisions a developer makes when building on the Anthropic platform. The choice ripples through every request: it sets your latency floor, your cost ceiling, and the quality of every answer your users see. Get it right and your application feels fast, smart, and affordable. Get it wrong and you either burn budget on a model more powerful than the task needs, or you ship a product that fails because the model could not keep up with the reasoning the task demanded.

This chapter treats model selection as an engineering discipline rather than a guess. We compare the three model families across the dimensions that matter, map task complexity to the right tier, and confront the versioning and breaking-change realities that catch teams off guard in production. By the end you should be able to look at a task and its constraints and name the model you would deploy — and defend why.

Learning Objectives

By the end of this chapter, you will be able to:

• Compare Opus, Sonnet, and Haiku across capability, latency, and cost, and articulate the tradeoffs each tier makes.
• Select an appropriate model tier for a given task and constraint set, using a repeatable decision framework rather than intuition.
• Anticipate breaking behavior changes across model releases — including version-pinning nuances and parameter deprecations that return 400 errors — and design migration and regression-testing strategies that mitigate them.

The Claude Model Families

Anthropic organizes its models into three named tiers, and the names are chosen deliberately to signal the tradeoff each one represents. Understanding the shape of that tradeoff is more durable knowledge than memorizing any single version number, because the version numbers change but the tiering philosophy does not [Source: https://platform.claude.com/docs/en/about-claude/models/choosing-a-model].

Before we define the tiers, one important term. A model tier is a capability-and-cost band within a model family — Opus, Sonnet, and Haiku are the three tiers, and each new generation (4.5, 4.6, 4.8, 5, and so on) ships models within these same tiers. When someone says “use Sonnet,” they mean the balanced tier, regardless of which specific Sonnet version is current.

Opus for Maximum Capability

Opus is Anthropic’s most capable tier — built for the hardest reasoning. Think of Opus as the senior specialist you bring in for the problems no one else on the team can crack: multi-hour autonomous agents, complex code refactors spanning 50 or more files, financial modeling, and research synthesis where a wrong answer has serious downstream consequences [Source: https://valueaddvc.com/blog/claude-opus-vs-sonnet-vs-haiku-which-model-to-use-and-when-in-2026]. It is engineered for sustained logical coherence across long outputs — 10,000-plus tokens — where the reasoning has to hold together from the first token to the last [Source: https://platform.claude.com/docs/en/about-claude/models/overview].

That capability comes with a cost. Opus carries the highest latency of the three tiers, which makes it a poor fit for real-time, user-facing chat where a person is waiting on every keystroke. Its natural home is batch and asynchronous workflows — the overnight code migration, the deep-research pipeline, the analysis job — where quality is the overriding concern and a few extra seconds (or minutes) of thinking is a fair trade [Source: https://valueaddvc.com/blog/claude-opus-vs-sonnet-vs-haiku-which-model-to-use-and-when-in-2026].

The current Opus model is Claude Opus 4.8 (claude-opus-4-8), which launched May 28, 2026. It is priced at $5.00 per million input tokens and $25.00 per million output tokens, with an optional “Fast Mode” premium at $10 / $50 for teams that need Opus-tier reasoning at up to 2.5x the output speed [Source: https://www.metacto.com/blogs/anthropic-api-pricing-a-full-breakdown-of-costs-and-integration].

Analogy: If your models were a hospital, Opus is the attending surgeon. You do not page the surgeon to take a patient’s temperature — you reserve them for the operation that only they can perform. Paging them for routine work is expensive and slow, and it means they are unavailable when the hard case walks in.

Sonnet for Balanced Performance

Sonnet is the balanced middle tier and the recommended default for most production use cases. It delivers near-Opus quality at faster latency and significantly lower cost, which is exactly the profile most applications actually need [Source: https://www.sitepoint.com/claude-model-selection-framework/]. Coding, analysis, writing, customer-facing applications, and retrieval-augmented generation (RAG) pipelines all sit comfortably in Sonnet’s sweet spot.

The current Sonnet model, Claude Sonnet 4.6 (claude-sonnet-4-6), is priced at $3.00 / $15.00 per million tokens and ships with a distinctive advantage: a 1M-token context window at no surcharge, versus the 200K windows of Opus and Haiku [Source: https://platform.claude.com/docs/en/about-claude/pricing]. For long-document analysis, large-codebase RAG, or any workload where you need to stuff a lot of context into a single request, that larger window can make Sonnet the right choice even when the reasoning itself is not especially demanding.

The practical guidance is simple: start with Sonnet. It is usually the best starting point for cost effectiveness and general capability. Only move up to Opus when you measure a quality gap, and only move down to Haiku when you measure a cost or latency problem [Source: https://platform.claude.com/docs/en/about-claude/models/choosing-a-model].

Analogy: Sonnet is the experienced generalist physician. They handle the overwhelming majority of cases well, quickly, and affordably. They know when to escalate to the surgeon and when a nurse can take over — but for the bulk of the work, they are the right answer.

Haiku for Speed and Cost Efficiency

Haiku is the fastest and most cost-efficient tier, purpose-built for high-volume, latency-sensitive workloads. Classification, routing, extraction, summarization, and content moderation are its bread and butter — tasks where response time matters more than deep reasoning [Source: https://www.ayautomate.com/blog/opus-vs-sonnet-vs-haiku].

The numbers tell the story. Claude Haiku 4.5 (claude-haiku-4-5) is priced at $1.00 / $5.00 per million tokens and averages roughly 110 tokens per second — about 2–3x faster than Sonnet and 4–5x faster than Opus. It consistently hits sub-1.5-second p95 end-to-end latency for chat workloads, which is what makes it viable for interactive, real-time experiences [Source: https://valueaddvc.com/blog/claude-opus-vs-sonnet-vs-haiku-which-model-to-use-and-when-in-2026].

A striking figure from the research: roughly 70–80% of typical production queries can be handled by Haiku. Most of what real applications do is not deep reasoning — it is triage, extraction, and formatting. Haiku is designed to do exactly that work at a fraction of the cost of the larger tiers [Source: https://www.sitepoint.com/claude-model-selection-framework/].

Analogy: Haiku is the triage nurse at the front desk. They see every patient, quickly sort who needs what, handle the routine cases on the spot, and pass the complicated ones up the chain. They are fast, always available, and inexpensive — and they keep the specialists free for the work only specialists can do.

The Comparison Matrix

The following table consolidates the three tiers. This is the single most important reference in the chapter — an Opus/Sonnet/Haiku matrix is the mental model to carry into any selection decision.

Key Takeaway: The three tiers trade capability against speed and cost along a single axis: Opus maximizes reasoning at the expense of latency and price, Haiku maximizes speed and economy at the expense of reasoning depth, and Sonnet sits in the middle as the sensible default. Learn the shape of these tradeoffs, not just the current version numbers — the tiers persist across generations even as the specific model IDs change.

Quality, Latency, and Cost Tradeoffs

Selecting a tier is really about balancing three competing pressures: how good the answer needs to be (quality), how fast it needs to arrive (latency — the time between sending a request and receiving a usable response), and how much you can afford to spend at your traffic volume (cost). No model optimizes all three; every choice is a position on this triangle.

Mapping Task Complexity to Model Tier

The governing principle is stated crisply in the research: match model capability to task complexity. Using a more powerful model than the task requires is waste; using a less powerful model than the task demands is a product failure [Source: https://www.sitepoint.com/claude-model-selection-framework/]. Both errors cost you — one in dollars, the other in broken user experiences — but they are not symmetric. An over-provisioned model quietly drains your budget; an under-provisioned one visibly breaks in front of users.

The mapping below turns that principle into concrete guidance.

A worked example makes the mapping vivid.

Worked Example — A Customer Support Pipeline. Imagine a support system that ingests incoming tickets. The pipeline has three stages, and each stage has a different complexity profile:

1. Classify the ticket (billing? technical? account?). This is a low-complexity classification task on a short input. Haiku handles it in well under 1.5 seconds at $1/$5 per million tokens.
2. Draft a reply for a routine billing question using retrieved policy documents. This is medium complexity — it needs coherent writing and light reasoning over context. Sonnet is the natural fit, and its 1M-token window comfortably holds the retrieved policy corpus.
3. Handle an escalated technical case that requires reasoning across the customer’s full history, three log files, and a config diff. This is high complexity with real consequences. Opus is worth the extra cost and latency because a wrong answer here means a frustrated customer and an engineer’s wasted afternoon.

The same pipeline uses all three tiers, each matched to the complexity of its stage. This is the essence of right-sizing — and it is why “which model should I use?” is often the wrong question. The better question is “which model for this step?”

Latency-Sensitive vs. Quality-Sensitive Workloads

A useful way to cut the decision is to ask which pressure dominates: is this a latency-sensitive workload or a quality-sensitive one?

Latency-sensitive workloads put a human in the loop, waiting. Live chat, autocomplete, interactive assistants, voice interfaces — anywhere a person perceives the delay. Here, Haiku’s sub-1.5-second p95 latency is a feature you can feel, and Opus’s high latency is disqualifying no matter how good its answers are. If the user is tapping their foot, you cannot afford Opus in the request path [Source: https://valueaddvc.com/blog/claude-opus-vs-sonnet-vs-haiku-which-model-to-use-and-when-in-2026].

Quality-sensitive workloads run in the background, where a few extra seconds — or minutes — cost nothing perceptible. Overnight code migrations, batch document analysis, research synthesis, report generation. Here Opus shines, because the whole point of the job is to get the best answer, and no one is watching the clock [Source: https://platform.claude.com/docs/en/about-claude/models/overview].

The mistake to avoid is applying the wrong lens. Teams sometimes reach for Opus on a real-time feature “for quality” and ship a laggy product; others run a nightly batch job on Haiku “for cost” and get results that are not good enough to act on. Name the dominant pressure first, then choose.

Adaptive Thinking Support by Model

Modern Claude models support adaptive thinking — a mode in which the model dynamically decides when and how much internal reasoning to perform before it answers, rather than the developer specifying a fixed thinking budget in advance. On the current models you enable it with thinking: {"type": "adaptive"}, and you steer overall depth and token spend with an effort parameter (output_config: {"effort": "low" | "medium" | "high" | "max"}).

This matters for model selection because thinking depth is another lever on the quality/latency/cost triangle. A higher effort setting buys more reasoning — and more tokens, latency, and cost — while a lower setting buys speed and economy. Adaptive thinking lets a single model flex along this axis instead of forcing you to jump tiers.

There is a critical versioning wrinkle here that we will return to in the next section, but it is worth flagging now because it directly affects which models support which thinking configuration:

• On the newest models (Opus 4.8, Opus 4.7, Sonnet 5), the old fixed-budget syntax — thinking: {"type": "enabled", "budget_tokens": N} — is rejected with a 400 error. You must use adaptive thinking.
• On Opus 4.6 and Sonnet 4.6, the fixed-budget syntax is deprecated but still functional as a transitional escape hatch — adaptive thinking is nonetheless the recommended path.
• On older models, the fixed-budget syntax is still the correct approach (with budget_tokens required to be less than max_tokens).

The design lesson: adaptive thinking is not just a feature toggle — it is a capability that varies by model version, and writing code that assumes the old fixed-budget API will break the moment you migrate to a newer model.

Key Takeaway: Every model choice is a position on a quality–latency–cost triangle; name the dominant pressure (is a human waiting, or is quality the whole point?) before you pick a tier. Adaptive thinking adds a within-model lever — via the effort parameter — that lets a single model flex along this axis, but the thinking configuration itself differs by model version, so the “right” way to enable it depends on which model you are calling.

Model Releases and Versioning

The tradeoffs above assume you know which model you are talking to. In production, that assumption is where things quietly go wrong. Anthropic ships new models regularly, and the way model identifiers, versions, and deprecations work is not intuitive. This section is where careful developers separate themselves from careless ones.

Model IDs Are Pinned Snapshots

Start with the single most important — and most misunderstood — fact about Claude versioning: each model ID identifies a pinned version of the model. When you use a model ID in an API request, the underlying model weights and configuration remain constant for the lifetime of that ID. Anthropic does not update the weights of an existing model ID. When an updated model is available, it ships under a new model ID [Source: https://platform.claude.com/docs/en/about-claude/models/model-ids-and-versions].

This is version pinning — the practice of targeting a specific, immutable model snapshot so that your application’s behavior does not change out from under you. And on modern Claude, it is largely automatic, because the IDs themselves are the pins.

Here is the nuance that trips people up. There are two ID formats:

• Dateless format (4.6 generation and later): claude-{name}-{major}[-{minor}] — for example, claude-sonnet-4-6, claude-opus-4-8, claude-sonnet-5. Major-version releases (like Sonnet 5) omit the minor segment.
• Dated snapshot format (before the 4.6 generation): claude-{name}-{major}-{minor}-{YYYYMMDD} — for example, claude-sonnet-4-5-20250929, claude-haiku-4-5-20251001.

The common misconception is that a dateless ID like claude-sonnet-4-6 is an evergreen pointer that always routes to the latest and greatest. It is not. For the 4.6 generation and later, the dateless ID is the canonical, single, fixed model snapshot [Source: https://platform.claude.com/docs/en/about-claude/models/model-ids-and-versions].

This differs from the older-generation aliases on the Claude API. An alias such as claude-sonnet-4-5 (no date) is a convenience pointer that resolves to the most recent dated snapshot for that minor version. So claude-sonnet-4-5 behaves like an evergreen pointer, but claude-sonnet-4-6 does not — the latter is a snapshot, not an alias.

There is one more subtlety worth knowing for the exam and for debugging. Model weights are fixed for a given ID, but the serving infrastructure is not. The request router, safety classifiers, and sampling logic can change over time. Occasionally an infrastructure update produces minor observable behavioral differences even though the model ID and weights are unchanged. If you notice unexpected behavioral drift on a previously stable model ID, an infrastructure update is the most likely cause — not a weight change, because weights do not change for an existing ID [Source: https://platform.claude.com/docs/en/about-claude/models/model-ids-and-versions].

Breaking Behavior Changes Across Releases

Pinning protects you from silent weight changes, but it does not protect you from breaking changes when you deliberately move to a new model. Newer models sometimes remove or change request parameters, and the failure modes are specific enough to memorize.

The canonical example — the one most likely to appear on an exam and most likely to bite you in production — is sampling parameter deprecation. The temperature, top_p, and top_k parameters are deprecated on Claude Opus 4.7 and later (including Opus 4.8 and Sonnet 5). Setting any of them to a non-default value returns a 400 error. The recommended replacement is to omit them entirely and use prompting to guide the model’s behavior [Source: https://platform.claude.com/docs/en/about-claude/models/model-ids-and-versions].

The related breaking change, already introduced above, is the thinking configuration: thinking: {"type": "enabled", "budget_tokens": N} also returns a 400 on Opus 4.7 and later. Use adaptive thinking ({"type": "adaptive"}) instead.

Why do these matter so much? Because they are silent in your source code and loud at runtime. The deprecated parameters often remain in the SDK’s request types, so existing code continues to type-check and compile without complaint — but the behavior changes per model, and a value that worked on Sonnet 4.6 throws a 400 on Sonnet 5. A migration that only swaps the model-ID string, without auditing for these parameters, will pass code review and fail on the first live request.

The Model Lifecycle and Deprecation Policy

To reason about when you will be forced to migrate, you need Anthropic’s model lifecycle model. Every model ID — dated or dateless — moves through four stages [Source: https://platform.claude.com/docs/en/about-claude/model-deprecations]:

The retired-stage detail is the one that causes outages. A retired model does not gracefully fall back to a successor — requests return errors. Code that hard-codes a model ID which then retires will simply start failing [Source: https://dev.to/flarecanary/claude-3-haiku-20240307-just-started-returning-errors-heres-what-happened-57he].

Figure 3.2: The four-stage model lifecycle and deprecation path

stateDiagram-v2
    [*] --> Active
    Active --> Legacy: No longer updated
    Legacy --> Deprecated: Retirement date assigned<br/>(60+ days notice)
    Deprecated --> Retired: Retirement date reached
    Retired --> [*]

    note right of Active
        Fully supported — build freely
    end note
    note right of Deprecated
        Still works but not recommended;
        migrate before the deadline
    end note
    note right of Retired
        Requests fail with errors,
        not redirects
    end note

Anthropic gives you runway. For publicly released models, customers with active deployments receive at least 60 days’ notice before retirement, delivered by email and in the documentation [Source: https://platform.claude.com/docs/en/about-claude/model-deprecations]. A real deprecation chain from the research illustrates the cadence: claude-opus-4-20250514 was deprecated on April 14, 2026 and retired June 15, 2026, with claude-opus-4-8 as the recommended replacement.

One platform caveat that matters for multi-cloud teams: the retirement dates on Anthropic’s deprecation page apply to Anthropic-operated platforms (the Claude API, Claude Platform on AWS, Microsoft Foundry). Partner-operated platforms — Amazon Bedrock and Google Cloud — set their own retirement schedules, so a model’s lifecycle status and dates can differ across platforms. If you run on Bedrock or Vertex, you cannot assume the first-party dates apply [Source: https://platform.claude.com/docs/en/about-claude/model-deprecations].

Key Takeaway: A Claude model ID is a pinned, immutable snapshot — dateless IDs like claude-opus-4-8 are not evergreen pointers, and Anthropic never changes the weights behind an existing ID. Breaking changes surface only when you deliberately migrate: on Opus 4.7 and later, non-default temperature/top_p/top_k and the old budget_tokens thinking syntax return 400 errors, and deprecated models eventually retire (with at least 60 days’ notice) into hard request failures. Pin deliberately, and treat every migration as a code audit, not a string swap.

Decision Frameworks for Model Choice

The previous sections give you the raw materials — tier characteristics, the tradeoff triangle, versioning realities. This section assembles them into repeatable frameworks you can apply under pressure, whether on an exam or in a design review.

The Four Decision Dimensions

Every model-selection decision can be reduced to four dimensions [Source: https://www.sitepoint.com/claude-model-selection-framework/]:

1. Task complexity — how much reasoning depth does the task genuinely require?
2. Latency sensitivity — is a human waiting on the response in real time?
3. Cost at scale — what does this cost when multiplied by your traffic volume?
4. Output quality requirements — what is the cost of a wrong answer?

Walk these four in order for any workload and the tier usually names itself. High complexity plus tolerance for latency plus high cost-of-error points to Opus. Low complexity plus high latency sensitivity plus enormous volume points to Haiku. Everything in between defaults to Sonnet.

Figure 3.1: Model-selection decision tree across the four dimensions

flowchart TD
    Start["New workload"] --> Q1{"Task complexity?"}
    Q1 -->|Low: classify, extract, route, summarize| Haiku["Choose Haiku"]
    Q1 -->|High: cross-file refactor, long-chain planning, high-stakes synthesis| Q2{"Is a human waiting<br/>in real time?"}
    Q1 -->|Medium: coding, analysis, RAG, support chat| Sonnet["Choose Sonnet (default)"]
    Q2 -->|"Yes: latency-sensitive path"| Sonnet
    Q2 -->|"No: batch / async job"| Opus["Choose Opus"]
    Haiku --> Check{"Quality gap or<br/>volume/latency issue?"}
    Sonnet --> Check
    Opus --> Check
    Check -->|"Measure, then escalate or drop a tier"| Route["Right-sized tier"]

Cost Modeling Per Tier

Cost is where abstract tier choices become concrete dollars, so it pays to model it explicitly. The base rates, per million tokens (input / output):

Two cost-optimization features apply across all tiers and belong in every model, because they change the economics dramatically [Source: https://www.finout.io/blog/anthropic-api-pricing]:

• Batch processing is 50% cheaper. If your workload is asynchronous and can tolerate the batch turnaround, you halve your token cost on any model. This is especially powerful with Opus, where the per-token price is highest — a quality-sensitive batch job on Opus effectively runs at half rate.
• Prompt caching cuts cached input cost by up to 90%. For workloads with a large, stable prefix — a long system prompt, a retrieved document reused across many questions, few-shot examples — caching turns most of your input tokens into near-free reads.

Worked Example — Modeling a High-Volume Extraction Job. Suppose you need to extract structured fields from 1,000,000 documents, each averaging 2,000 input tokens and 200 output tokens.

• On Opus 4.8: input cost ≈ 2B tokens × $5/M = $10,000; output ≈ 200M × $25/M = $5,000. Total ≈ $15,000 — and this task does not need Opus-level reasoning.
• On Haiku 4.5: input ≈ 2B × $1/M = $2,000; output ≈ 200M × $5/M = $1,000. Total ≈ $3,000 — a 5x saving with no quality loss, because extraction is a low-complexity task.
• On Haiku 4.5 with the Batch API (50% off): ≈ $1,500. An order of magnitude cheaper than the naive Opus approach.

The lesson: right-sizing the tier and layering in batch and caching can turn a $15,000 job into a $1,500 one — a 90% reduction — without degrading results, because the task never needed the expensive tier in the first place.

The research quantifies the aggregate impact: a tiered routing approach can cut costs by 50–70% compared to running everything through Opus [Source: https://www.sitepoint.com/claude-model-selection-framework/].

Escalation and Fallback Patterns

The most sophisticated production systems do not pick one model — they route between models based on runtime signals. Two patterns dominate.

The cascade pattern. Attempt the task with Haiku first. If the response has low confidence or fails a validation check, escalate to Sonnet or Opus. Most requests resolve at the cheap tier; only the hard cases pay for the expensive one [Source: https://aiforanything.io/blog/claude-model-selection-guide-haiku-sonnet-opus-2026].

First-pass filtering. Run Haiku as a cheap classifier or filter, then send only the qualified inputs to Sonnet or Opus for full generation. This combines the economics of the cheapest tier with the quality of the more capable tiers — you pay Opus rates only for the small slice of inputs that actually reach it [Source: https://www.sitepoint.com/claude-model-selection-framework/].

Analogy: The cascade pattern is exactly how a well-run hospital operates. Every patient meets the triage nurse (Haiku) first. Most are handled or routed on the spot. The generalist physician (Sonnet) sees those the nurse escalates. The surgeon (Opus) is reserved for the handful of cases that genuinely need surgery. No one wastes the surgeon’s time on a routine visit, and no one sends a surgical case home with the nurse. The system is fast, cheap, and safe — because it right-sizes at every step.

Figure 3.3: The cascade (escalation/fallback) routing flow

flowchart TD
    Req["Incoming request"] --> H["Haiku attempt"]
    H --> HC{"Confident & valid?"}
    HC -->|Yes| Done1["Return answer"]
    HC -->|"No / low confidence / failed validation"| S["Sonnet attempt"]
    S --> SC{"Confident & valid?"}
    SC -->|Yes| Done2["Return answer"]
    SC -->|"Still insufficient: high-stakes reasoning"| O["Opus attempt"]
    O --> Done3["Return answer"]

A textual sketch of the cascade (this is where a flow diagram would help a visual learner — imagine a top-to-bottom flowchart):

Incoming request
      │
      ▼
  Haiku attempt ──── confident & valid? ──── yes ──▶ Return answer
      │                                    
      │ no / low confidence / failed validation
      ▼
  Sonnet attempt ─── confident & valid? ─── yes ──▶ Return answer
      │
      │ still insufficient (high-stakes reasoning)
      ▼
  Opus attempt ──────────────────────────────────▶ Return answer

One caution when building cascades and routing: changing the model mid-conversation invalidates the prompt cache, because caches are model-scoped. If you route a follow-up turn to a different tier, you pay a fresh cache write. A common workaround is to spawn a cheaper-model sub-agent for the sub-task while keeping the main conversation loop pinned to a single model — preserving the cached prefix on the main thread.

Right-Sizing the Model to the Workload

Pulling the threads together, right-sizing is the discipline of matching model capability to workload demand — neither over- nor under-provisioning. The checklist:

1. Default to Sonnet. It is the best starting point for most production work.
2. Measure before you escalate to Opus. Move up only when you can demonstrate a quality gap Sonnet cannot close. Opus is worth it for cross-file engineering, long-chain planning, and high-consequence synthesis — and wasteful everywhere else.
3. Measure before you drop to Haiku. Move down when you have a cost or latency problem and the task is low-complexity enough that Haiku maintains quality. Remember: Haiku can serve 70–80% of production queries.
4. Route, don’t guess. Use cascade and first-pass-filtering patterns to let each request find its cheapest sufficient tier automatically.
5. Layer in batch and caching wherever the workload shape allows — they cut cost across every tier.

The anti-pattern to internalize: running everything through Opus “to be safe.” It is the single most expensive mistake in Claude application design, and the research is blunt about the fix — tiered routing recovers 50–70% of that spend with no quality loss on the tasks that never needed Opus.

Key Takeaway: Reduce every model choice to four dimensions — complexity, latency sensitivity, cost at scale, and quality requirements — and let them name the tier. Then right-size aggressively: default to Sonnet, measure before escalating to Opus or dropping to Haiku, and use cascade and first-pass-filtering patterns plus batch (50% off) and prompt caching (up to 90% off cached input) to route each request to its cheapest sufficient tier. Running everything through Opus is the costliest default there is.

Chapter Summary

Model selection on the Claude platform is a discipline, not a guess. The three tiers — Opus, Sonnet, Haiku — occupy fixed positions on a single tradeoff axis: Opus buys maximum reasoning at the cost of latency and price, Haiku buys speed and economy at the cost of reasoning depth, and Sonnet sits in the middle as the production default. Because the tiering philosophy outlives any specific version, learning the shape of these tradeoffs matters more than memorizing the current model IDs.

Every selection is a position on a quality–latency–cost triangle. Name the dominant pressure first: if a human is waiting, latency rules and Opus is disqualified from the request path; if quality is the whole point of a background job, Opus’s high latency is a non-issue. Adaptive thinking adds a within-model lever for trading depth against speed and cost, but its configuration differs by model version — a reminder that model choice and API surface are entangled.

That entanglement is the heart of the versioning story. A Claude model ID is a pinned, immutable snapshot — and critically, dateless IDs like claude-opus-4-8 are not evergreen pointers; they are fixed snapshots. Anthropic never changes the weights behind an existing ID, though serving infrastructure can shift and cause minor drift. Breaking changes surface only when you deliberately migrate: on Opus 4.7 and later, non-default temperature/top_p/top_k and the legacy budget_tokens thinking syntax return 400 errors, and deprecated models eventually retire into hard request failures (with at least 60 days’ notice). Treat every migration as a code audit and every deprecation notice as a deadline.

Finally, the decision frameworks turn all of this into repeatable practice. Reduce each choice to four dimensions — complexity, latency sensitivity, cost at scale, quality requirements — and right-size aggressively. Default to Sonnet, measure before escalating or dropping tiers, and route intelligently with cascade and first-pass-filtering patterns. Layer in batch processing and prompt caching to cut cost across every tier. The discipline of right-sizing, backed by tiered routing, recovers 50–70% of the spend that a naive “everything on Opus” architecture would waste — with no loss of quality on the tasks that never needed the expensive tier.

Key Terms

Chapter 4: Cost, Tokens, Caching, and Batch Processing

Every request you send to Claude has a price tag, and that price tag is denominated in tokens. A developer who understands how tokens are counted, priced, cached, and batched can run the same workload for a fraction of the cost of a developer who does not — sometimes an order of magnitude less. This chapter is about turning the Claude platform’s cost levers from invisible surprises on a monthly invoice into deliberate engineering decisions you make on every request.

By the end of this chapter you will be able to track token usage and model costs across an application, apply prompt caching and cache checkpointing to cut cost and latency, and choose between realtime and batch processing based on how much latency your workload can tolerate. Along the way we will work through concrete cost-calculation walkthroughs — the kind of break-even math you can carry directly into a production budget.

Think of your token budget the way a household thinks about a utility bill. You can leave every light on and pay whatever arrives at the end of the month, or you can meter your usage, insulate the parts that leak, and shift heavy loads to off-peak hours. Claude gives you a meter (the usage object), insulation (prompt caching), and off-peak pricing (the Batches API). This chapter shows you how to use all three.

Token Budgeting and Cost Modeling

Before you can optimize cost, you have to measure it. That means understanding what a token is, how input and output tokens are priced differently, how to read the token accounting the API hands back on every response, and how to forecast a workload’s cost before you ever run it at scale.

Input vs. output token pricing

A token is the unit into which text is broken before the model processes it — roughly three-quarters of an English word on average, though the exact ratio varies with language and content type. Claude’s pricing is quoted per million tokens (MTok), and — this is the single most important pricing fact in the chapter — input tokens and output tokens are priced at different rates. Output is consistently the more expensive of the two.

Here is the standard (non-cached, non-batched) pricing for the current model lineup:

[Source: https://claude.com/pricing#anthropic-api]

Notice the pattern: across every model, output costs five times as much as input. This is the central lever of cost modeling. A prompt that pours in 50,000 tokens of context but asks for a one-sentence answer is cheap; a prompt with a tiny question that triggers a 20,000-token essay is expensive. Two design implications follow immediately:

1. Bound your output. Right-size the max_tokens parameter so a runaway generation can’t quietly cost 5× what you expected. max_tokens is a hard ceiling the model cannot exceed.
2. Don’t over-fear input. Large context is comparatively cheap — and, as we’ll see, prompt caching makes repeated input cheaper still.

An analogy: input tokens are like the ingredients you bring to a restaurant kitchen, and output tokens are the finished, plated dishes the chef sends back. You pay a modest price to stock the pantry, but the labor of cooking — the output — is where the bill really accrues.

Tracking token usage in responses

Every Messages API response carries a usage object, and it is the source of truth for what a request actually cost. It reports four buckets of tokens [Source: https://platform.claude.com/docs/en/docs/build-with-claude/token-counting]:

The crucial subtlety — and a frequent source of confusion — is that input_tokens is not the total prompt size when caching is in play. It is only the uncached remainder after the last cache breakpoint. To reconstruct the full prompt size you must add all three input buckets:

total input = input_tokens + cache_creation_input_tokens + cache_read_input_tokens

If your agent ran for hours and input_tokens reads a mere 4,000, don’t conclude your prompts were tiny — the rest was almost certainly served from cache. Check the sum, not the single field.

In Python, reading the meter looks like this:

response = client.messages.create(
    model="claude-opus-4-8",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Summarize the report."}],
)
u = response.usage
print(u.input_tokens, u.output_tokens,
      u.cache_creation_input_tokens, u.cache_read_input_tokens)

When streaming, the final message_delta event carries the usage totals (including output_tokens), and stream.get_final_message() / .finalMessage() returns the complete usage object even though you consumed the response incrementally. It is also good practice to log response._request_id (the request-id header) so you can trace any individual request for auditing or support.

To compute exact spend, multiply each bucket by its per-token rate and sum. Wrapping that in a helper makes per-request cost logging trivial:

RATES = {  # $ per token for claude-opus-4-8
    "input":  5.00 / 1_000_000,
    "output": 25.00 / 1_000_000,
    "write":  6.25 / 1_000_000,   # 1.25x, 5-minute cache write
    "read":   0.50 / 1_000_000,   # 0.1x, cache hit
}

def cost(u):
    return (u.input_tokens              * RATES["input"]
          + u.output_tokens             * RATES["output"]
          + u.cache_creation_input_tokens * RATES["write"]
          + u.cache_read_input_tokens   * RATES["read"])

Modeling and forecasting cost

The usage object tells you what a request did cost, after the fact. To budget before you run — to decide which model to route to, or whether a prompt fits your target — you use the Token Counting endpoint, POST /v1/messages/count_tokens. It accepts the same structured inputs as message creation (system prompt, messages, tools, images, PDFs) and returns {"input_tokens": N} without actually running the model [Source: https://platform.claude.com/docs/en/docs/build-with-claude/token-counting].

resp = client.messages.count_tokens(
    model="claude-opus-4-8",
    system="You are a scientist.",
    messages=[{"role": "user", "content": "Hello, Claude"}],
)
print(resp.input_tokens)  # e.g. 14
estimated_input_cost = resp.input_tokens * (5.00 / 1_000_000)

Token counting is free, but it has its own requests-per-minute rate limits by usage tier (Start = 2,000 RPM, Build = 4,000 RPM, Scale = 8,000 RPM), and those limits are independent of message-creation limits — counting tokens does not draw down your message quota, and vice versa. Three production uses stand out: proactively managing rate limits and cost, making model-routing decisions by prompt size, and fitting a prompt to a target length or context window.

Two warnings are worth committing to memory:

• Never estimate Claude tokens with tiktoken. That is OpenAI’s tokenizer; it undercounts Claude tokens by roughly 15–20% on typical text and considerably more on code or non-English input. Use count_tokens, which is model-specific.
• Recount against the exact model you’ll use. Claude Opus 4.7 and later Opus models, Fable 5, and Sonnet 5 use a newer tokenizer that produces roughly 30% more tokens for the same text than earlier models. A count measured against an older model will mislead you. (You are never billed for system-optimization tokens the platform may add to the estimate — billing reflects only your content.)

A simple cost forecast for a batch of similar requests, then, is a matter of arithmetic. Suppose you plan to run 100,000 support-ticket classifications on Haiku 4.5, each with a ~400-token input and a ~20-token label output:

Input:  100,000 × 400 tok × ($1.00 / 1M)  = $40.00
Output: 100,000 ×  20 tok × ($5.00 / 1M)  = $10.00
Total (realtime, no caching)              = $50.00

That $50 baseline is the number we’ll drive downward in the next two sections — first with caching, then with batching. The platform also offers finer-grained spend controls worth knowing exist: an effort parameter (low through max) that trades intelligence for token spend, and beta Task Budgets, which give an agentic loop a self-moderating token countdown (distinct from max_tokens, which is an enforced ceiling the model is unaware of).

Key Takeaway: Output tokens cost 5× input tokens across every Claude model, so bounding output with max_tokens and modeling the input/output split is the foundation of cost control. The response usage object is your source of truth for actual spend (remember to sum all three input buckets), and the free count_tokens endpoint lets you forecast and route before you spend — never estimate with tiktoken.

Prompt Caching

Most production prompts are mostly repetition. A chatbot resends the same 5,000-token system prompt on every turn; a document-analysis tool re-reads the same 50-page contract for each of a dozen questions; an agent replays its entire tool set and conversation history on every step. Prompt caching lets Claude reuse those identical leading segments across requests, so you pay full price to process them once and roughly a tenth of the price every time after.

How prompt caching works

The mechanism rests on a single invariant, and everything else follows from it:

Prompt caching is an exact prefix match. Any byte that changes at or before a cache breakpoint invalidates that cache entry and every entry after it. [Source: https://platform.claude.com/docs/en/docs/build-with-claude/prompt-caching]

The cache key is derived from the exact bytes of the rendered prompt up to each breakpoint. Prompts are always rendered in a fixed order — tools → system → messages — so a breakpoint placed on the last system block caches the tools and the system prompt together.

Figure 4.1: Fixed prompt render order and where a cache prefix ends

flowchart LR
    T["tools (rendered first)"] --> S["system prompt"]
    S --> BP{{"cache_control breakpoint"}}
    BP -->|"caches tools + system as one prefix"| CACHE["Cached prefix (exact-byte match)"]
    BP --> M["messages (volatile: user's question)"]
    M --> UNCACHED["Billed as input_tokens each request"]
    CACHE -.->|"hit on next request"| READ["cache_read_input_tokens (~0.1x)"]

The economics are what make it worth the trouble. Using Claude Opus 4.8 (base input $5/MTok) as the worked example:

[Source: https://platform.claude.com/docs/en/docs/build-with-claude/prompt-caching]

A cache hit costs about 90% less than reprocessing the same tokens. The catch is that a cache write costs slightly more than a plain request — 1.25× for the default 5-minute time-to-live (TTL), or 2× for the optional 1-hour TTL. So caching is a bet: you pay a small premium to write the cache, and you win only if you get enough reads before it expires.

The analogy here is a photocopier versus retyping. Retyping a 50-page document every time (no cache) is slow and expensive. Making one photocopy costs a little more than a single read of the original (the write premium), but every subsequent copy is nearly free (the read discount). If you only ever need the document once, don’t photocopy it. If you need it repeatedly, the copier pays for itself almost immediately.

Figure 4.2: Prompt cache read/write decision path per request

flowchart TD
    START["Request arrives with cache_control prefix"] --> MIN{"Prefix >= model minimum? (1024-4096 tok)"}
    MIN -->|No| NOCACHE["No caching (silent): both cache buckets = 0"]
    MIN -->|Yes| HIT{"Matching entry still live in cache?"}
    HIT -->|"Miss (first request or TTL expired)"| WRITE["Cache WRITE: 1.25x (5-min) or 2x (1-hour)"]
    HIT -->|Hit| READ["Cache READ: ~0.1x (about 90% cheaper)"]
    WRITE --> TTL["Entry lives for TTL (5 min or 1 hour)"]
    READ --> TTL

Two ways to turn it on [Source: https://platform.claude.com/docs/en/build-with-claude/prompt-caching]:

# 1. Automatic caching — simplest; caches the last cacheable block
response = client.messages.create(
    model="claude-opus-4-8",
    max_tokens=1024,
    cache_control={"type": "ephemeral"},
    system=large_document_text,
    messages=[{"role": "user", "content": "Summarize the key points"}],
)

# 2. Explicit breakpoint — fine-grained placement on a specific block
response = client.messages.create(
    model="claude-opus-4-8",
    max_tokens=1024,
    system=[{
        "type": "text",
        "text": large_document_text,
        "cache_control": {"type": "ephemeral"},        # 5-minute TTL (default)
        # "cache_control": {"type": "ephemeral", "ttl": "1h"},  # 1-hour TTL
    }],
    messages=[{"role": "user", "content": "Summarize the key points"}],
)

ephemeral is the only supported cache type. There is one more silent gotcha: the cacheable prefix must clear a minimum token threshold — 1,024 tokens for Opus 4.8 and Sonnet 5, 2,048 for Fable 5, 4,096 for Haiku 4.5. A prompt below the minimum simply won’t cache, with no error raised — both cache_creation_input_tokens and cache_read_input_tokens come back as 0.

Cache breakpoints and checkpointing

A cache breakpoint (also called a cache checkpoint) is a marker — the cache_control parameter on a content block — that says “the reusable prefix ends here.” It defines where a cache entry gets written and where a later request will try to read a match. A few mechanical rules govern them:

• Maximum 4 breakpoints per request. You place cache_control on up to four content blocks (system text blocks, tool definitions, or message blocks).
• A 20-block lookback window. Each breakpoint walks backward at most 20 content blocks looking for a matching cached entry from a prior request. Long agentic turns that add more than 20 blocks at once can silently miss the cache; the fix is to place an intermediate breakpoint roughly every 15 blocks.
• Place the breakpoint on the last block that stays identical across requests. A breakpoint sitting on content that changes every request — a timestamp, a per-request UUID — hashes differently every time and never produces a hit.

The invalidation rules form a hierarchy: changes only invalidate their own tier and everything after it. Changing a tool definition or switching models invalidates everything; changing message content leaves the cached tools and system prompt intact. This is why swapping models mid-conversation or reordering your tool list is so costly — it throws away the entire cached prefix.

Verify your cache is actually hitting. The single most common caching bug is a silent invalidator — something in the prefix that changes on every request. If cache_read_input_tokens is 0 across repeated, seemingly-identical requests, hunt for one of these:

[Source: https://platform.claude.com/docs/en/build-with-claude/prompt-caching]

Structuring prompts to maximize cache hits — a worked example

The architectural rule is simple: stable content first, volatile content last. Freeze the system prompt, keep the tool list deterministic (sort tools by name), and inject anything dynamic — the user’s current question, the current timestamp — after the last breakpoint.

Now the promised break-even math. Return to our document-analysis tool: a 10,000-token contract, asked 5 questions in quick succession, on Opus 4.8. Compare caching the contract with a 5-minute TTL against not caching at all.

Without caching, all 10,000 tokens are processed at the base input rate on every request:

5 requests × 10,000 tok × ($5.00 / 1M) = $0.250 in input cost

With 5-minute caching, the first request writes the cache (1.25×) and the next four read it (~0.1×):

Request 1 (write): 10,000 tok × ($6.25 / 1M) = $0.0625
Requests 2–5 (read): 4 × 10,000 tok × ($0.50 / 1M) = $0.0200
Total input cost                              = $0.0825

Caching cut the repeated-context input cost from $0.250 to $0.0825 — a 67% reduction — and it would only widen with more questions. (Output cost is unaffected either way; caching applies to input only.)

When does caching lose? The general break-even rule falls straight out of the arithmetic. With a 5-minute TTL, a write costs 1.25× and each read costs 0.1×, so caching pays off after just two requests: 1.25× + 0.1× = 1.35× beats the 2× you’d pay to process the prefix twice uncached. With a 1-hour TTL, the write costs 2×, so you need at least three requests to come out ahead (2× + 0.2× = 2.2× vs. 3×). Use the 5-minute default for bursty, back-to-back traffic; reach for the 1-hour TTL only when requests are spaced far enough apart that the entry would otherwise expire between them. And if your prefix changes from the very first token every time, don’t cache at all — you’d pay the write premium for zero reads.

Key Takeaway: Prompt caching is an exact prefix match — put stable content (frozen system prompt, sorted tools) before your cache_control breakpoints and volatile content (timestamps, per-request IDs, the user’s question) after. Cache reads cost ~90% less than reprocessing; a 5-minute cache breaks even after just two requests, a 1-hour cache after three. Always confirm hits via usage.cache_read_input_tokens, because a silent invalidator produces zero savings and zero errors.

Batch Processing

Caching attacks cost by eliminating repeated work. Batch processing attacks it from a different angle entirely: by trading latency for a flat 50% discount. If a workload doesn’t need its answers right now, you can hand a pile of requests to the platform, let it process them asynchronously, and pay half.

The Message Batches API and its 24-hour window

The Message Batches API (POST /v1/messages/batches) asynchronously processes large volumes of standard Messages API requests. You submit a list of requests — each with a unique custom_id and a params object of normal Messages API parameters — and poll for completion [Source: https://platform.claude.com/docs/en/docs/build-with-claude/batch-processing].

from anthropic.types.message_create_params import MessageCreateParamsNonStreaming
from anthropic.types.messages.batch_create_params import Request

batch = client.messages.batches.create(requests=[
    Request(custom_id="req-1", params=MessageCreateParamsNonStreaming(
        model="claude-opus-4-8", max_tokens=1024,
        messages=[{"role": "user", "content": "Hello, world"}])),
    # ... up to 100,000 requests
])

# Poll until done
while True:
    b = client.messages.batches.retrieve(batch.id)
    if b.processing_status == "ended":
        break
    time.sleep(60)

# Stream results — match by custom_id, NOT by position
for result in client.messages.batches.results(batch.id):
    if result.result.type == "succeeded":
        msg = result.result.message
        ...

The defining constraint is the 24-hour processing window. Most batches finish in less than an hour, but results are guaranteed available when all requests complete or after 24 hours, whichever comes first. A batch that hasn’t completed within 24 hours expires — and expired requests are not billed. High demand or a very large batch can push processing toward that ceiling, so treat 24 hours as your worst case, not your expectation.

Figure 4.3: Message Batches API lifecycle

stateDiagram-v2
    [*] --> in_progress: create (up to 100k requests / 256 MB)
    in_progress --> ended: all requests complete
    in_progress --> ended: 24-hour window elapsed
    in_progress --> canceling: cancel requested
    canceling --> ended
    ended --> results: poll processing_status == "ended"
    results --> [*]: retained 29 days from creation
    note right of results
        Per-request result type:
        succeeded (billed) | errored | canceled | expired (not billed)
        Match by custom_id, never by position
    end note

A few operational rules matter for the exam and for production:

• Size limits: a batch holds at most 100,000 requests or 256 MB, whichever comes first. Exceeding 256 MB returns a 413 request_too_large.
• custom_id: 1–64 characters matching ^[a-zA-Z0-9_-]{1,64}$, unique within the batch.
• Results come back in any order. Always match a result to its request by custom_id — never by position. This is the single most common batch bug.
• Results are retained for 29 days after batch creation (not completion).
• Batches are Workspace-scoped and are not eligible for Zero Data Retention (ZDR).

There are four possible result types, and only one of them costs you money:

[Source: https://platform.claude.com/docs/en/docs/build-with-claude/batch-processing]

Cost savings for asynchronous workloads

All usage in a batch — input, output, and special tokens alike — is charged at 50% of standard prices [Source: https://claude.com/pricing#anthropic-api]:

Recall our 100,000-classification forecast, which cost $50.00 at realtime Haiku pricing. Run it as a batch instead:

Input:  100,000 × 400 tok × ($0.50 / 1M) = $20.00
Output: 100,000 ×  20 tok × ($2.50 / 1M) = $ 5.00
Total (batch)                            = $25.00

The batch discount cut the bill exactly in half, from $50 to $25 — the only cost being that the answers may take up to (but rarely) 24 hours instead of arriving in seconds.

Better still, batching and caching stack. Prompt caching works inside a batch and the two discounts compound. Because batch requests run concurrently and in any order, cache hits are best-effort — real-world hit rates typically land between 30% and 98% depending on traffic. Three tactics maximize them: include identical cache_control blocks in every request, keep a steady stream of requests flowing so the cache doesn’t expire, and share as much cached content as possible across requests. Because a batch can easily run longer than five minutes, the 1-hour cache TTL is recommended for batches that share a large context.

Realtime vs. batch selection criteria

The decision reduces to a single question: can this workload tolerate latency?

Figure 4.4: Choosing realtime vs. batch processing

flowchart TD
    Q1{"Is a human or user-facing system waiting on the response?"}
    Q1 -->|Yes| RT["Realtime Messages API (seconds; streaming supported)"]
    Q1 -->|No| Q2{"Can it tolerate up to 24h latency? (usually < 1 hour)"}
    Q2 -->|No| RT
    Q2 -->|Yes| BATCH["Message Batches API (50% of standard price)"]
    BATCH --> USES["Evals, bulk generation, moderation, data analysis"]
    RT --> USES2["Chat, autocomplete, voice, interactive apps"]

Use the synchronous Messages API whenever a human — or a user-facing system — is waiting on the response: chat, autocomplete, voice assistants, anything interactive. Reach for the Batches API when responses are not time-sensitive and cost matters: processing a nightly queue of documents, running a 50,000-item evaluation set, moderating a backlog of user content, or generating product descriptions for an entire catalog. The mental model is a courier versus standard shipping — pay a premium for same-hour delivery when someone is waiting at the door; use the cheaper overnight service when tomorrow is fine.

Key Takeaway: The Message Batches API processes up to 100,000 requests asynchronously at 50% of standard prices, with results guaranteed within 24 hours (usually under an hour). Choose batch over realtime whenever your workload can tolerate that latency — bulk generation, evals, moderation, analysis. Always match results by custom_id (never by position), and remember that only succeeded results are billed; caching stacks on top of the batch discount, so prefer the 1-hour TTL for batches sharing context.

Optimization Patterns

The three levers — token budgeting, caching, and batching — are most powerful in combination. This section pulls them together into the patterns you’ll actually deploy, and closes with how to keep an eye on spend once your application is live.

Reducing context bloat to save tokens

The cheapest token is the one you never send. Before optimizing how you pay for tokens, cut the tokens you don’t need:

• Right-size max_tokens. Because output costs 5× input, a loose output ceiling is the fastest way to overspend. Set it to the smallest value that comfortably fits your expected response.
• Trim the prompt itself. Long conversation histories, redundant instructions, and duplicated context all cost input tokens on every request. For long-running agents, the platform offers context editing (clearing stale tool results and thinking blocks) and compaction (summarizing earlier history) to keep the transcript lean without losing coherence.
• Count before you commit. Use count_tokens to verify a prompt fits your target and to route by size — a small prompt to a cheaper model, a large one to a more capable model only when warranted.

Combining caching, batching, and model choice

The three levers multiply. Consider a realistic pipeline: analyze 10,000 documents against a large, fixed 8,000-token rulebook, one question each. Start from the naive baseline on Opus 4.8 and stack optimizations:

Each lever attacks a different axis — caching removes repeated cost, batching halves all cost, model choice lowers the per-token cost — so they compound rather than overlap. A workload optimized on all three axes routinely runs for a small fraction of the naive bill. The engineering discipline is to apply each lever where it fits: cache the shared rulebook (it’s identical across requests), batch the whole run (nobody’s waiting), and route by difficulty (don’t pay Opus prices for a task Haiku handles).

A practical caution when combining caching with concurrency: a cache entry only becomes readable after the first response begins streaming. Fire N identical-prefix requests in parallel and all N pay the full write price — none can read what the others are still writing. For fan-out, send one request, await its first token, then fire the rest so they read the cache the first one just wrote. (Inside a batch this is handled for you as best-effort, which is why batch cache hits are a range rather than a guarantee.)

Monitoring spend in production

Optimization without measurement is guesswork. Build cost observability in from the start:

• Log the usage object on every response. Persist all four token buckets plus the computed dollar cost and the request_id. This is your ground-truth ledger, and it’s the only way to catch a regression — say, a cache that quietly stopped hitting after a refactor introduced a datetime.now() into the system prompt.
• Alert on cache-hit rate. If cache_read_input_tokens trends toward zero on a workload that should be caching, a silent invalidator has crept in. Diff the rendered prompt bytes between two requests to find it.
• Forecast with count_tokens before large runs. Estimate the cost of a 100,000-item batch before submitting it, so a surprise is a caught bug rather than a monthly invoice.
• Watch spend limits. Batch processing runs concurrently and may slightly exceed a Workspace’s configured spend limit before it takes effect — plan headroom accordingly.

The discipline mirrors any good operations practice: instrument first, then optimize, then verify the optimization actually landed by watching the meter. Because Claude hands you the meter on every single response, closing that loop costs almost nothing.

Key Takeaway: The optimization levers compound — caching removes repeated input cost, batching halves all cost, and right-sizing max_tokens plus routing to cheaper models lowers per-token cost, so apply each where it fits. Cut context bloat first (the cheapest token is the one never sent), then log the usage object on every response and alert on cache-hit rate, so a silent regression shows up on your dashboard rather than your invoice.

Chapter Summary

This chapter turned Claude’s cost structure from an opaque monthly surprise into a set of deliberate engineering controls.

• Tokens are the unit of both work and billing, and output costs 5× input across every model. Bounding output with max_tokens and modeling the input/output split is the foundation of cost control.
• The response usage object is your source of truth for actual spend, reporting four buckets: input_tokens, output_tokens, cache_creation_input_tokens, and cache_read_input_tokens. Total input is the sum of the three input buckets — input_tokens alone is only the uncached remainder.
• The count_tokens endpoint forecasts cost before you spend, is free (with independent rate limits), and must be used instead of tiktoken, which undercounts Claude tokens. Recount against your exact model, since newer models tokenize ~30% larger.
• Prompt caching is an exact prefix match. Stable content goes before the cache_control breakpoint, volatile content after. Reads cost ~90% less; a 5-minute cache breaks even after two requests, a 1-hour cache after three. Verify hits via cache_read_input_tokens and hunt for silent invalidators when it reads zero.
• The Message Batches API processes up to 100,000 requests asynchronously at 50% of standard prices, with results guaranteed within 24 hours. Match results by custom_id, never by position; only succeeded results are billed.
• Realtime vs. batch is a latency-tolerance decision: synchronous Messages API for anything a human waits on, batch for cost-sensitive bulk work.
• The levers compound. Caching, batching, and model choice each attack a different cost axis, and disciplined spend monitoring — logging usage, alerting on cache-hit rate, forecasting with count_tokens — keeps the optimizations honest.

Master these and you can run the same Claude workload for a fraction of the cost of a developer who treats the invoice as inevitable.

Key Terms

Chapter 5: Claude API Mechanics

Every capability you have studied so far — reasoning, tool use, vision, extended thinking — reaches your code through a single doorway: the Messages API. There is no separate “vision endpoint,” no dedicated “tools service,” no distinct “streaming server.” There is one endpoint, POST /v1/messages, and everything else is a feature of that one request [Source: https://platform.claude.com/docs/en/api/messages/create]. If you internalize that single fact, most of the API’s apparent complexity collapses into a manageable set of request fields and response blocks.

Think of the Messages API like a restaurant kitchen with one order window. You don’t walk to a different counter for drinks, appetizers, and dessert — you write everything on one ticket (the request), and the kitchen sends back a tray of plates (the response), each plate a different kind of content block. This chapter teaches you how to write that ticket correctly, how to read the tray, and how to do both when the kitchen is operated by a third party like Amazon or Google.

By the end you should be able to construct a well-formed request with a system prompt, messages, and tools; use streaming, vision, and thinking features correctly; and invoke Claude through Amazon Bedrock and Google Vertex AI. We will call out the single most common beginner mistake — confusing the system message role with the top-level system parameter — early and repeatedly, because it trips up nearly everyone once.

The Messages API

Request/response structure and roles

Every request is an HTTP POST to https://api.anthropic.com/v1/messages carrying three mandatory headers: x-api-key: <your key>, anthropic-version: 2023-06-01, and content-type: application/json. Beta features add a fourth, anthropic-beta: <feature-id> [Source: https://platform.claude.com/docs/en/api/messages/create].

The request body has three required top-level fields:

Here is a minimal, well-formed request expressed as JSON:

{
  "model": "claude-opus-4-8",
  "max_tokens": 1024,
  "messages": [
    { "role": "user", "content": "What is the capital of France?" }
  ]
}

The response comes back as a JSON object whose most important field is content — an array of content blocks, not a plain string:

{
  "id": "msg_01ABC...",
  "type": "message",
  "role": "assistant",
  "model": "claude-opus-4-8",
  "content": [
    { "type": "text", "text": "The capital of France is Paris." }
  ],
  "stop_reason": "end_turn",
  "usage": {
    "input_tokens": 14,
    "output_tokens": 9,
    "cache_creation_input_tokens": 0,
    "cache_read_input_tokens": 0
  }
}

Two response fields deserve immediate attention. The stop_reason tells you why Claude stopped generating — a topic we return to below. The usage object reports token accounting: input_tokens (billed at full rate), output_tokens, and two cache-related counters (cache_creation_input_tokens, cache_read_input_tokens) covered in the caching section [Source: https://platform.claude.com/docs/en/api/messages/create].

Figure 5.1: Messages API request/response round-trip

sequenceDiagram
    participant App as Your Application
    participant API as Messages API (POST /v1/messages)
    App->>API: POST request (model, max_tokens, messages)
    Note over API: Claude generates the reply
    API-->>App: Response (content[] blocks, stop_reason, usage)
    Note over App: Iterate content[] and branch on each block's type

Because content is always an array, robust code never assumes content[0] is text. When thinking or tool use is enabled, the first block may be a thinking or tool_use block instead. Always iterate the array and branch on each block’s type.

System prompts vs. user/assistant messages

Here is the mistake that catches almost every newcomer. The system prompt — the standing instructions that define Claude’s role and behavior — is a top-level system parameter, not an entry in the messages array [Source: https://platform.claude.com/docs/en/api/messages/create].

{
  "model": "claude-opus-4-8",
  "max_tokens": 1024,
  "system": "You are a terse assistant. Answer in one sentence.",
  "messages": [
    { "role": "user", "content": "Explain photosynthesis." }
  ]
}

The tempting error is to write the system prompt as a message, like {"role": "system", "content": "..."}, inside the messages array. Historically the API rejected that outright. Think of it this way: the system parameter is the job description posted on the wall before anyone walks in; the messages array is the transcript of the actual conversation. You don’t paste the job description into the middle of the dialogue.

Note (visual aid): A simple two-column diagram helps here — left column labeled “Top-level parameters” listing model, max_tokens, system, tools; right column labeled “messages[]” showing alternating user / assistant bubbles. It makes the separation visually unmistakable.

The system parameter accepts either a plain string (as above) or an array of text content blocks. You use the array form when you want to attach cache_control for prompt caching (covered later). Within messages, the valid roles are user and assistant. There is one modern exception worth knowing for the exam: newer models such as Claude Opus 4.8 allow a mid-conversation system message — a {"role": "system", "content": "..."} entry appended to messages[] — as a cache-preserving, prompt-injection-safe operator channel [Source: https://platform.claude.com/docs/en/api/messages/create]. This is a deliberate, model-gated feature, not a license to put your main system prompt in the messages array. On models that don’t support it, that entry returns a 400 with the message role 'system' is not supported on this model. The rule to memorize: the primary system prompt is always the top-level system parameter.

Data access patterns and multi-turn conversations

The single most important operational fact about the Messages API is that it is stateless. The server keeps no memory of your prior calls. To hold a multi-turn conversation, you resend the full conversation history — every user and assistant turn — on every request [Source: https://platform.claude.com/docs/en/api/messages/create].

A conversation therefore grows like an accumulating ledger:

{
  "model": "claude-opus-4-8",
  "max_tokens": 1024,
  "messages": [
    { "role": "user", "content": "My name is Alice." },
    { "role": "assistant", "content": "Nice to meet you, Alice." },
    { "role": "user", "content": "What's my name?" }
  ]
}

Three rules govern the array:

1. The first message must be user. A conversation cannot open with an assistant turn.
2. Roles conventionally alternate user/assistant, though the API will combine consecutive same-role messages into one turn.
3. content may be a plain string (shorthand) or an array of content blocks. Block type values include text, image, document, tool_use, tool_result, thinking, and redacted_thinking [Source: https://platform.claude.com/docs/en/api/messages/create].

Because history is resent every turn, long conversations grow in cost and eventually approach the model’s context window. This is precisely why prompt caching (later in this chapter) matters so much for multi-turn workloads — you cache the stable prefix so you aren’t paying full price to re-process it on every turn.

Key Takeaway: Everything flows through one endpoint, POST /v1/messages, with three required fields: model, max_tokens, and messages. The system prompt is a top-level system parameter, never a {"role":"system"} entry in messages (except as a model-gated mid-conversation feature on newer models). The API is stateless, so you resend the full conversation history on every request.

Streaming and Realtime Behavior

Server-sent events and streaming responses

By default, a Messages request blocks until Claude has finished generating, then returns the whole response at once. For a long reply that can mean many seconds of silence followed by a wall of text. Streaming solves this: set stream: true on the request (or use an SDK helper such as client.messages.stream()), and the server pushes the response back incrementally as a sequence of Server-Sent Events (SSE) — a standard where the server holds one HTTP connection open and emits a series of small event:/data: chunks over time [Source: https://platform.claude.com/docs/en/api/messages-streaming].

The analogy is a live sports commentator versus a next-day newspaper report. Non-streaming is the newspaper: complete, but you wait. Streaming is the commentator: you hear each play as it happens.

The canonical event sequence is worth memorizing, because the exam and real debugging both lean on it:

[Source: https://platform.claude.com/docs/en/api/messages-streaming]

Figure 5.2: SSE streaming event sequence

sequenceDiagram
    participant Client as Streaming Client
    participant Server as Messages API (stream: true)
    Client->>Server: POST request with stream: true
    Server-->>Client: message_start (metadata, empty content)
    Server-->>Client: content_block_start (index, block shell)
    loop Per chunk
        Server-->>Client: content_block_delta (text_delta / input_json_delta / thinking_delta)
    end
    Server-->>Client: content_block_stop (index)
    Server-->>Client: message_delta (stop_reason, cumulative usage)
    Server-->>Client: message_stop (stream complete)

The heart of the stream is the content_block_delta event, whose delta.type tells you what kind of increment you received:

• text_delta — a chunk of visible text (in .text)
• input_json_delta — a chunk of a tool call’s JSON input, streamed piecemeal
• thinking_delta — a chunk of extended-thinking reasoning (in .thinking)
• signature_delta — the cryptographic signature that closes a thinking block

Handling partial output

Because the response arrives in pieces, streaming code must accumulate. In practice you almost never hand-assemble the deltas: the SDKs expose helpers — stream.get_final_message() in Python, stream.finalMessage() in TypeScript — that reconstruct the complete Message object for you while still letting you render text as it arrives. This gives you the best of both worlds: live output for the user, and a clean, fully-assembled object once the stream ends [Source: https://platform.claude.com/docs/en/api/messages-streaming].

Two practical cautions. First, a stream can be interrupted mid-flight (a dropped connection), in which case you hold only a partial response — design your UI to tolerate that. Second, track the message_delta event, because that is where the final stop_reason and cumulative output-token count arrive; if you only watch the text deltas you will miss why the model stopped.

Realtime vs. batch tradeoffs

Streaming is one axis of “how you get output”; the other is realtime versus batch. Realtime — whether streamed or returned in one shot — gives you an answer now, and is what you want for anything interactive: chat, agents, anything with a user waiting.

The Message Batches API (POST /v1/messages/batches) is the asynchronous alternative. You submit up to 100,000 requests at once, wrapped as {custom_id, params} pairs, and Claude processes them in the background at 50% of the standard token price on both input and output [Source: https://claude.com/blog/message-batches-api]. You poll the batch’s processing_status until it reads "ended", then stream the results. Most batches finish within an hour; the maximum is 24 hours, and results are retained for 29 days. Critically, results arrive in any order, so you must key them by custom_id, never by position [Source: https://platform.claude.com/docs/en/build-with-claude/batch-processing].

The essential exam point: there is no quality difference between batch and realtime — you are trading latency for cost and throughput, nothing more [Source: https://claude.com/blog/message-batches-api].

Key Takeaway: Streaming (stream: true) delivers the response as a sequence of SSE events — message_start → content_block_start → content_block_delta → content_block_stop → message_delta (carrying stop_reason) → message_stop — and is strongly recommended for large max_tokens to avoid HTTP timeouts. Realtime returns answers immediately; the Batch API trades latency for a 50% discount with identical quality, keyed by custom_id.

Multi-Format Input and Vision

Text, image, and document inputs

The content array is not limited to text. You compose a multi-format input by mixing content blocks of different type values inside a single user message. The most common non-text blocks are image and document.

Figure 5.3: Composing a multi-format user message

flowchart LR
    Img["image block: source (base64 / url / file)"] --> Msg["Single user message: content[]"]
    Doc["document block: PDF / text source"] --> Msg
    Txt["text block: the question about the media"] --> Msg
    Msg --> Req["Messages API request"]

An image block carries a source of one of three kinds:

1. base64 — the raw image bytes, base64-encoded, with a media_type. Supported everywhere.
2. url — a link Claude fetches.
3. file — a file_id from the Files API, letting you upload once and reference many times.

Here is a vision request that sends an image and asks about it:

{
  "model": "claude-opus-4-8",
  "max_tokens": 1024,
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "image",
          "source": {
            "type": "base64",
            "media_type": "image/png",
            "data": "<base64-encoded bytes>"
          }
        },
        { "type": "text", "text": "What's in this image?" }
      ]
    }
  ]
}

Supported image formats are JPEG, PNG, GIF, and WebP. Claude works best with image-then-text ordering — put the image block before the text that asks about it [Source: https://platform.claude.com/docs/en/build-with-claude/vision].

Documents work the same way, via a document block with the same three source types (url, base64 with media_type: "application/pdf", or file). Place the document block before the text block. A key subtlety: PDF support relies on Claude’s vision — each page is processed as both text and image, so PDFs inherit vision’s costs and limits. Plain-text files (.txt, .csv, .md) can be sent as documents too, but binary office formats like .xlsx and .docx are not supported and must be converted to text or PDF first [Source: https://platform.claude.com/docs/en/build-with-claude/pdf-support].

Vision capabilities

Understanding how vision is priced prevents surprise bills. Claude views images in patches: each 28×28-pixel block equals one visual token. An image therefore costs approximately ⌈width/28⌉ × ⌈height/28⌉ visual tokens [Source: https://platform.claude.com/docs/en/build-with-claude/vision].

Resolution capability comes in two tiers:

[Source: https://platform.claude.com/docs/en/build-with-claude/vision]

Images larger than a tier’s limit are downscaled automatically. As a concrete example, a 1000×1000 image costs about 1,296 tokens; a 1920×1080 image on the high-resolution tier costs roughly 2,691 tokens. High-res images can cost about 3× the tokens of the same image at standard tier — real fidelity, real cost. Other limits to know: a maximum of 100 images per request on 200K-context models (600 on others), maximum dimensions of 8000×8000 px, and up to 10 MB per image on the first-party Claude API [Source: https://platform.claude.com/docs/en/build-with-claude/vision].

Thinking blocks in responses

When extended thinking is enabled, Claude reasons internally before answering, and that reasoning surfaces in the response as thinking content blocks alongside the normal text blocks. A thinking block has three fields: type: "thinking", a thinking string (the reasoning), and a signature — an opaque, encrypted string that lets the API verify the reasoning across turns [Source: https://platform.claude.com/docs/en/build-with-claude/extended-thinking].

On current models (Opus 4.8, Opus 4.7, Sonnet 5, Fable 5), you enable thinking with adaptive thinking: thinking: {"type": "adaptive"}. The older thinking: {"type": "enabled", "budget_tokens": N} form is removed on these models and returns a 400 — an easy exam trap. You control reasoning depth with output_config: {"effort": "low" | "medium" | "high" | "xhigh" | "max"}, and a display sub-field controls visibility: "summarized" returns a readable summary, while "omitted" (the default on the newest models) leaves the thinking field empty. Crucially, display affects visibility only — you are billed for thinking tokens regardless [Source: https://platform.claude.com/docs/en/build-with-claude/extended-thinking].

{
  "model": "claude-opus-4-8",
  "max_tokens": 8000,
  "thinking": { "type": "adaptive", "display": "summarized" },
  "output_config": { "effort": "high" },
  "messages": [
    { "role": "user", "content": "Solve this step by step: 27 * 453" }
  ]
}

There is one rule you must not violate: in a multi-turn or tool-use conversation on the same model, you must pass thinking blocks back unchanged. Echo the [thinking_block, tool_use_block] in the assistant turn before returning your tool_result. Modifying a thinking block returns an invalid_request_error stating that thinking blocks in the latest assistant message cannot be modified [Source: https://platform.claude.com/docs/en/build-with-claude/extended-thinking]. Think of the signature as a wax seal: you may read the letter, but if you break the seal the API won’t accept it back.

Key Takeaway: You compose multi-format input by mixing image and document blocks (each with url, base64, or file sources) alongside text in a user message, placing media before the text that references it. Vision is priced in 28×28-pixel visual tokens with high-resolution and standard tiers. Extended thinking surfaces as thinking blocks (with a signature) that must be echoed back unchanged in multi-turn/tool-use flows; enable it with adaptive thinking, not the deprecated budget_tokens.

Tools and Third-Party Access

Tool use in the Messages API

Tool use lets Claude call functions you define — fetching weather, querying a database, sending email. You declare a tools array on the request. Each tool has a name, a description (Claude reads this to decide when to call the tool), and an input_schema — a JSON Schema object describing the arguments [Source: https://platform.claude.com/docs/en/api/messages/create].

{
  "model": "claude-opus-4-8",
  "max_tokens": 1024,
  "tools": [
    {
      "name": "get_weather",
      "description": "Get the current weather for a location.",
      "input_schema": {
        "type": "object",
        "properties": {
          "location": { "type": "string", "description": "City name" }
        },
        "required": ["location"]
      }
    }
  ],
  "messages": [
    { "role": "user", "content": "What's the weather in Paris?" }
  ]
}

The tool-use loop is a conversation, not a single call. It proceeds like a relay:

1. Claude decides to call the tool and returns a tool_use block — {"type":"tool_use","id":"toolu_...","name":"get_weather","input":{"location":"Paris"}} — and the response stop_reason becomes "tool_use".
2. Your code executes the tool.
3. You send a new user message containing a tool_result block whose tool_use_id matches the id from step 1: {"type":"tool_result","tool_use_id":"toolu_...","content":"72°F and sunny"}.
4. Claude reads the result and produces its final answer.

Figure 5.4: The tool_use / tool_result loop

sequenceDiagram
    participant App as Your Code
    participant Claude as Claude (Messages API)
    App->>Claude: user message + tools[]
    Claude-->>App: tool_use block (stop_reason: "tool_use")
    Note over App: Execute the requested tool
    App->>Claude: user message with tool_result (matching tool_use_id)
    Claude-->>App: final answer (stop_reason: "end_turn")

You control tool selection with tool_choice: {"type":"auto"} (the default), {"type":"any"} (must use some tool), {"type":"tool","name":"..."} (force a specific one), or {"type":"none"}. When Claude requests several tools at once, execute them all and return all tool_result blocks in a single user message — splitting them across messages silently trains Claude to stop making parallel calls [Source: https://platform.claude.com/docs/en/api/messages/create]. Anthropic also offers server-side tools (web search, web fetch, code execution) that run on Anthropic’s own infrastructure; you declare them in tools and the results come back as content blocks in the same response.

The stop_reason field is your loop’s control signal. Learn the full set:

stop_details is populated only when stop_reason == "refusal"; for every other stop reason it is null, so guard before reading it [Source: https://platform.claude.com/docs/en/api/messages/create].

Invoking Claude via Amazon Bedrock and Google Vertex AI

You need not call Anthropic’s servers directly. Two major clouds host Claude, and each SDK provides a dedicated client class for them — you do not point the first-party client at a custom URL.

Amazon Bedrock. Use the Mantle client — AnthropicBedrockMantle(aws_region="...") in Python, new AnthropicBedrockMantle({ awsRegion }) in TypeScript. Region is required (there is no default). Model IDs take an anthropic. prefix — anthropic.claude-opus-4-8, not the bare claude-opus-4-8, which returns a 400. Authentication is AWS-native (SigV4 / IAM), not an Anthropic API key [Source: https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-anthropic-claude-messages.html].

Google Vertex AI. Use AnthropicVertex(project_id="...", region="...") in Python (or new AnthropicVertex({ projectId, region }) in TypeScript). Two constructor arguments are required: the GCP project_id and a region (often "global"). Vertex model IDs take no prefix — current-generation models use the bare first-party ID (claude-opus-4-8), while dated snapshots use an @ separator (claude-opus-4-5@20251101). Authentication is GCP Application Default Credentials — no Anthropic API key [Source: https://platform.claude.com/docs/en/build-with-claude/claude-on-vertex-ai].

The payoff of these dedicated clients is uniformity: after construction, all three expose the same messages.create / .stream surface. The mental model is a franchise. The recipe (the Messages API) is identical; only the sign over the door and the way you badge in differ.

Feature parity is close but not total. Core Messages, streaming, tool use, extended thinking, PDF input, and explicit prompt caching all work on Bedrock and Vertex. The notable gaps: automatic prompt caching is not available on either (explicit cache_control still works); the Message Batches API, Files API, and Models API are first-party only; web fetch and code execution are unavailable on both; and web search is Vertex-only in its basic variant [Source: https://platform.claude.com/docs/en/build-with-claude/claude-on-vertex-ai].

Caching within API requests

Prompt caching lets you store a stable prefix of your request so that repeated calls don’t re-process — and re-pay for — the same tokens. The mechanism is a prefix match: Claude renders your request in the order tools → system → messages, and any byte change anywhere in that prefix invalidates the cache from that point forward [Source: https://platform.claude.com/docs/en/build-with-claude/prompt-caching].

You mark a cache breakpoint by attaching cache_control to the last block of a stable prefix — typically the last system text block:

{
  "model": "claude-opus-4-8",
  "max_tokens": 1024,
  "system": [
    {
      "type": "text",
      "text": "<large shared instructions / document>",
      "cache_control": { "type": "ephemeral" }
    }
  ],
  "messages": [
    { "role": "user", "content": "Summarize the key points." }
  ]
}

The default cache lifetime is 5 minutes; pass {"type":"ephemeral","ttl":"1h"} for one hour. You may set up to 4 breakpoints per request, and the minimum cacheable prefix is model-dependent (for example, 4096 tokens on Opus 4.8 — shorter prefixes silently fail to cache) [Source: https://platform.claude.com/docs/en/build-with-claude/prompt-caching].

Verify caching with the usage counters: cache_creation_input_tokens reports tokens written to the cache (about 1.25× cost for the 5-minute TTL), and cache_read_input_tokens reports tokens served from it (about 0.1× cost — a ~90% saving). If cache_read_input_tokens stays zero across requests that share an identical prefix, a silent invalidator is at work: a datetime.now() or UUID interpolated into the system prompt, unsorted JSON, or a tool set that changes between requests. The fix is to keep the prefix byte-identical — freeze the system prompt, serialize JSON deterministically, and hold the tool list stable [Source: https://platform.claude.com/docs/en/build-with-claude/prompt-caching].

Key Takeaway: Tool use is a loop: declare tools, receive a tool_use block with stop_reason: "tool_use", execute the tool, and reply with a matching tool_result block (all parallel results in one user message). Invoke Claude on Amazon Bedrock (AnthropicBedrockMantle, anthropic.-prefixed IDs, SigV4 auth) or Google Vertex AI (AnthropicVertex, bare IDs, GCP ADC) via dedicated clients that share the same Messages surface. Prompt caching is a prefix match (tools → system → messages) verified by the cache_read_input_tokens usage counter — automatic caching is first-party only.

Chapter Summary

The Claude API is unified around a single endpoint, POST /v1/messages. A valid request always carries model, max_tokens, and a non-empty messages array; the system prompt lives in the separate top-level system parameter, and confusing it with a {"role":"system"} message is the classic beginner error. Because the API is stateless, you resend the full conversation history on every turn, which makes prompt caching a first-class cost lever rather than an afterthought.

Output can be delivered in realtime (optionally streamed as SSE events, which you should use for large max_tokens to avoid timeouts) or via the asynchronous Batch API at half price for non-latency-sensitive bulk work — with identical quality. The response is always an array of content blocks: text, plus image/document you sent in, thinking blocks when extended thinking is on (echo them back unchanged), and tool_use/tool_result blocks that drive the tool-use loop. Vision and PDFs share the same block machinery and the same visual-token pricing.

Finally, Claude runs the same Messages surface whether you call Anthropic directly or through Amazon Bedrock (anthropic.-prefixed IDs, SigV4) or Google Vertex AI (bare IDs, GCP credentials) — with a handful of first-party-only features (automatic caching, Batches, Files, and Models APIs) that the exam expects you to recognize as gaps on the third-party platforms.

Key Terms

Chapter 6: Software Engineering Foundations for Claude Integration

Building an application that talks to Claude is, at its core, an exercise in classic software engineering. Before you ever write a clever prompt, your code must speak the language of the web (REST and JSON), stay responsive while waiting on the network (asynchronous programming), lean on well-built tools (client SDKs), and evolve safely over time (version control, code review, and refactoring). This chapter grounds each of those foundations in the concrete reality of integrating Anthropic’s Claude API, so that the model becomes just one well-behaved component in an application you can reason about, test, and maintain.

Think of it this way: Claude is a brilliant consultant who only takes calls through a very specific phone system. The consultant’s insight is only as useful as your ability to dial correctly, wait patiently for the answer, handle a busy signal gracefully, and keep good records of every conversation. This chapter is about the phone system, not the consultant.

6.1 REST APIs and JSON

Every request your application sends to Claude travels over a REST API — an application programming interface that follows the conventions of Representational State Transfer, using standard HTTP methods and URLs to operate on resources. Understanding REST and its data format, JSON (JavaScript Object Notation, a lightweight, human-readable text format for structured data), is the bedrock of Claude integration.

REST principles and HTTP methods

REST treats everything the server exposes as a resource addressed by a URL, and it uses a small, fixed vocabulary of HTTP methods to act on those resources. The analogy that helps most learners: a REST API is like a restaurant menu with a stable address for each dish, where you place orders using a handful of universal verbs.

The single most important endpoint for Claude integration is the Messages API: POST https://api.anthropic.com/v1/messages [Source: https://platform.claude.com/docs/en/api/client-sdks]. You use POST because you are submitting new work — a conversation turn — for Claude to process. REST also inherits HTTP’s stateless property: each request carries everything the server needs, which is why you resend the full conversation history on every turn rather than relying on the server to remember it.

A diagram would help here: a simple request/response arrow between “Your App” and “api.anthropic.com/v1/messages,” labeled with the HTTP method and the JSON payload flowing each direction.

Figure 6.1: Stateless request/response flow for a Messages API call

sequenceDiagram
    participant App as Your App
    participant API as api.anthropic.com/v1/messages
    App->>API: POST /v1/messages
    Note right of App: Headers: x-api-key,<br/>anthropic-version, content-type<br/>Body: full messages array (JSON)
    API-->>App: 200 OK
    Note left of API: JSON: content array,<br/>token usage, stop_reason
    Note over App,API: Stateless: every turn resends<br/>the entire conversation history

JSON request and response handling

The body of a Claude request and the body of its response are both JSON. A minimal request looks like this:

{
  "model": "claude-sonnet-5",
  "max_tokens": 1024,
  "messages": [{ "role": "user", "content": "Hello, Claude" }]
}

Notice the structure: a top-level object with typed fields. model selects which Claude model answers; max_tokens caps the response length (and therefore cost); messages is an ordered array of turns, each with a role (user or assistant) and content. The response is likewise JSON — an object with a content array holding the assistant’s reply, plus metadata such as token usage and a stop reason.

Worked example — the same call as raw cURL, exactly as your application (or an SDK) constructs it under the hood:

curl https://api.anthropic.com/v1/messages \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{
    "model": "claude-sonnet-5",
    "max_tokens": 1024,
    "messages": [{"role": "user", "content": "Hello, Claude"}]
  }'

[Source: https://platform.claude.com/docs/en/api/client-sdks]

Your job as an integrator is serialization (turning your in-memory data structures into a JSON string for the request) and deserialization (parsing the JSON response back into objects you can use). As you will see in Section 6.3, the client SDKs do this for you, but knowing the shape of the JSON is essential for debugging, logging, and reading the docs.

Authentication headers and API keys

Claude requests carry authentication in HTTP headers — key/value metadata sent alongside the body. Three headers matter for a basic call [Source: https://platform.claude.com/docs/en/api/client-sdks]:

The API key is a bearer credential: anyone who holds it can spend from your account. It belongs in an environment variable (conventionally ANTHROPIC_API_KEY), never hardcoded in source and never committed to version control — a discipline we return to in Section 6.4. The version header is a subtle but important REST practice: by pinning anthropic-version, your integration keeps behaving the same way even as Anthropic ships new capabilities behind newer version dates.

Key Takeaway: Claude integration rides on standard REST over HTTP: you POST JSON to https://api.anthropic.com/v1/messages, sending a model, max_tokens, and a messages array, and receive JSON back. Authentication and versioning live in headers — x-api-key, anthropic-version, and content-type — and the API key must be treated as a secret pulled from the environment, never hardcoded.

6.2 Asynchronous Programming

A call to Claude is a network round-trip that can take seconds. If your program simply stops and waits, it wastes time it could spend doing other work — serving other users, prefetching data, or issuing more Claude requests. Asynchronous programming is the technique of starting an operation and continuing to do other work while it completes, rather than blocking. It is the difference between a chef who stares at the oven until the cake is done and one who preps the next three dishes while the cake bakes.

Async/await patterns for API calls

The modern expression of asynchronous programming is async/await: language syntax that lets you write non-blocking code that reads like ordinary sequential code. You mark a function as asynchronous and use await at the point where you need the result; the runtime is free to run other tasks until the awaited operation finishes.

Anthropic’s SDKs provide dedicated async clients for exactly this purpose [Source: https://platform.claude.com/docs/en/api/client-sdks]. In Python, the async client is AsyncAnthropic, and both the sync and async clients are powered by the httpx HTTP library:

import anthropic
import asyncio

client = anthropic.AsyncAnthropic()  # reads ANTHROPIC_API_KEY from the environment

async def ask(question: str) -> str:
    message = await client.messages.create(
        model="claude-sonnet-5",
        max_tokens=1024,
        messages=[{"role": "user", "content": question}],
    )
    return message.content[0].text

print(asyncio.run(ask("Hello, Claude")))

In TypeScript, calls are promise-based by default, so await is all you need [Source: https://platform.claude.com/docs/en/api/sdks/typescript]:

import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic();
const message = await client.messages.create({
  model: "claude-sonnet-5",
  max_tokens: 1024,
  messages: [{ role: "user", content: "Hello, Claude" }],
});

Concurrency and parallel requests

The real payoff of async clients is concurrency: firing off many Claude requests that overlap in flight rather than running strictly one after another. If you need to summarize 20 documents, running them concurrently can be dramatically faster than looping sequentially, because most of each request’s duration is spent waiting on the network — time that overlaps for free.

Worked example — fan out several requests and await them together in Python:

async def summarize_all(docs: list[str]) -> list[str]:
    tasks = [ask(f"Summarize: {d}") for d in docs]
    return await asyncio.gather(*tasks)   # all requests in flight at once

There is a catch the exam expects you to know. Concurrency raises the risk of hitting rate limits — the per-minute caps Anthropic enforces on requests and tokens. The guidance is to ramp traffic up gradually and keep usage patterns consistent rather than sending sudden bursts, which can trip both standard limits and acceleration limits triggered by sharp spikes [Source: https://platform.claude.com/docs/en/api/rate-limits]. Sudden bursts produce 429 errors (covered in Section 6.3), so effective concurrency means bounding how many requests are in flight, not simply launching everything at once.

Websockets and streaming connections

Sometimes you want Claude’s answer to appear token-by-token as it is generated — the “typing” effect in chat UIs — rather than waiting for the complete response. This is streaming, and it is a place where a common misconception must be corrected.

A websocket is a persistent, bidirectional connection that stays open so client and server can send messages to each other at any time, in both directions — well suited to interactive, back-and-forth protocols like live multiplayer or collaborative editors. It is tempting to assume Claude streaming uses websockets. It does not. Anthropic’s SDKs stream via Server-Sent Events (SSE), a simpler, unidirectional mechanism where the server pushes a sequence of events to the client over a single long-lived HTTP response [Source: https://platform.claude.com/docs/en/build-with-claude/streaming]. SSE fits the Claude use case precisely: the client makes one request, and the server streams incremental chunks of the answer back down that one connection — a one-way flow, not the two-way conversation a websocket is built for.

One important consequence of SSE: because the HTTP status is 200 the moment the stream opens, an error can occur after that success code. Mid-stream errors arrive as SSE error events and do not follow the standard status-code error mechanism, so streaming code must watch for error events within the stream, not just the initial response status [Source: https://platform.claude.com/docs/en/build-with-claude/streaming].

Key Takeaway: Async/await lets a Claude integration stay responsive by doing other work while requests are in flight, and concurrency (e.g., Python’s asyncio.gather) can process many requests in parallel — provided you ramp up gradually to respect rate limits. Streaming Claude responses uses Server-Sent Events (SSE), not websockets; SSE is a one-way server-to-client push, and mid-stream errors arrive as SSE events after the initial 200, not as HTTP status codes.

6.3 SDKs That Wrap REST

You could hand-build every HTTP request, header, and JSON parse yourself. In practice you use a client SDK — a software development kit, a language-specific library that packages the REST calls behind idiomatic, typed functions. Anthropic publishes official SDKs in seven languages — Python, TypeScript, C#, Go, Java, PHP, and Ruby — plus an ant command-line tool, each acting as a general-purpose Messages API client with built-in support for streaming, retries, and error handling [Source: https://platform.claude.com/docs/en/api/client-sdks].

Python and TypeScript client SDKs

The two SDKs you are most likely to be tested on are Python and TypeScript.

The Python SDK (anthropic) is installed with pip install anthropic and imported as import anthropic. It ships both a synchronous client (Anthropic) and an asynchronous one (AsyncAnthropic), both built on httpx, and it uses Pydantic models so that request parameters and response fields are typed. It also supports batch processing via client.messages.batches.create, where you submit a list of requests, each carrying a unique custom_id and a params object mirroring messages.create [Source: https://platform.claude.com/docs/en/cli-sdks-libraries/sdks/python].

client = anthropic.Anthropic()          # reads ANTHROPIC_API_KEY from env
message = client.messages.create(
    model="claude-sonnet-5",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Hello, Claude"}],
)
print(message.content[0].text)

The TypeScript SDK (@anthropic-ai/sdk) is installed with npm install @anthropic-ai/sdk and imported as import Anthropic from '@anthropic-ai/sdk'. Calling new Anthropic() automatically reads the API key from the ANTHROPIC_API_KEY environment variable. It runs on Node.js, Deno, Bun, and browsers, and ships full TypeScript type definitions for every request parameter and response field [Source: https://platform.claude.com/docs/en/api/sdks/typescript].

How SDKs abstract REST endpoints

The SDK’s central trick is to turn the raw POST /v1/messages into a single typed method call. When you invoke client.messages.create(...) (or client.Messages.New in Go, .messages().create() in Java), the SDK [Source: https://platform.claude.com/docs/en/api/client-sdks]:

1. Serializes your typed parameters into the JSON request body.
2. Adds the required headers — x-api-key, anthropic-version: 2023-06-01, content-type: application/json.
3. Sends the HTTP POST to https://api.anthropic.com/v1/messages.
4. Deserializes the JSON response into a typed Message object.

In other words, everything in Section 6.1 that you would otherwise write by hand is handled for you.

Figure 6.2: How the SDK wraps the raw REST call

flowchart LR
    A["Your code: client.messages.create(...)"] --> B["Serialize typed params to JSON body"]
    B --> C["Add headers: x-api-key,<br/>anthropic-version, content-type"]
    C --> D["POST https://api.anthropic.com/v1/messages"]
    D --> E["Deserialize JSON into typed Message object"]
    E --> F["Typed Message returned to your code"]

Streaming is likewise abstracted over SSE. In Python you write with client.messages.stream(...) as stream: and can call stream.get_final_message(); in TypeScript you can use a raw stream: true async iterable (lowest memory) or the client.messages.stream(...) helper, which wraps events in a MessageStream, accumulates the full Message, exposes typed callbacks like .on('contentBlock') and .on('message'), and offers await stream.finalMessage() [Source: https://deepwiki.com/anthropics/anthropic-sdk-typescript/3.2.3-streaming-responses]. Every response also carries a unique request-id header (for example req_018EeWyXxfu5pfWkrYcMdjWG), which the Python and TypeScript SDKs expose as a _request_id property — invaluable for debugging and support tickets [Source: https://platform.claude.com/docs/en/api/errors].

Error handling and retries in SDKs

Networks fail and servers get busy, so robust error handling is not optional. Claude’s API returns errors as JSON with a top-level error object containing a type and message, plus a request_id. The status-code taxonomy is worth memorizing [Source: https://platform.claude.com/docs/en/api/errors]:

A distinction the exam loves: 429 versus 529. A 429 means your account exceeded its own per-minute limits (or acceleration limits from a sharp usage spike) — the fix is to slow your own traffic. A 529 means Anthropic’s servers are temporarily at capacity for everyone, independent of your usage — the fix is to back off and retry, possibly with failover. Same “too busy” feeling, different cause and different remedy [Source: https://ofox.ai/blog/claude-api-error-529-overloaded-fix-2026/].

The SDKs make much of this automatic. They retry transient failures — connection errors, 429s, and 5xx server errors — using exponential backoff, twice by default, and they honor the retry-after header when the server sends one. Each client accepts a maximum-retries option to raise, lower, or disable that behavior [Source: https://platform.claude.com/docs/en/api/errors]. Exponential backoff means each retry waits longer than the last (roughly doubling), giving a struggling server room to recover instead of being hammered.

Figure 6.3: SDK retry and exponential-backoff state flow

stateDiagram-v2
    [*] --> Sending
    Sending --> Success: 200 OK
    Sending --> Transient: connection error / 429 / 5xx
    Sending --> Fatal: non-retryable (400, 401, 403, 404)
    Transient --> RetriesLeft: retries remaining?
    RetriesLeft --> Wait: yes
    RetriesLeft --> RaiseException: no (retries exhausted)
    Wait --> Sending: honor retry-after,<br/>else exponential backoff
    Success --> [*]
    Fatal --> RaiseException
    RaiseException --> [*]: typed exception

If you ever hand-roll retries, follow Anthropic’s priority order [Source: https://www.aifreeapi.com/en/posts/claude-api-429-error-fix]: (1) respect the retry-after header first — wait exactly that many seconds, since retrying early just yields more 429s; (2) fall back to rate-limit reset-header math; (3) otherwise use jittered exponential backoff starting around one second. The jitter — adding a small random delay — matters because it breaks the “thundering herd,” where many clients that failed at the same instant would otherwise all retry in lockstep and collide again.

Finally, the SDKs raise typed exceptions rather than returning raw JSON — a 404 surfaces as anthropic.NotFoundError in Python, for instance. Best practice is to catch these typed classes, most-specific first, instead of string-matching error messages [Source: https://platform.claude.com/docs/en/api/errors]:

try:
    message = client.messages.create(...)
except anthropic.RateLimitError:
    # 429 — the SDK already retried; back off further or queue the work
    ...
except anthropic.APIStatusError as e:
    # broader catch-all, checked after the specific cases
    log.error("Claude call failed", request_id=e.request_id)

For long-running work, note that the SDKs validate that non-streaming Messages requests are not expected to exceed a 10-minute timeout and set a TCP keep-alive; for genuinely long outputs, use streaming (SSE) or the Message Batches API and avoid oversized max_tokens on non-streaming calls [Source: https://platform.claude.com/docs/en/api/errors].

Key Takeaway: Client SDKs wrap the raw REST call so that one typed method — client.messages.create(...) — serializes parameters to JSON, adds the required headers, POSTs to /v1/messages, and deserializes a typed Message. They auto-retry transient failures (connection, 429, 5xx) with exponential backoff twice by default while honoring retry-after, and raise typed exceptions you should catch most-specific-first. Remember the error taxonomy, and that 429 is your limit while 529 is Anthropic’s servers overloaded for everyone.

6.4 Engineering Practices and the SDLC

An AI feature is still software, and it lives inside a software development lifecycle (SDLC) — the end-to-end process of planning, building, reviewing, testing, releasing, and maintaining an application. Three practices deserve special attention when Claude is in the mix: version control, code review, and refactoring.

Version control and code review

Version control is a system (in practice, Git) that records the full history of changes to a codebase, letting you diff, revert, branch, and collaborate. The distinctive move for Claude integrations is to treat prompts as code. Prompt templates are development artifacts, so they belong in your Git repository — typically in a dedicated directory such as prompts/ or prompt_templates/, organized by feature, with each prompt in its own file [Source: https://apxml.com/courses/prompt-engineering-llm-application-development/chapter-3-prompt-design-iteration-evaluation/version-control-for-prompts]. This makes a prompt change reviewable, diffable, revertible, and testable exactly like source code: a prompt regression can be traced and rolled back the same way a code regression is.

Concrete Git habits that pay off here:

• Write meaningful commit messages recording what changed and why; your history becomes the documentation of how a prompt evolved.
• Use branches to explore alternative prompt or model variations in parallel, then merge the winners back to the main line.
• Use Git as a state log with checkpoints — Anthropic notes that Claude’s recent models perform especially well at using Git to track state across multiple sessions, so you can experiment and cleanly revert when an AI-generated change does not pan out [Source: https://code.claude.com/docs/en/best-practices].

A non-negotiable: never commit secrets. Keep ANTHROPIC_API_KEY and other credentials in environment variables, and add .env and credential files to .gitignore. The SDKs read the key from the environment by default, which is exactly what makes this pattern painless [Source: https://code.claude.com/docs/en/best-practices].

Code review — having another person (or an automated tool) inspect changes before they merge — should cover the LLM-specific concerns alongside the usual ones: correct handling of the full status-code taxonomy (401/403/429/500/529), retry and backoff configuration, timeout handling, no hardcoded secrets, prompt-injection and untrusted-input handling, and cost/token controls such as max_tokens and batching [Source: https://platform.claude.com/docs/en/api/errors]. Claude can also participate in review: running the SDK or CLI in non-interactive mode lets you wire it into CI pipelines and pre-commit hooks — for example, feeding a diff to a structured security-review prompt to flag exploitable vulnerabilities before merge [Source: https://code.claude.com/docs/en/best-practices].

Small- and large-scale refactoring

Refactoring is improving the internal structure of code without changing its external behavior. For Claude integrations the guiding principle is to isolate the integration behind a thin, well-typed abstraction layer, so that model IDs, prompts, retry policy, and timeouts are centralized and swappable rather than scattered across the codebase [Source: https://apxml.com/courses/prompt-engineering-llm-application-development/chapter-3-prompt-design-iteration-evaluation/version-control-for-prompts].

The analogy is a building’s electrical panel: rather than splicing every appliance directly into the wiring in the wall, you route everything through one panel with labeled breakers. When you need to change a fuse — swap claude-sonnet-5 for a newer model, tighten a timeout, or bump the retry count — you touch one place, not fifty.

• Small-scale refactoring is local and incremental: rename a prompt file, extract a repeated request into a helper, tighten a type. Because the SDKs deliver typed request/response objects and typed exceptions, these refactors can lean on the type system to catch mistakes at compile time.
• Large-scale refactoring restructures whole modules: introducing that central “Claude client” wrapper, moving all inline prompt strings into versioned files under prompts/, or replacing ad-hoc try/except blocks with a consistent typed-exception strategy.

Keeping prompts externalized in versioned files delivers a clean separation: you can refactor prompt wording without touching application logic, and refactor application logic without disturbing prompts. Combined with tests and Git checkpoints, this lets you evolve code and prompts safely and independently [Source: https://apxml.com/courses/prompt-engineering-llm-application-development/chapter-3-prompt-design-iteration-evaluation/version-control-for-prompts].

SDLC integration for AI features

Pulling it together, an AI feature threads through the SDLC like any other, with a few Claude-specific reinforcements at each stage:

Figure 6.4: SDLC integration pipeline for a Claude feature

flowchart TD
    A["Plan: choose model, budget max_tokens,<br/>set rate-limit expectations"] --> B["Build: official SDK, prompts in versioned files,<br/>API key in environment"]
    B --> C["Review: human + Claude-in-CI check<br/>error handling, retries, secrets, injection, cost"]
    C --> D["Test: exercise typed layer,<br/>Git checkpoints to revert experiments"]
    D --> E["Maintain: refactor behind thin client layer,<br/>swap models/prompts via central config"]
    E -.-> A

Because the integration is stateless REST behind a typed wrapper, and because prompts are versioned artifacts under review, an AI feature stays as traceable and revertible as the rest of your application — which is precisely the goal.

Key Takeaway: Treat prompts as code: version them in Git (e.g., a prompts/ directory) with meaningful commits and branches, and never commit ANTHROPIC_API_KEY — keep it in the environment and .gitignore the .env. Review Claude code for the full error taxonomy, retries, secrets, prompt injection, and cost, and consider running Claude in non-interactive mode inside CI. Refactor to isolate the integration behind a thin, typed layer that centralizes model IDs, prompts, retry policy, and timeouts, so both code and prompts can evolve safely across the SDLC.

Chapter Summary

Integrating Claude is applied software engineering built on four foundations. First, the REST API and JSON: you POST JSON to https://api.anthropic.com/v1/messages, sending a model, max_tokens, and a messages array, with authentication and versioning carried in the x-api-key, anthropic-version, and content-type headers — and the API key always sourced from the environment as a secret. Second, asynchronous programming: async/await keeps your app responsive, concurrency (e.g., asyncio.gather) processes many requests in parallel provided you ramp up to respect rate limits, and streaming uses Server-Sent Events (SSE), not websockets — a one-way server-to-client push whose mid-stream errors arrive as events after the initial 200. Third, SDKs that wrap REST: the Python (anthropic) and TypeScript (@anthropic-ai/sdk) clients turn the raw POST into a single typed messages.create call, auto-retry transient failures with exponential backoff (twice by default, honoring retry-after), and raise typed exceptions you catch most-specific-first — with 429 signaling your rate limit and 529 signaling Anthropic’s servers overloaded for everyone. Fourth, engineering practices and the SDLC: version prompts as code in Git, never commit secrets, review for the LLM-specific concerns (optionally using Claude-in-CI), and refactor behind a thin typed layer that centralizes model IDs, prompts, retry policy, and timeouts. Together these practices make Claude a well-behaved, maintainable component of any application.

Key Terms

Chapter 7: Designing Claude Applications

Building a real product on Claude is less about writing one clever prompt and more about design: translating a business goal into concrete requirements, deciding where instructions live, drawing firm lines between trusted and untrusted content, and keeping conversation state clean over time. This chapter walks through that design discipline end to end. Think of it as the difference between cooking a single meal and running a restaurant kitchen — the recipe matters, but so do the supply chain, the food-safety rules, and the cleaning schedule between services.

By the end of this chapter you will be able to translate business requirements into functional and infrastructure requirements, design schemas and content boundaries with proper session hygiene, and explain why the same instruction can be interpreted differently depending on which Claude surface reads it.

7.1 Requirements and the Systems Life Cycle

Every well-designed Claude application starts before a single line of prompt is written, with a clear statement of what the system must do and how it must run.

Functional vs. Infrastructure Requirements

A functional requirement describes what the system must do — the observable behavior a stakeholder cares about. For a Claude application these include the task itself, the quality bar, the tone of voice, the structured-output contract the app must return, the set of tools or actions the agent is permitted to take, and the safety and compliance policies it must honor. If you can phrase it as “the system shall…,” it is functional.

An infrastructure requirement describes how the system runs — the operational scaffolding a user never sees directly but that determines whether the product survives contact with production. These include authentication, rate limits, connection pooling, monitoring, cost budgets, and data-retention rules [Source: https://platform.claude.com/docs/en/build-with-claude/context-windows].

An analogy: functional requirements are the menu (what the restaurant serves and how good it must taste); infrastructure requirements are the kitchen and plumbing (whether the ovens can keep up at the dinner rush and whether the health inspector will pass you). A great menu backed by a broken kitchen fails just as surely as a working kitchen with a menu nobody wants.

On the functional side, Anthropic’s ecosystem guidance frames the first design decision as choosing the right building block: start with the API SDK for simple request/response integrations, graduate to the Agent SDK when you need autonomous, multi-step capabilities, and extend with MCP (Model Context Protocol) servers when the application needs to reach external tools and data. On the infrastructure side, the Claude API enforces tier-based rate limits measured in requests-per-minute and tokens-per-minute. Production applications should therefore request limit increases in advance, implement client-side rate limiting and retry logic with exponential backoff (progressively longer waits between retries), and budget token usage at the architecture stage rather than discovering the bill after launch [Source: https://platform.claude.com/docs/en/build-with-claude/context-windows].

Solution Architecture from Business Needs

Solution architecture is the bridge that turns those requirements into a concrete design — the choice of components, control flow, and integration patterns that satisfy the business need. Anthropic’s canonical reference here is the “Building Effective Agents” guidance, which draws a sharp line between two shapes of system [Source: https://www.anthropic.com/research/building-effective-agents].

• Workflows use a predefined control flow: the LLM is called at fixed, predictable steps that you designed in advance. Workflows are testable and predictable, and they are the right choice when the task structure is known ahead of time.
• Agents let the LLM dynamically decide what to do next, choosing its own path through tools and steps. Agents are flexible and suited to novel, open-ended, or adaptive tasks.

The core recommendation is deceptively simple: start with workflows and graduate to agents only when the task genuinely requires dynamic routing or open-ended exploration. Complexity is a cost, not a feature.

Figure 7.2: Choosing between workflows and agents from requirements

flowchart TD
    A["Business need + requirements"] --> B{"Is the task structure\nknown in advance?"}
    B -->|Yes| C["Workflow: predefined control flow\n(testable, predictable)"]
    C --> D["Compose five patterns:\nprompt chaining, routing,\nparallelization, orchestrator-worker,\nevaluator-optimizer"]
    B -->|"No: novel / open-ended / adaptive"| E["Agent: LLM dynamically\ndecides next step"]
    D --> F["Start simple, build modularly,\nmake reasoning visible"]
    E --> F

Within the workflow family, the guidance names five composable patterns you combine like building blocks rather than follow as rigid prescriptions: prompt chaining (output of one step feeds the next), routing (classify the input, then send it down the right branch), parallelization (run subtasks concurrently and aggregate), orchestrator–worker (a lead model delegates to specialized workers), and evaluator–optimizer (one model generates, another critiques and refines) [Source: https://www.anthropic.com/research/building-effective-agents].

Overarching design principles: keep the architecture simple, start small and build modularly, and make the reasoning process visible so users can see how the agent plans and decides. Production guidance layers on three more: progressive disclosure (let the agent discover information through tool calls as it needs it, rather than dumping everything into context up front — with the exception of always-relevant data like the user’s own instructions), per-tenant system prompts with clear isolation in multi-tenant deployments, and connection pooling plus request queuing to absorb high-concurrency traffic spikes [Source: https://resources.anthropic.com/building-effective-ai-agents].

(A diagram would help here: a decision tree that starts at “Is the task structure known in advance?” — Yes leads to Workflows and the five patterns; No leads to Agents — visually reinforcing the “start with workflows” default.)

The Systems Life Cycle: Develop → Implement → Operate → Maintain

The systems life cycle is the end-to-end sequence a Claude application moves through, from first design to ongoing upkeep. Framing your project against these four phases keeps design decisions in their proper place.

• Develop. Choose the building block (API SDK → Agent SDK → MCP), define the structured-output contract and tool definitions, engineer system prompts (including an untrusted-content policy, covered in 7.3), and design how conversation state will be handled. Budget tokens and rate limits here, at design time — not after launch.
• Implement. Wire up authentication through your identity provider (IdP), add input sanitization and prompt-injection defenses, and run adversarial testing (red-teaming) before go-live. Establish CI/CD with security scanning.
• Operate. Deploy behind rate-limit handling, retries with backoff, and connection pooling. Crucially, LLM monitoring needs both traditional signals (latency, error rates, uptime) and AI-specific quality signals; without comprehensive observability you cannot detect quality degradation, safety failures, or cost anomalies until they surface as user-visible incidents [Source: https://platform.claude.com/docs/en/build-with-claude/context-windows].
• Maintain. Continuously monitor for prompt injection and quality drift, iterate on prompts and filters, track cost and ROI, and manage model migration deliberately — because each model ID is a fixed snapshot, an upgrade is a conscious event, not an automatic drift. Keep plugin and marketplace hygiene tidy, and rotate and scope secrets.

Figure 7.1: The systems life cycle — Develop → Implement → Operate → Maintain

stateDiagram-v2
    direction LR
    [*] --> Develop
    Develop --> Implement
    Implement --> Operate
    Operate --> Maintain
    Maintain --> Develop: iterate on prompts,\nfilters, model migration
    Develop: Develop\nBuilding block, output contract,\nprompts, state design, token budget
    Implement: Implement\nAuth via IdP, input sanitization,\nred-teaming, CI/CD security
    Operate: Operate\nRate limits, retries+backoff,\npooling, dual observability
    Maintain: Maintain\nMonitor injection & drift,\ncost/ROI, secret rotation

Key Takeaway: Separate what the system must do (functional requirements) from how it must run (infrastructure requirements), then let solution architecture bridge the two — defaulting to simple workflows and escalating to agents only when dynamic behavior is truly needed. The develop → implement → operate → maintain life cycle ensures token budgets, injection defenses, and observability are designed in from the start rather than bolted on after an incident.

7.2 Instruction Interpretation Across Interfaces

The single most surprising fact for new Claude developers is that the same instruction behaves differently depending on which surface reads it. Understanding why is foundational.

What a System Prompt Is, and Where It Lives

A system prompt is a block of natural-language instructions delivered to Claude before any conversation begins. It sits at the top of the context window, above everything the user types, and Claude reads it first. It is written in plain language, not code. But where those instructions come from — and whether they are applied at all — differs sharply across Anthropic’s surfaces, and that is where designers get tripped up [Source: https://platform.claude.com/docs/en/release-notes/system-prompts].

An interface here means a distinct Claude surface — claude.ai and the mobile apps, the Claude API, the Agent SDK, Claude Code, and Claude Cowork/Desktop — each with its own convention for where instructions live and how they are read.

The Five Surfaces

claude.ai (web) and mobile apps. When a person chats on claude.ai, Anthropic automatically loads a full, Anthropic-authored system prompt at the start of every conversation. This prompt sets the current date, encourages formatting conventions (for example, Markdown for code), and defines Claude’s default behaviors. Anthropic periodically updates this prompt and publishes the text in its release notes. On top of that base prompt, claude.ai supports organization instructions and individual instructions — preferences set at the account or organization level. Chat interfaces like claude.ai can also manage the context window on a rolling “first in, first out” basis, dropping the oldest turns as the conversation grows [Source: https://platform.claude.com/docs/en/release-notes/system-prompts].

The Claude API. A developer using the Messages API gets no default system prompt at all. The published claude.ai system-prompt updates do not apply, and neither do the organization or individual instructions — those live only inside claude.ai. The API’s system parameter operates independently because it is set per application, not per user account. If you build on the API, your system prompt is the primary control surface: you are responsible for setting the current date, enabling web search separately, and supplying any behavioral defaults. The API is also stateless, so you must resend the full instruction set (and history) on every request. Starting with the Claude 4.6 generation, each model ID is a single fixed snapshot with one associated system-prompt entry [Source: https://platform.claude.com/docs/en/release-notes/system-prompts].

Claude Code. Claude Code ships with its own extensive built-in system prompt geared toward agentic coding — roughly 600 tokens across about twelve separate instructions, each reading like a compressed senior-engineer code-review rule. It defines which actions require user permission and which do not, instructing Claude to weigh the “reversibility and blast radius” of every action before acting. When building on the Agent SDK, you opt into this full preset with systemPrompt: { type: "preset", preset: "claude_code" } (TypeScript) or system_prompt={"type": "preset", "preset": "claude_code"} (Python). Note a common gotcha: the claude_code preset does not automatically load CLAUDE.md project files. You must explicitly set settingSources: ['project'] (TS) or setting_sources=["project"] (Python) for those instructions to be included [Source: https://code.claude.com/docs/en/agent-sdk/modifying-system-prompts].

Claude Cowork / Desktop. Claude Cowork is an agentic knowledge-work tool for non-developers, available as a desktop app; Claude Code is likewise available via CLI, desktop, and mobile. These products inherit Claude’s base capabilities, and their behavior is determined by the underlying model plus any custom instructions provided.

Consistency Across Surfaces

The recurring, exam-relevant theme: safety guardrails are applied uniformly at the model level across all surfaces, but the system-prompt layer differs. It is automatic and Anthropic-managed on claude.ai, developer-owned and per-call on the API, and a rich built-in preset in Claude Code [Source: https://platform.claude.com/docs/en/release-notes/system-prompts].

The practical consequence for designers: never assume a prototype that “just works” in claude.ai will behave the same way when you port it to the API. In claude.ai, Claude already knows the date, already prefers Markdown, and already obeys your org instructions — all invisibly. On the API, none of that exists until you write it. A useful mental model is that claude.ai is a furnished apartment (the lights, appliances, and house rules come pre-installed) while the API is a bare shell where you supply every fixture yourself. Building for consistency means writing your system prompt explicitly enough that the surface’s defaults become irrelevant.

Key Takeaway: The same instruction is read differently because the system-prompt layer differs by surface — claude.ai auto-loads an Anthropic-managed prompt plus org/individual instructions, while the stateless API supplies nothing until the developer’s system parameter does. Safety guardrails, however, are uniform at the model level everywhere, so behavior differences come from instructions, not from the model relaxing its rules.

7.3 Content Boundaries and Schema Design

Once instructions are in place, the next design task is to draw firm boundaries: which content in the context window is trusted and which must be treated as data to be reported, never commands to be obeyed.

Separating Trusted Instructions from Untrusted Content

Content boundaries are the design decisions that keep trusted instructions (your system prompt, the user’s genuine request) separate from untrusted third-party content (web pages, emails, documents, tool results). Anthropic’s guidance splits the threat into two models: jailbreaks / direct prompt injection, where the user is the adversary, and indirect prompt injection, where the user is trusted but Claude processes adversarial content that arrived from somewhere else [Source: https://platform.claude.com/docs/en/test-and-evaluate/strengthen-guardrails/mitigate-jailbreaks].

Claude is inherently resilient — Anthropic hardens it through reinforcement learning that exposes it to injections during training and rewards correct refusals — but application-level design is still required. The documented techniques:

• Put untrusted content only in tool_result blocks — never in system prompts or plain user text blocks. Claude is trained to treat instructions inside tool results with appropriate skepticism.
• Label the content’s nature and source in the tool description or result structure (for example, “body of an inbound email from an unknown sender” or “OCR text from a user-uploaded image”) so Claude can calibrate trust.
• State an untrusted-content policy in the system prompt — for instance, a <untrusted_content_policy> block telling Claude that tool, file, and search content is data to report, not commands to follow, and must never override the system prompt or the user’s original request.
• JSON-encode untrusted content: wrap third-party strings in a JSON object so escaping provides unambiguous delimiters, preventing an attacker from “breaking out” of a quote or tag into an instruction context.
• Do not put your own instructions in tool results — they may be ignored or flagged as injection. Send your instructions in a user turn following the tool_result; on Claude Opus 4.8+ you can also use mid-conversation system messages.
• Apply least privilege: scope permissions narrowly, sandbox tools, and withhold secrets Claude does not need, so a successful injection does minimal damage.
• Screen inputs and tool outputs with a lightweight model (Claude Haiku 4.5) plus structured outputs to constrain the verdict to a boolean such as is_harmful or injection_suspected. Validate inputs for known injection patterns, and throttle or ban repeat offenders.
• Red-team before deploying with deliberately malicious documents, emails, and tool outputs, and continuously monitor outputs for signs of successful injection [Source: https://platform.claude.com/docs/en/test-and-evaluate/strengthen-guardrails/mitigate-jailbreaks].

Think of it like a newsroom. The editor’s standards (your system prompt) are trusted. A tip that arrives in the mail (untrusted tool result) might contain useful facts — but a professional newsroom reports on the tip; it does not let an anonymous letter rewrite the editorial policy. Putting untrusted content in tool_result blocks and labeling its source is how you keep the anonymous letter in the “tips” folder rather than on the masthead.

Figure 7.3: The trusted / untrusted content boundary in the context window

flowchart TD
    subgraph Trusted["Trusted zone — instructions to obey"]
        SP["System prompt\n(incl. untrusted-content policy)"]
        UR["User's genuine request\n(user text block)"]
    end
    subgraph Untrusted["Untrusted zone — data to report, never obey"]
        TR["tool_result blocks:\nweb pages, emails, docs,\nOCR, search results"]
    end
    TR -->|"labeled + JSON-encoded"| CLAUDE["Claude reads with skepticism"]
    SP --> CLAUDE
    UR --> CLAUDE
    CLAUDE --> OUT["Reports on untrusted content;\nnever lets it override\nsystem prompt or user request"]

Designing Input and Output Schemas

Schema design is the practice of defining explicit, machine-checkable structures for what goes into and comes out of Claude. On the output side, structured outputs constrain Claude’s responses to follow a specific schema, guaranteeing valid, parseable output. The feature has two complementary halves [Source: https://platform.claude.com/docs/en/build-with-claude/structured-outputs]:

• JSON outputs via output_config.format control what Claude says — the shape of the response.
• Strict tool use via strict: true on a tool definition validates how Claude calls your functions, guaranteeing that tool names and arguments match the declared input_schema.

These can be used independently or together. The mechanism underneath is constrained decoding: the JSON schema is compiled into a grammar that actively restricts token generation during inference. This is fundamentally stronger than merely prompting “please return valid JSON,” because the model is structurally prevented from emitting a token that would break the schema.

Worked example — a customer-support triage app. Suppose the business requirement is “classify each incoming ticket and route it.” The output schema might be:

{
  "type": "object",
  "additionalProperties": false,
  "properties": {
    "category": { "type": "string", "enum": ["billing", "technical", "account", "other"] },
    "priority": { "type": "string", "enum": ["low", "medium", "high", "urgent"] },
    "summary": { "type": "string", "description": "One-sentence summary of the customer's issue" },
    "needs_human": { "type": "boolean", "description": "True if the ticket requires human escalation" }
  },
  "required": ["category", "priority", "summary", "needs_human"]
}

With additionalProperties: false and constrained decoding, the app can call JSON.parse on the result with confidence and route on category without defensive retry logic. That last point matters commercially: schema-design best practices from the docs include avoiding recursive schemas (flatten hierarchies or cap nesting depth), using additionalProperties: false for strict validation, writing descriptive field names and descriptions (Claude interprets them effectively), setting generous max_tokens buffers, and monitoring stop_reason to catch refusals or truncations early. Expect a small system-prompt overhead (roughly 50–200 tokens depending on schema complexity, about 2–3% cost at scale) — an overhead more than offset by eliminating retry code. Structured outputs are a Claude API beta available on the Sonnet 4.5 / Opus 4.1 and 4.5 Sonnet generations noted in production [Source: https://platform.claude.com/docs/en/build-with-claude/structured-outputs].

Structured Data Contracts

A schema is more than validation plumbing — it is a contract between Claude and the rest of your system. Downstream services, dashboards, and databases can all depend on the shape being exactly as promised. This is the same discipline as a typed API boundary in traditional software: once the contract is fixed, each side can evolve independently as long as the contract holds. Constrained decoding is what makes the contract enforceable at generation time rather than merely hoped for.

Key Takeaway: Keep trusted instructions and untrusted data on opposite sides of a firm boundary — untrusted third-party content belongs only in labeled, JSON-encoded tool_result blocks, governed by an explicit untrusted-content policy in the system prompt. On the output side, structured outputs (JSON outputs plus strict tool use) turn schemas into contracts enforced by constrained decoding, giving you parseable results without retry logic.

7.4 Session Hygiene and Plugins

The final design dimension is time: how state accumulates across a conversation, how it is cleaned up, and how you avoid one session’s context leaking into another.

Managing Conversation State and Session Hygiene

Recall from 7.2 that the Messages API is stateless: Claude stores no conversation history, and each request is fully independent. To sustain a multi-turn conversation, the developer must resend the entire message array each turn. Messages use two roles — user and assistant — which must alternate, starting with a user message. This design makes the API horizontally scalable, predictable, and auditable, but it places state management squarely on the developer [Source: https://platform.claude.com/docs/en/build-with-claude/context-windows].

Session hygiene is the discipline of keeping that accumulating state clean, relevant, and bounded. It matters because everything counts toward the context window: the system prompt, every message (including tool results, images, and documents), the tool definitions, and Claude’s own generated output (including extended thinking). Context windows now reach up to 1M tokens on models such as Opus 4.6/4.7/4.8, Sonnet 5, Sonnet 4.6, Fable 5, and Mythos 5, while others like Sonnet 4.5 remain at 200k. But capacity is not the whole story: as token count grows, accuracy and recall degrade — a phenomenon called context rot — so curating what sits in context matters as much as raw capacity [Source: https://platform.claude.com/docs/en/build-with-claude/context-windows].

The session-hygiene toolkit:

• Server-side compaction (beta) automatically summarizes earlier conversation so it can continue past the window limit.
• Context editing offers tool-result clearing and thinking-block clearing to reclaim space from content no longer needed.
• Context awareness (Sonnet 5/4.6/4.5, Haiku 4.5) injects the model’s remaining token budget so it can pace long tasks; newer models use explicit task budgets (beta).
• If input alone exceeds the window, the API returns a 400 invalid_request_error; on 4.5+ models an overflow during generation stops with stop_reason: "model_context_window_exceeded". Use the token counting API to estimate before sending. For multi-session agents, design state artifacts for fast context recovery — for example, the memory-tool multi-session pattern [Source: https://platform.claude.com/docs/en/build-with-claude/context-windows].

An analogy: the context window is a whiteboard, not a filing cabinet. It has finite space, and once it fills, the oldest or least relevant notes must be erased (compaction, context editing) or you run out of room mid-thought. Session hygiene is the habit of wiping stale scribbles so the important lines stay legible — and context rot is what happens when the board gets so crowded that even the model struggles to read its own writing.

Figure 7.4: Session hygiene — curating a bounded context window

flowchart TD
    A["Context window (bounded budget)\nsystem prompt + all messages +\ntool defs + generated output"] --> B{"Approaching\ntoken limit?"}
    B -->|No| C["Continue conversation"]
    B -->|Yes| D["Apply hygiene toolkit"]
    D --> E["Server-side compaction:\nsummarize earlier turns"]
    D --> F["Context editing:\nclear tool results / thinking"]
    D --> G["Token counting API:\nestimate before sending"]
    E --> C
    F --> C
    G --> C
    B -->|"Overflow anyway"| H["400 invalid_request_error\nor stop_reason:\nmodel_context_window_exceeded"]
    C --> I["Guards against context rot\n& state leakage"]

Plugin Management and Dependencies

In Claude Code, plugins extend the tool with skills, agents, hooks, MCP servers, and LSP servers. Marketplaces are catalogs of plugins, and using one is a two-step process: add the marketplace (which registers the catalog but installs nothing), then install individual plugins. The official claude-plugins-official marketplace is available automatically at startup; a community marketplace (anthropics/claude-plugins-community, pinned per commit SHA and safety-screened) and a demo marketplace are added manually [Source: https://code.claude.com/docs/en/discover-plugins].

Run /plugin to open a four-tab manager — Discover, Installed, Marketplaces, Errors. Plugin details show a context-cost estimate (tokens added per turn — a direct tie back to session hygiene), a last-updated date, and a “Will install” inventory of components. Installation scopes are user (default, all projects), project (all repo collaborators, via .claude/settings.json), local (self, this repo only), and admin-set managed. Commands include /plugin install <name>@<marketplace>, /plugin disable|enable|uninstall, /plugin marketplace add|update|remove|list, and /reload-plugins to apply changes without a restart. Auto-update is on by default for official marketplaces and off for third-party or local ones [Source: https://code.claude.com/docs/en/discover-plugins].

Security is central. Plugins can execute arbitrary code with the user’s privileges, so only install from trusted sources. Organizations can restrict allowed marketplaces via managed settings — strictKnownMarketplaces, extraKnownMarketplaces, and pluginSuggestionMarketplaces. Treating plugins as dependencies — each with a trust level, a context cost, and an update cadence — is the mature posture: an unused plugin that still loads at startup is both an attack surface and a silent tax on your context budget.

Avoiding State Leakage Between Sessions

Because the API is stateless, the developer owns the boundaries between sessions — and getting that wrong is how one user’s data ends up in another user’s conversation. Two design practices prevent this. First, per-tenant system prompts with clear isolation (from 7.1) ensure that tenant A’s instructions and history never share a context window with tenant B’s. Second, session hygiene tooling — clearing tool results, compacting old turns, and starting fresh message arrays for new sessions — ensures that stale state from a completed task does not bleed into the next one. The statelessness of the API is a gift here: because nothing persists automatically, leakage only happens if you carry it over. Build each session’s message array deliberately, scope secrets to the minimum, and never reuse a context window across tenants, and state leakage becomes a design error you can simply avoid.

Key Takeaway: Since the Messages API is stateless, the developer both resends full history each turn and owns the boundaries between sessions — making session hygiene (compaction, context editing, token counting) essential to fight context rot and prevent state leakage. In Claude Code, treat plugins as dependencies with a trust level and a context cost: install only from trusted marketplaces, disable what you do not use, and keep per-tenant isolation firm.

Chapter Summary

Designing a Claude application is a discipline of translation and boundaries. You begin by translating a business goal into functional requirements (what the system does — task, tone, output contract, permitted actions, safety policy) and infrastructure requirements (how it runs — auth, rate limits, pooling, monitoring, cost). Solution architecture bridges the two, and Anthropic’s “Building Effective Agents” guidance sets the default: prefer predictable workflows built from five composable patterns, and graduate to dynamic agents only when the task truly demands it. The systems life cycle — develop, implement, operate, maintain — keeps token budgets, injection defenses, and observability in their proper phases.

A recurring theme is that instructions are interpreted differently across interfaces. claude.ai auto-loads an Anthropic-managed system prompt plus organization and individual instructions; the stateless Claude API supplies nothing by default, making the developer’s system parameter the primary control surface; Claude Code ships a rich built-in preset. Safety guardrails, though, are uniform at the model level everywhere.

Robust applications enforce content boundaries — untrusted third-party content belongs only in labeled, JSON-encoded tool_result blocks under an explicit untrusted-content policy — and use schema design with structured outputs and constrained decoding to turn output shapes into enforceable contracts. Finally, because the API is stateless, session hygiene (compaction, context editing, token counting) fights context rot and prevents state leakage, while disciplined plugin management treats each plugin as a trust-scoped, context-costed dependency.

Key Terms

Chapter 8: Configuration Management and Claude Code

Every capable agent needs a memory and a rulebook. Left with nothing but a fresh context window, Claude Code begins each session as a talented but amnesiac colleague — brilliant in the moment, but forgetting your coding standards, your build commands, and your architectural decisions the instant the conversation ends. Configuration management is how you cure that amnesia. Through a small family of plain-text files — CLAUDE.md for instructions, settings.json for behavior, plus rules, skills, commands, and agents — you turn a general-purpose assistant into one that knows your project, follows your conventions, and behaves consistently across your whole team.

This chapter maps that configuration system end to end. You will learn the CLAUDE.md hierarchy and how files layer from organization-wide policy down to your personal scratch notes; how settings.json merges across scopes with a strict precedence order; how to pin model versions and version your prompts for reproducibility; the core components that make Claude Code extensible; the session and headless modes that let it run everywhere from your terminal to your CI pipeline; and finally, how to manage all of this at scale across a team using version control and a plugin marketplace. Think of it as learning the wiring diagram of a house before you start rearranging the furniture.

Configuration Files

Configuration in Claude Code rests on two file-based pillars that reload at the start of every session: CLAUDE.md files, which carry human-authored instructions, and settings.json files, which carry machine-readable behavior settings. Understanding how each layers across scopes is the foundation for everything else in this chapter.

CLAUDE.md files and the memory hierarchy

CLAUDE.md is a plain-text markdown file that gives Claude persistent instructions — coding standards, workflows, architecture notes, project quirks. Claude reads it in full at the start of every session and treats it as context, not enforced configuration. This distinction matters: CLAUDE.md is delivered as a user message after the system prompt, so it strongly influences behavior but does not guarantee it. To hard-block an action, you need a PreToolUse hook (covered under settings.json), not a line in CLAUDE.md [Source: https://code.claude.com/docs/en/memory].

Think of CLAUDE.md as the employee handbook you hand a new hire on day one. It shapes how they work and what they prioritize, but it is guidance a thoughtful person follows — not a locked door. The locked door is the hook.

Claude Code actually runs two complementary memory systems, both loaded at conversation start:

• CLAUDE.md files — written by you. Instructions and rules, loaded in full every session.
• Auto memory (also called agent memory) — written by Claude itself. Learnings and patterns Claude discovers, such as build commands or debugging insights. The first 200 lines or 25 KB of its MEMORY.md index load per session.

CLAUDE.md files live at four scopes. Crucially, they do not override one another — every file Claude discovers is concatenated into context, ordered from the broadest scope down to the most specific. Because more-specific instructions are read last, they carry more weight in Claude’s attention.

Table 8.1 — The CLAUDE.md Scope Hierarchy (broadest to most specific)

[Source: https://code.claude.com/docs/en/memory]

How the files load. When a session starts, Claude walks up the directory tree from your current working directory, loading every CLAUDE.md and CLAUDE.local.md it finds. Discovered files are concatenated from filesystem root down to the working directory; within a single directory, CLAUDE.local.md is appended after CLAUDE.md. Files in subdirectories below your working directory load on demand — only when Claude actually reads files there. The managed-policy CLAUDE.md loads before all others and cannot be excluded by any individual setting [Source: https://code.claude.com/docs/en/memory].

Figure 8.1: CLAUDE.md concatenation order from broadest to most specific scope

graph TD
    A["Managed policy CLAUDE.md: org-wide, cannot be excluded"] --> B["User instructions: ~/.claude/CLAUDE.md"]
    B --> C["Project instructions: ./CLAUDE.md or ./.claude/CLAUDE.md"]
    C --> D["Local instructions: ./CLAUDE.local.md (gitignored)"]
    D --> E["Concatenated context: more-specific files read last, carry more weight"]

Imports. A CLAUDE.md file can pull in other files with @path/to/import syntax (relative paths resolve against the importing file). Imports expand into context at launch and can recurse up to a maximum depth of four hops. A backtick-wrapped path such as `@README` is treated as literal text, not an import. The first time Claude encounters external imports, it shows an approval dialog. This import mechanism is also how you reuse an existing AGENTS.md: Claude Code reads CLAUDE.md, not AGENTS.md, so you create a CLAUDE.md that does @AGENTS.md (or symlink it) [Source: https://code.claude.com/docs/en/memory].

Rules. For large projects, you can split instructions into topic files under .claude/rules/ (project) or ~/.claude/rules/ (user). By default these load at launch with the same priority as .claude/CLAUDE.md. Add a paths: YAML frontmatter field with glob patterns and a rule becomes path-scoped — it loads only when Claude touches matching files [Source: https://code.claude.com/docs/en/memory].

Worked example — a starter CLAUDE.md. Suppose you run /init on a Python project. Claude analyzes the codebase and generates something like this:

# Project: payments-service

## Commands
- Test: `pytest -q`
- Lint: `ruff check .`
- Run locally: `uvicorn app.main:app --reload`

## Conventions
- Use 2-space indentation in YAML, 4-space in Python.
- All monetary values are integer cents, never floats.
- New endpoints require a corresponding test in tests/.

@docs/architecture.md

Notice the specificity — “integer cents, never floats” is enforceable guidance; “format code nicely” would not be. The @docs/architecture.md import folds the architecture doc into context without you re-pasting it. Anthropic’s guidance is to keep each CLAUDE.md under 200 lines: longer files consume more context and, paradoxically, reduce adherence [Source: https://code.claude.com/docs/en/memory].

Two commands manage all of this. /init generates a starter CLAUDE.md by analyzing your codebase. /memory lists and edits every loaded CLAUDE.md, CLAUDE.local.md, and rules file, toggles auto memory, and opens the auto-memory folder. Simply asking Claude to “remember” something saves it to auto memory. The project-root CLAUDE.md is re-read from disk after /compact, so its instructions stay stable; nested subdirectory files do not auto-reload after compaction [Source: https://code.claude.com/docs/en/memory].

settings.json configuration

Where CLAUDE.md carries instructions, settings.json carries behavior — which model to use, which tools are allowed, what hooks fire, what environment variables get injected. Claude Code reads JSON settings from four hierarchical scopes and merges them, with more-specific and higher-authority scopes winning field by field.

Table 8.2 — settings.json Precedence (lowest to highest authority)

[Source: https://vineetagarwal-code-claude-code.mintlify.app/configuration/settings]

Read the table as a chain of command. A plugin proposes a default; your user settings can override it; the project’s committed settings override those for everyone on the team; your local file overrides again for just you; and managed enterprise policy sits on top of all of it, immovable. Settings merge from lowest to highest priority, so a key set only in your user file still applies unless a higher scope explicitly changes it.

Figure 8.2: settings.json merge precedence from lowest to highest authority

graph TD
    A["1. Plugin defaults: baseline from installed plugins"] --> B["2. User global: ~/.claude/settings.json"]
    B --> C["3. Project shared: .claude/settings.json (version-controlled)"]
    C --> D["4. Local personal: .claude/settings.local.json (gitignored)"]
    D --> E["5. Managed enterprise: platform paths, cannot be overridden"]
    E --> F["Merged settings: higher scope wins field by field"]

The most important keys include:

• model — which Claude model the session uses.
• permissions — tool access, expressed through allow, deny, ask, and defaultMode. This is where you grant or block specific commands.
• hooks — custom shell commands that run before or after tool use. A PreToolUse hook is the mechanism for enforcement (blocking an action), which CLAUDE.md alone cannot do.
• env — environment variables injected into every session.
• cleanupPeriodDays — transcript retention (default 30).
• availableModels — an enterprise model allowlist (managed scope only).
• autoMemoryEnabled / autoMemoryDirectory — control agent memory (user/local only).
• claudeMd — a managed-only key that embeds managed CLAUDE.md content directly in the policy file.
• claudeMdExcludes — skip specific CLAUDE.md files by path or glob (arrays merge across layers; the managed CLAUDE.md can never be excluded) [Source: https://claudefa.st/blog/guide/settings-reference].

The division of labor is clean: managed settings.json handles technical enforcement (blocking tools via permissions.deny, forcing sandbox.enabled, setting env, pinning a login method), while managed CLAUDE.md handles behavioral guidance. Run /config to open the settings UI; edits reload automatically [Source: https://vineetagarwal-code-claude-code.mintlify.app/configuration/settings].

Worked example — a project settings.json:

{
  "model": "claude-opus-4-8",
  "permissions": {
    "allow": ["Bash(pytest:*)", "Bash(ruff:*)"],
    "deny": ["Bash(rm -rf:*)"],
    "defaultMode": "acceptEdits"
  },
  "env": { "PYTHONWARNINGS": "error" }
}

Committed to .claude/settings.json, this gives the whole team a consistent model, pre-approved test and lint commands, a blanket block on destructive deletes, and a strict warning environment — no per-developer setup required.

Model version pinning and prompt versioning

Reproducibility depends on knowing exactly which model and which instructions produced a result. Both are versionable.

Model version pinning. Every Claude model ID identifies a pinned version. When you use a model ID in a request, the underlying weights stay constant for the lifetime of that ID — Anthropic never updates an existing ID’s weights or configuration. When a newer version ships, it ships under a new model ID [Source: https://platform.claude.com/docs/en/about-claude/models/model-ids-and-versions].

Naming conventions changed with the 4.6 generation:

• 4.6 and later — dateless and pinned: claude-{name}-{major}[-{minor}], e.g., claude-sonnet-4-6, claude-opus-4-8. A common misconception is that a dateless ID is an “evergreen” pointer to the latest model. It is not — for 4.6 and later, the dateless ID is the fixed, canonical snapshot.
• Before 4.6 — dated snapshots: claude-{name}-{major}-{minor}-{YYYYMMDD}, e.g., claude-sonnet-4-5-20250929. These also have shorter aliases (claude-sonnet-4-5) that resolve to the most recent dated snapshot — and those aliases genuinely are evergreen pointers, unlike the 4.6-generation dateless IDs [Source: https://platform.claude.com/docs/en/about-claude/models/model-ids-and-versions].

The analogy: a pinned model ID is like a Docker image referenced by digest, not by the latest tag. latest can shift under you; a digest cannot.

In Claude Code, unpinned aliases (fable, opus, sonnet, haiku) resolve to a built-in provider default. To take control, set model environment variables to specific version IDs during setup. When deploying via Amazon Bedrock, Google Cloud, Microsoft Foundry, or the Claude Platform on AWS, pin explicit model versions before rolling out to users — pinning is what lets you decide when your team moves to a new model rather than being moved automatically [Source: https://code.claude.com/docs/en/model-config].

Prompt versioning. Prompts deserve the same discipline as code. Treat CLAUDE.md, rules, and imported instruction files as version-controlled artifacts: commit them, review changes through pull requests, and tie each prompt version to the model ID it was validated against. Because these files live in source control, an instruction change flows through the same review process as a code change — and because project-root CLAUDE.md is re-read after /compact, your pinned instructions remain stable across a long session [Source: https://platform.claude.com/docs/en/about-claude/models/model-ids-and-versions].

Key Takeaway: CLAUDE.md files carry human-authored instructions and are concatenated across four scopes (managed → user → project → local), while settings.json carries machine-readable behavior and merges across scopes with managed policy always winning. Pin model IDs explicitly (dateless IDs from 4.6 on are already pinned snapshots, not evergreen pointers) and treat CLAUDE.md and rules as version-controlled artifacts tied to the model they were validated against.

Claude Code Core Components

Beyond the two configuration files, Claude Code is extensible through a small set of components — Rules, Skills, Commands, Agents, and Agent Memory. Each answers a different question: when should instructions load, how should a repeatable workflow be triggered, and who remembers what across sessions.

Rules, Skills, Commands, Agents, and Agent Memory

Rules are .claude/rules/*.md files (project) or ~/.claude/rules/*.md (user) holding modular instructions. Without paths: frontmatter they load unconditionally at launch, at the same priority as .claude/CLAUDE.md. With a paths: glob they become path-scoped, loading only when Claude works with matching files — ideal for “when editing anything under migrations/, always add a rollback” [Source: https://code.claude.com/docs/en/memory].

Skills package repeatable workflows — often with supporting files — that load on demand, either when you invoke them explicitly or when Claude decides they are relevant to your prompt. The distinction from rules is when they load: rules load every session (or on matching-file access), whereas skills stay dormant until needed. The distinction from a plain slash command is richness: reach for a slash command when you want an explicit, repeatable terminal entry point; reach for a skill when you want Claude to auto-apply a fuller workflow [Source: https://code.claude.com/docs/en/commands].

Commands (slash commands) control the session and are recognized only at the start of a message, with any trailing text becoming arguments. They come in four types, detailed in the next sub-topic.

Agents (subagents) are defined as markdown files with YAML frontmatter in .claude/agents/ (project) or ~/.claude/agents/ (personal). A subagent is a specialized helper Claude can delegate to, and it can maintain its own persistent auto memory [Source: https://alexop.dev/posts/claude-code-customization-guide-claudemd-skills-subagents/].

Agent Memory (auto memory) is Claude’s own notebook. Claude writes learnings across sessions into ~/.claude/projects/<project>/memory/ — a MEMORY.md index plus topic files. The first 200 lines or 25 KB of MEMORY.md load at each session start; topic files load on demand. It is machine-local and shared across all git worktrees of the same repo, since the <project> path derives from the repo. Auto memory requires Claude Code v2.1.59 or later, and is toggled via autoMemoryEnabled (or CLAUDE_CODE_DISABLE_AUTO_MEMORY=1) [Source: https://code.claude.com/docs/en/memory].

The mental model: CLAUDE.md is what you tell Claude; auto memory is what Claude tells itself. The two are complementary — one is your handbook, the other is Claude’s field notes.

Figure 8.3: Claude Code core components and how each activates

flowchart LR
    CC["Claude Code session"]
    CC --> R["Rules: load every session or on matching paths"]
    CC --> S["Skills: load on demand when relevant or invoked"]
    CC --> CMD["Commands: explicit entry points at message start"]
    CC --> AG["Agents/subagents: delegated specialized helpers"]
    CC --> AM["Agent Memory: Claude's cross-session notebook"]

Repository initialization

The fastest way to give a repo a memory is /init. Run it in a project and Claude analyzes the codebase — languages, build tooling, directory layout, conventions — and generates a starter CLAUDE.md tailored to what it finds. Setting the environment variable CLAUDE_CODE_NEW_INIT=1 upgrades this to an interactive, multi-phase flow that walks you through the generated content [Source: https://code.claude.com/docs/en/memory].

Initialization is a starting point, not a finish line. The generated file captures obvious structure, but the high-value context — the “monetary values are integer cents” rules, the deployment gotchas, the reason a module is structured oddly — comes from you editing it afterward. Once committed, that file becomes the shared, reviewable memory the whole team inherits.

Custom and built-in slash commands

Slash commands come in four flavors [Source: https://code.claude.com/docs/en/commands]:

• Built-in commands — coded into the CLI: /model, /effort, /clear, /diff, /permissions, /init, /memory, /mcp, /compact, /context, /resume, /status, /help, and more.
• Skills (bundled) — prompts handed to Claude that Claude can also invoke automatically; marked [Skill] (e.g., /code-review, /debug, /loop, /run, /batch).
• Workflows (bundled) — dynamic workflows that fan work across subagents; marked [Workflow] (e.g., /deep-research).
• Custom commands — skills you write, stored project-level in .claude/commands/ or personally in ~/.claude/commands/.

Commands are recognized only at the start of a message. A useful exception arrived in v2.1.199: skills can be chained. Typing /skill-a /skill-b do XYZ loads every named skill — up to six — and passes the trailing text to each as arguments [Source: https://code.claude.com/docs/en/commands].

A custom command is just a markdown file. Drop .claude/commands/ship-check.md into your repo describing a pre-deploy checklist, and /ship-check becomes available to everyone who clones it — a team-shared macro living in version control alongside the code it serves.

Key Takeaway: Rules, Skills, Commands, Agents, and Agent Memory differ mainly in when and how they activate — rules load every session (or on matching files), skills load on demand, commands are explicit entry points, subagents are delegated helpers, and auto memory is Claude’s own cross-session notebook. /init bootstraps a repo’s CLAUDE.md by analyzing the codebase, and custom slash commands stored in .claude/commands/ become shareable team macros.

Claude Code Features and Modes

The same agent loop that powers the interactive terminal can run headlessly in a script, stream JSON events in real time, or drive a large-scale modernization. Knowing which mode fits which job is the difference between a chat tool and an automation platform.

Session management

Interactive session mode is the default terminal UI, driven by slash commands. You start a session, work conversationally, and steer with commands like /init, /memory, /model, /plan, /compact, and /resume. Two session-management commands deserve emphasis: /compact summarizes the conversation to reclaim context window space (re-reading project-root CLAUDE.md afterward so your pinned instructions survive), and /resume reopens a prior session so long-running work spans multiple sittings [Source: https://code.claude.com/docs/en/commands].

Headless, streaming, and auto modes

Headless mode (also called non-interactive or print mode) runs the identical agent loop without the terminal UI. Add the -p (or --print) flag with a prompt: Claude runs its full loop, prints a result, and exits. Because it reads stdin (data can be piped in) and writes stdout (output can be redirected), headless mode is the primary mechanism for automation, CI/CD, and scripting. As of v2.1.205 a subset of slash commands works in -p mode (/effort, /model, /mcp, /config key=value, /rename, /fast, /color), but changes apply to the current session only and do not persist as defaults [Source: https://www.buildthisnow.com/blog/guide/development/claude-code-headless-mode].

If interactive mode is a phone call, headless mode is sending a letter: you compose the whole request up front, hand it off, and receive one complete reply — perfect for a machine that cannot chat back and forth.

Figure 8.4: Choosing between interactive and headless modes and their output formats

flowchart TD
    A["Same agent loop"] --> B{"Terminal UI needed?"}
    B -->|"Yes: conversational work"| C["Interactive mode: slash commands like /compact, /resume"]
    B -->|"No: automation, CI/CD"| D["Headless mode: -p / --print over stdin/stdout"]
    D --> E{"--output-format"}
    E --> F["text: plain result string"]
    E --> G["json: structured object, optional --json-schema"]
    E --> H["stream-json: one JSON event per line, real-time"]

Output formats and streaming. Headless mode supports --output-format with three values [Source: https://www.buildthisnow.com/blog/guide/development/claude-code-headless-mode]:

• text — a plain result string.
• json — one structured JSON object with the result plus request metadata. Combined with --json-schema (a JSON Schema definition), Claude Code returns output conforming to your schema in a structured_output field — invaluable when a downstream program must parse the result.
• stream-json — every line is a self-contained JSON object describing one event, enabling real-time streaming consumption as the agent works.

Worked example — headless in a pipeline:

cat error.log | claude -p "Summarize the root cause and suggest a one-line fix" \
  --output-format json --json-schema schema.json

Here the log is piped in via stdin, Claude runs non-interactively, and the result comes back as schema-conforming JSON your CI job can act on programmatically.

Operating Claude Code for codebase modernization

Headless mode plus streaming output is the engine behind large-scale codebase modernization — framework upgrades, dependency migrations, or systematic refactors applied across many files. Rather than shepherding each change by hand, you script Claude to run the same well-specified task repeatedly, capture structured output for each file, and feed results into your review pipeline. The stream-json format lets a monitoring layer observe progress event by event, while committed CLAUDE.md rules keep every invocation aligned to the same standards. Pin the model version first (see the previous section) so a migration that spans days produces consistent results throughout [Source: https://hidekazu-konishi.com/entry/claude_code_cicd_and_headless_automation.html].

Key Takeaway: Interactive mode is the conversational terminal UI managed with slash commands like /compact and /resume, while headless mode (-p/--print) runs the same agent loop non-interactively over stdin/stdout — the foundation for CI/CD and scripting. Output formats (text, json with optional --json-schema, and stream-json) make results machine-consumable, powering automated codebase modernization when paired with a pinned model and committed rules.

Managing Configuration at Scale

A configuration that works on your laptop is only useful if it works for the whole team, survives across upgrades, and can be audited. Scaling configuration means managing plugin dependencies reproducibly, sharing settings deliberately, and keeping everything in version control.

Plugin dependencies

Claude Code ships a full plugin marketplace ecosystem — a distribution system with centralized discovery, version pinning, automatic updates, permission controls, and multiple source backends (GitHub, npm, GitLab, and local paths). A plugin bundles reusable configuration — commands, skills, agents, even default settings — into an installable unit [Source: https://claudefa.st/blog/tools/mcp-extensions/plugins-distribution].

For dependencies, pinning strategy is everything:

• A catalog/marketplace can pin a plugin entry to a specific commit SHA via a sha field on the source.
• A plugin manifest can pin itself to a semver string.
• Best practice: pin to a SHA in production. Referencing a ref (branch or tag) means a branch push or a moved tag can silently change what gets installed; a SHA guarantees reproducibility. The catalog can override the manifest by pinning to a SHA [Source: https://claudefa.st/blog/tools/mcp-extensions/plugins-distribution].

This is exactly why lockfiles exist in package managers: a semver range like ^2.1.0 is convenient but mutable, whereas a pinned SHA is a fingerprint that resolves to one and only one version. Anthropic’s official/community marketplace works this way — its CI pins each approved plugin to a commit, and the pin advances only after re-review. Anthropic also maintains an official managed directory of high-quality plugins at anthropics/claude-plugins-official [Source: https://github.com/anthropics/claude-plugins-official].

Sharing configuration across a team

The scope model from earlier sections is precisely what makes team sharing work. The rule of thumb: commit what the team should share; gitignore what is personal.

• .claude/settings.json, ./CLAUDE.md, .claude/rules/, .claude/commands/, .claude/agents/ → commit. Everyone who clones the repo inherits the same model, permissions, instructions, rules, and custom commands.
• .claude/settings.local.json, ./CLAUDE.local.md → gitignore. Your sandbox URLs, local test data, and personal overrides stay yours.
• Managed policy (settings.json and CLAUDE.md at OS-specific paths) → deployed by IT/DevOps for organization-wide enforcement that no user or project can override.

This layering lets an organization set immovable security policy at the top, teams set shared project conventions in the middle, and individuals keep private tweaks at the bottom — all without the layers fighting each other.

Keeping configuration in version control

Version control is what turns configuration into a governed, auditable artifact rather than tribal knowledge. Because CLAUDE.md, rules, settings.json, custom commands, and subagent definitions are all plain-text files, they diff cleanly and review naturally through pull requests. An instruction change — say, tightening a coding standard — goes through the same review, blame, and rollback machinery as a code change [Source: https://platform.claude.com/docs/en/about-claude/models/model-ids-and-versions].

Combine this with the two other pinning practices from this chapter and you get end-to-end reproducibility: pin the model ID so behavior is stable, pin plugin SHAs so dependencies are fixed, and commit CLAUDE.md/rules tied to the model they were validated against so instructions are reviewed. Together these make a Claude Code setup that behaves the same for every teammate today and the same again six months from now.

Key Takeaway: Scale configuration by pinning plugin dependencies to commit SHAs (not mutable branch/tag refs) for reproducibility, and by committing shared files (.claude/settings.json, CLAUDE.md, rules, commands, agents) while gitignoring personal ones (*.local.json, CLAUDE.local.md). Keeping configuration in version control lets instruction changes flow through code-review discipline, and pinning the model, plugins, and prompts together delivers reproducible behavior across the whole team.

Chapter Summary

Configuration management is how Claude Code stops being an amnesiac assistant and becomes a consistent, team-aware collaborator. The system rests on two layered file families. CLAUDE.md files carry human-authored instructions and are concatenated across four scopes — managed policy, user, project, and local — from broadest to most specific, with more-specific files carrying more weight; they are context, not enforcement, and stay effective when kept under about 200 lines. settings.json files carry machine-readable behavior and merge across the same conceptual scopes with a strict precedence (plugin defaults → user → project → local → managed), where managed enterprise policy always wins and provides true enforcement through keys like permissions.deny and PreToolUse hooks.

Reproducibility runs through the whole chapter. Every Claude model ID is a pinned snapshot — Anthropic ships updates as new IDs, never by mutating an existing one — and from the 4.6 generation on, dateless IDs like claude-opus-4-8 are themselves fixed snapshots rather than evergreen pointers. Pinning the model, treating prompts and CLAUDE.md as version-controlled artifacts, and pinning plugins to commit SHAs together deliver behavior that stays consistent across teammates and across time.

Claude Code’s extensibility comes from a small set of components that differ by when and how they activate: Rules (every session or on matching files), Skills (on demand), Commands (explicit entry points, including custom ones in .claude/commands/), Agents/subagents, and Agent Memory (Claude’s own cross-session notebook). /init bootstraps a repo’s memory. And the same agent loop runs everywhere: interactive mode for conversational terminal work, and headless mode (-p) with text, json, and stream-json output formats for CI/CD, scripting, and large-scale codebase modernization. Managed with version control and deliberate sharing, this configuration system scales cleanly from one developer to an entire organization.

Key Terms

Chapter 9: Prompt Engineering

Prompt engineering is the practice of designing the text you send to Claude so that the model reliably produces the behavior you want. It sits at the center of the Claude Certified Developer curriculum because almost every other capability — tool use, structured outputs, agents, RAG — ultimately rests on the quality of the instructions you write. A well-engineered prompt is cheaper than fine-tuning, faster to iterate on, and portable across models.

A useful analogy runs through all of Anthropic’s official guidance: treat Claude as a brilliant but brand-new employee who lacks context on your norms and workflows [Source: https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/claude-prompting-best-practices]. A new hire may be extraordinarily capable, but on day one they don’t know your company’s conventions, your preferred formats, or the unspoken rules of your domain. You wouldn’t hand them a two-word task and expect telepathy; you’d explain what you want, why it matters, and what “good” looks like. Claude works the same way. This chapter walks through four pillars of that craft — writing clear instructions, using examples and placement well, constraining output, and iterating safely — each grounded in Anthropic’s published best practices.

Two key terms to define at the outset. A system prompt is the top-level instruction passed via the API’s dedicated system parameter; it sets Claude’s role, persona, and high-level operating rules. A user prompt is the actual query or task, passed in the messages array with role: "user". Understanding the division of labor between these two is a recurring theme below.

Section 1: Instruction Clarity

Writing unambiguous instructions

The foundational principle of prompting Claude is to be clear and direct. Claude responds well to explicit, specific instructions, and vague prompts produce vague results. Anthropic frames the standard with what it calls the Golden Rule of Clear Prompting: “Show your prompt to a colleague with minimal context on the task and ask them to follow it. If they’d be confused, Claude will be too” [Source: https://docs.claude.com/en/docs/build-with-claude/prompt-engineering/be-clear-and-direct]. Instruction clarity — the degree to which a prompt unambiguously specifies the task, format, and constraints — is the single highest-leverage lever a developer controls.

A concrete consequence of this rule: if you want “above and beyond” behavior, you must explicitly request it rather than hoping Claude infers it. Anthropic’s classic contrast makes the point [Source: https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/claude-prompting-best-practices]:

• Less effective: “Create an analytics dashboard.”
• More effective: “Create an analytics dashboard. Include as many relevant features and interactions as possible. Go beyond the basics to create a fully-featured implementation.”

A second clarity technique is to provide instructions as sequential steps — numbered lists or bullet points — whenever the order or completeness of the steps matters. A model asked to “summarize, then translate, then flag risks” as three numbered steps is far less likely to skip a stage than one handed a run-on paragraph.

A third, subtler technique is to add the motivation behind an instruction — the why. Explaining the reason helps Claude generalize the intent to cases you didn’t anticipate. Anthropic’s example: instead of a blunt “NEVER use ellipses,” write “Your response will be read aloud by a text-to-speech engine, so never use ellipses since the text-to-speech engine will not know how to pronounce them” [Source: https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/claude-prompting-best-practices]. Claude is “smart enough to generalize from the explanation,” so it will also avoid other constructs a TTS engine would mispronounce — not just the one you named.

Structuring prompts with delimiters and sections

Once a prompt grows beyond a single instruction — once it mixes instructions, background context, examples, and variable input — plain prose becomes ambiguous. Where do the instructions end and the data begin? The solution is XML tags: wrapping each distinct type of content in its own named tag so Claude can parse the structure unambiguously [Source: https://docs.claude.com/en/docs/build-with-claude/prompt-engineering/use-xml-tags]. A prompt might use <instructions>, <context>, <input>, and <examples> blocks so that Claude never confuses the document it is analyzing with the instructions about how to analyze it.

Two best practices govern tag use. First, use consistent, descriptive tag names across your prompts — pick sensible names that match their content and reuse them. Second, nest tags when content has a natural hierarchy, for example multiple documents inside a <documents> wrapper, each in its own <document index="1"> block. A crucial exam-relevant nuance: Claude has no reserved or special XML tags. There is no magic vocabulary the model was trained on; <instructions> carries no more inherent power than <the_rules>. What matters is that the names are sensible and used consistently.

Delimiters also help at scale. For long-context prompting (inputs of roughly 20,000+ tokens), Anthropic recommends placing the longform data at the top of the prompt, above the query, instructions, and examples. Putting the query at the end can improve response quality by up to 30% on complex, multi-document inputs. Documents should be structured with <document> tags containing <document_content> and <source> subtags, and for grounding you can ask Claude to first extract relevant passages into <quotes> tags before answering [Source: https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/claude-prompting-best-practices].

A diagram would help here: picture a vertical prompt “stack” for a long-context task — a large <documents> block at the top, then <instructions>, then a <query> at the bottom — with an arrow showing that the query’s position at the end is what drives the quality gain.

Figure 9.1: Long-context prompt stack (query-at-end ordering)

graph TD
    A["Top: <documents> block (20K+ tokens of longform data)"] --> B["Middle: <instructions> (how to analyze)"]
    B --> C["Bottom: <query> (the actual question)"]
    C -. "position at end drives up to +30% quality" .-> A

Specifying role and task

Setting a role focuses Claude’s tone and behavior for your use case, and Anthropic notes that “even a single sentence makes a difference” [Source: https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/claude-prompting-best-practices]. The mechanism is the system parameter, for example system="You are a helpful coding assistant specializing in Python.", while the user’s actual question goes in the messages array. Role prompting is the primary purpose of the system prompt. Task-specific instructions, by contrast, can live in either the system prompt or the user turn — a placement decision explored in the next section.

Consider a compact before/after improvement that combines every technique above:

Before:

Summarize this feedback.
{customer_feedback}

After:

System: You are a customer-insights analyst for a SaaS product team.

User:
<instructions>
Summarize the customer feedback below. Because this summary feeds our
weekly triage meeting, follow these steps in order:
1. Identify the single most-requested feature.
2. List up to three recurring complaints, most frequent first.
3. Flag any feedback that mentions billing or security.
</instructions>

<customer_feedback>
{customer_feedback}
</customer_feedback>

The “after” prompt assigns a role, states the motivation (“feeds our weekly triage meeting”), gives ordered steps, and cleanly delimits the data from the instructions. A colleague with no context could execute it — which is exactly the Golden Rule’s test.

Key Takeaway: Clarity is the highest-leverage prompting skill: be explicit about the desired output, give ordered steps when order matters, and always explain the why so Claude can generalize. Use XML-style delimiters to separate instructions from data, remembering that tag names carry no special power — only consistency and descriptiveness do.

Section 2: Examples and Placement

Few-shot examples

After clarity, the most reliable way to steer Claude’s output format, tone, and structure is to show it what you want. Few-shot examples (also called multishot prompting) are worked input/output pairs embedded in the prompt that demonstrate the target behavior. Anthropic calls examples “one of the most reliable ways to steer Claude’s output” and recommends 3–5 examples for best results [Source: https://docs.claude.com/en/docs/build-with-claude/prompt-engineering/multishot-prompting].

The analogy again is onboarding: telling a new employee “format the report professionally” is far weaker than handing them three past reports and saying “like these.” Examples collapse a paragraph of description into a demonstrated pattern.

For examples to work well, they should be:

A powerful bootstrapping move: you can ask Claude itself to evaluate your examples for relevance and diversity, or to generate additional examples based on an initial seed set. Examples also compose with reasoning — if you place <thinking> tags inside your few-shot examples to show the reasoning pattern, Claude will generalize that style to its own thinking blocks [Source: https://docs.claude.com/en/docs/build-with-claude/prompt-engineering/multishot-prompting].

System vs. user prompt placement

Where content goes matters as much as what it says. The guiding principle: the system prompt is best for high-level scene-setting and role assignment (“You are AcmeCorp’s ethical AI assistant…”), while the bulk of task instructions, examples, and the actual query go in the user turn [Source: https://docs.claude.com/en/docs/build-with-claude/prompt-engineering/multishot-prompting]. This mirrors an org chart: the system prompt is the job description that defines who Claude is; the user turn is the daily ticket that defines what to do right now.

That last row is a security rule, not merely a style preference. For untrusted or third-party content — web pages, emails, documents, search results — placement is a security control. Such content belongs only in tool_result blocks, never in the system prompt or as plain user text, because Claude is trained to treat instructions embedded in tool results with appropriate skepticism. Section 4 develops this defense in full.

Placement across components and interfaces

The same content can appear in different “surfaces,” and its authority shifts with the surface. An instruction in the system prompt is treated as an operating rule; the identical sentence sitting inside a tool_result is treated as untrusted data. This is the conceptual bridge to input sanitization: a developer chooses placement deliberately to encode trust level. Content the developer authored and trusts goes high in the hierarchy (system, then user); content that arrives from the outside world goes into tool results, labeled with what it is and where it came from.

Figure 9.2: Placement decision by trust level

flowchart TD
    A["New content to add to the prompt"] --> B{"Who authored it?"}
    B -->|"Developer-trusted"| C{"Role, persona, or policy?"}
    B -->|"Outside world (web, email, docs, search)"| D["tool_result block ONLY"]
    C -->|"Yes"| E["System prompt (operating rule)"]
    C -->|"No: task, examples, query"| F["User turn"]
    D --> G["Label nature and origin; treat as data, not instructions"]

Key Takeaway: Show, don’t just tell — 3 to 5 relevant, diverse, <example>-wrapped few-shot examples steer format and tone more reliably than description alone. Place role and policy in the system prompt, task instructions and examples in the user turn, and route all untrusted third-party content exclusively through tool_result blocks, since placement encodes how much Claude should trust the content.

Section 3: Output Constraints

Constraining format and length

Output constraints are the parts of a prompt that govern the shape of the response — its format, length, structure, tone, and style — as opposed to its content. Anthropic’s single most important steering rule here is counterintuitive: tell Claude what to do instead of what not to do. Negative instructions are weaker than positive ones. Instead of “Do not use markdown in your response,” write “Your response should be composed of smoothly flowing prose paragraphs” [Source: https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/claude-prompting-best-practices]. A positive target gives Claude something to aim at; a prohibition only tells it where not to go.

Anthropic lists four complementary format-steering methods:

1. Tell Claude what to do instead of what not to do (the rule above).
2. Use XML format indicators. Instruct Claude to write output inside a specific tag, e.g. “Write the prose sections of your response in <smoothly_flowing_prose_paragraphs> tags.” You can then programmatically extract exactly the content between those tags.
3. Match your prompt style to the desired output. The formatting of your prompt influences Claude’s response formatting — removing markdown from your prompt tends to reduce markdown in the output.
4. Use detailed prompts for specific formatting preferences. A dedicated block such as <avoid_excessive_markdown_and_bullet_points> can spell out that Claude should write in flowing prose, reserve markdown for inline code and simple headings, and avoid unnecessary bold, italics, or lists.

On length, current Claude models are already more concise than earlier generations and may skip verbal summaries after tool calls. If you want more visibility into an agent’s work, ask for it explicitly: “After completing a task that involves tool use, provide a quick summary of the work you’ve done” [Source: https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/claude-prompting-best-practices]. For mathematical output, current models default to LaTeX; a plain-text instruction overrides that when needed.

Enforcing structured output

For truly machine-readable output — JSON, YAML, a fixed classification label — prompting alone is not the most reliable path. Anthropic’s guidance is to enforce structure via the Structured Outputs feature (an output_config carrying a JSON schema) or via tool calling with an enum field, rather than by manual prompt instructions [Source: https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/claude-prompting-best-practices]. These features constrain the model at the decoding level, guaranteeing schema conformance instead of merely requesting it.

This connects to a significant model-behavior change developers must know. Historically, prefilling Claude’s response — placing text in the assistant turn so the model “continues” from it, colloquially “speaking for Claude” or “putting words in Claude’s mouth” — was a core control technique. Developers prefilled { to force a JSON object with no preamble, or a bracketed [ROLE_NAME] to maintain character in role-play. Important deprecation nuance: starting with Claude 4.6 models and Claude Mythos Preview, prefilled responses on the last assistant turn are no longer supported and return a 400 error [Source: https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/prefill-claudes-response]. Adding assistant messages elsewhere in a conversation is unaffected, and earlier models still support last-turn prefills.

Anthropic’s migration paths for the former prefill use cases:

Guiding tone and style

Tone and style are steered with the same toolkit. A role in the system prompt establishes a baseline voice (“You are a warm, patient customer-support agent”). Positive format targets refine it (“Respond in short, reassuring sentences”). And prompt-style matching applies: a terse, formal prompt tends to elicit terse, formal output, while a conversational prompt loosens the response. Because these levers interact, tone is best treated as an emergent property tuned through iteration rather than dictated by one magic sentence.

Key Takeaway: Steer format by stating what Claude should do (positive targets beat prohibitions), and use XML format indicators for programmatic extraction. For guaranteed machine-readable output, reach for Structured Outputs or tool-call enums rather than manual prompting — and remember that last-turn prefilling is deprecated on Claude 4.6+ / Mythos Preview (it returns a 400), so migrate to structured outputs, direct system instructions, or user-turn continuations.

Section 4: Iteration and Input Sanitization

Iterative refinement and prompt adjustment

No prompt is perfect on the first draft. Iterative refinement is the disciplined cycle of write → test → gather feedback → refine that turns a rough prompt into a production-grade one. Anthropic operationalizes this in the Console’s Prompt Improver, a tool that takes an existing prompt — even one originally written for a different AI model — and uses Claude to automatically rewrite it with best-practice techniques [Source: https://claude.com/blog/prompt-improver]. The improver applies five refinement methods:

1. Chain-of-thought reasoning — adds a dedicated section for Claude to reason through the problem before answering.
2. Example standardization — converts examples into consistent XML formatting.
3. Example enrichment — augments examples with aligned reasoning steps.
4. Rewriting — clarifies structure and corrects grammar and spelling.
5. Prefill addition — directs actions and enforces output formats (for models that still support it).

The improver’s reported gains quantify why iteration pays off: a 30% increase in accuracy on a multilabel classification task and 100% word-count adherence on a summarization task after optimization [Source: https://claude.com/blog/prompt-improver]. The Console Workbench also offers structured example management — add, edit, and auto-generate synthetic input/output pairs — so example curation becomes part of the same loop.

Chain-of-thought (CoT) prompting deserves special mention as a refinement technique. The simplest trigger is a phrase like “Think step by step,” which prompts Claude to reason before answering. Current guidance is nuanced by newer reasoning modes: on models with adaptive thinking (thinking: {type: "adaptive"}), Claude dynamically decides when and how much to reason, calibrated by an effort parameter and query complexity, and Anthropic reports adaptive thinking “reliably drives better performance than extended thinking.” Two refinements follow: prefer general instructions over prescriptive steps (“think thoroughly” often beats a hand-written plan, because Claude’s reasoning can exceed what a human would prescribe), and ask Claude to self-check by appending “Before you finish, verify your answer against [test criteria]” — especially valuable for coding and math [Source: https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/claude-prompting-best-practices]. When extended thinking is disabled, some models are sensitive to the word “think,” so alternatives like “consider,” “evaluate,” or “reason through” can help.

Testing prompt variants

Refinement is only meaningful if you can tell whether a change helped. The practical pattern is to test prompt variants against a fixed evaluation set — a collection of representative inputs with known-good outputs — and compare candidates on measurable criteria such as accuracy, format adherence, or word count. Anthropic’s own metrics (the +30% and 100% figures) are exactly this kind of measured comparison. For pipelines where you need to inspect intermediate results, prompt chaining helps: generate a draft in one call, have Claude review it against criteria in a second, and refine in a third, with each stage a separate API call whose output you can inspect and gate.

A diagram would help here: a simple loop — Draft → Evaluate against test set → Refine → Draft — with a branch out of “Evaluate” to “Ship” once the variant clears its threshold.

Figure 9.3: Iterative refinement loop

flowchart TD
    A["Draft prompt"] --> B["Evaluate against fixed test set"]
    B --> C{"Clears threshold? (accuracy, format, word count)"}
    C -->|"No"| D["Refine (Prompt Improver, CoT, examples)"]
    D --> A
    C -->|"Yes"| E["Ship"]

Input sanitization for safety

The final pillar is defensive. Input sanitization is the practice of validating, labeling, and neutralizing untrusted input so that adversarial text cannot hijack Claude’s behavior. Anthropic distinguishes two threat models [Source: https://platform.claude.com/docs/en/test-and-evaluate/strengthen-guardrails/mitigate-jailbreaks]:

• Jailbreaks / direct prompt injection — the user is the adversary, trying to make Claude violate its guidelines.
• Indirect prompt injection — the user is trusted, but Claude processes adversarial third-party content (web pages, emails, documents, tool results) that carries hidden instructions.

Defenses against direct attacks:

• Harmlessness screens — pre-screen user input with a lightweight model (e.g. Claude Haiku 4.5) before it reaches the main conversation, using structured outputs to constrain the classifier to a simple boolean such as {"is_harmful": bool}.
• Input validation — filter input for known injection patterns before it reaches Claude; an LLM can build a generalized validation screen using known jailbreak language as examples.
• Prompt engineering — craft system prompts that emphasize ethical and legal boundaries and tell Claude exactly how to refuse (e.g. “I cannot perform that action as it goes against AcmeCorp’s values”).
• Respond to repeat offenders — throttle or ban users who repeatedly trigger the same refusals.

Defenses against indirect injection all share one goal: make untrusted content unambiguously data, not instructions.

Figure 9.4: Indirect prompt-injection defense pipeline

flowchart TD
    A["Untrusted third-party content (web, email, doc, tool output)"] --> B["Route into labeled tool_result block"]
    B --> C["JSON-encode content (unambiguous delimiters)"]
    C --> D["Screen with lightweight classifier: {injection_suspected: bool}"]
    D -->|"Flagged"| E["Return error or stripped summary"]
    D -->|"Clean"| F["Claude processes under untrusted_content_policy + least privilege"]
    F --> G["Report embedded instructions rather than act on them"]

A subtle corollary: don’t put your own instructions in tool results either — they may be ignored or flagged. Send instructions in a following user turn instead (or, on Opus 4.8+, a mid-conversation system message). Anthropic recommends chaining safeguards — layering several of these defenses — and for the computer-use tool it runs additional classifiers that detect injections in screenshots and steer Claude to seek user confirmation before acting [Source: https://www.anthropic.com/research/prompt-injection-defenses].

Key Takeaway: Treat prompting as an empirical loop — draft, test variants against a fixed evaluation set, and refine, using the Console Prompt Improver and chain-of-thought techniques to accelerate the cycle. Sanitize input by matching placement to trust: screen direct user input with lightweight classifiers and ethical system prompts, and neutralize indirect injection by routing untrusted content through labeled, JSON-encoded tool_result blocks governed by an explicit untrusted-content policy, backed by least privilege and red-teaming.

Chapter Summary

Prompt engineering is the developer’s primary lever for shaping Claude’s behavior, and it rests on four connected pillars. Instruction clarity comes first: be explicit about the desired output, give ordered steps when sequence matters, explain the motivation so Claude can generalize, and use consistent XML-style delimiters to separate instructions from data — remembering that no tag name is special, only clear and consistent. Examples and placement come second: 3–5 relevant, diverse, <example>-wrapped few-shot demonstrations steer format and tone more reliably than description, while the system-versus-user division encodes trust — role and policy up top, task and query in the user turn, and untrusted third-party content confined to tool_result blocks. Output constraints come third: state positive format targets rather than prohibitions, use XML indicators for extraction, and reach for Structured Outputs or tool-call enums when you need guaranteed machine-readable output — a necessity heightened by the deprecation of last-turn prefilling on Claude 4.6+ and Mythos Preview, which now returns a 400 error. Iteration and input sanitization come fourth: prompting is empirical, so test variants against fixed evaluation sets and lean on the Console Prompt Improver, while defending against both direct jailbreaks (harmlessness screens, input validation, ethical system prompts) and indirect injection (labeled, JSON-encoded tool results, an untrusted-content policy, least privilege, and red-teaming). Held together, these pillars turn the “brilliant new employee” analogy into a repeatable engineering discipline.

Key Terms

Chapter 10: Context Engineering, Output Handling, and Debugging

Building a Claude-powered application that works in a demo is easy. Building one that stays reliable across a long agentic run, parses cleanly under edge cases, and can be debugged when something goes wrong is the real work of a certified developer. This chapter covers three disciplines that separate a toy from a production system: context engineering (curating what the model sees), output handling (safely consuming what the model produces), and debugging (finding out why a failure happened and where it originated).

Think of these three as the input side, the output side, and the diagnosis loop of every request. If you manage context well, the model reasons better and cheaper. If you handle output defensively, your application doesn’t crash on the one response in a thousand that breaks the pattern. And if you can read a trace, you can tell the difference between a problem you must fix in your own code and one you simply have to wait out on Anthropic’s side.

By the end of this chapter you will be able to manage the context window and prevent drift and bloat; produce, validate, and defensively parse Claude output; and debug failure modes through trace analysis, isolating whether the problem lives in your integration layer or in the model’s output.

Context Engineering

Context window management

Context engineering is the discipline of curating the optimal set of tokens the model sees at inference time. Anthropic frames it as the natural evolution beyond prompt engineering: where prompt engineering focuses on writing one good instruction, context engineering focuses on “curating and maintaining the optimal set of tokens (information) during LLM inference,” optimizing “the utility of those tokens against the inherent constraints of LLMs” [Source: https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents]. The guiding principle is to find “the smallest possible set of high-signal tokens that maximize the likelihood of some desired outcome.”

Crucially, context is not just the current user message. It is the whole state the model sees: system instructions, tool definitions, MCP (Model Context Protocol) server definitions, external data you inject, and the full message history. Every one of those competes for the same finite window.

An analogy: think of the context window as a workbench, not a warehouse. A warehouse can hold everything you own; a workbench can only hold what you’re actively using. A skilled craftsperson keeps the workbench clear of clutter so the tools that matter are within reach. Context engineering is the practice of keeping the model’s workbench clear.

Key Takeaway: Context engineering is curating the smallest set of high-signal tokens for inference, managing the entire context state — system prompt, tools, MCP definitions, injected data, and message history — not just a single prompt. Treat the window as a finite, degradable resource, and reach for the smallest high-signal set that gets the job done.

Preventing context drift and bloat

Two failure modes threaten a long-running context. Context bloat is the simple accumulation of low-value tokens — verbose tool outputs, redundant reasoning, stale data — that crowd out signal and raise cost. Context drift is subtler: as the conversation grows, the model’s focus wanders from the original task, anchored by irrelevant or outdated material still sitting in the window.

The root cause of degradation is a phenomenon Anthropic calls context rot: as the number of tokens in the context window grows, the model’s ability to accurately recall information from that context decreases [Source: https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents]. Retrieval performance drops progressively before the hard token limit is ever reached — so even a window that isn’t “full” can still be underperforming. The transformer architecture is a root cause: it creates n² pairwise relationships for n tokens, so longer contexts stretch the model’s ability to maintain focus. Anthropic describes this as an “attention budget,” analogous to human working memory. The blunt practical takeaway is that “more tokens makes agents worse.”

The defense against drift and bloat is progressive disclosure with just-in-time retrieval. Rather than pre-loading all data up front, the agent maintains lightweight identifiers — file paths, record IDs, URLs — and loads the underlying content only when it’s needed, using tools at runtime [Source: https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents]. The agent assembles understanding layer by layer, keeping only what’s necessary in working memory. A human researcher does the same thing: you don’t memorize an entire library, you keep a stack of call numbers and pull a book when you need it.

The table below contrasts the two problems and their remedies.

Key Takeaway: Context bloat is the buildup of low-value tokens; context drift is the loss of task focus; context rot is the model’s declining recall as the window grows — and rot begins before the hard limit. Prevent all three by loading data just in time through progressive disclosure rather than pre-loading everything.

Tool output pruning and compaction

Anthropic’s cookbook documents three complementary, first-party-supported strategies for managing a growing window. The recommended workflow is diagnosis first: determine whether the bottleneck is accumulated dialogue and reasoning (use compaction), re-fetchable tool output (use clearing), or cross-session knowledge (use memory) — then compose them together [Source: https://platform.claude.com/cookbook/tool-use-context-engineering-context-engineering-tools].

1. Compaction. Compaction takes a conversation nearing the context limit, summarizes its contents, and reinitializes a new window with that summary. It is “typically the first lever” and distills the window in a high-fidelity way. It is a beta feature (edit type compact_20260112, beta header compact-2026-01-12), triggered at a configurable input-token threshold — minimum 50K, default 150K. It preserves architectural decisions, key facts, task progress, and unresolved questions or bugs, while discarding redundant tool outputs, verbatim detail, and obscure specifics such as exact statistical values or appendix table cells. In Claude Code, compaction additionally preserves the five most recently accessed files. Because a summarizer model runs, compaction costs inference — but it handles every kind of context growth. A critical implementation note: you must append the full response.content (including the compaction block) back to your messages on each turn, because the API uses that block to replace the compacted history on the next request. Extracting only the text and appending that will silently lose the compaction state.

2. Tool-result clearing (pruning). Tool output pruning — clearing — is a mechanical, no-inference-cost edit that replaces old tool_result bodies with a short placeholder such as "[cleared to save context]", while keeping the tool_use record so the model still knows the call happened and with what inputs. It uses edit type clear_tool_uses_20250919 with beta header context-management-2025-06-27. Key parameters:

• trigger — fires above a token threshold (default 100K)
• keep — number of most-recent tool uses whose results survive
• clear_at_least — minimum tokens to clear per firing
• exclude_tools — tools whose results are never cleared (e.g., the memory tool)
• clear_tool_inputs — also clear the recorded inputs

Because the cleared content is re-fetchable, the agent can simply re-call a tool if it needs that data again. In Anthropic’s research-agent example, clearing dropped context roughly 128K → 43K tokens (about a 67% reduction) while keeping the most recent read [Source: https://platform.claude.com/cookbook/tool-use-context-engineering-context-engineering-tools]. One caveat: clearing invalidates the prompt cache on each firing, since it edits the prefix.

3. Memory. The memory tool (memory_20250818) lets the agent store notes in persistent external files (e.g., a /memories directory or a NOTES.md) that live outside the context window and are retrieved just in time. It is the only one of the three that solves cross-session persistence. It is client-side: the API defines the protocol (commands view, create, str_replace, insert, delete, rename) and you implement the file backend — so you must guard against path traversal on every model-supplied path.

Figure 10.1: Diagnosis-first decision flow for choosing a context-management lever

flowchart TD
    A["Window nearing the limit"] --> B{"What is the bottleneck?"}
    B -->|"Accumulated dialogue and reasoning"| C["Compaction: summarize + reinitialize window"]
    B -->|"Re-fetchable tool output"| D["Clearing: replace tool_result bodies with placeholder"]
    B -->|"Cross-session knowledge"| E["Memory tool: store notes in external files"]
    C --> F["Compose levers together on long-running agents"]
    D --> F
    E --> F

A single benchmark makes the value concrete. On a biology research agent working over eight documents of ~40K tokens each, the baseline with no management climbed to 335K tokens and hard-stopped after three turns on a 200K window (96% of the context was file-read results). With compaction, the agent completed in seven turns holding peak context at 169K; with clearing, it completed in seven turns at 173K peak [Source: https://platform.claude.com/cookbook/tool-use-context-engineering-context-engineering-tools].

Key Takeaway: Diagnose the bottleneck first, then apply the right lever: compaction summarizes accumulated dialogue at a cost of inference, clearing mechanically prunes re-fetchable tool output at no inference cost (a ~67% reduction in Anthropic’s example), and the memory tool persists knowledge across sessions in external files. They compose — many long-running agents use all three.

Context Isolation

Isolation through subagents

Context isolation is the practice of giving separate parts of a workload separate context windows so no single window ever has to hold everything. The canonical mechanism is a multi-agent architecture in which a lead (orchestrator) agent delegates sub-tasks to subagents, each with its own isolated context window [Source: https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents].

The key design property is the return contract: a subagent does deep work in its own window — potentially reading many files or running many tool calls — but returns only a condensed summary, typically 1,000–2,000 tokens, to the orchestrator. The orchestrator’s window therefore accumulates distilled results, not raw exploration.

Anthropic’s multi-agent research system is the reference example: many agents with isolated contexts outperformed a single agent. There are three reasons. First, each subagent can devote its entire window to a narrow sub-task, so it works with high signal density. Second, no single window ever holds everything, which directly counters distraction and context rot. Third, independent sub-tasks run in parallel, improving latency.

Figure 10.2: Subagent context isolation — deep work in isolated windows, thin summaries returned

flowchart LR
    O["Orchestrator (lead agent)<br/>accumulates distilled results"]
    S1["Subagent 1<br/>isolated window"]
    S2["Subagent 2<br/>isolated window"]
    S3["Subagent 3<br/>isolated window"]
    M["Shared memory file"]
    O -->|"task assignment"| S1
    O -->|"task assignment"| S2
    O -->|"task assignment"| S3
    S1 -->|"1K-2K token summary"| O
    S2 -->|"1K-2K token summary"| O
    S3 -->|"1K-2K token summary"| O
    S1 -.->|"read / write"| M
    S3 -.->|"read / write"| M

An analogy: a research director doesn’t read every primary source personally. They assign topics to analysts, each of whom reads deeply and reports back a one-page brief. The director synthesizes the briefs. Each analyst’s “context” is their own; the director never drowns in raw material.

One practical caution the Claude Code error reference raises: subagents inherit every parent MCP tool definition, which can fill a subagent’s window before its first turn. Disable unused MCP servers before spawning subagents so their windows start clean [Source: https://code.claude.com/docs/en/errors].

Key Takeaway: Context isolation gives each subagent its own window and returns only a 1,000–2,000-token summary to the orchestrator, so no single window holds everything. Many isolated agents beat one monolithic agent — but disable unused MCP servers first, since subagents inherit every parent tool definition.

Multi-step agentic workflows

Isolation is the enabling primitive for multi-step agentic workflows. In a well-designed workflow, the orchestrator decomposes a large goal into sub-tasks, dispatches each to a subagent (often in parallel), collects the summaries, and synthesizes a result. Because each step runs in a fresh, focused window, a ten-step workflow does not accumulate the tokens of all ten steps in one place — a structural defense against both bloat and rot.

This composes naturally with the context-management techniques from the previous section. Within a single long-running agent, compaction and clearing keep one window lean; across a workflow, isolation keeps the number of large windows down. And the memory tool provides the connective tissue: a subagent can write findings to an external file that a later step reads, passing knowledge between steps without passing raw tokens.

A diagram would help here: picture an orchestrator node at the top, three subagent nodes beneath it each with its own bounded box (its window), arrows down carrying task assignments and arrows up carrying thin 1–2K-token summaries, with a shared memory file off to the side that subagents read and write.

Key Takeaway: Multi-step agentic workflows use isolation as their structural defense — decomposing a goal so each step runs in a fresh, focused window rather than accumulating every step’s tokens in one place. Within-window techniques (compaction, clearing) and cross-window techniques (isolation, memory) compose into a single context strategy.

Keeping trusted and untrusted context separate

Isolation matters for correctness; it also matters for security. Not all tokens in a window carry the same authority. Trusted context is content you control — your system prompt, your tool definitions, your operator instructions. Untrusted context is anything that originated outside your control: a fetched web page, a user-supplied document, a tool result from a third-party API, the raw text of an email. Untrusted content can contain prompt-injection attempts — text that tries to pose as instructions to the model.

The defensive posture is to never let untrusted content masquerade as an operator instruction. On Claude Opus 4.8, the API provides a purpose-built channel for this: operator instructions delivered mid-conversation should be sent as a {"role": "system", ...} message appended to the messages array, rather than embedded as text in a user turn [Source: https://platform.claude.com/docs/en/build-with-claude/structured-outputs]. That system role is a non-spoofable operator channel — anything that merely writes to user-visible input cannot forge it, whereas instructions buried in user or tool content can be imitated by injected text. Subagent isolation reinforces this: routing untrusted document processing into a subagent with a narrow tool surface limits the blast radius if that content tries to hijack the agent.

The practical rule: treat every tool result, fetched page, and uploaded document as data to be reasoned about, never as instructions to be obeyed — and keep the channel that does carry authority (the system role) separate and protected.

Key Takeaway: Trusted context (your system prompt and tools) must be kept separate from untrusted context (fetched pages, user documents, third-party tool results), which may carry prompt-injection attempts. Use the non-spoofable system-role channel for operator instructions and treat all external content as data, not commands.

Output Handling

Structured output patterns

Once the model produces a response, your application has to consume it — reliably. Structured output constrains Claude’s responses to follow a specific JSON schema, and Claude supports it through two complementary features [Source: https://platform.claude.com/docs/en/build-with-claude/structured-outputs]:

• JSON outputs (output_config.format, formerly output_format) control what Claude says, forcing the response body into a specific JSON shape.
• Strict tool use (strict: true on a tool definition) validates how Claude calls your functions, guaranteeing schema-valid tool names and inputs.

The mechanism behind both is grammar-constrained sampling: the JSON schema is compiled into a grammar that actively restricts token generation during inference. This is fundamentally different from merely prompting “please return valid JSON” — the model is prevented from emitting a token that would break the schema, rather than being politely asked not to. The payoff is direct: no JSON.parse() errors, guaranteed field types and required fields, and no retries needed for schema violations. Both features can be combined in one request to get a structured response format and guaranteed-valid tool parameters.

The schema has firm requirements. Every object must set "additionalProperties": false, and every property must be listed in the required array. Supported constructs include basic types, enum, const, anyOf/allOf (with limits), internal $ref/$def, string formats (date, email, uri, uuid), and array minItems of 0 or 1 only. Not supported: recursive schemas, numeric constraints (minimum/maximum/multipleOf), string-length constraints (minLength/maxLength), and complex regex patterns. Limits: at most 20 strict tools per request, 24 optional parameters, and 16 parameters with union types. Required properties always appear before optional ones regardless of schema ordering.

The SDKs are defensive by design. In Python, client.messages.parse(..., output_format=Model) takes a Pydantic model, validates the response, and returns response.parsed_output as a typed object; TypeScript uses zodOutputFormat() with a Zod schema; Java, Ruby, and PHP have equivalent typed models. So you define schemas in your language’s native validation library instead of hand-writing raw JSON schema.

Structured outputs are generally available on the Claude API for Claude Fable 5, Mythos 5, Opus 4.8/4.7/4.6/4.5, Sonnet 5/4.6/4.5, and Haiku 4.5, plus Amazon Bedrock, Google Cloud Vertex AI, Microsoft Foundry, and Claude Platform on AWS. On performance: the first request with a new schema incurs a one-time grammar-compilation latency; compiled grammars are then cached for 24 hours (invalidated by schema or tool-set changes); and the injected schema adds input tokens. The feature qualifies for Zero Data Retention.

Here is the strict-tool-use pattern in Python:

response = client.messages.create(
    model="claude-opus-4-8",
    max_tokens=16000,
    messages=[{"role": "user", "content": "Book a flight to Tokyo for 2 on March 15"}],
    tools=[{
        "name": "book_flight",
        "description": "Book a flight to a destination",
        "strict": True,
        "input_schema": {
            "type": "object",
            "properties": {
                "destination": {"type": "string"},
                "date": {"type": "string", "format": "date"},
                "passengers": {"type": "integer", "enum": [1, 2, 3, 4, 5, 6, 7, 8]},
            },
            "required": ["destination", "date", "passengers"],
            "additionalProperties": False,
        },
    }],
)

Note that strict: True is a sibling of name/description/input_schema on the tool itself — not something you put on tool_choice, a common mistake.

Key Takeaway: Structured outputs = JSON outputs (output_config.format) plus strict tool use (strict: true), both driven by grammar-constrained sampling that restricts token generation to schema-valid output at inference time — eliminating JSON.parse() errors and schema-violation retries. Every object needs "additionalProperties": false and all properties in required; define schemas with Pydantic or Zod and let the SDK’s parse() return a validated, typed object.

Response validation and defensive parsing

Grammar-constrained sampling removes most parse failures — but not all. Defensive parsing is the practice of never assuming the schema held, because in specific edge cases the output legitimately will not match it [Source: https://platform.claude.com/docs/en/build-with-claude/structured-outputs]. Three cases matter:

• Refusals (stop_reason: "refusal"): a safety refusal takes precedence over the schema. The content may be empty (pre-output) or partial (mid-stream), and it will not conform.
• Truncation (stop_reason: "max_tokens"): output cut off at the token limit is incomplete and therefore won’t parse. Increase max_tokens or stream.
• Enum casing: string enums may differ in capitalization from the schema.

The rule is: always branch on stop_reason before you touch content. Code that reads response.content[0].text unconditionally will throw on a refusal that returns an empty content array. The defensive pattern:

response = client.messages.create(...)  # with output_config.format
if response.stop_reason == "refusal":
    handle_refusal(response.stop_details)   # do not parse
elif response.stop_reason == "max_tokens":
    handle_truncation()                     # incomplete — retry with more tokens
else:
    data = response.parsed_output           # safe to consume

Figure 10.3: Defensive parsing — branch on stop_reason before touching content

flowchart TD
    A["Response received"] --> B{"stop_reason?"}
    B -->|"refusal"| C["handle_refusal(stop_details)<br/>do not parse — content may be empty/partial"]
    B -->|"max_tokens"| D["handle_truncation()<br/>incomplete — retry with more tokens or stream"]
    B -->|"other"| E["data = response.parsed_output<br/>safe to consume"]
    E --> F["Validate parsed object<br/>fallback path if schema did not hold"]

Two supporting practices complete the discipline. First, catch the SDK’s typed exceptions (e.g., anthropic.RateLimitError) rather than string-matching error messages — string matching is brittle and breaks when wording changes. Second, treat schema conformance as best-effort under the edge cases above: a robust consumer branches on stop_reason, validates the parsed object, and has a fallback path for the cases where the schema didn’t hold.

Key Takeaway: Even with structured outputs, the response may not match the schema on stop_reason: "refusal" (safety takes precedence) or "max_tokens" (truncation), and enum casing may differ — so always branch on stop_reason before parsing content, and catch typed SDK exceptions rather than string-matching messages.

Skepticism toward confident output

Structured, well-formed output can still be wrong. Anthropic’s “Reduce hallucinations” guidance stresses that fluent, confident prose can carry false information, and that developers and users should be skeptical of specific numbers, dates, and citations and cross-check them against trusted primary sources [Source: https://docs.anthropic.com/en/docs/test-and-evaluate/strengthen-guardrails/reduce-hallucinations]. A schema guarantees the shape of an answer, never its truth.

An analogy: a beautifully typeset invoice is not proof the amounts are correct. Formatting is orthogonal to accuracy. The same is true of a schema-valid JSON object full of confident-sounding figures.

The guidance offers five techniques to reduce and audit hallucination:

1. Allow uncertainty. Explicitly give Claude permission to say “I don’t know.” This alone drastically reduces fabricated information — a model forced to answer will confabulate; a model permitted to abstain will decline.
2. Ground in direct quotes. For long documents (>20K tokens), have Claude extract word-for-word quotes first, anchoring its answer to the actual text rather than a paraphrase from memory.
3. Verify with citations. Make responses auditable by having Claude cite quotes and sources per claim, or find a supporting quote after generating each claim. The Citations API generates responses with precise citations tied to the specific source chunks you provided.
4. Chain-of-thought verification. Ask Claude to reason step by step before giving a final answer, which exposes faulty logic to inspection.
5. Best-of-N verification. Run the same prompt multiple times and compare outputs for consistency; divergence flags low-confidence answers.

The through-line: build auditability into the pipeline. Don’t just consume the answer — consume the answer plus its evidence, and verify the load-bearing facts against a source you trust.

Key Takeaway: A valid schema guarantees an answer’s shape, never its truth — fluent, confident output can still be wrong, so cross-check numbers, dates, and citations against trusted sources. Reduce and audit hallucination by allowing “I don’t know,” grounding answers in direct quotes, citing sources (Citations API), and using chain-of-thought and best-of-N verification.

Debugging and Error Handling

Error type identification and recovery strategies

When a request fails, the first job is to identify what kind of failure it is. The Claude API returns errors as JSON with a top-level error object containing a type and message, plus a request_id, mapped onto predictable HTTP status codes [Source: https://platform.claude.com/docs/en/api/errors]. The full taxonomy:

The recovery strategy follows directly from the “retryable?” column. Non-retryable 4xx errors are your responsibility to fix in code or configuration — a 400 needs a corrected request, a 401 needs a valid key, a 404 usually means a mistyped model ID. Retryable errors (429, 5xx, 529) call for exponential backoff. The official SDKs automatically retry transient failures — connection errors, rate limits, and 5xx server errors — with exponential backoff, twice by default, honoring the retry-after header; a max_retries option configures or disables this [Source: https://platform.claude.com/docs/en/api/errors]. So for most transient failures, the SDK already does the right thing and you should only add custom retry logic when you need behavior beyond the default.

Figure 10.4: Error recovery decision tree driven by the retryable column

flowchart TD
    A["Request fails with error type + status code"] --> B{"Retryable?"}
    B -->|"No — 400/401/402/403/404/413"| C["Fix in your code or configuration<br/>(correct request, valid key, model ID)"]
    B -->|"Yes — 409"| D["Resolve resource-state conflict, then retry"]
    B -->|"Yes — 429/500/504/529"| E["Exponential backoff, honor retry-after"]
    E --> F{"Custom behavior needed?"}
    F -->|"No"| G["SDK auto-retries (twice by default)"]
    F -->|"Yes"| H["Configure max_retries / custom logic"]

One streaming subtlety: mid-stream errors on an SSE (Server-Sent Events) stream can occur after a 200 response and don’t follow standard error handling — they arrive as stream error events. For requests that may run over ~10 minutes, use the streaming Messages API or the Message Batches API; the SDKs validate that a non-streaming request won’t exceed a 10-minute timeout and set TCP keep-alive.

Key Takeaway: Claude API errors follow a predictable HTTP taxonomy with a typed JSON error object and a request_id; the retryable column drives recovery — fix non-retryable 4xx errors in your own code/config, and let the SDK’s automatic exponential backoff (twice by default, honoring retry-after) handle transient 429/5xx/529 failures.

Trace analysis to identify failure modes

A failure mode is a characteristic way a system goes wrong; trace analysis is the practice of reconstructing what happened from the artifacts a request leaves behind. The single most important artifact is the request ID. Every response includes a unique request-id header (e.g., req_018EeWyXxfu5pfWkrYcMdjWG), mirrored as request_id in error bodies [Source: https://platform.claude.com/docs/en/api/errors]. Python and TypeScript expose it as _request_id (public despite the underscore); other SDKs surface it via raw-response accessors. Log it on every failure and include it in support tickets — it lets Anthropic trace the request end to end.

message = client.messages.create(...)
print(message._request_id)   # req_018EeWyXxfu5pfWkrYcMdjWG

On Claude Platform on AWS there are two IDs to capture: the AWS request ID (x-amzn-requestid, the primary one, indexed in CloudTrail) and the Anthropic request-id (secondary, for Anthropic support). When classifying errors programmatically, catch the SDK’s typed exceptions most-specific-first (e.g., anthropic.NotFoundError before the broader anthropic.APIStatusError), rather than string-matching messages — a chain that distinguishes retryable from non-retryable failures preserves information a single catch-all discards.

Beyond error responses, Claude Code exposes context-state diagnostics that are themselves a form of trace analysis. Running /context gives a breakdown of what’s filling the window — system prompt, tools, memory files, messages — which tells you why a window is bloated rather than merely that it is [Source: https://code.claude.com/docs/en/errors]. That breakdown is the trace you read to choose a remedy: /compact to summarize earlier turns, /clear to start fresh, /mcp disable to drop unused MCP tool definitions, or trimming an oversized CLAUDE.md.

Key Takeaway: Trace analysis reconstructs a failure from its artifacts — chiefly the request-id header (mirrored as request_id in errors), which you should log on every failure and include in support tickets. Catch typed SDK exceptions most-specific-first, and use context-state diagnostics like /context to read why a window is bloated before choosing a remedy.

Isolating integration-layer vs. model-output problems

The most important debugging judgment is where a problem lives. There are three broad origins, and the fix differs for each:

1. Integration-layer problems — 4xx errors, authentication failures, rate limits, malformed requests, MCP/context bloat. These are yours to fix in your own code and configuration.
2. Provider-side problems — 5xx and 529. These are Anthropic’s infrastructure, not your request.
3. Model-output problems — quality degradations, “the responses seem off.” These are usually conversation state, not the model itself.

The 429 vs. 529 distinction is the sharpest expression of the first-versus-second split, and a core exam concept. A 429 rate_limit_error is your responsibility: your organization exceeded its tier limits (it can also signal an acceleration limit from a sharp usage spike, which is why you ramp traffic gradually). Check for the retry-after header and wait exactly that duration [Source: https://platform.claude.com/docs/en/api/rate-limits]. A 529 overloaded_error is not your fault: it reflects Anthropic-wide capacity, high traffic across all users. It typically resolves within ~30–300 seconds (occasionally 5–15 minutes on model-launch days or peak US hours, 14:00–22:00 UTC). Recommended recovery for 529 is a bounded retry with jitter, a ~60-second circuit breaker, and tier/model fallback — but first verify you’re hitting the same model, route, auth path, and request shape before changing anything else [Source: https://platform.claude.com/docs/en/api/rate-limits].

The third origin — quality degradation with no error at all — is where Claude Code’s error reference is most instructive. When “responses seem lower quality than usual,” the cause is “usually conversation state rather than the model itself”; Claude Code does not silently change model versions [Source: https://code.claude.com/docs/en/errors]. Check, in order: model selection (/model — a prior choice or an ANTHROPIC_MODEL env var may have you on a smaller model), effort level (/effort — raise it for hard work), context pressure (/context, then /compact or /clear), and stale instructions (an oversized or outdated CLAUDE.md and bloated MCP tool definitions steer responses; /doctor flags oversized memory files).

Figure 10.5: Isolating the origin of a failure — integration-layer vs. provider-side vs. model-output

flowchart TD
    A["Something went wrong"] --> B{"Is there an error code?"}
    B -->|"4xx (except 409)"| C["Integration-layer — yours to fix<br/>429: honor retry-after, ramp traffic"]
    B -->|"5xx or 529"| D["Provider-side — Anthropic's<br/>retry w/ jitter, circuit breaker, /model fallback, check status page"]
    B -->|"No error, output off"| E["Model-output / conversation state"]
    E --> F["/context, /compact, /clear"]
    E --> G["/model, /effort"]
    E --> H["/rewind — remove bad turn instead of correcting in-thread"]

One recovery technique deserves emphasis. When a response goes wrong, rewinding — pressing Esc twice or running /rewind — works better than correcting in-thread. Correcting in-thread keeps the wrong attempt in context and anchors later answers to it, whereas rewinding removes the bad turn entirely. This is context engineering applied to debugging: the mistaken tokens are themselves the problem.

Claude Code retries transient failures — server errors, overloaded, timeouts, temporary 429s, dropped connections — up to 10 times with exponential backoff. Two classes are deliberately not retried because a retry can’t succeed: certificate-validation (TLS) failures, and server errors that arrive after visible output has already streamed (the partial response is kept with an incomplete-response notice, since re-running could double-execute tools). For 500/529 specifically, check status.claude.com (or the relevant provider status page), wait and retry, and run /model to switch models — capacity is tracked per model, which is why the guidance “Opus is experiencing high load, please use /model to switch to Sonnet” appears [Source: https://code.claude.com/docs/en/errors].

This maps directly onto the exam concept of isolating integration-layer versus model-output problems. The disciplined mental checklist: Is there an error code? If it’s a 4xx (except 409), it’s an integration-layer issue you fix in code/config. If it’s a 5xx or 529, it’s provider-side — retry and check the status page. If there’s no error but the output is off, it’s almost always conversation state or a model/effort misconfiguration — reach for /context, /compact, /clear, /model, /effort, and /rewind before you conclude the model itself has failed.

Key Takeaway: Isolate failures by origin: 4xx/auth/rate-limit/malformed-request and MCP/context bloat are integration-layer issues you fix (429 is your responsibility — honor retry-after); 5xx/529 are provider-side (529 is not your fault — retry with jitter, circuit breaker, and model fallback); and “off but no error” is almost always conversation state, best fixed with /context, /compact, /clear, and /rewind rather than assuming the model failed.

Chapter Summary

This chapter tied together the three disciplines that make a Claude application production-grade. Context engineering treats the window as a finite, degradable resource: context rot means recall declines as tokens grow, even before the hard limit, so you keep the window small through progressive disclosure and manage growth with three composable, first-party levers — compaction (summarize accumulated dialogue at an inference cost), clearing (prune re-fetchable tool output for free), and the memory tool (persist knowledge across sessions). Context isolation scales this further: subagents each get their own window and return only 1,000–2,000-token summaries, so multi-step workflows never accumulate every step’s tokens in one place — and the same separation keeps trusted operator instructions apart from untrusted, injectable content.

On the output side, structured outputs use grammar-constrained sampling — via JSON outputs and strict tool use — to guarantee schema-valid responses and eliminate parse failures, but defensive parsing still requires branching on stop_reason (refusals and truncation legitimately break the schema), and skepticism toward confident output demands you verify facts against sources regardless of how well-formed the JSON is. Finally, debugging turns on identifying error type from the HTTP taxonomy, reading traces through the request-id, and — above all — isolating whether a problem is integration-layer (yours: 4xx, 429), provider-side (Anthropic’s: 5xx, 529), or a model-output degradation (usually conversation state, best fixed by rewinding rather than correcting in-thread). The 429-versus-529 distinction is the single sharpest test of that judgment.

Key Terms

Chapter 11: Agent Architecture, Patterns, and Frameworks

By this point in the study guide you can prompt Claude, hand it tools, and manage its context. This chapter is about the layer that sits on top of those skills: how to structure a system that uses Claude to accomplish multi-step work. When should the developer control the sequence of steps, and when should the model? How do you split a large problem across several coordinated Claude instances? And which off-the-shelf framework — if any — should you reach for? These are exactly the architectural judgment calls the Claude Certified Developer exam probes, because they separate someone who can call an API from someone who can design a reliable agentic system.

Anthropic has published an unusually clear set of guidance on this topic, and the exam leans heavily on it. Three documents anchor everything in this chapter: “Building Effective Agents,” the “How we built our multi-agent research system” engineering post, and the “when and how to use multi-agent systems” guidance. We will work through their definitions, decision criteria, patterns, and framework recommendations in order — with tables, worked examples, and analogies to make the distinctions stick.

Workflows vs. Agents

The single most important vocabulary distinction in this domain comes from Anthropic’s “Building Effective Agents.” The article introduces an umbrella term and then splits it in two.

An agentic system is any system that uses an LLM together with tools to accomplish a task. Anthropic divides agentic systems into two architectural sub-types [Source: https://www.anthropic.com/engineering/building-effective-agents]:

• A workflow is a system “where LLMs and tools are orchestrated through predefined code paths.” The developer writes the control flow; the LLM fills in slots inside a structure the engineer designed. The sequence of steps is known before the program runs.
• An agent is a system “where LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks.” The model decides the path at runtime — which tool to call next, whether it is finished, whether to try again.

A useful analogy: a workflow is a railway and an agent is a self-driving car. The railway’s track is laid down in advance; the train is powerful but it can only go where the rails go, and that predictability is a feature — you know exactly which stations it will visit. The self-driving car chooses its own route in response to live traffic; it can reach destinations the railway never anticipated, but it is harder to predict, more expensive to run, and can take a wrong turn. Neither is “better.” You pick based on the trip.

Decision criteria for workflow versus agent

Anthropic states the core rule directly: “When more complexity is warranted, workflows offer predictability and consistency for well-defined tasks, whereas agents are the better option when flexibility and model-driven decision-making are needed at scale” [Source: https://www.anthropic.com/engineering/building-effective-agents]. Reframed as two decision rules:

• Choose a workflow when the task decomposes into fixed, predictable subtasks and you value predictability, consistency, lower cost, and lower latency.
• Choose an agent for open-ended problems “where it’s difficult or impossible to predict the required number of steps, and where you can’t hardcode a fixed path.” The canonical examples are complex coding tasks and computer-use automation.

The most reliable exam heuristic is the “can you predict the steps?” test. If you can draw the flowchart before running the program, build a workflow. If the number and order of steps depend on what the model discovers along the way, you need an agent.

Predictability vs. autonomy tradeoffs

The reason this is a genuine tradeoff, and not just “agents are more advanced,” is cost. Anthropic is explicit that agentic systems “often trade latency and cost for better task performance,” and that autonomy in particular “means higher costs, and the potential for compounding errors” [Source: https://www.anthropic.com/engineering/building-effective-agents]. An agent that loops ten times to solve a problem makes ten model calls, each carrying the growing context of the previous ones — and any mistake early in the loop can propagate and amplify through every later step.

Because agents plan and operate independently once a task is clear, Anthropic recommends extensive testing in sandboxed environments and appropriate guardrails before granting real autonomy [Source: https://www.anthropic.com/engineering/building-effective-agents]. The autonomy that makes an agent powerful is the same property that makes it capable of doing expensive or damaging things unsupervised.

When each approach fits — and the “start simple” principle

The article’s central thesis outranks the workflow-versus-agent choice itself: start simple. The discipline is to “find the simplest solution possible, and only increase complexity when needed.” For many applications no agentic system is required at all — “optimizing single LLM calls with retrieval and in-context examples is usually enough” [Source: https://www.anthropic.com/engineering/building-effective-agents].

This produces a clear escalation ladder that the exam expects you to know:

1. Single LLM call (optionally with retrieval and few-shot examples) — try this first.
2. Workflow — when the task genuinely needs multiple coordinated steps that you can predefine.
3. Agent — only when the path cannot be predicted in advance.

Figure 11.1: The “start simple” escalation ladder and the predictability test

flowchart TD
    Start["New task"] --> Q1{"Does a single well-crafted<br/>prompt (with retrieval /<br/>few-shot) solve it?"}
    Q1 -->|Yes| Single["Single LLM call<br/>(simplest — try first)"]
    Q1 -->|No| Q2{"Can you predict the<br/>steps and draw the<br/>flowchart in advance?"}
    Q2 -->|Yes| Workflow["Workflow<br/>predefined code paths<br/>predictable, lower cost"]
    Q2 -->|No| Agent["Agent<br/>model directs its own path<br/>flexible, higher cost"]
    Agent --> Guard["Requires sandboxing<br/>+ guardrails before autonomy"]

The phrase to memorize is “workflows before agents.” Reaching for an autonomous agent when a single well-crafted prompt would do is the most common architectural mistake, and it costs money, latency, and reliability for no benefit.

Every pattern above the single-call baseline is built on one foundational component: the Augmented LLM — a model enhanced with retrieval, tools, and memory. Anthropic recommends tailoring these capabilities to your use case and giving the model “an easy, well-documented interface” to each [Source: https://www.anthropic.com/engineering/building-effective-agents]. The Augmented LLM is the atom; workflows and agents are molecules assembled from it.

Key Takeaway: An agentic system is either a workflow (LLM steps on developer-defined code paths) or an agent (the LLM directs its own path at runtime). Choose a workflow for predictable, well-defined tasks and an agent for open-ended tasks whose steps you cannot predict — but always start with the simplest thing that works, escalating single-call → workflow → agent only as needed.

Agent Hierarchies

When a single agent is not enough, the next architectural step is to coordinate several agents in a hierarchy. Anthropic’s reference implementation is its internal multi-agent research system, and its design choices are the exam’s model answer for how hierarchies should work [Source: https://www.anthropic.com/engineering/multi-agent-research-system].

Manager/supervisor hierarchies

A manager/supervisor hierarchy (also called an orchestrator or lead-agent architecture) has one coordinating agent at the top that decomposes the problem, delegates pieces to subordinate agents, and synthesizes their results into a final answer. In Anthropic’s research system the structure is concrete:

• A lead agent (running Claude Opus 4) analyzes the incoming query, develops a research strategy, and spawns subordinate agents to pursue different aspects of the problem in parallel.
• The lead agent thinks through its approach and saves its plan to memory before delegating — a crucial safeguard, because the context window can exceed 200,000 tokens and get truncated; persisting the plan means it survives even if the conversation history is trimmed.
• After subordinates report back, the lead agent synthesizes their findings and decides whether the research is complete or whether it should spawn more subordinates and refine the strategy — an iterative loop, not a single fan-out.

The manager, in other words, owns strategy and integration; it does not do the detailed digging itself. This is the same shape as a research director assigning topics to analysts and then writing the executive summary from their memos.

Figure 11.2: Manager/supervisor hierarchy with parallel subagents in isolated contexts

graph TD
    Query["Incoming query"] --> Lead["Lead agent (Claude Opus 4)<br/>analyze, plan, delegate, synthesize"]
    Lead -->|"saves plan to memory"| Mem[("External memory")]
    Lead -->|"delegate in parallel"| S1["Subagent 1 (Sonnet 4)<br/>own context window + tools"]
    Lead -->|"delegate in parallel"| S2["Subagent 2 (Sonnet 4)<br/>own context window + tools"]
    Lead -->|"delegate in parallel"| S3["Subagent 3 (Sonnet 4)<br/>own context window + tools"]
    S1 -->|"condensed findings"| Lead
    S2 -->|"condensed findings"| Lead
    S3 -->|"condensed findings"| Lead
    Lead --> Decide{"Research complete?"}
    Decide -->|"No — spawn more,<br/>refine strategy"| Lead
    Decide -->|"Yes"| Answer["Final synthesized answer"]

The role of subagents

A subagent is a subordinate agent spawned by a manager to handle a delegated piece of the task. In the research system the subagents run Claude Sonnet 4, and the defining property is that each subagent operates in its own separate context window [Source: https://www.anthropic.com/engineering/multi-agent-research-system]. A subagent explores its slice of the problem, uses its own tools, and passes condensed findings back to the lead agent — not its entire raw working context.

That separate-context property is the whole point of the architecture, and it delivers three distinct benefits that the “when and how” guidance names explicitly [Source: https://claude.com/blog/building-multi-agent-systems-when-and-how-to-use-them]:

1. Context protection — delegating a lookup (say, retrieving a customer’s order history) keeps the manager’s own context clean, so it isn’t polluted with raw data it doesn’t need to reason over directly.
2. Parallelization — independent lines of inquiry run simultaneously across a much larger information space than one context window could hold.
3. Specialization — each subagent can carry a focused toolset and system prompt. This matters because agents given 20+ tools struggle to choose among them; a narrow, purpose-built toolset is more reliable, and conflicting behavioral modes can be separated into different agents.

The payoff is large but expensive. On internal research evaluations the multi-agent system achieved a 90.2% performance improvement over a single-agent Opus baseline [Source: https://www.anthropic.com/engineering/multi-agent-research-system]. The cost side is equally memorable: single agents use roughly 4× more tokens than a chat interaction, and multi-agent systems use roughly 15× more tokens than a chat. Token usage alone explained about 80% of the variance in performance across the evals — capability and token spend are tightly coupled. (The “when and how” guidance cites a more conservative 3–10× token multiplier for multi-agent over single-agent; either figure supports the same conclusion: multi-agent is powerful and costly.)

Delegating and improving task execution

Good delegation is a prompt-engineering problem, and vague delegation is the primary failure mode — it causes subagents to duplicate each other’s work or leave gaps. Anthropic’s rule is that every subagent task must include four things: a clear objective, an output format, guidance on which tools and sources to use, and explicit task boundaries [Source: https://www.anthropic.com/engineering/multi-agent-research-system].

The lead agent should also embed effort-scaling heuristics so it doesn’t over- or under-invest:

The most important design principle for building a good hierarchy is context-centric decomposition: split the work by shared context requirements, not by problem type [Source: https://claude.com/blog/building-multi-agent-systems-when-and-how-to-use-them]. A tempting but poor split is into “planning,” “implementation,” and “testing” agents — because those roles all need the same evolving context, forcing constant coordination and “telephone game” information loss. You should only split off a subagent when its context can be truly isolated.

The classic example of a clean split is the verification subagent: checking whether the work is correct is inherently low-context, so it isolates well. One caveat the exam may test: verification agents “often declare success prematurely,” so you must instruct them explicitly — for example, “Run the complete test suite before marking as passed.”

Key Takeaway: A manager/supervisor hierarchy has a lead agent that plans, delegates to subagents, and synthesizes results, while each subagent works in its own isolated context window to protect context, parallelize, and specialize. Delegate with clear objective, output format, tool guidance, and boundaries — and decompose by shared context, not by problem type.