What is context engineering for AI agents?
Context engineering for AI agents is the system-level work of curating the tokens available during inference. It goes beyond writing a prompt: the agent must decide which instructions, tool definitions, external data, memories, examples and conversation turns matter for this step while fitting a finite attention budget.
Anthropic describes the goal as finding the smallest set of high-signal tokens that maximize the chance of the desired outcome. Larger context windows help, but they do not eliminate context rot, stale information, irrelevant tool output or conflicting facts.
What belongs in an agent’s context?
Treat context as an assembled view, not a single prompt string. Each layer has a different owner, freshness requirement and security boundary. The agent should receive the smallest combination that helps with the current decision.
| Context layer | Question it answers | Typical owner |
|---|---|---|
| Instructions | How should the agent behave? | Product and system prompt |
| Tools | What can the agent do next? | Tool registry and authorization |
| Retrieved documents | What external knowledge is relevant? | RAG or search system |
| Durable memory | What do we know about this user, account or project? | Memory layer |
| Execution state | What has happened in this run? | Agent runtime or checkpointer |
| Source-of-truth data | What is true and authorized right now? | Application database or API |
Do not copy one layer into another by default. An account plan returned from the billing API is current truth, not a memory the model should be allowed to overwrite.
How is context engineering different from prompt engineering or memory?
Prompt engineering writes and organizes instructions. Memory stores selected information outside the model. Context engineering decides which instructions, memories and other sources enter this particular inference call, and in what form.
A memory system can provide excellent facts and still be used badly if every memory is injected into every turn. Context engineering turns that store into a small, relevant view with provenance and a token budget.
| Practice | Main output | Failure when isolated |
|---|---|---|
| Prompt engineering | Clear task and behavioral instructions | Good wording cannot supply missing or stale facts |
| RAG | Relevant external document chunks | Documents do not remember changing user decisions |
| Agent memory | Durable facts, preferences and episodes | Unbounded retrieval floods the context |
| Context engineering | The complete token view for the next step | Requires reliable retrieval, state and evaluation |
How should an agent assemble context for each step?
Start with the current task, then retrieve only the evidence and memory that can change the next decision. Keep stable instructions ahead of volatile data and mark the source and trust level of every model-readable block. Budget for output and tool calls as well as input.
| Step | Question | Action |
|---|---|---|
| 1. Frame | What decision is the agent making now? | State the task and success condition |
| 2. Resolve | Which user, tenant and permissions apply? | Resolve identity before retrieval |
| 3. Retrieve | What facts could change the answer? | Query memory, RAG and live tools |
| 4. Rank | Which evidence is current and relevant? | Deduplicate and rerank |
| 5. Compress | What is the smallest faithful representation? | Summarize or select extracts |
| 6. Delimit | Where does untrusted data begin and end? | Label sources and instructions |
| 7. Verify | What remains missing or uncertain? | Ask, call a tool or stop |
Should context be retrieved up front or just in time?
Use up-front retrieval for small, stable, frequently needed context. Use just-in-time retrieval when the task requires exploration or when the available corpus is too large to preload. Many production agents use a hybrid: a compact known profile at the start and tools for deeper exploration.
Just-in-time retrieval is slower and depends on good navigation heuristics. Give the agent precise tools and identifiers so it can inspect the right file, record or query without loading whole systems. Keep a compact evidence ledger for long-running work.
- Preload the authenticated profile, current task and critical constraints.
- Use tools for large datasets, current status and deep source material.
- Return identifiers and concise summaries before full records.
- Record which sources were consulted so later steps do not repeat work.
How should long-running agents compact context?
Compaction preserves the meaning needed to continue while discarding redundant messages and tool output. It should retain decisions, constraints, unresolved failures, important identifiers and evidence. Aggressive summaries save tokens but can erase a detail whose significance appears several steps later.
Structured notes are a second persistence layer. The agent writes durable task state outside the window, then retrieves that note when needed. A managed memory layer is the product-level version of the same idea when facts must persist across sessions and users.
What are the most common context engineering failures?
Most failures are not caused by too little context. They come from retrieving the wrong source, injecting too much context, losing the boundary between instructions and untrusted data, or treating a summary as a permanent source of truth.
| Failure | Symptom | Fix |
|---|---|---|
| Tool flood | The agent chooses similar tools unpredictably | Reduce and rename the tool set |
| Memory flood | Every known fact appears in every prompt | Retrieve per task with a token budget |
| Stale truth | The model repeats an old plan | Version or supersede the fact; read live sources |
| Context poisoning | A document or tool output changes agent behavior | Delimit untrusted data and require confirmation |
| Compaction loss | The agent forgets a constraint after a reset | Preserve decisions and identifiers in structured notes |
| Hidden authorization | Tenant scope disappears from a sub-agent call | Pass trusted principal context outside prompt text |
How do you evaluate context engineering?
Evaluate the final answer and the context path that produced it. The useful metrics are required-evidence recall, irrelevant context, source freshness, prompt tokens, retrieval latency, tool selection accuracy and task success. Keep tenant-boundary and deletion failures as hard gates.
- Did the context contain every required fact and no foreign-tenant records?
- Was the latest valid fact selected over a superseded value?
- How many tokens were sent compared with a full-history baseline?
- Did adding more context improve the action, or only increase cost?
- Could the agent identify missing evidence and request it explicitly?
What is a practical context architecture?
Separate stable instructions, source-of-truth APIs, document retrieval, durable memory and execution state. Assemble them per step under a token budget. Compress long work into structured notes, keep authorization outside the model and evaluate both the selected context and the resulting action.
Frequently asked questions
Is context engineering just prompt engineering?
No. Prompt engineering organizes instructions. Context engineering decides which instructions, tools, documents, memories and state enter each model call.
Does a larger context window remove the need for context engineering?
No. More tokens can increase cost and introduce irrelevant or conflicting evidence. Curating high-signal context remains necessary even with larger windows.
How does memory fit into context engineering?
Memory supplies durable facts about users, accounts and projects. Context engineering decides which memories are relevant now and how much of the memory graph to inject.
Should I use RAG, memory or both?
Use RAG for external documents and memory for user or agent-specific facts. Many agents use both and keep source-of-truth application data in a third layer.
What should be compacted first?
Remove redundant tool output and low-value chatter first. Preserve decisions, constraints, unresolved work, identifiers and source references.
How do I know the context is helping?
Compare task success, required-evidence recall, irrelevant context, tokens and latency against no-context and full-history baselines on the same evaluation set.