Effective context engineering for AI agents

생각 덩어리

프롬프트 엔지니어링에서 컨텍스트 엔지니어링으로 — 질문 자체가 바뀌었다

Building with language models is becoming less about finding the right words and phrases for your prompts, and more about answering the broader question of "what configuration of context is most likely to generate our model's desired behavior?"

Context engineering refers to the set of strategies for curating and maintaining the optimal set of tokens (information) during LLM inference, including all the other information that may land there outside of the prompts.

Context engineering is the art and science of curating what will go into the limited context window from that constantly evolving universe of possible information.

Context rot — 토큰이 늘수록 회상 정확도가 줄어든다

Studies on needle-in-a-haystack style benchmarking have uncovered the concept of context rot: as the number of tokens in the context window increases, the model's ability to accurately recall information from that context decreases.

Context, therefore, must be treated as a finite resource with diminishing marginal returns.

어텐션 예산 — n² 관계가 만드는 구조적 희소성

Like humans, who have limited working memory capacity, LLMs have an "attention budget" that they draw on when parsing large volumes of context. Every new token introduced depletes this budget by some amount, increasing the need to carefully curate the tokens available to the LLM.

LLMs are based on the transformer architecture, which enables every token to attend to every other token across the entire context. This results in n² pairwise relationships for n tokens.

models develop their attention patterns from training data distributions where shorter sequences are typically more common than longer ones.

These factors create a performance gradient rather than a hard cliff: models remain highly capable at longer contexts but may show reduced precision for information retrieval and long-range reasoning compared to their performance on shorter contexts.

지도 원리 — 최소의 고신호 토큰 집합

good context engineering means finding the smallest possible set of high-signal tokens that maximize the likelihood of some desired outcome.

시스템 프롬프트 — 골디락스 존의 적정 고도

System prompts should be extremely clear and use simple, direct language that presents ideas at the right altitude for the agent. The right altitude is the Goldilocks zone between two common failure modes.

At one extreme, we see engineers hardcoding complex, brittle logic in their prompts to elicit exact agentic behavior. ... At the other extreme, engineers sometimes provide vague, high-level guidance that fails to give the LLM concrete signals for desired outputs or falsely assumes shared context.

Note that minimal does not necessarily mean short; you still need to give the agent sufficient information up front to ensure it adheres to the desired behavior.

툴 — 사람이 못 고르면 에이전트도 못 고른다

Because tools define the contract between agents and their information/action space, it's extremely important that tools promote efficiency, both by returning information that is token efficient and by encouraging efficient agent behaviors.

One of the most common failure modes we see is bloated tool sets that cover too much functionality or lead to ambiguous decision points about which tool to use. If a human engineer can't definitively say which tool should be used in a given situation, an AI agent can't be expected to do better.

예시 — 엣지케이스 나열 대신 대표 예시 큐레이션

teams will often stuff a laundry list of edge cases into a prompt in an attempt to articulate every possible rule the LLM should follow for a particular task. We do not recommend this.

we recommend working to curate a set of diverse, canonical examples that effectively portray the expected behavior of the agent. For an LLM, examples are the "pictures" worth a thousand words.

에이전트의 단순한 정의 — 루프 안에서 자율적으로 툴 쓰는 LLM

we've gravitated towards a simple definition for agents: LLMs autonomously using tools in a loop.

As the underlying models become more capable, the level of autonomy of agents can scale: smarter models allow agents to independently navigate nuanced problem spaces and recover from errors.

Just-in-time 컨텍스트 — 데이터 대신 가벼운 식별자만 들고 있기

agents built with the "just in time" approach maintain lightweight identifiers (file paths, stored queries, web links, etc.) and use these references to dynamically load data into context at runtime using tools.

The model can write targeted queries, store results, and leverage Bash commands like head and tail to analyze large volumes of data without ever loading the full data objects into context.

This approach mirrors human cognition: we generally don't memorize entire corpuses of information, but rather introduce external organization and indexing systems like file systems, inboxes, and bookmarks to retrieve relevant information on demand.

메타데이터가 행동을 다듬는다 — 점진적 노출(progressive disclosure)

To an agent operating in a file system, the presence of a file named test_utils.py in a tests folder implies a different purpose than a file with the same name located in src/core_logic/

Each interaction yields context that informs the next decision: file sizes suggest complexity; naming conventions hint at purpose; timestamps can be a proxy for relevance.

This self-managed context window keeps the agent focused on relevant subsets rather than drowning in exhaustive but potentially irrelevant information.

하이브리드 전략 — CLAUDE.md는 미리, glob·grep은 그때그때

Of course, there's a trade-off: runtime exploration is slower than retrieving pre-computed data.

CLAUDE.md files are naively dropped into context up front, while primitives like glob and grep allow it to navigate its environment and retrieve files just-in-time, effectively bypassing the issues of stale indexing and complex syntax trees.

Given the rapid pace of progress in the field, "do the simplest thing that works" will likely remain our best advice for teams building agents on top of Claude.

Compaction — 무엇을 버릴지가 기술이다

But it's likely that for the foreseeable future, context windows of all sizes will be subject to context pollution and information relevance concerns—at least for situations where the strongest agent performance is desired.

Compaction is the practice of taking a conversation nearing the context window limit, summarizing its contents, and reinitiating a new context window with the summary.

The art of compaction lies in the selection of what to keep versus what to discard, as overly aggressive compaction can result in the loss of subtle but critical context whose importance only becomes apparent later.

once a tool has been called deep in the message history, why would the agent need to see the raw result again?

먼저 recall을 최대로 끌어올려 트레이스의 핵심 정보를 빠짐없이 담게 한 뒤, 군더더기를 깎아 precision을 높이는 순서를 권한다. 가장 안전하고 가벼운 형태가 tool result clearing — Claude Developer Platform에 기능으로 출시됐다.

구조화된 노트 — Pokémon을 며칠씩 이어 플레이하는 기억법

Structured note-taking, or agentic memory, is a technique where the agent regularly writes notes persisted to memory outside of the context window. These notes get pulled back into the context window at later times.

tracking objectives like "for the last 1,234 steps I've been training my Pokémon in Route 1, Pikachu has gained 8 levels toward the target of 10."

Without any prompting about memory structure, it develops maps of explored regions, remembers which key achievements it has unlocked, and maintains strategic notes of combat strategies that help it learn which attacks work best against different opponents.

After context resets, the agent reads its own notes and continues multi-hour training sequences or dungeon explorations.

서브에이전트 — 탐색은 격리하고 증류만 돌려받는다

Each subagent might explore extensively, using tens of thousands of tokens or more, but returns only a condensed, distilled summary of its work (often 1,000-2,000 tokens).

the detailed search context remains isolated within sub-agents, while the lead agent focuses on synthesizing and analyzing the results.

세 기법의 선택 기준은 과업의 성격이 정한다:

• Compaction maintains conversational flow for tasks requiring extensive back-and-forth;
• Note-taking excels for iterative development with clear milestones;
• Multi-agent architectures handle complex research and analysis where parallel exploration pays dividends.

결론 — 컨텍스트는 귀하고 유한한 자원

the challenge isn't just crafting the perfect prompt—it's thoughtfully curating what information enters the model's limited attention budget at each step.

We're already seeing that smarter models require less prescriptive engineering, allowing agents to operate with more autonomy.

But even as capabilities scale, treating context as a precious, finite resource will remain central to building reliable, effective agents.