Context Engineering for AI Coding: Context Windows, Compaction and Fresh Chats

Published On: July 22nd, 2026|Categories: AI, Programming|31 min read|

Most bad answers from a coding agent trace back to its context: too much, too little or the wrong things. This guide covers context windows and token limits across GPT, Claude and Gemini, what context engineering is, which parts make up an agent’s context, how compaction works, and when you should simply start a fresh chat.

Every language model has a hard ceiling on how much text it can consider at one time, and that ceiling is called the context window. It governs how long a conversation can run, how much code a model can read at once, and where an agent starts to forget. The size of that window varies dramatically between models, and the numbers change often. Understanding what the window is, and what it is not, matters more than memorizing any single figure.

Table of Contents

The window is measured in tokens

A context window is the maximum number of tokens a model can take as input at once. Since tokens are chunks of text usually smaller than a word, a window is not a word count or a character count but a token count. A window of two hundred thousand tokens holds roughly a hundred and fifty thousand words, give or take. This is the unit that actually matters, and thinking in tokens rather than pages is the right instinct. The window is a token budget, full stop.

Everything shares the window

The window is not just for your prompt, it has to hold the entire input context plus the model’s reply. That means the system prompt, tool descriptions, memory, conversation history, any documents, your request, and the generated output all draw from the same pool. A long conversation eats into it steadily, which is why very long chats eventually strain the limit. The reply competes for space too, so a huge input can leave little room to answer. Every token you spend on input is a token not available for anything else.

Current limits vary widely

The headline numbers differ by a lot across the major models, as documented in references like the Claude context window docs. GPT-5.2 offers a context window of about four hundred thousand tokens, with output capped separately around a hundred and twenty-eight thousand. Claude Sonnet 4.5 uses a two hundred thousand token window as standard, with a one million token option available in beta for large jobs. Gemini, in its 2.5 Pro form, leads the mainstream field at around two million tokens. These figures shift with each release, so always confirm the current number for the exact model you are using.

Bigger is not automatically better

It is tempting to assume the largest window wins, but that is a trap. Models suffer from a well-known lost-in-the-middle effect, where facts buried in the center of a very long input get less attention and are more easily missed. Broader writing on large language models notes that a huge window does not guarantee reliable recall across all of it. So a two million token window does not mean two million tokens of dependable memory. The advertised size and the effective size are different numbers, and the gap grows as inputs get longer.

Effective context is smaller than advertised

This distinction is worth dwelling on because it shapes real decisions. The effective context is the amount a model can actually use reliably, and it is typically well below the advertised ceiling. Stuffing a window to the brim often produces worse answers than a smaller, focused input, because the signal gets diluted. Treating the limit as a hard maximum to avoid, rather than a target to fill, is the healthier mindset. The window tells you what is possible, not what is wise.

Cost and latency scale with context

Filling a large window is not free in money or time. Because pricing is per token and the model does more work the more it reads, a bigger input costs more and takes longer to process. A prompt that fills a million-token window can be dramatically more expensive and slower than a focused one. This is a direct consequence of how the model reads its whole input before responding. Large windows are a capability to use sparingly, not a resource to max out by default.

Why the limits exist

The window has a ceiling because the computation grows sharply as the input gets longer. Processing attention across a very long context takes disproportionately more resources, which is why bigger windows are hard and expensive to build. The steady march from tiny early windows to millions of tokens reflects real engineering effort, not a simple dial. Understanding this explains why window size is a headline feature that companies compete on. It is a genuine technical achievement, with genuine costs attached.

Use a big window wisely

Having a large window changes what is possible but not what is smart. The skill is to use the space deliberately, filling it with high-signal, relevant content rather than everything you have, which is the heart of context engineering. A big window lets you include a whole codebase or a long document when you truly need to, without forcing you to always. Reaching for the maximum by reflex wastes money and can hurt quality. The window is a tool, and good use of a tool is selective.

When you hit the limit

Every long task eventually bumps against the ceiling, and something has to give. The usual moves are to drop the oldest content, summarize earlier parts into a compact note, or selectively keep only what is relevant, the same management that lets a stateless model hold a long conversation. Handling this gracefully is what keeps a long session coherent instead of confused. The limit is not the end of a task, it is the point where context management takes over. How you handle the ceiling matters more than how high it is.

Why this matters for coding agents

Coding agents hit the window faster than almost anything, because they read files and tool output constantly. An agent working through a real task can fill a large context in a single session, which is why context management is a core agent skill rather than an afterthought. A bigger window buys an agent more room, but it does not remove the need to manage that room. This is why serious agent work pays close attention to what stays in context. The window is the agent’s whole world, and it is always finite.

What Is Context Engineering and How Does It Differ From Prompt Engineering?

A couple of years ago, the hot skill was prompt engineering, the art of phrasing a request just right. That skill still matters, but the frontier has moved to something larger called context engineering. The shift is not just a rebrand, it reflects a real change in how you get good results from models, especially agents. Understanding the difference is one of the most useful upgrades you can make to how you work with AI.

Prompt engineering crafts the ask

Prompt engineering focuses on the instruction you write. It is about wording, structure, and framing, finding the phrasing that coaxes the best answer out of a model for a single request. Small changes genuinely matter here, since how you ask can change what you get. For one-off questions in a chat box, a well-crafted prompt is often all you need. It is a real skill, and it is not going away.

Context engineering manages everything the model sees

Context engineering zooms out to the whole input, not just your sentence. A model only knows what is in its context, as anyone who has grasped how a stateless model uses its context understands, so managing that context becomes the real lever. That means shaping the system prompt, the tool descriptions, the memory, the conversation history, and any documents you feed in. Your prompt is one piece of that, but only one piece. Context engineering is the discipline of getting the entire input right.

The context is far more than the prompt

When a modern model responds, it is reading much more than the message you just typed. There is a hidden system prompt setting its behavior, descriptions of the tools it can use, memory of earlier facts, the running history of the conversation, and often retrieved documents. All of that is the context, and all of it shapes the output. Focusing only on your one message ignores most of what actually determines the result. Context engineering is about deliberately designing that whole bundle.

Why the focus shifted

The move from prompts to context tracks the rise of agents. A single clever prompt is enough for a single reply, but an agent running for many steps needs a well-managed context to stay on track. As tasks grew longer and more tool-heavy, the bottleneck stopped being the phrasing of one instruction and became the management of a large, evolving input. Guides to building effective agents reflect this, spending far more energy on context than on wording. The problem got bigger, so the skill had to grow to match.

Prompt engineering is a subset

The cleanest way to relate the two is that prompt engineering is part of context engineering. Writing the instruction well is one component of getting the whole context right, not a competing discipline. A brilliant prompt buried in a messy, bloated context will still produce poor results. So the newer skill contains the older one rather than replacing it. You still craft the ask, but now you also curate everything around it.

How Do System Prompts, Tool Descriptions, Memory, and Conversation History Form the Input Context for AI Agents?

When an AI agent takes a step, the model behind it receives one big block of text and nothing else. That block, the input context, is not a single thing you wrote but an assembly of several distinct parts stitched together on every call. Knowing what those parts are, and how they combine, is the foundation of controlling how an agent behaves. It is the anatomy lesson behind all of context engineering.

The context is assembled every call

Because a model is stateless, the whole context has to be rebuilt and resent on every single step, exactly as conversation memory works under the hood. Nothing persists inside the model between calls, so the application gathers all the relevant pieces and concatenates them into one input each time. This assembly is invisible to you, but it is happening constantly. Understanding that the context is freshly built each call is the key to understanding everything else. The agent is only ever as good as the pile it is handed this turn.

The system prompt sets the stage

First in the assembly is usually the system prompt, a hidden instruction that defines who the model is and how it should behave. It sets the role, the tone, the rules, and the goals before the agent reads anything else. Because it comes first and frames everything after, it has outsized influence over the whole run. Two agents with identical everything else but different system prompts will behave very differently. This is the layer where much of an agent’s character and reliability is decided.

Tool descriptions advertise what it can do

Next come the descriptions of the tools the agent may use. Each tool is presented with a name, a purpose, and the inputs it expects, so the model knows what actions are available. These descriptions are not just documentation, they actively steer when and how the model reaches for each tool. Vague descriptions lead to misused tools, while precise ones lead to correct choices. In effect, the tool descriptions are part of the context that programs the agent’s behavior.

Memory carries facts forward

Memory is the part that lets an agent seem to remember things across time. Stored facts, whether about the user, the project, or earlier decisions, get pulled in and inserted into the context so the model can use them. This memory lives outside the model, in the application, and is added back into the input when relevant. It is how an agent recalls your preferences or a decision made yesterday. Without it, every session would start from nothing, since the model itself remembers nothing.

Conversation history provides continuity

The running history of the current session is another major component. Previous messages, actions, and their results are included so the model can follow the thread and build on what already happened. This is what makes an ongoing task coherent rather than a series of disconnected steps. As the session grows, this history is often the largest and fastest-growing part of the context. Managing it well is central to keeping a long agent run on track.

What Is Context Compacting, and Should You Trust It or Manually Manage Your Context?

Run a coding agent long enough and it will fill its context window, at which point it cannot simply keep piling on more text. The common solution is context compacting, where the agent summarizes its own history to free up room and keep going. It is a clever trick that lets long tasks continue, but it comes with a real cost. Deciding when to trust it and when to override it is a genuine skill in serious agent work.

What compacting actually does

Context compacting is the process of condensing a long history into a shorter summary. When the context window approaches its limit, the agent takes the older parts of the conversation and tool output and replaces them with a compact summary that captures the gist. This frees space for new work while preserving, in theory, what mattered from before. Some tools do this automatically in the background, so the agent can run far longer than the raw window would allow. It is essentially the agent taking notes and throwing away the transcript.

Why it is necessary

Compacting exists because context is finite and long tasks are not. An agent reading files, running tools, and iterating fills its window steadily, and without compaction it would simply hit a wall and stop. By summarizing, it can keep working on a task that would otherwise be too large to fit, which is the same pressure behind all context management. For long-running agents, some form of compaction is not optional, it is what makes extended sessions possible at all. The alternative is a hard stop the moment the window fills.

The cost of compacting

The catch is that summarizing is lossy by nature. When the agent compresses its history, details get dropped, and sometimes the detail that gets lost is the one that mattered later. A summary is an interpretation, so it can also subtly distort what happened, carrying a small error forward. After a compaction, the agent is working from a sketch of the past rather than the real record. This is why a long agent session can sometimes seem to forget a decision it made an hour ago. Compression buys room by trading away fidelity.

When automatic compacting is fine

For many tasks, trusting the automatic compaction is perfectly reasonable. If the work is long but not deeply dependent on precise early details, a good summary preserves enough to keep going well. Routine, exploratory, or loosely coupled tasks tend to survive compaction without trouble. In these cases, fighting the automatic behavior just adds friction for little gain. Letting the tool manage its own memory is the path of least resistance, and often the right one.

When to manage it yourself

For high-stakes or detail-sensitive work, leaning on automatic compaction is riskier. If a task hinges on specifics established early, a summary that blurs them can send the agent astray in ways that are hard to notice. Here it pays to manage context deliberately, keeping the crucial details explicit and not trusting them to a summary. This is the same instinct behind the broader discipline of not trusting agent output blindly. When the details matter, you want to control what survives rather than delegate it.

Why Does Putting Less in the Context Often Produce Better AI Coding Results?

There is a natural instinct, when working with a model, to give it everything that might possibly help. More information feels safer, like it can only improve the odds of a good answer. In practice, the opposite is often true, and a leaner context regularly beats a stuffed one. Understanding why turns context from something you pile up into something you carefully curate.

Models get lost in too much

The core reason is that models do not attend evenly to everything you give them. A large input suffers from the lost-in-the-middle effect, where important details placed among a lot of other text get overlooked. Writing on large language models documents this degradation on long inputs directly. So a fact the model needs can be present in the context and still effectively ignored, drowned out by everything around it. Presence is not the same as attention. A model can have the answer sitting right there in its context and still miss it entirely. The window guarantees the model can see something, not that it will actually look.

Noise dilutes the signal

Every irrelevant thing you add is not neutral, it is noise. Extra files, tangential history, and just-in-case documents all compete for the model’s attention with the parts that actually matter. The more filler surrounds the key information, the harder the model has to work to find and use it. Cutting the noise raises the concentration of signal, making the important content easier to act on. A focused context is not just smaller, it is clearer.

Irrelevant context can mislead

Worse than diluting, off-topic context can actively steer the model wrong. If you include code or instructions that are not relevant to the current task, the model may latch onto them and produce an answer shaped by the wrong material. It cannot always tell what you meant to be relevant, so everything you include is a hint it might follow. Removing the irrelevant is not just tidying, it is preventing the model from being misled. What you leave out protects the answer as much as what you put in.

Precision beats completeness

The goal of context is not to be complete, it is to be precise. A small set of exactly the right files and facts gives the model a clear target, while a complete dump of the whole project gives it a haystack. This is the core insight of good context engineering, which is about selection far more than accumulation. Feeding a model well is an act of editing, choosing what to leave out as much as what to include. The best context is the minimum that fully answers the task.

Less context costs less

There is a practical bonus on top of the quality gain. A smaller context means fewer tokens, which means lower cost and faster responses, since the model does less work reading less input. So curating down is that rare choice that improves quality and reduces cost at the same time. Overstuffing pays twice, once in worse answers and once in a bigger bill. Trimming context is a win with no real downside when done well. Few optimizations improve quality and cost at the same time, and this is one of them.

Should You Meticulously Craft Your Context Engineering Strategy or Let AI Agents Self-Correct: The 2025 vs. 2026 Mindset?

There is a real tension in how people approach context today. One camp says you should meticulously engineer every piece of context an agent sees, controlling it tightly. The other says the models are now good enough to manage their own context and correct their own mistakes, so heavy hand-crafting is wasted effort. Both were right at different moments, and the shift between them is one of the more interesting mindset changes in the field.

The 2025 mindset: craft everything

In the earlier phase, careful context engineering was close to mandatory. Models were more easily thrown off by clutter, so getting good results meant curating exactly what the agent saw, as the discipline of context engineering describes. You hand-picked files, wrote precise instructions, and managed the window closely, because the model would not reliably recover from a messy context on its own. This was real, skilled work, and it made the difference between success and failure. Control was the safest strategy when models were less robust.

The 2026 mindset: trust self-correction

As models improved, a different attitude gained ground. Newer agents got better at noticing their own mistakes, asking for missing information, and managing their own context through summarization and retrieval. This led some to argue that meticulous manual crafting is now over-engineering, since the agent will sort out much of it itself. In this view, you give the agent a goal and let it figure out the context it needs, intervening less. The rise of more capable, self-managing agents made this a credible position rather than wishful thinking.

Both contain a real truth

The honest read is that neither extreme is right, and both capture something true. Models genuinely have gotten better at self-correction, so a lot of the fussy manual work that once mattered now buys less. But context still shapes results powerfully, and an agent handed a bad context will confidently do the wrong thing no matter how capable it is. The pendulum swung, but it did not swing all the way. The skill is knowing how much crafting each situation actually warrants.

Self-correction has real limits

Trusting an agent to fix its own context works right up until it does not. An agent can only self-correct within what it can perceive, so if a crucial fact was never provided, no amount of autonomy will conjure it. Self-management like context compaction also loses detail, and an agent confidently reasoning from a flawed summary will not notice the flaw. Over-trusting self-correction quietly reintroduces the very failures careful context was meant to prevent. Autonomy is not the same as omniscience.

Crafting still pays where it counts

At the same time, meticulous crafting everywhere is genuinely wasteful now. Spending an hour perfecting the context for a throwaway task is effort the agent would have handled fine on its own. The move is to concentrate your crafting where stakes are high and let the agent self-manage where they are low. This mirrors the lesson that less, well-targeted context beats more, applied to your own effort rather than the tokens. Craft deliberately, not reflexively.

When to Reset the Context Window and Start a New Chat With an AI Coding Agent

How you manage an AI coding agent’s context window decides how sharp, fast and expensive it is. As a conversation grows, the window fills with history, some useful and much not, and output quality quietly degrades. This guide explains why that happens, the signs that tell you to reset and start a new chat, what a reset keeps and loses, and the everyday habits that keep the window lean between resets.

What the context window holds

The context window is everything the agent currently has in view: your instructions, the conversation so far and the code it is working with. It is the agent’s working memory for the task, and it has a limit. Everything in that window shapes the next response, for better or worse, as the context window documentation explains.

Why context drives quality, speed and cost

Because the window drives everything the agent does, managing it is managing efficiency. What sits in the window shapes the quality of the output and the speed of the response. Models also bill by tokens, so a bigger context means more tokens and a higher cost for every request. A lean, relevant context gives you better answers faster and cheaper, while a crowded one hurts all three, and under heavy use this adds up in your overall tool cost.

Why the window fills up and output degrades

Over a long session the window accumulates. Every message, every piece of code, every wrong turn and correction stays in it. A conversation that started focused becomes crowded with history that no longer matters, as dead ends, abandoned approaches and resolved detours dilute the signal the agent needs. That noise is why less context often works better. A crowded window is a distracted agent.

This buildup is natural, but it is costly. The noise competes with the signal, so the same agent that was sharp an hour ago starts drifting, and every extra token of stale history also makes each request slower and more expensive. Understanding why the window fills explains why a reset helps: the problem is not the model getting tired, it is the model reading a cluttered page.

Signs it is time to start a new chat

The clearest trigger is degrading quality. When the agent starts giving vaguer answers, forgetting earlier instructions, repeating mistakes or contradicting itself, the context has likely become cluttered enough to hurt. The decline is gradual and easy to miss, so learning to notice it is the practical skill. The moment output quality slips is the signal, and the situations below are where it shows up most often.

You switch to a new task

Moving to unrelated work is the most natural reset point, because a finished task’s context no longer helps and can even mislead the agent on the new one. A fresh chat for each new task is one of the easiest and most effective habits you can build.

Quality drops in the middle of a task

If the agent was doing well and has started producing worse code or losing the thread, clearing the context and re-establishing the task often restores its earlier quality, because the accumulated noise is gone. Do not push a degrading agent harder. Resetting is often faster than fighting a cluttered context. A fresh start beats a struggling one, and a mid-task reset can rescue a session that is going downhill.

The agent loops on the same mistake

A strong, specific trigger is the agent repeating an error no matter how you correct it. The context, including its own wrong attempts, keeps steering it back to the mistake, and a fresh chat removes that pull. Clearing the context is often the fastest way out of a loop, as with guiding a stuck agent. When corrections stop working, start fresh.

The session has run long

Even without obvious problems, a very long session is worth resetting periodically. After many exchanges the context is crowded whether or not quality has visibly dropped, so starting fresh for the next chunk of work keeps the agent sharp before decline sets in. Preventive resets avoid the slow degradation entirely, which makes a periodic clean slate good hygiene on any marathon session.

What a reset keeps and what it loses

Resetting loses the conversation, not your work. Your code, files and any context file remain, so a fresh chat still has the project and only drops the accumulated history. To carry forward what matters, restate the current task and point the agent at the relevant code or the context file. You keep the substance and shed the noise, which makes a reset painless.

A short handover message works well in practice. State what you are building, what is already done, what the next step is and which files matter, then let the agent read those files and the context file. The new chat starts clean but informed, and because none of the earlier dead ends come along, it often solves in a few messages what the old session kept circling.

Keep the context lean between resets

Resets are only part of the job. The core habit is giving the agent what it needs, the relevant code and the current goal, and leaving out unrelated history and files. A focused window gives the agent a clear target and less room to drift, improving quality and speed at once. Resist dumping everything in.

Within a session, be willing to trim. When a conversation has piled up abandoned approaches and resolved tangents, refocus it or start fresh rather than letting that history linger and dilute the signal. A window full of resolved detours is a less efficient one.

Use a context file instead of repeating yourself

Re-explaining your project in every session wastes effort and tokens. Record your stack, conventions and structure in a context file the agent reads, and you set them once while the agent applies them in every chat. This keeps guidance consistent and is part of good context engineering. Use the file for durable context and the conversation for the task at hand. It also makes resets cheaper, since each fresh session starts informed.

Scope each request tightly

A tight, well-scoped task keeps the relevant context small and the agent focused, so it responds faster and more accurately. Asking for one clear thing at a time is context management at the level of each interaction.

Learn your tool’s context controls

Modern agents and editors let you start fresh chats, reference specific files and control what the agent sees. Learning these controls lets you manage the window deliberately instead of letting it fill unchecked. The better you know them, the easier it is to turn good context habits into everyday actions.

Common questions

What is a context window?

The maximum number of tokens a model can take as input at once. It must hold the whole input, including system prompt, history, and documents, plus the model’s reply.

What are the token limits for GPT-5.2, Claude Sonnet 4.5, and Gemini?

GPT-5.2 is around 400,000 tokens, Claude Sonnet 4.5 is 200,000 with a 1 million beta option, and Gemini 2.5 Pro is about 2 million. These change with each release, so verify the current figure.

Is a bigger context window always better?

No. Models suffer a lost-in-the-middle effect where facts buried in long inputs get missed, so effective context is smaller than advertised. Large windows also cost more and run slower.

Why does a large context cost more?

Pricing is per token and the model does more work the more it reads, so a bigger input is both more expensive and slower to process. Filling a huge window by default wastes money.

What happens when you reach the context limit?

The application drops the oldest content, summarizes earlier parts, or keeps only what is relevant. Handling this gracefully is what keeps a long session coherent.

What is context engineering?

The practice of managing everything a model sees, including the system prompt, tool descriptions, memory, conversation history, and documents, so that its limited context holds exactly the right information.

Why did the focus shift from prompts to context?

Because agents run for many steps and need a well-managed, evolving context to stay on track. A clever one-off prompt is not enough once tasks get long and tool-heavy.

Why is more context not always better?

Models can lose track of key facts buried in a long input, so padding the context with everything can hurt results. The goal is high signal, giving the model what it needs and little else.




Related Articles

If you enjoyed reading this, then please explore our other articles below:

More Articles

If you enjoyed reading this, then please explore our other articles below: