Context is Everything

One of the first things I learned when I started working with agents and harnesses is that context is finite and must be protected at all costs.

Agent amnesia

When I built my first personal agent, in what now feels like a lifetime ago, I immediately wanted to tailor and customise everything about it. Every day I had a new instruction for it. From now on, let’s work this way. Always write to me like this. Add X, then Y, then Z, over and over. I knew nothing about how context is structured or what polluting it does, so I kept adding.

Then, past some point, the agent started forgetting. Not one rule. Every instruction I had given it. So I did the obvious thing and asked it why it kept forgetting things that were, in theory, sitting in its system prompt.

The model was not being stubborn. Anthropic’s guidance for Claude Code says so plainly: if the agent keeps breaking a rule that is in the file, the file is probably too long and the rule is getting lost. Nothing in the model had changed. I had changed what it was looking at, one reasonable line at a time.

Protect it at all costs

That is when I started reading. What context is, how it works, how to protect it, and how to get around the limits of a window that only holds so many instructions an agent can reasonably be expected to follow. The agent does not read a bucket of tokens. It works on a surface, and what is on that surface, in what order and at what density, decides what it notices and what it ignores.

Once you accept that context has to be protected at all costs, a lot of design decisions stop being optional. Most of how we build agents now, if we want them to do real work, follows from that one idea. The rest of this piece is the conclusions I drew from it.

One job, one context

The first conclusion is sub-agents: small agents with one specific task each and a clean context of their own, so nothing they read lands on the main thread. Mine do the code exploration, the memory lookups and the research. A research run fans out across several of them, and their findings come back through a hierarchy of summaries, so the main thread reads only the final output. That thread runs the strongest model, the one that judges and makes the decisions, and its context should go on judging.

The same shape wraps verbose tools. A sub-agent runs the tool, reads the forty lines it prints, and returns the one line that matters. The forty lines never reach the main thread. I found this helped a lot with the quality and consistency of what the agent does.

Compaction has got much better in the latest models and harnesses. It still does not beat not dumping everything into one thread in the first place. Compaction cleans the junk up later. A sub-agent means it never arrives.

Only what the task needs

Once each sub-agent has a role, a second conclusion follows: its context can be tailored. Each one gets just the information its task needs. The code explorer gets the repository and its conventions, not how I like to be written to. The researcher gets the question and when to stop. Nothing else competes for its attention.

The same care goes into what comes back. The distillation is the product, not the search. A sub-agent that returns its transcript has only moved the pollution one hop. The brief has to say what shape the answer takes: a conclusion, the three files that matter, and the dead ends by name so nobody walks them twice.

What you get back

Costs drop, and that is real. On a local harness a single thread fills up if you keep using it, and you compact; protecting the context stretches that thread a long way. But the bigger win is consistency. The main session reads a higher-quality conversation, with less noise and less junk that has nothing to do with the task, so its behaviour holds. It does the thing you asked, not the thing it read three thousand tokens ago.

None of this is fully solved. Models still regress in how they handle compaction, and huge context windows still cost what they cost.

Two agents that share a context share a bias.

Suppose compaction and cost were both solved tomorrow. Protecting context would still pay, because of what it does to judgment. An agent that reviews, checks or critiques work does it better from a clean context than from the thread that produced the work. The builder’s thread carries every assumption the builder made. A reviewer that starts fresh, with the diff and the brief and nothing else, does not inherit them.

The same logic leads to mixing models and harnesses. A second model, in a second harness, reading the same work brings a different level of abstraction and a different set of blind spots. Two agents that share a context share a bias. Two that do not, do not.

Context is everything

So this is all to say that context is everything. If you are building an agent, or customising your own coding agent, those are my two cents. I would like to know the first thing you stopped letting into the main thread, and what changed when you did.