Drew Breunig's "State of Context Engineering" talk: four ways long contexts fail and why the job is knowing when to be deterministic
- Breunig names four failure modes of long context: context poisoning (a hallucination or bad fact the model cannot get past), context distraction (the model leans on the context over its base knowledge), context confusion (extraneous facts skewing answers), and context clash (new information conflicting with what is already there).
- Context windows jumped from about 128K tokens with GPT-4o to 1M with Gemini 1.5 Pro, which teased a 10M-token model in the lab; Llama 4 Scout shipped 10M first but inference was a challenge, it never caught on, and no further Llama followed.
- His one-line summary of the job: when building harnesses and workflows, choose when to be deterministic (what goes in code) and when to be probabilistic (what the model drives).
- Breunig argues the problem will not age out. He told O'Reilly in June 2025, while pitching The Context Engineering Handbook, that giving clear, consistent, low-noise instructions stays hard no matter how good models get, because models are not people and their construction produces its own failure modes.
Hacker News opinions
Someone tells me their coding agent is acting weird and the model got nerfed, and my first question is always how big their context is. Though too little information also makes it behave strangely. Most agent anti-patterns work fine at a medium amount of context and only fall apart at the extremes.
The FireRed tea task was baked into the weights, so how is that context poisoning? That seems like mixing two different things. Poisoned context should mean junk sitting in the context window, not something in the model itself.
I thought about using a fast decision model to prune before compaction and dropped it. I am not bottlenecked on speed, and pruning turn C out of A through E kills the cache for D and E. Correct me if I am wrong, I have not tested it.
Decision models for pruning really are not worth it. You pay to load the data into the decision model's context to save loading it into the generation model's context, and that math only works in a niche that barely exists. I just use a harness extension that auto-compacts right before the cache expires, combined with compact-when-full.
I use dynamic compaction in an opencode plugin and added eviction on top. Everything runs local, so keeping context near 128k is a real speed win and gets one agent through multiple tasks in the same repo. It breaks around 6M, but that is more probabilistic anyway.
It is because the cache comes from the prefix. f(f(f(A,B),C),D) is a very different thing from f(f(A,B),D).
Sample size of one, but as my context grows the model genuinely seems to get dumber. I only use the chat interface, no harness, and after a few turns the frontier models do stupider things.
Bump that to sample size two, I have seen the same thing and it shaped how I built my coding harness.
DeepSeek v4.1 flash starts telling me we should wrap up and resume next session, that the context budget is nearly gone, as the window fills. I just make it write a handoff file and clear the context myself now.
This is already well documented, there are papers on it and Anthropic says in their docs you want the minimum context needed for the goal. Quality drops the more you dump in.
That plus Claude's stingy five-minute cache TTL, unless you pay a lot more for the one-hour option, means some sessions cost ten times what they should on cold caches.
The 200k to 300k reset point does not hold for newer models in my experience. Context fills to 250k fast just from reading the codebase, and the models do worse without it, and I do not notice much degradation until 700k or 800k. Long-horizon RLVR tasks probably helped here.
Giving the agent an index of the code it can search beats grep, in my experience it works wonders.
Until a few months ago I built my own harness, but OpenAI and Anthropic shipped theirs and now mine mostly conflicts with them. Running with no setup at all performed better, and if I do add anything it is logging and flow, not rules or guardrails.
That last line about deterministic versus probabilistic is the best part of the talk for me. I keep asking myself whether to have the AI do the task or have the AI build something that does the task, and the answer mostly comes down to how deterministic the task is.
I like the deliberate forgetting angle. Checkpointing into artifacts, code plus design and an execution plan, and starting fresh context from those works well. Oddly the design doc and plan have become outputs of the coding process rather than inputs.
The human analogy is what convinces me. Perfect memory seems to impede creativity in people, so strategic forgetting is not a bug to fix in models. The computer already handles perfect recall, and context engineering stays relevant even with loop engineering.