Context is attention, not memory.

The spec sheets make it sound like storage. 128,000 tokens. 256,000. A million. Bigger number, more room, surely better.

Here’s the mistake I watch people make with that number: they treat it like a filing cabinet. Stuff everything in — the full case history, every attachment, the whole knowledge base — because the window is big enough to hold it.

But a context window — the text available to a model for one task — isn’t memory. It’s closer to attention. And attention has a shape.

There’s a well-documented behaviour called “lost in the middle”: models can pay more attention to what is at the beginning and end of a long input, and miss what is buried in the centre. The document was in the window. The model had it. It still didn’t use it — the same way you can read every page of a forty-page pre-read at 11pm and retain the first two and the last one.

Once you think of it as attention, the practical questions change. Not “will it fit?” but “will it be noticed?”

Does the agent need the whole case history, or the three facts that decide this case? Should the critical constraint sit at the top of the input rather than page seventeen? Is a bigger window actually the fix, or is the fix sending less, better-ordered material?

For an operator, this has a direct cost consequence too: every token — a piece of text, roughly three-quarters of a word — that you put into the window may be billed, attended to or not. Paying for a million tokens of context so the model can miss the fact in the middle is the worst deal in software.

A good agent isn’t the one with the biggest window. It’s the one whose builder decided what deserves the window.

Try this

Try it: bury the decisive fact at the start, middle and end of the same document, then watch what changes.