A common misconception is that an agent "has" a conversation in the way a person does. It does not. Every turn, a context window is assembled from scratch and handed to a model that has no memory of the previous call.
Instructions
- role
- constraints
- tone
- escalation rules
Tools
- available tool schemas
Retrieved knowledge
- relevant chunks for this turn
Conversation
- recent turns verbatim
- earlier turns compacted
Current input
- what the user just said
Why bigger windows did not solve this
Three reasons. Cost scales with what you put in. Attention is not uniform across a long context, so buried instructions lose influence. And a large window filled indiscriminately contains contradictions - an old answer that has since been superseded is still, as far as the model can tell, part of the record.
Compaction and what it costs
Compaction replaces older turns with a summary. It is lossy in a specific direction: it keeps what happened and drops what was ruled out. That is why an agent twenty turns into a conversation will cheerfully re-offer something the customer already rejected.
Summarising toward a constraint set - established, rejected, decided, pending - rather than toward a narrative preserves exactly the material later turns need. It is a small change with a disproportionate effect.
Retrieval is context management
Retrieval is often filed under knowledge, but its function is contextual: it puts a small, relevant, *current* slice of a large corpus into the window instead of a large stale one. Freshness is the underrated half of that.