Memory, Context, and State: Why AI Agents Rot Over Time

Most agent failures that get blamed on the model are actually confusion between three different things: context, memory, and state. Builders use the words interchangeably, so they build one bucket where three systems are needed. The bucket works fine for a demo. Over weeks of production traffic, each of the three decays in its own distinct way, and because they were never separated, the failures are impossible to isolate.

This post defines the three cleanly, then walks through how each one goes bad in the long run. The failure modes are the point. Anyone can draw the taxonomy; the interesting part is what breaks.

What context, memory, and state actually are

Context is the set of tokens the model sees during a single inference call. Anthropic's definition is exactly this: the tokens included when sampling from the model, spanning system prompt, tool definitions, message history, and retrieved data. Context is rented, not owned. It exists for one forward pass, you pay for it per call, and it is rebuilt every turn.

Memory is what persists across sessions: user preferences, learned facts, past decisions. It lives in a store outside the model, gets written by some extraction process, and gets read back into context when judged relevant. Memory is curated and lossy by design. That lossiness is a feature until it isn't.

State is the ground truth of the task or system. Which files changed. Which subtasks are done. What the current account balance is. State is not an LLM concept at all; it is a software concept, and it should be authoritative regardless of what any model believes.

A useful compression: context is what the model sees, memory is what the system recalls, state is what is true.

The conflation happens because all three arrive at the model the same way, as tokens in the prompt. From inside a single inference call they are indistinguishable. The differences only show up over time, which is exactly why demos hide them and production exposes them.

How context goes bad: rot, not overflow

The intuitive failure mode for context is overflow: you run out of window. The real failure mode arrives much earlier. Chroma's context rot research tested 18 frontier models, including GPT-4.1, Claude 4, and Gemini 2.5, and found that every one of them became less reliable as input length grew, even on deliberately simple tasks where the answer was verifiably present. One controlled study using the FLenQA dataset held the reasoning problem constant and only padded the input, and accuracy fell from 0.92 to 0.68 as inputs grew from roughly 250 to 3,000 tokens. Not 300,000 tokens. Three thousand.

Position matters too. The lost-in-the-middle finding (Liu et al., TACL 2024) showed models retrieve well from the start and end of context and poorly from the middle. Anthropic frames the underlying cause as an attention budget: transformer attention creates n² pairwise relationships between tokens, so every added token dilutes the attention available for the ones that matter.

The long-run consequence for agents is specific. Agents append as they work: tool outputs, search results, error traces, dead ends. A coding agent that explored three wrong approaches carries all three corpses in context, and that noise degrades every subsequent decision. The model did not get dumber. Its context got dirtier. Teams that respond by buying a bigger context window are treating context as storage, which is the exact conflation this post is about. Context is a working set, and working sets need eviction.

How memory goes bad: staleness, contradiction, poisoning

Memory fails differently. It does not degrade within a session; it corrupts across them, through three mechanisms.

Staleness. A fact written six months ago ("user prefers weekly billing") was true at write time. Nothing in most memory systems re-verifies it. Retrieval by embedding similarity happily surfaces a fact whose truth expired, and the agent asserts it with full confidence.

Contradiction. Memories written at different times disagree, and naive stores keep both. Retrieval then becomes a lottery: which of the two contradictory entries is semantically closer to today's query? The agent's behavior becomes inconsistent in ways no single log line explains, because the bug is distributed across two writes made weeks apart.

Poisoning. This one is now well documented as a security problem. Research on memory injection attacks found that MINJA achieved over 95% injection success against memory-based agents under test conditions, and follow-up work showed the root cause plainly: agents treat retrieved memories as ground truth because retrieval is pure embedding similarity with no provenance checks. A poisoned entry, whether adversarial or just a hallucination the agent wrote down about itself, keeps getting retrieved and keeps steering behavior. Memory turns prompt injection into a persistent infection with a time delay.

The common thread: memory systems are almost always write-optimized and read-naive. Everyone builds the extraction pipeline. Almost nobody builds eviction, expiry, provenance, or contradiction resolution. A memory store without a forgetting policy is not a memory; it is a landfill with a search index.

How state goes bad: drift and the photocopy problem

The third failure is the least discussed and, in my experience building multi-agent systems, the most expensive: treating context as if it were state.

An agent that tracks task progress only in its message history has no state, only a narrative about state. When the context gets compacted or summarized, as every long-horizon agent eventually requires, the narrative gets rewritten. Summaries of summaries compound errors the way a photocopy of a photocopy compounds blur. A subtask marked "mostly done" in one summary becomes "done" in the next, and the agent skips the remaining work with complete confidence. Nothing in the system can catch this, because the only record of the truth was the thing that got compressed.

The teams shipping serious agents have converged on the fix. Manus treats the file system as the durable layer: the agent writes state to files, keeps context recoverable rather than complete (drop a web page's content, keep its URL), and maintains a todo.md it rewrites every turn so the plan sits at the end of context where attention is strongest. Claude Code externalizes progress the same way. The pattern generalizes: state lives outside the model, in a form the model reads and writes but does not own. The context holds a pointer to the truth, never the only copy of it.

Inference from the pattern, not a sourced fact: I would go further and say an agent's reliability over long horizons is mostly a function of how little it needs to remember. Every fact the agent must carry in tokens is a fact that can rot, drift, or be poisoned. Every fact it can re-derive from external state on demand is a fact that stays true.

The separation test

Three rules fall out of the taxonomy, and each maps to one decay mode:

1. Context is a working set, so curate it every turn. Anthropic's phrasing is the right target: the smallest set of high-signal tokens that produces the desired behavior. Evict dead ends. Keep the active goal near the end.
2. Memory is a database, so give it database hygiene. Timestamps, provenance, expiry, and a contradiction policy. If your memory layer has a write path but no deletion path, you have scheduled a future outage.
3. State is ground truth, so keep it out of the model. Files, a database, a task ledger the agent reads and writes. If the honest answer to "where does the truth live?" is "in the conversation," the system will drift, and you will not notice until it matters.

A quick test for any agent architecture: delete the entire conversation history and ask whether the system still knows what is true, what it has learned, and what it was doing. If yes, the three layers are separated. If no, you have one bucket, and time is already working on it.

Conclusion

Context, memory, and state fail on different clocks. Context rots within a session, memory corrupts across sessions, and state drifts across compactions. A system that conflates them will look healthy in every demo and decay in production, because each decay mode hides inside the other two. Separate them explicitly, give each its own hygiene rules, and most of what gets blamed on "the model being unreliable" turns out to be an architecture bug you can actually fix.