Forgetting as a feature: aging memory in LLM agents

The default failure mode of an LLM agent that has been running for a few weeks is not amnesia. It is the opposite. The agent remembers an early misconfiguration, a stale preference the user has since changed, a deleted file’s contents, a one-off detour the user asked about and forgot. Every retrieval is a noisier neighbourhood than the last. The cosine numbers still look fine, but the answers degrade.

This is the part of memory that everyone has an opinion about and nobody wants to build: forgetting. Human memory is built on forgetting as much as on recall; the brain’s hippocampal-cortical pipeline is, among other things, a brutal filter that drops most of what it sees. LLM agents inherited their memory architecture from search engines, and search engines do not forget on purpose. The first version of every memory layer is too retentive; the second version is a panic-tagged set of delete_memory tool calls; the third is something that ages on its own. Memorg tries to be the third version.

What forgetting actually has to do

Useful forgetting needs to do four things at once. It should make stale facts harder to surface without making them irretrievable. It should let important things stay sharp for longer. It should let unimportant things fade quickly. It should be deterministic enough that you can debug it.

That last constraint is the one that rules out the obvious answer of “have the LLM decide what to keep.” LLM-driven retention is fashionable and we are not against it as a feature on top, but as the primary retention policy it has two problems. First, every write becomes an LLM call, which is expensive and slow. Second, debugging “why was this forgotten” becomes a transcript-reading exercise. If you cannot answer that question from a SQL query, you cannot operate the system.

The three signals

Memorg’s retrieval ranking blends three signals: semantic similarity, recency, and importance. It is worth saying out loud that those signals do not produce forgetting on the write path — Memorg does not throw away exchanges as they age. What they produce is a retrieval-time forgetting effect: stale, unimportant items rank below fresher or more important ones and, for any given token budget, are statistically very unlikely to be returned. The exchange is still there, but the agent does not see it in normal operation.

This is the right place to apply the forgetting policy. Hard deletes are a sledgehammer; they make audits impossible, they make “undo” impossible, they make compliance asks harder. Soft, ranking-driven forgetting gets the benefits of forgetting (fresher recall, less noise) without the irreversibility.

Recency

Recency is the simplest signal. An exchange that happened a minute ago is more likely to be relevant than one from a month ago. There are exceptions — a user’s allergy mentioned three years ago is still relevant — but they are exceptions. The default policy should treat newer as better, and the exceptions should be handled by importance.

The shape of the recency decay is a design choice. A linear decay forgets too fast for long-running agents and too slowly for short-lived chat apps. A logarithmic decay is closer to what humans do and what most retrieval workloads want. Memorg uses a decay that is slow over hours and steep over months, with a floor so very old items can still be retrieved if nothing newer is relevant.

Importance

Importance is the override for recency. An importance score is set when the item is written and can be updated later. Items above a threshold are essentially exempt from recency decay; items below decay faster. The bar for “important” is set by the application, not by the memory layer. A medical app will mark allergies, dosages, and conditions as high-importance. A code assistant will mark the user’s stack, their preferred frameworks, and any commit-blocking constraints. A general chat assistant will mark stated preferences and revealed facts.

The hardest part of importance is filling it in. Memorg ships defaults that work — exchanges in topics flagged as important inherit higher importance; explicit notes default high; documents default mid — and the API lets you override per-item. We are deliberately not running an LLM in the write path to score importance, because the cost-and-determinism trade ruled it out. If you want LLM-scored importance, you can compute it once asynchronously and call update_importance().

Semantic similarity

Semantic similarity is the third signal and the one most people start with. It is necessary but not sufficient. Memorg’s scorer multiplies similarity by an aging weight (a function of recency and importance) so that a high-similarity stale item ranks below a moderate-similarity fresh-and-important one. The weights are exposed in the configuration; the default produces good results out of the box for chat-shaped workloads.

Forgetting policies that come for free

Once the scorer exists, several useful forgetting behaviours fall out of it without any additional code.

Topic switching. When a conversation switches topics, the new topic gets its own ID and new exchanges are written under it. Recency is per-exchange but importance can be inherited from the topic; switching topics naturally lowers the chance that the previous topic’s items show up in retrieval.

Stale-preference handling. When the user updates a stated preference (“actually use Postgres, not MySQL”), the new statement is more recent and ranks higher than the old one. The old statement is still there for audit, but the agent’s normal retrieval surfaces the new one.

Episodic vs. semantic. Topics that are short and bursty (an episode) age out fast on recency. Topics that are long-running and high-importance (semantic, in the cognitive-science sense) stay accessible. The application gets the distinction for free.

What we deliberately did not build (yet)

There are two forgetting features we have left out of the v1 because they are harder to do well than they look.

The first is automatic consolidation — periodically running over the store and rewriting groups of exchanges as condensed memories. Mem0 does a version of this on the write path. We think it is a reasonable feature, but it is essentially the rolling-summary problem at a larger scale, and getting the consolidation policy right requires either an LLM call (expensive, non-deterministic) or a careful rule system (brittle). We have left a hook for it; we have not committed to a default.

The second is TTL-based hard deletion. We may add it for compliance reasons (a per-user “forget all memory older than N days” policy is a legitimate ask) but we are not adding it as a retention strategy. Hard deletion as a retention strategy is a worse version of recency-weighted retrieval.

The cognitive analogy

The reason to take forgetting seriously is that it is what makes memory work. A system that returns every previous turn that has any similarity to the current turn is not memory; it is a log search with extra steps. A system that decides what to surface based on freshness, importance, and similarity is closer to how usable memory actually behaves.

LLMs are bad at reasoning over noisy context. Even with long-context models, the signal-to-noise ratio of retrieved memory matters more than the absolute amount you can fit. Forgetting on purpose — at retrieval time, deterministically, with a paper trail you can audit — is how that ratio stays high as your agent runs into year two and beyond.

Memorg is built on the assumption that you will be running the same store for longer than you currently plan. The scoring defaults are tuned for that. If you treat forgetting as a feature from the start, the store stays useful. If you do not, you will be writing a migration script the first time a user complains that the bot remembers something they said in error six months ago.