Designing memory for agents vs memory for chatbots

“LLM memory” is a single phrase covering two products with different physics. A chatbot’s memory is mostly about the user — preferences, history, facts. An agent’s memory is mostly about the task — what was tried, what failed, what the current plan looks like. Both can be backed by the same library, but only if the library is built knowing they are not the same shape. This post is about the shape difference and how Memorg’s hierarchy was chosen so neither use case has to pretend to be the other.

Chatbot memory is a slow, user-shaped stream

A chat assistant talks to one user across many sessions. The interesting unit of memory is the user, not the session — when the user opens a new chat tomorrow, they expect the assistant to remember today. Writes are bursty but rare: a few dozen exchanges per day, max, for a normal user. Reads are constant: every turn requires a memory lookup to know what the user is on about.

The dominant access pattern is “what does this user know, and what do I know about them?” The scoping is mostly “this user, all conversations.” The retrieval ranking wants to be biased toward stated preferences and revealed facts. Stale generic chitchat should disappear. The user’s allergy, frameworks, project name, dog’s name — those should be sticky.

Three things follow from this shape. First, the store needs a strong notion of user-and-session separately, so retrieval can cross conversation boundaries but stay inside a user. Second, importance scoring matters more than recency, because the high-value items are precisely the durable preferences. Third, the read budget per turn is small — you do not need fifty results, you need the right five.

Agent memory is a fast, task-shaped stream

An agent working on a task talks to itself dozens or hundreds of times in a single session. The interesting unit of memory is the run — the current plan, the current scratchpad, the recent tool results, the constraint that the last subgoal violated. Writes are constant and dense. Reads are constant and shaped by the agent’s current step.

The dominant access pattern is “what relevant thing happened in the last N steps of this task?” The scoping is “this run, this task.” The retrieval ranking wants to be biased toward recency and toward the current topic, because the agent is reasoning step-by-step and an event from six steps ago is more relevant than one from three runs ago. Long-term cross-task memory matters too — “the last time we tried this approach it failed” — but it is a secondary lookup, not the dominant one.

Three things follow from this shape. First, the store needs strong sub-conversation scoping — a topic per subgoal, a topic per tool, whatever the runtime decides. Second, recency matters more than importance, because the time-locality of useful info is much tighter. Third, the read budget per step is larger — agents benefit from more context per step than chatbots do, because they have less external information to work with.

The shape Memorg picked

The reason for the four-level hierarchy — session, conversation, topic, exchange — is that it lets both shapes fit naturally.

For a chatbot: a user gets a session. Each chat thread is a conversation. Topics emerge from the conversation (the app can create them explicitly, or they can be implicit). Exchanges are message pairs. Retrieval scopes to the session by default, and the scorer’s importance weight carries the load. The query “what does the assistant know about this user” is search_context(session_id=user.session_id).

For an agent: a long-running agent gets a session that may outlive any single task. Each task is a conversation. Each subgoal or tool-call cluster is a topic. Each step is an exchange. Retrieval scopes to the conversation or the topic, and the scorer’s recency weight carries the load. The query “what relevant thing happened in this run” is search_context(conversation_id=run.id). The cross-task query “have we tried this before” is search_context(session_id=agent.session_id) with appropriate filters.

Same library, same scorer, same storage. The hierarchy is what lets the scoping be honest.

What changes per use case

The three signals (semantic similarity, recency, importance) are the same. The defaults differ.

For chatbot deployments, the recommended defaults are a slower recency decay (preferences are durable), a higher importance ceiling (stated facts dominate), a smaller k (returns of 3–5 items per turn are usually enough), and a per-session token budget tuned to the model.

For agent deployments, the recommended defaults are a faster recency decay (last step matters most), a lower importance ceiling (most things are not durable), a larger k (returns of 10–20 items per step are reasonable), and a per-conversation budget tuned to the agent’s loop.

The scorer weights are exposed in the configuration. You can override them per session if you are running both shapes against the same store.

The MCP layer matters more for one of them

The MCP server (memorg-mcp) is useful in both cases, but the value proposition differs.

For chatbots, MCP is mostly a developer ergonomics win — you can pop open Claude Desktop or Cursor and inspect, edit, or add facts to a user’s memory without writing a one-off admin tool. This is genuinely useful but not the core path.

For agents, MCP is closer to a core path. An agent built on an MCP-aware host can read Memorg directly as a tool. The host runtime does not need to know how Memorg works; it sees a tool that exposes search_context, add_exchange, create_memory_item, and so on. You get a shared memory across multiple agents on the same host, with the host’s tool calling discipline applied to memory operations.

What does not fit Memorg

Two access patterns will outgrow Memorg’s defaults.

Very high-throughput agent runs. If you are running thousands of steps per second across many concurrent agents, a single-file SQLite + USearch store is going to bottleneck. The vector store interface is pluggable; the storage adapter is pluggable; but at that throughput you are probably better off with a purpose-built infrastructure.

Graph-shaped queries. If your dominant query is “what are all the entities related to entity X via relationship Y,” Memorg does not have a graph store. Mem0 has added one. We may add one. Today, if your memory is fundamentally an entity-relationship graph, a graph store is the right choice.

The case for a single library

The reason we wrote Memorg as one library that handles both shapes is that most teams run both shapes at the same time. The product is a chatbot, the product is also an agent that runs background tasks, the chat history feeds the agent’s context, the agent’s results feed the chat history. Forcing two memory libraries with two scoring models and two storage backends produces glue you do not want.

The hierarchy is the shared substrate. The scorer is the shared retrieval primitive. The MCP server is the shared integration surface. What changes between the use cases is configuration: which level of the hierarchy you scope to, which weights you set, which budget you allocate.

If you are starting an LLM-backed product today and you know that “we will have a chatbot and we will have agents,” you do not need to pick a memory model now. You need a memory layer that can be configured to behave like both. Memorg is built around that bet.