Managing Memory for AI Agents

Chapter 2. Long-Term Memory: Building Persistent Learning Agents

There’s no single, universal definition for the different types of memory in agent systems—every company seems to have its own take. For example, Anthropic, OpenAI, and Google all use slightly different terminology and approaches. But the more interesting question is: how does an agent actually decide what counts as procedural, semantic, or episodic memory? Or put another way: what gets treated as short-term, long-term, or contextual memory?

Along these lines, semantic caching might play a big role. Sometimes short-term memories can be promoted to long-term if they’re accessed frequently enough. Conversely, long-term memories that aren’t used much might get summarized, become less detailed, or even be dropped from the system altogether. These are the kinds of trade-offs and decisions that go into designing agent memory. Ultimately, it all comes down to how the system is built to manage and retain information. Since it’s nearly impossible to program an agent to handle every scenario, we rely on constraints and parameters to guide what gets treated as episodic, semantic, or procedural memory.

Types of Long-Term Memory

The industry has converged on three primary types of long-term memory, although implementations vary significantly:

Episodic memory

Stores specific past experiences and events, functioning like human autobiographical memory. Companies typically implement this through RAG systems on conversation histories, ...