Quick answer
A language model has no memory of its own: every request starts from nothing, and it only knows what is in front of it right now. When an assistant "remembers" your name or your preferences, software around the model stored that information and put it back into the prompt. Short-term memory is the context window; long-term memory is a database of facts and past interactions, retrieved when relevant. Getting it right is the difference between an assistant that knows you and one that remembers the wrong things.
This surprises people: the model that wrote you a thoughtful reply yesterday has no idea who you are today. Everything that feels like memory in ChatGPT, Claude, Meta Muse, or a customer-service agent is engineering layered on top of a system that forgets completely between calls.
The two kinds of memory
- Short-term: the context window — the conversation so far and any documents you shared. Large (up to a million tokens now) but temporary and increasingly expensive as it fills
- Long-term: facts extracted and stored outside the model — your preferences, past decisions, project details — then searched and re-inserted into the prompt when they seem relevant
- Working memory for agents: the plan, the steps completed, and the results of tool calls during a long task
How long-term memory actually works
After a conversation, a process decides what is worth keeping — "user prefers concise answers", "project deadline is October 3" — and writes it to a store, often as embeddings in a vector database. On the next conversation, the system searches that store for anything related to what you just said and includes the hits in the prompt, invisibly. Products like ChatGPT's memory and developer libraries such as Mem0 and Letta do this; agents like Meta Muse depend on it to complete tasks across days.
Where it goes wrong
- Stale facts: a preference you changed months ago keeps resurfacing
- Wrong retrieval: an irrelevant memory gets pulled in and distorts the answer
- Privacy: the assistant remembers something you said once and did not want kept
- Cost and quality: cramming history into every prompt raises bills and can degrade responses — context rot
- Opacity: you often cannot see what it remembers, which is why the better products show and let you edit memories
Related reading
Bottom line
Memory is not a property of AI models; it is a product decision made around them. When an assistant remembers you well, thank the retrieval system. When it remembers you badly, look for the setting that lets you see and delete what it kept.
