Understanding How Memory Works in Large Language Models

Every call to a large language model (LLM) starts from scratch, as the model has no inherent memory. Each request is independent, meaning the model does not retain information from previous requests. This is suitable for one-off questions but becomes problematic for systems that require historical context, such as support agents or sales assistants. Agent memory typically operates on two timescales: short-term memory tracks current events, while long-term memory retains user preferences and past task outcomes. This information is stored in an external layer, allowing the agent to retrieve relevant details before each call.