LlamaIndex memory: blocks, chat stores and long-term context

Does LlamaIndex have built-in memory?

Yes, but the current API is more specific than the old “chat memory” examples. The `Memory` class assembles the context an agent sees: recent chat history within a token limit, plus optional memory blocks that inject static facts, extracted facts or vector-retrieved messages. A `ChatStore` persists the underlying messages so a later session can load them.

That is a memory toolkit, not a managed memory product. LlamaIndex gives you the interfaces and the retrieval mechanics. Your application still decides which facts become durable, how conflicting facts are merged, which tenant owns a record, how long it is retained and how deletion propagates to every derived copy.

What is the difference between Memory, ChatStore and VectorStoreIndex?

These three layers answer different questions. `Memory` decides what enters the model context for this run. `ChatStore` saves messages so a later run can reload them. `VectorStoreIndex` retrieves documents or passages for retrieval-augmented generation. A document index is not a durable memory of the user, and a chat history is not automatically a corrected fact store.

How does the LlamaIndex Memory class work?

The current guidance uses `Memory.from_defaults()` rather than the deprecated `ChatMemoryBuffer`. The class keeps short-term chat history under `token_limit`, with `chat_history_token_ratio` controlling how much of that budget belongs to recent messages. When the ratio is exceeded, the oldest messages are dropped from the active context. You can also call `memory.put_messages()` and `memory.get()` directly when you want to manage the history yourself.

The important detail is that the token limit is a context budget, not a retention policy. Dropping the oldest message from the active prompt does not delete it from a chat store, a vector index or an external memory service. Keep the context budget, the persistence layer and the deletion policy separate.

from llama_index.core.agent.workflow import FunctionAgent
from llama_index.core.memory import Memory

memory = Memory.from_defaults(
    session_id="support-user-4821",
    token_limit=40000,
    chat_history_token_ratio=0.7,
)

agent = FunctionAgent(llm=llm, tools=tools)
response = await agent.run(
    "What did we decide about the migration?",
    memory=memory,
)

Which LlamaIndex memory blocks should you use?

The Memory class can compose blocks with different priorities. A static block injects fixed context, such as a product name or a core instruction. A fact-extraction block uses an LLM to turn chat history into a bounded set of facts. A vector-memory block stores and retrieves batches of past messages by similarity to the current query. Each block has a different failure mode, so enable only the ones that support a measured behavior.

A memory block changes what the model sees; it does not create an authorization boundary, a retention rule or a deletion guarantee.

How do you persist LlamaIndex memory across sessions?

Pass a `chat_store` to `Memory` when messages must survive a process restart. LlamaIndex documents a SQLAlchemy-backed chat store for SQL databases plus the `AsyncDBChatStore` interface for custom storage. A stable `session_id` selects the conversation; the store holds the messages that belong to it.

Do not treat `session_id` as a security token. It is a lookup key. Resolve the authenticated user and tenant before choosing or accepting a session ID, and test that a caller cannot read another tenant’s history by guessing or reusing a key.

from llama_index.core.memory import Memory
from llama_index.storage.chat_store.postgres import PostgresChatStore

chat_store = PostgresChatStore.from_uri(
    uri="postgresql+asyncpg://app:secret@db/app",
)

memory = Memory.from_defaults(
    session_id=f"tenant:{tenant_id}:user:{user_id}",
    token_limit=30000,
    chat_store=chat_store,
)

How should LlamaIndex memory map users and tenants?

Use the narrowest safe scope and make the tenant the outer boundary. A practical order is tenant, user, agent and task. Store the scope with every message or fact, not only inside the session ID string. Then filter on it when loading memory, retrieving from a vector index or writing to an external service.

LlamaIndex does not know which tenant is allowed to read a session. The application must resolve identity from its own authentication context and reject a mismatched tenant before the memory layer is called. If the same memory is shared with another framework, keep the scope names consistent across both implementations.

When do you need an external memory layer with LlamaIndex?

The native Memory class and chat store are enough when one LlamaIndex application owns the data, the team is comfortable owning extraction and retrieval quality, and the product already has a complete lifecycle for identity, correction and deletion. Add an external memory service when several frameworks, products or agent runtimes need the same durable user context.

How do you test LlamaIndex memory before production?

Test the context assembly and the persistence separately. First verify that a new session for the same user loads the intended history and facts. Then change a fact, start another session and confirm the new value wins. Finally, use a similar user in another tenant and prove that no message, fact or vector result crosses the boundary.

Measure retrieval quality independently from answer quality. A model change can hide a memory regression, and a larger token budget can make a bad retrieval look better by including more noise. Record which block contributed which context so a failure can be attributed.

What is the recommended LlamaIndex memory architecture?

Use the Memory class for context assembly, a database-backed ChatStore for message persistence and a VectorStoreIndex only for document retrieval. Add an external memory contract when durable facts must follow a user across agents, services or frameworks. Resolve tenant identity outside the model, keep the token budget separate from retention and treat deletion as a workflow that covers every layer.