How to add memory to an AI agent (without stuffing the prompt)

Step 0: decide what deserves to be remembered

A memory layer that stores everything is a transcript with a search box. The useful version stores the things that change future behavior: a fact about the user, a preference, a decision, a constraint or a meaningful episode.

Start with the questions your agent gets wrong today. If it keeps asking for the same account id, store the account. If it recommends a workaround that failed last week, store the failed attempt. If it forgets a commitment, store the commitment and its date.

Step 1: define the scopes before the first write

Retrofitting scopes is painful because every stored memory needs an owner. Decide the four ids up front, even if one of them is constant in your first release.

Step 2: write the events you already have

Do not build a new event pipeline for memory. Send the messages, tool results or documents that already exist in your application. Attach the ids you already trust.

• Write after the turn, not before it, so the extraction sees the outcome.
• Batch when the source is a document or a long tool result.
• Keep the source id in metadata. You will need it when a memory is disputed.

from aimemory import Memory

memory = Memory(api_key="aim_live_...")

memory.add(
    messages=[
        {"role": "user", "content": "we moved to the Scale plan"},
        {"role": "assistant", "content": "I updated the billing note."},
    ],
    user_id="dana",
    agent_id="support-bot",
    metadata={"tenant": "northwind", "source": "support-thread-4821"},
)

Step 3: extract, merge and resolve contradictions

The write path should not append every sentence. It should decide which parts are durable, merge them with what is already known, and mark the older version as superseded when the truth changes.

This is where most homegrown memory systems fail. A summary table can hold one row per user, but it cannot explain why the row changed. A versioned graph can.

• Deduplicate near-identical memories before they accumulate.
• Resolve conflicts to the newest reliable source, not the newest message.
• Keep the old value as history when the change matters for an audit.
• Drop ephemeral details that do not change a future decision.

Step 4: retrieve with a scope and a token budget

A search that returns everything is not retrieval. Ask a natural-language question, filter by the ids you trust, and set a limit that fits the prompt budget.

• Retrieve before assembling the prompt, not after the model has started.
• Keep the top five, not the top fifty. More context is not automatically better.
• Log which memories were used so you can debug a bad answer later.

import { Memory } from '@aimemory/sdk';

const memory = new Memory({ apiKey: process.env.AI_MEMORY_KEY });

const context = await memory.search({
  query: 'how does Northwind like to be contacted?',
  userId: 'dana',
  agentId: 'support-bot',
  tenantId: 'northwind',
  limit: 5,
  rerank: true,
});

Step 5: make forgetting a first-class operation

A memory layer without deletion is a liability. If a user asks to be forgotten, the raw event, the derived memory and the audit trail all need a defined path.

• Define a TTL for session memories and short-lived episodes.
• Support deletion by memory, user and tenant.
• Redact sensitive fields before they reach the memory store.
• Test deletion with a real user id and then search again.
• Keep an audit entry that proves the deletion happened without retaining the content.

A minimum test suite for memory

Common mistakes

• Storing every message and calling it memory.
• Using the model to summarize the transcript on every turn.
• Forgetting to version a changed fact.
• Filtering only by user id and leaking across tenants.
• Treating deletion as a database task with no audit path.
• Choosing a memory provider before defining the scopes.