OpenAI Responses API memory: conversation state vs long-term recall

Is the OpenAI Responses API stateful?

The Responses API supports several ways to manage conversation state, but the model itself remains stateless. The application can send previous messages manually, chain responses with `previous_response_id`, or use the Conversations API to persist a long-running conversation object.

This state is conversation infrastructure. It preserves items such as messages, tool calls and tool outputs. It does not define which user preferences should survive a new conversation, how a corrected fact supersedes an old one or when a tenant must be deleted.

How do you manage state in the Responses API?

Manual state gives the application complete control over which messages are resent. A previous response ID is simpler for a threaded chain. A Conversations object gives the thread a durable identifier that can be reused across sessions, devices or jobs.

How does previous_response_id work?

A response can be linked to the previous turn by passing its ID. The API uses that relationship to continue the thread. With the Responses API, the stored parameter controls whether the response is retained for later retrieval and use with `previous_response_id`.

The convenience does not erase the context-window economy. The official documentation notes that previous input tokens in the chain are still billed as input tokens. A long chain eventually needs compaction, summarization or a smaller externally retrieved context.

const first = await openai.responses.create({
  model: process.env.OPENAI_MODEL!,
  input: 'What did we decide about the API migration?',
  store: true,
});

const second = await openai.responses.create({
  model: process.env.OPENAI_MODEL!,
  previous_response_id: first.id,
  input: 'Turn that decision into an implementation checklist.',
  store: true,
});

What does the Conversations API add?

The Conversations API persists conversation state as an object with its own identifier. The application can reuse that conversation across sessions, devices or jobs, and the API can store messages, tool calls, outputs and other items.

A conversation object is still bounded by its purpose. It provides continuity for one thread. It does not automatically create a user profile, reconcile facts across two threads or decide that an old dependency is no longer valid.

• Use a conversation when the active thread must survive a device or process change.
• Use external memory when selected facts must follow the user outside that thread.
• Keep account permissions in the source system rather than in conversation items.
• Define how a conversation is deleted independently from durable user memory.

How should context-window limits affect memory design?

Treat the context window as a budget shared by instructions, input, tool schemas, retrieved data, reasoning and output. A conversation can be persistent and still become too large, repetitive or contradictory to use efficiently.

Compact completed work into a short state summary, retrieve only facts relevant to the next action and keep source records outside the active prompt. Compaction is a context-management operation; durable memory is the policy that decides what should survive the conversation at all.

How do you add long-term memory around the Responses API?

Retrieve a small scoped memory block before the request, add it as labeled data and write the meaningful exchange or confirmed outcome after the response. Keep extraction outside the critical path so a failed memory write does not break the user turn.

Do not ask the model to choose the tenant. Resolve the tenant, user and agent from authenticated application state, then pass those values to the memory service. Memory retrieval should never grant a tool permission or override current account data.

const memories = await memory.search({
  tenantId: principal.tenantId,
  userId: principal.userId,
  agentId: 'billing-assistant',
  query: userMessage,
  limit: 8,
});

const response = await openai.responses.create({
  model: process.env.OPENAI_MODEL!,
  input: [
    { role: 'user', content: userMessage },
    {
      role: 'developer',
      content: memory.renderAsUntrustedData(memories, { maxTokens: 1200 }),
    },
  ],
});

await memory.add({
  tenantId: principal.tenantId,
  userId: principal.userId,
  agentId: 'billing-assistant',
  source: 'responses-api',
  conversationId: response.conversation_id ?? null,
  messages: [
    { role: 'user', content: userMessage },
    { role: 'assistant', content: response.output_text },
  ],
});

Which scopes should Responses API memory use?

Use conversation scope for the active thread, user scope for durable preferences, project or account scope for shared decisions and tenant scope for isolation. A user fact may be visible across several agents while an agent’s working conclusion should remain private.

Store source and validity with every durable fact. When account data changes, supersede the remembered value rather than keep injecting it. High-impact actions should query the billing, permission or inventory system directly.

• Conversation: temporary history and current task.
• User: stable preferences and confirmed personal context.
• Project or account: approved shared decisions.
• Agent: behavior or task-specific state.
• Tenant: mandatory isolation boundary.
• Source system: current permissions, plans and other truth.

How do you test Responses API memory?

Test conversation continuity separately from durable recall. A response chain should resume the current thread; a memory test should retrieve a fact after the original conversation is gone. Combining the two hides which subsystem caused the result.

Run the same scoped queries with no memory, conversation replay and durable retrieval. Compare answer quality, prompt tokens, latency and boundary failures. Then change a fact, delete a user and retry the same request.

Which mistakes break Responses API memory?

The first mistake is assuming a stored response or conversation contains the user’s durable profile. A second is carrying every prior token forward because chaining is convenient; long histories increase cost and can bury the instruction that matters now. The third is asking the model to decide tenant scope from the conversation.

Treat conversation state and durable memory as two contracts with different retention. A conversation may be deleted while selected preferences remain, or a user may clear memory while continuing the current thread. Encode both cases explicitly instead of coupling every delete action to the whole account.

• Do not confuse conversation items with extracted facts.
• Do not ignore context-window and input-token cost.
• Do not let model output select tenant identity.
• Do not inject every old memory into every request.
• Do not assume memory write and retrieval are immediately consistent.

What is the recommended Responses API memory architecture?

Use manual state, `previous_response_id` or Conversations for the active thread. Use a scoped memory service for selected facts and decisions that should survive across conversations. Compact long context, retrieve a small relevant subset and keep current business truth in the source systems.