Memory Architecture

Overview

An autonomous agent that works over long horizons needs memory that survives more than a single prompt. Driftless agents build their context once per turn, before the user's prompt enters the agent loop, and they organize that context into a stack ordered for both correctness and provider-cache efficiency. The model separates what is happening right now from what happened earlier in this session from what the agent was told to remember across sessions, and gives both the agent and the user a way to curate which earlier work persists and which fades.

The result is memory that does not silently grow without bound and does not silently lose the thread. Short, casual exchanges fade away. Durable decisions and the conversations behind them are promoted to cross-session memory and injected into every future turn — unless a human decides otherwise.

The three-layer context model

Every turn, the agent assembles a context stack in cache-optimized order: a stable prefix that rarely changes (maximizing provider cache hits), a semi-stable prefix fetched from the database each turn, and a volatile suffix that changes every turn. Three logical layers map onto that stack — an active window, prior topics, and persistent memory — each with a distinct purpose, retention policy, and scope.

How the stack is ordered

The layers are placed stable-first to maximize cache hits across turns within a session; placing volatile content at the top would blow the provider cache on every call. The assembled stack is:

1. Stable prefix (cached, never compacted):
     SOUL identity, skills, tools list, persistent memory,
     durable facts, drift result.

2. Semi-stable prefix (per-turn database fetch):
     notes for the current topic, prior topic summaries.

3. Volatile suffix (changes every turn, trailing 20 requests max):
     current topic conversation thread, active tool results,
     working memory.

What persists and what fades is the central design choice. The active window is intentionally lossy — only the trailing window is kept, so a long single-topic conversation stays bounded. Prior topics persist as summaries for the session, but a session is not a permanent record; only promoted topics cross the session boundary. Persistent memory is the only layer that outlives the session, and it is curated rather than automatic.

Topics, tool calls & summarization

Conversations are organized into sessions, topics, and messages. A session is a conversation; it holds many topics, but exactly one is active at a time. A topic is a coherent line of work within that conversation and owns its own message thread, its own summary, and its own memory flags.

A topic is created in one of two ways. A user can start a new topic explicitly, or the agent can create one with a silent fork when drift detection decides the conversation has moved on to a new subject. On a fork, messages that belong to the new subject are relocated to the new topic (within a bounded scan-back window), the prior topic is summarized, and the prior topic's notes are archived — all without interrupting the user.

Tool calls are logged and surfaced in the thread

The agent has a small set of built-in runtime tools that read and write its own memory stores — notes, important_info, and manage_topic_memory — dispatched ahead of any MCP tools. Tool calls and their results are part of the conversation thread, and active tool results are injected into the volatile suffix of the context stack so the agent reasons over what it just did. Because tool calls live in the message history, they are summarized alongside the rest of the topic when it is closed or forked.

Topic summaries are generated by just-in-time summarization

A topic does not carry a summary from birth. Summaries are generated just in time — when the topic is exhausted by a fork or when a summarization trigger fires — in a single LLM call that regenerates the topic's title, summary, and memory flag from its complete message history. This is covered in detail in the next section.

Just-in-time summarization & promotion

When a topic is summarized, one LLM call regenerates its title, summary, and the include_in_memory flag together, from the topic's full message history. The summary is deliberately terse — roughly half the token count of a verbose paragraph — to keep both the session-scoped prior-topics block and the cross-session memory block cheap. The prompt instructs the model to keep all durable decisions, preferences, and key context while cutting filler, so the summary reads like a sharp note left for a future self.

Summarize this conversation. Return a JSON object with:
- 'title' (short descriptive label)
- 'summary' (concise note — roughly half the length of a verbose
  paragraph; keep all durable decisions, preferences, and key context,
  but cut filler; lead with the topic, use terse phrasing and
  parenthetical lists where natural; it should read like a sharp note a
  smart person left for themselves — same voice, zero bloat)
- 'include_in_memory' (boolean: true if this conversation contains durable
  decisions, preferences, or context the agent should remember across
  sessions; false if it's casual or ephemeral)

The include_in_memory boolean is what drives promotion. When it is true, the topic's keepInMemory flag is set, and from that point the topic's summary is queried from the memory cache and injected into the stable prefix on every session — it has crossed from session-scoped recall into cross-session memory. When it is false, the topic stays session-scoped and fades when the session ends.

The minimum turn threshold

Not every conversation is worth remembering. A topic with fewer than five messages (user and agent combined) is forced to keepInMemory: false regardless of what the model returns. Short exchanges are either trivial, or important enough that the key facts already live in durable facts. The threshold keeps casual chatter out of long-term memory.

// Topics with fewer than this many messages are forced to keepInMemory: false
// regardless of the LLM's include_in_memory response. Short conversations are
// either trivial or important enough that key facts would already be in
// important_info.
const MIN_MEMORY_TURNS = 5;

Safe defaults and cache invalidation

The flag defaults to false when the model omits it or returns anything other than a strict boolean true, so a malformed response never accidentally promotes a topic. If the summarization LLM call fails entirely, the fallback summary is generated deterministically from the first user message, and its keepInMemory is always false — fallbacks never promote to memory.

Promotion and demotion are cache-aware. When the keepInMemory flag changes — a promotion (false→true) or a demotion (true→false) — the Redis memory cache for the agentId:userId:orgId scope is invalidated, so the next turn re-fetches the updated memory list rather than serving a stale entry. The token cost of the memory block is also bounded: it is capped at an approximate budget with FIFO truncation, so a very long memory cannot crowd out the rest of the context.

Memory curation: the tri-state model

Memory is not write-only. Every topic lives in one of three curation states, and both the agent and the user can move a topic between them. The model keeps memory accurate and current rather than letting it accumulate noise.

A deprecated topic still leaves a trace: it appears in the memory prefix flagged [DEPRECATED] with its title only — no summary. The agent learns that a prior decision was reversed without paying the token cost of the full summary, so a reversed decision does not silently reassert itself later.

Who curates: the agent and the user

The two curators have different and complementary roles:

• The agent curates automatically. Promotion and demotion are automatic: during just-in-time summarization, the model sets the include_in_memory flag, which promotes or demotes the topic on its own. The agent can also curate explicitly through its manage_topic_memory tool. Deprecation, the strongest signal, is the user's call to confirm — the agent cannot reverse a user's deprecation, and un-deprecation is not exposed to the agent at all.
• The user curates through the UI. A tri-state brain toggle on each topic row cycles the state: gray (not in memory) → teal (promoted) → red (deprecated). This lets a person override what the agent decided, in either direction.

The user-decision guard

The agent never silently overrides a human curation choice. Two manual-set flags enforce this:

• keepInMemoryManuallySet — set when the user toggles keep-in-memory. Once true, the JIT summarization call must not override keepInMemory back to false; the user's manual setting persists across summarization cycles. The agent's manage_topic_memory promote/demote is rejected with “keepInMemory was set manually by the user; the agent cannot override it.”
• deprecatedManuallySet — set when the user deprecates (or un-deprecates) a topic. The agent's manage_topic_memory deprecate is rejected when the user has already set it.

Un-deprecation is reserved for the user — it is not exposed to the agent at all. So deprecation, the strongest signal, requires a human to reverse it. The guard is asymmetric by design: the agent may curate freely until a human takes a position, at which point the human decision wins and sticks.

Durable facts

Beside topic summaries, the agent keeps a separate store of durable facts — long-lived key-value entries that persist indefinitely and are never summarized. These are the facts the agent should be able to rely on across every session and task: entity references, configuration values, user identity, and standing decisions.

Durable facts live in a single document scoped per user + agent + organization triple — one isolated store of important information per agent per user. They are injected into the stable prefix, so they are present across every turn regardless of which session or topic is active, always at full fidelity. The agent manages them through the important_info tool with three operations: $set (create or update an entry by key), read (retrieve all or by key), and $unset (delete by key).

How durable facts differ from memory entries

The split is deliberate: durable facts are for information that must always be exactly correct and always present; memory entries are for the gist of what was discussed, kept compact and curatable. Topic summaries and discussion recaps belong in memory; facts the agent should never lose belong in durable facts.

Notes

Each topic carries its own notes — ephemeral working context scoped to that topic, not to the user. Notes capture decisions, reasoning, and drafts as they are made, giving the agent a scratchpad for the current line of work without polluting the durable layers.

Notes are fetched at the start of each turn and injected into the semi-stable prefix, so the agent starts every turn with its working context for the active topic. The notes tool offers three operations: $set to append working context (decisions are logged as they happen, not overwritten), read to retrieve the notes for the current topic or any topic in the session by id, and update to replace the full content when understanding has changed and the notes should reflect the corrected state.

Notes survive compaction within a topic's lifetime, but when the topic closes or drift-forks, the notes are archived — and only after the topic summary has been generated from them. One active (non-archived) notes document exists per topic; an archived document stays on disk for traceability but is no longer injected into context.

How notes differ from memory

In short: notes are the agent's scratchpad for the current topic; memory is the agent's long-term recollection across topics and sessions. When a note proves to be a durable fact, the agent promotes it to important_info rather than leaving it in the ephemeral scratchpad.

Session lifecycle triggers

Summarization does not wait for a clean exit. The same evaluation pipeline — drift detection, then either a silent fork (on drift) or an in-place title/summary/keepInMemory refresh (no drift) — fires from several triggers, so a topic is summarized even when the user simply walks away mid-conversation.

Topic creation. When a new topic starts (manually or by drift fork), the prior topic is summarized before the new topic becomes active, so closing a line of work and starting a new one are atomic from the memory's perspective.

The 10-message drift interval. A drift evaluation fires every five user-assistant turn pairs — ten messages — inside the message-send handler. This is the heartbeat that detects when a conversation has drifted to a new subject and forks it.

Page navigation. When the chat page unmounts on an in-app route change, a deferred flush fires the evaluation for the active topic.

Window visibility change. When the tab is switched or minimized (visibilityState → hidden), the evaluation is scheduled.

Socket disconnect. When the real-time socket disconnects, the evaluation is scheduled.

The debounce mechanism

The three exit triggers — page navigation, visibility change, and socket disconnect — rarely fire in isolation. Leaving a session often produces a rapid-fire sequence (tab switch → socket disconnect → navigation), and running the pipeline once per event would waste work. They are routed through a single 2000 ms trailing-edge debounced entry point that coalesces the burst into one pipeline execution.

A hasEvaluated guard then suppresses further triggers for the active topic until it is reset — which happens when the user sends a new message or the active topic changes. An unchanged topic is not re-evaluated. The evaluation runs server-side at POST /agents-api/sessions/:id/evaluate and is fire-and-forget: it is best-effort, mirroring the 10-message path, and never throws to block the user's exit.

Summary

The driftless memory architecture is a layered, curated, lifecycle-driven system. The three-layer context model keeps the active window full-fidelity but bounded, prior topics session-scoped but compact, and persistent memory cross-session but curated. Just-in-time summarization generates titles, summaries, and the promotion flag in one call, with a minimum-turn threshold and safe defaults that keep trivial chatter out of long-term memory. The tri-state curation model lets both the agent and the user promote, demote, and deprecate, with a user-decision guard that ensures a human override sticks. Durable facts hold full-fidelity key-value information that is never summarized, while notes give each topic an ephemeral scratchpad. Lifecycle triggers and a debounce ensure topics are summarized whenever the user stops engaging — not only on a clean exit.

The agent remembers what matters, forgets what does not, and lets a human settle the difference when they disagree. That is advanced long-horizon memory management — and it is open for inspection.