ADR-006: Memory Strategy — Two Tiers, Fallback-Only, User-Scoped

Status

Accepted

Context

Phase 2 adds conversational memory: the ability for the system to recall earlier turns in a session and durable facts/preferences across sessions. Two questions had to be answered before this could be designed in detail: where memory sits in the request flow (which components can reach it), and how many tiers it has and what each tier persists. This ADR decides both, mirroring how ADR-007-rag-strategy.md decided RAG's shape before rag-design.md detailed its internals. The specific storage technology (SQLite via SQLAlchemy) is decided separately in ADR-010-rag-memory-tech-stack.md — this ADR is about strategy, not implementation.

Decision

Memory has two tiers, and both are reachable only through the Context Builder, on the same low-confidence/no-route branch RAG already occupies. Memory is never available to high-confidence Skills, and the Router never depends on it.

Router (scores the RAW query)
   │
   ├── HIGH CONFIDENCE ──────→ Skill (no Context Builder, no memory, no RAG)
   │
   └── LOW CONFIDENCE / NO ROUTE ──→ Context Builder (RAG + Memory) ──→ Fallback Agent Skill

RequestContext (domain-model-design.md §3) gains a memory_items: list[MemoryItem] field alongside the existing retrieved_context: list[RetrievedItem] (RAG) and conversation_history: list[ConversationTurn] (already typed, populated for the first time in Phase 2) — populated by the Context Builder the same way retrieval is, and empty on every high-confidence request.

Rationale: Why Fallback-Only, Not Always-On

The alternative — the Orchestrator attaching memory to every request, including high-confidence ones — was considered and rejected for Phase 2:

The residual gap this creates: a query with an explicit entity and an implicit reference to something else — e.g. "Compare AAPL's P/E to the one I asked about earlier" — clears the entity gate and routes high-confidence to FinancialSkill, which never sees memory. This is closed by a Router-side fix, not a memory-scope change: routing-design.md §5b (Phase 2) adds a reference/comparison-cue gate that demotes such queries to low_confidence=true even when an entity matched, so they still reach the Context Builder + FallbackAgentSkill. Memory's own scope stays fallback-only; the Router gets better at recognizing which queries actually need that branch. As a backstop for whatever the gate still misses, every Skill's system prompt (skills-design.md §2, Phase 2 addition) is instructed to explicitly flag — not silently drop — any part of the query it lacks data or context for, so a missed case degrades visibly instead of quietly.

Rationale: Why Two Tiers, Not One

A single persisted tier (everything in SQLite) would pay a durable-write cost for data that's only ever useful within one session (e.g., "the company I just asked about" two turns ago) and would need explicit expiry logic to avoid unbounded growth. A single in-process tier (nothing persisted) can't answer "what did I tell you about my portfolio last week," which is squarely what "long-term memory" is supposed to mean. Splitting matches the actual access pattern: short-term is read far more often and needs to be fast and disposable; long-term is read less often but must survive a restart.

Rationale: Why user_id, Not session_id, for Long-Term Scoping

Long-term memory is durable preference/fact storage that should follow a person across sessions and devices — a new session_id is minted per conversation (AgentContext.__init__), so scoping long-term storage to it would make memory effectively session-scoped, defeating the point of a "long-term" tier. This requires Phase 2 to introduce real user identity, which is why Milestone 3 (Auth Foundation — FastAPI Users, JWT, per security-design.md §4) is sequenced immediately before the Memory milestone in phases/phase2-implementation-plan.md, rather than deferred to a later phase. Short-term memory keeps session_id scoping — it has no reason to survive past the session it buffers.

Alternatives Considered

A. Always-on memory (Orchestrator-level, every branch) — rejected; see rationale table above.

B. Single persisted tier only — rejected; see "Why Two Tiers" above.

C. session_id-scoped long-term memory, no auth in Phase 2 — considered and rejected during this session's planning; without real user identity, "long-term" memory can't outlive the session it was created in, which contradicts what the tier is for. Rejecting this option is what pulled Auth Foundation into Phase 2 rather than deferring it.

Consequences

Positive:

• No invariant changes; memory and RAG share one seam, one mental model, one set of anti-patterns (context-builder-design.md §8).
• Long-term memory is genuinely durable and personal from the moment it exists, not a session-scoped placeholder that needs a later migration once real accounts arrive.

Negative / accepted tradeoffs:

• Phase 2 grows by one milestone (Auth Foundation) beyond the original ~4-milestone estimate, specifically to make user_id scoping meaningful now rather than later.
• High-confidence Skills still don't get memory automatically; the Router reference-gate + Skill backstop-prompt combination is a deliberate mitigation, not a guarantee of catching every phrasing.

Revisit Trigger

If observability data shows the Router's reference/comparison-cue gate (routing-design.md §5b) is missing a specific, recurring phrasing pattern, extend the gate's cue list first. If a high-confidence Skill is later found to have a genuine, recurring need for memory (not just an edge case), revisit via the same small/local retrofit path already documented for RAG (architecture-overview.md §8, "What if a high-confidence Skill someday needs retrieved context?") — not by making the Context Builder unconditional.