Memory Design — Short-Term Buffer and Long-Term Store

Implements the what behind ADR-006-memory-strategy.md (two tiers, fallback-only, user_id-scoped long-term) and ADR-010-rag-memory-tech-stack.md (SQLite via SQLAlchemy). Delivered by the Memory milestone, after Auth Foundation, in phases/phase2-implementation-plan.md.

1. Position in the Architecture

Memory is the Context Builder's second Phase 2+ responsibility, alongside RAG (rag-design.md) — same conditional call site, same ContextBuilder.build() interface, never reachable from a high-confidence Skill. See ADR-006 for the full rationale on why memory is fallback-only rather than always-on, and how the Router's reference/comparison gate (routing-design.md §5b) plus the per-Skill backstop prompt instruction (skills-design.md §2) mitigate the resulting gap for high-confidence queries that implicitly depend on prior context.

LOW CONFIDENCE / NO ROUTE
        │
        ▼
Context Builder
 ├── Short-term: in-process TTL buffer, keyed by session_id
 └── Long-term:  SQLite, keyed by user_id
        │
        ▼
RequestContext { conversation_history, memory_items, ... }
        │
        ▼
Fallback Agent Skill

2. Package Layout

backend/src/memory/
├── short_term.py     # in-process TTL buffer, session-scoped
├── long_term.py        # SQLAlchemy models + repository, user-scoped
└── service.py             # MemoryService — the single seam the Context Builder and Orchestrator call (§5)

config/memory/
├── short_term.yaml    # TTL / max-turns-per-session
└── long_term.yaml       # SQLite file path, retention policy

3. Short-Term Memory

An in-process structure (e.g. a bounded deque per session_id, evicted on TTL expiry) — never written to SQLite or any other store. Holds recent conversation turns for the current session only, feeding RequestContext.conversation_history (already typed since Phase 1 — domain-model-design.md §3 — populated for the first time here).

Isolation key: session_id (AgentContext.session_id, minted per session since Phase 1).
Lifetime: bounded by TTL/max-turns config (config/memory/short_term.yaml); lost on process restart — this is by design, not a gap, since short-term memory's job is same-session continuity, not durability.
Cost: an in-memory read, no I/O — safe to keep on the fallback-only seam without needing to justify against a latency budget the way RAG's vector search does.

4. Long-Term Memory

SQLite via SQLAlchemy models, scoped to user_id. Requires Auth Foundation (Milestone 3) to exist first — before real user identity is established, there is no durable, cross-session key to store long-term memory against (ADR-006 §"Why user_id, Not session_id").

# backend/src/memory/long_term.py — illustrative shape, not final schema
class MemoryRecord(Base):
    __tablename__ = "memory_records"
    id: Mapped[int] = mapped_column(primary_key=True)
    user_id: Mapped[str] = mapped_column(index=True)
    content: Mapped[str]
    created_at: Mapped[datetime]
    # retention/expiry columns per config/memory/long_term.yaml — exact schema is
    # implementation-time detail, not an architecture decision

MemoryService (service.py) is the only thing the Context Builder and Orchestrator call — it internally decides what's worth persisting from a completed fallback-branch interaction and what to retrieve for a new one. Neither short_term.py nor long_term.py is called directly by anything outside memory/.

5. Interface — Called by the Context Builder and Orchestrator

# backend/src/memory/service.py
class MemoryService:
    async def load(self, agent_context: AgentContext) -> MemorySnapshot:
        """Returns short-term conversation_history (session_id) + long-term memory_items (user_id)."""

    async def record(self, agent_context: AgentContext, turn: ConversationTurn) -> None:
        """Appends to the short-term buffer; long-term persistence is a MemoryService-internal policy
        decision (e.g. periodic summarization), not triggered per-turn by the caller."""

class MemorySnapshot(BaseModel):
    conversation_history: list[ConversationTurn]
    memory_items: list[MemoryItem]

RagMemoryContextBuilder.build() (rag-design.md §4) calls MemoryService.load() alongside RAG retrieval and maps the result directly onto RequestContext.conversation_history / .memory_items — both already-typed fields (domain-model-design.md §3, §8 forward-compat table), so this is populating existing contract shape, not a schema change.

Caller of record(): the Orchestrator, not a Skill and not the Context Builder. build() runs before FallbackAgentSkill.execute(), so it has no result to record yet; no Skill — including FallbackAgentSkill — ever calls MemoryService directly (§9's anti-pattern). The Orchestrator calls record() twice, both after FallbackAgentSkill returns its SkillResult, on the same fallback branch where it called build():

record(agent_context, ConversationTurn(role="user", content=request.context.user_query))
record(agent_context, ConversationTurn(role="assistant", content=skill_result.content))

Both calls happen post-Skill, not one before and one after, because record()'s single-turn signature (§5 below) has no way to express "this pair belongs together" across two calls made at different points in the request lifecycle — recording them back-to-back after the Skill returns keeps insertion order (user then assistant) correct in the short-term buffer with a single, easy-to-place call site, and avoids a partial buffer write if the Skill invocation fails before producing a SkillResult. This is a Milestone 4 implementation task (phases/phase2-implementation-plan.md Milestone 4 checklist), not left to the implementer to improvise — symmetric pre/post hooks around the one Skill invocation that has memory as a dependency, both owned by the Orchestrator, mirroring how it already owns the conditional build() call (context-builder-design.md §2). High-confidence Skills are never in this path, so this adds no new call for them.

6. `RequestContext.memory_items` — New Field

class MemoryItem(BaseModel):
    content: str
    created_at: datetime
    # Phase 1: type does not exist. Phase 2: populated by MemoryService.load() via the Context Builder.

Added to RequestContext per domain-model-design.md (Revision 5) — additive only, no change to any other model or to SkillRequest/BaseSkill.execute().

7. `user_id` Dependency

AgentContext gains user_id: str in Milestone 3 (Auth Foundation), resolved once at the API layer from the JWT-authenticated request (security-design.md §4) and carried through exactly like session_id/trace_id are today (backend/src/agents/base/agent_context.py). Long-term memory is the first consumer of this field; RAG and short-term memory don't need it. RequestContext mirrors it (user_id: str | None = None, domain-model-design.md §3) so FallbackAgentSkill can reference it if it ever needs to (e.g. for its own logging), though MemoryService is the only component that actually queries by it.

8. Testing Approach

Unit tests: short_term.py's TTL/eviction logic with a fake clock, no I/O; long_term.py's SQLAlchemy models against an in-memory SQLite instance; MemoryService.load()/.record() with both sub-stores mocked.
Contract test: RagMemoryContextBuilder.build() (shared with rag-design.md §7) confirms memory_items/conversation_history are populated only on the fallback branch, never on the high-confidence path — an integration-level assertion that composes with the existing Milestone 8 Context-Builder-invocation-count test.
FallbackAgentSkill: never receives or calls MemoryService (§9) — its unit tests construct RequestContext with fixture conversation_history/memory_items already populated, the same way retrieved_context fixtures are used, and assert the agent prompt includes them (skills-design.md §3a). There is no MemoryService mock in these tests because the Skill has no such dependency to mock.
Orchestrator: integration test confirms MemoryService.record() is called exactly twice (user turn, then assistant turn, in that order) after FallbackAgentSkill returns on the fallback branch, and never called on the high-confidence branch.

9. Anti-Patterns Explicitly Rejected

❌ Any component other than MemoryService reading long_term.py's SQLAlchemy models directly — memory/ is a sealed package with one public seam, mirroring how Tools/llm/ are internal to their owning layer.
❌ Persisting short-term buffer contents to SQLite "in case it's useful later" — short-term is deliberately disposable; anything worth keeping long-term goes through MemoryService's own persistence policy, not an automatic write-through.
❌ Scoping long-term memory by session_id — defeats the purpose of the tier; see ADR-006.
❌ Any Skill — high-confidence or FallbackAgentSkill — calling MemoryService directly — memory reaches Skills only via RequestContext, produced by the Context Builder on the fallback branch, same rule as RAG (context-builder-design.md §8). FallbackAgentSkill is not an exception: it reads conversation_history/memory_items off RequestContext like retrieved_context, and never receives MemoryService as a dependency.