Context Builder Design — Preventing Orchestrator God-Object Growth

The Context Builder's position in the request flow: it is not an unconditional pre-Router step. The Router runs first, on the raw query, and the Orchestrator invokes the Context Builder only on the low-confidence/no-route branch, before FallbackAgentSkill. PassthroughContextBuilder (Phase 1, no-op) is superseded in two steps as Phase 2 lands: RagContextBuilder (Milestone 1, rag-design.md §4 — RAG retrieval only), then RagMemoryContextBuilder (Milestone 4, rag-design.md §4 / memory-design.md §5 — adds memory/service.py). The second step needs agent_context.user_id (present from the Auth Foundation milestone, security-design.md §4) to scope long-term memory lookups. The Context Builder is also reached more often from Phase 2 onward than in Phase 1 — routing-design.md §5b's reference/comparison gate routes here queries that would otherwise have gone high-confidence, specifically so they can reach memory. Position in the flow (§2) and every interface/anti-pattern below apply to all three implementations equally.

1. Why This Layer Exists

In the original design, the Orchestrator called the Router directly with a raw query. That's fine with zero context sources. It stops being fine the moment Phase 2 adds memory retrieval and RAG retrieval, which need to reach Skills as citations. Without a dedicated seam, that assembly logic has exactly one place to land: inside the Orchestrator, growing it into a God Object that knows about memory stores, vector search, user preferences, and routing all at once.

The Context Builder is introduced now — as a no-op — specifically so that seam exists architecturally before it's load-bearing, and so Phase 2 changes one small module instead of the Orchestrator.

Revision 3 narrows when that seam is load-bearing, not why it exists: performance analysis (langchain-rag-router-decision.md) found that routing doesn't need retrieval to make its decision — the Router's YAML pattern-matching is cheap and shouldn't pay for embedding + vector search on every request. Retrieval earns its cost specifically on the case where a query doesn't map to a clean Tool call and the Router already can't confidently route it — which is also the one case where the God-Object risk this layer prevents is real. High-confidence requests never need this seam at all.

2. Position in the Architecture (Revision 3 — conditional, post-Router)

API → Orchestrator → Router (scores the RAW query)
                          │
        ┌─────────────────┴─────────────────┐
        ▼                                    ▼
  HIGH CONFIDENCE                    LOW CONFIDENCE / NO ROUTE
        │                                    │
        ▼                                    ▼
      Skill                          Context Builder → Fallback Agent Skill

The Context Builder no longer sits unconditionally between the Orchestrator and the Router. It is invoked by the Orchestrator only on the low-confidence/no-route branch, after the Router has already produced a RoutingDecision — never before routing, and never on the high-confidence path. The Router still does not call it directly (unchanged from Revision 2) — that would break the Router's "pure function of the raw query" contract from routing-design.md §1, which Revision 5 of that document makes explicit.

On the high-confidence branch, the Orchestrator does not call the Context Builder at all — it constructs a trivial RequestContext inline instead (see architecture-overview.md §8, "How SkillRequest.context gets populated on each branch"). This is why high-confidence Skills can be described as never having the Context Builder "as a dependency" even in Phase 1, when its output would be identical either way: the component capable of doing retrieval is structurally absent from that path, not merely returning an empty result.

3. Responsibilities

4. Interface (Stable From Phase 1 Onward)

ContextBuilder (abstract):
  async def build(query: str, agent_context: AgentContext) -> RequestContext

RequestContext:
  user_query: str
  conversation_history: list[ConversationTurn]   # [] in Phase 1; Phase 2+ populated here
  retrieved_context: list[RetrievedItem]           # [] in Phase 1; Phase 2+ populated here
  memory_items: list[MemoryItem]                     # new, Phase 2 (domain-model-design.md Revision 5); [] until then
  session_id: str | None
  trace_id: str
  user_id: str | None                                    # new, Phase 2 (Auth Foundation milestone); None until then

Synced to domain-model-design.md §3, the authoritative Models layer definition: query → user_query; entities removed (never actually owned by the Context Builder — it now lives solely on RoutingDecision.matched_entities, computed by the Router, per routing-design.md §2); conversation_history added, typed and always empty in Phase 1, the same pattern already used for retrieved_context. memory_items/user_id are Revision 5 additions (Phase 2) — additive only, no signature change to build() itself.

5. Phase 1 Implementation

# context_builder.py — Phase 1 (no-op)
class PassthroughContextBuilder(ContextBuilder):
    async def build(self, query: str, agent_context: AgentContext) -> RequestContext:
        return RequestContext(
            user_query=query,
            conversation_history=[],
            retrieved_context=[],
            session_id=agent_context.session_id,
            trace_id=agent_context.trace_id,
            # memory_items/user_id aren't set here — neither field exists on
            # RequestContext or AgentContext yet in Phase 1; they're added in
            # Milestones 3/4 respectively (rag-design.md §4).
        )

No memory store, no vector index, no external calls — this exists purely to establish the interface boundary and give the Orchestrator and Skills (via SkillRequest.context) a stable contract to code against. The Router is not a consumer of this type at all (routing-design.md §2) — it scores the raw query directly.

6. Lifecycle (Revision 3 — conditional invocation)

• Instantiated once at application startup, like LLMService and the registries, and injected into the Orchestrator.
• Invoked zero or one times per incoming request — zero on the high-confidence branch (Orchestrator builds RequestContext inline instead, §2), exactly once on the low-confidence/no-route branch, after the Router has returned its RoutingDecision and before FallbackAgentSkill executes.
• Stateless across requests in Phase 1 (no caching, no session store) — statefulness arrives with Phase 2's memory implementation.

7. Extension Points for Phase 2+

PassthroughContextBuilder is swapped in two steps, each a one-line wiring change at the Orchestrator's single call site:

• Milestone 1 — RagContextBuilder (rag-design.md §4): implements the same ContextBuilder interface, internally calling rag/retriever.py (LangChain retriever/vectorstore/embedding components) — mapped to the typed RequestContext.retrieved_context field before leaving the component, so LangChain never leaks past this boundary (ADR-009-langchain-scope.md).
• Milestone 4 — RagMemoryContextBuilder (rag-design.md §4, memory-design.md §5): replaces RagContextBuilder, adding a call to memory/service.py (MemoryService) alongside the existing RAG retrieval, populating conversation_history/memory_items.

Both depend on backend/src/rag/ and/or backend/src/memory/, not on new Tools — retrieval and memory are Context-Builder-internal concerns, not Tool-layer ones (Tools remain external/optional data fetches per ADR-004-llm-service-layer.md). The conditional (fallback-only) call site does not change at either milestone — Phase 2 changes what happens inside the call, not when the call happens, and not how often: routing-design.md §5b's reference/comparison gate widens which queries reach this call site, but the Orchestrator's own branch logic (route_name is None or low_confidence) is untouched.

No change to Router, Skills, or Orchestrator control flow. RequestContext gains two additive fields over the course of Phase 2 (user_id in Milestone 3, memory_items in Milestone 4 — domain-model-design.md); retrieved_context/conversation_history were already typed and waiting to be populated.

8. Anti-Patterns Explicitly Rejected

• ❌ Orchestrator performing memory/RAG retrieval inline "for now, since it's simple" — this is precisely the God Object growth path this layer exists to prevent, even though Phase 1's own implementation is equally simple; the point is where the logic lives, not how complex it currently is.
• ❌ Orchestrator calling the Context Builder unconditionally, on every request including high-confidence ones — this was this document's own Revision 1–2 wiring, and Revision 3 corrects it specifically because an unconditional call would force high-confidence requests to pay Phase 2's real retrieval cost/latency for no benefit (langchain-rag-router-decision.md).
• ❌ Router calling the Context Builder itself, or depending on RequestContext at all — keeps the Router a pure function of the raw query, callable and testable in isolation, regardless of what the Context Builder does internally.
• ❌ Skills calling the Context Builder directly — Skills receive RequestContext from the Orchestrator only, preserving one-way data flow.