ADR-007: RAG Retrieval Is Conditional (Fallback-Only) and LangChain-Based Internally
Status
Accepted
Context
MarketCompass's Phase 2 roadmap (project-context.md, architecture-handbook.md) commits to retrieval-augmented generation over unstructured sources (company descriptions, earnings calls, analyst notes, financial definitions). Two questions had to be answered before that could be designed in detail: when in the request flow retrieval runs, and what it's built with internally. Both are decided together in langchain-rag-router-decision.md; this ADR records the decision in the project's standard ADR format, alongside ADR-002 (routing) and ADR-009 (LangChain scope), which it depends on.
This ADR does not design retrieval's internals (which vector store, which embedding model, chunking strategy) — those remain open Phase 2 implementation decisions. What's decided now is the shape: where retrieval sits in the control flow, and the boundary around the framework that implements it.
Decision
RAG retrieval is conditional, not unconditional — it runs only on the low-confidence/no-route branch, after the Router, via the Context Builder. It is never a pre-Router step and never reachable from a high-confidence Skill.
Router (scores the RAW query)
│
├── HIGH CONFIDENCE ──────→ Skill (no Context Builder, no retrieval)
│
└── LOW CONFIDENCE / NO ROUTE ──→ Context Builder (RAG) ──→ Fallback Agent Skill
Internally, the Context Builder uses LangChain's retriever/vectorstore/embedding components (Phase 2+), mapping their output to the existing typed RequestContext.retrieved_context field before it leaves the component — LangChain never leaks past that boundary (ADR-009).
Phase 1 ships the wiring, not the retrieval: ContextBuilder.build(query, agent_context) -> RequestContext is a no-op passthrough, invoked at the same conditional call site retrieval will occupy from Phase 2 onward, so Phase 2 changes the Context Builder's internals only, not its position or any caller.
Rationale: Why Conditional, Not Unconditional
| Consideration | Unconditional (RAG before every route) | Conditional, fallback-only (adopted) |
|---|---|---|
| Cost/latency on high-confidence requests | Every request pays embedding + vector search, even "AAPL's P/E ratio" | Zero — high-confidence requests never reach the Context Builder |
| Fit with current Tools | CompanyProfileTool/FinancialMetricsTool/NewsTool/SECFilingsTool are live, structured lookups — a vector search over stale embedded snapshots doesn't improve on a direct API call |
Retrieval is reserved for exactly the case it earns its cost: queries that don't map to a clean Tool call and need synthesis across unstructured documents |
| Router contract | Router would need RequestContext as input, coupling its cost to retrieval's cost/availability |
Router stays a pure function of the raw query (ADR-002) — retrieval's cost profile can change freely in Phase 2+ without touching the Router |
| Consistency with why a query was low-confidence in the first place | N/A | Low confidence already means "doesn't map cleanly to a deterministic route" — the same condition that makes retrieval-based synthesis valuable, not incidental |
Rationale: Why LangChain for the Retrieval Internals
Building a bespoke retriever/vectorstore/embedding integration layer would duplicate functionality LangChain already provides and maintains across multiple vector store and embedding provider backends — undifferentiated plumbing, not something this project's retrieval quality depends on being hand-rolled. Confined per ADR-009 to inside the Context Builder, behind the same typed RequestContext.retrieved_context contract that existed before this decision (domain-model-design.md §3) — nothing upstream (Router, Orchestrator, Skills other than FallbackAgentSkill) needs to change or even know LangChain is involved.
Alternatives Considered
A. Unconditional Context Builder before every route — rejected; see rationale table above. This was this project's own Phase 1 placeholder wiring in earlier revisions of context-builder-design.md, corrected by this decision specifically because Phase 2 would have made that placeholder's zero cost stop being true.
B. Skill-owned, injectable Context Builder (any Skill can request retrieval as a dependency) — deferred, not rejected; see langchain-rag-router-decision.md's Option A/B table and ADR-009. No Skill today has a concrete need for retrieved context; the retrofit if one arises is small and local.
C. Hand-rolled retrieval (no LangChain) — rejected per ADR-009: undifferentiated plumbing LangChain already solves, and LangChain is already a project dependency.
Consequences
Positive:
- The differentiator this project already claims — cheap, explainable, deterministic routing (
ADR-002) — stays true even after RAG is added; retrieval cost is opt-in by construction, not by discipline. - RAG's own placement adds no Router, Orchestrator, or high-confidence Skill code changes — it's scoped to the Context Builder's internals only. (Phase 2 as a whole does touch the Router and Orchestrator, but for reasons unrelated to RAG placement:
routing-design.md§5b's reference/comparison gate and the Milestone 2 Orchestrator wiring swap toFallbackAgentSkill— seephases/phase2-implementation-plan.md.)
Negative / accepted tradeoffs:
- A high-confidence route that later turns out to genuinely benefit from retrieved context (e.g., a company summary enriched with an analyst note) doesn't get it automatically — that's a deliberate scope boundary (Option B above), not an oversight, and the retrofit path is already designed.
- Retrieval quality/recall tuning (chunking, embedding model choice, vector store) is unresolved by this ADR — intentionally, since none of that is decidable before Phase 2's actual data sources exist.
Revisit Trigger
If observability data (once it exists, per observability-design.md) shows a specific, recurring high-confidence route whose answers would materially improve with retrieved context, revisit via the Option B retrofit path already designed — not by making the Context Builder unconditional again.