ADR-009: Scope LangChain to Three Isolated Pockets, Never the Orchestration Framework

Status

Accepted

Context

LangChain (and langchain-openai/langchain-mcp-adapters) were already project dependencies (backend/pyproject.toml) before this decision, but nothing in the architecture said where — or whether — they should be used. Meanwhile Phase 2 needs RAG retrieval, and the low-confidence/no-route case needs something to reason over which Tools to call when the sequence isn't fixed in advance. Both are exactly the kind of problem LangChain is built for. The risk is scope creep: "adopt LangChain" can silently become "LangChain becomes the orchestration framework," which would directly conflict with ADR-002-yaml-routing.md's YAML Router and this architecture's explicit Router/Orchestrator/Skill-Factory design.

Full decision record and reasoning: langchain-rag-router-decision.md.

Decision

LangChain is scoped to exactly three isolated implementation pockets, each inside a component whose external contract is already a typed Pydantic model. It is never adopted as the orchestration framework — the YAML Router and Orchestrator control flow are unchanged, and Router/Orchestrator/Skill Factory never import or know LangChain exists.

1. LLM Service (backend/src/llm/providers/*) — LangChain's provider wrappers (langchain-anthropic, langchain-openai, etc.) replace hand-rolled vendor SDK calls. BaseProvider.complete()/.stream() is unchanged.
2. Context Builder (Phase 2+) — LangChain's retriever/vectorstore/embedding components, used internally. Output is mapped to the existing typed RequestContext.retrieved_context before leaving the component.
3. Fallback Agent Skill (new, Phase 2+) — invoked only when RoutingDecision.low_confidence == true or no route matched. Its execute() uses a LangChain agent internally to decide which Tools to call and in what order — the one case where that sequence genuinely isn't known ahead of time. Still returns a typed SkillResult, indistinguishable to the Orchestrator from any other Skill.

A second, coupled decision: the Router runs first, on the raw query, and the Context Builder is invoked by the Orchestrator only on the low-confidence/no-route branch — never unconditionally, never before routing. See ADR-002 and routing-design.md §2 (Revision 5).

Alternatives Considered

A. Don't adopt LangChain at all; keep hand-rolled provider SDK calls and write a bespoke fallback tool-selection loop Rejected: LangChain is already a dependency, provider wrapper coverage (Anthropic/OpenAI/Ollama/OpenRouter-compatible) is exactly the kind of undifferentiated plumbing not worth hand-maintaining, and a bespoke agent loop for the fallback case would duplicate what LangChain's agent abstraction already does correctly (tool-calling loop, message formatting, stop conditions).

B. Adopt LangChain as the orchestration layer — replace the YAML Router and/or Orchestrator with LangChain/LangGraph constructs Rejected: directly contradicts ADR-002's cost/latency/explainability rationale for rule-based routing (a LangGraph or agent-based router would reintroduce LLM cost and non-determinism into a decision that's supposed to be free and instant) and this architecture's explicit non-goal of no dynamic/plugin discovery or graph orchestration frameworks (architecture-overview.md §18).

C. Run Context Builder (RAG) unconditionally, before the Router, for every request Rejected: performance-driven. The Router's YAML pattern-matching is cheap and shouldn't pay for embedding + vector search on every request, including trivial high-confidence ones. Current Tools (CompanyProfileTool, FinancialMetricsTool, NewsTool, SECFilingsTool) are live, structured lookups that a vector search doesn't improve on — RAG earns its cost specifically on the low-confidence case, where a query doesn't map to a clean Tool call.

D. Make Context Builder an injectable dependency any Skill can request (Skill-owned, not Orchestrator-owned) Considered and deferred, not rejected outright — see langchain-rag-router-decision.md's Option A/B comparison. Rejected for now: no Skill today has a genuine need for retrieved context, so building the more general injection pattern would solve a problem that doesn't exist yet. The retrofit path (promote one Skill to receive ContextBuilder via injection, same pattern FallbackAgentSkill uses) is small and local if a real need arises later.

Consequences

Positive:

• Every LangChain touchpoint sits behind a contract (BaseProvider, RequestContext, SkillResult) that already existed before this decision — nothing upstream needs to change to accommodate it.
• High-confidence requests (the common case) never pay LangChain's retrieval or agent-loop cost/latency — that machinery is structurally absent from that path, not merely unused.
• The Router, Orchestrator, and Skill Factory remain independently testable and LangChain-version-upgrade-proof, since none of them import it.

Negative / accepted tradeoffs:

• Two different "shapes" of Skill now exist — fixed hand-written Tool sequences (most Skills) vs. an internally agentic one (FallbackAgentSkill) — a real asymmetry, accepted because it's confined to exactly the one case where the sequence isn't known in advance, not a general pattern every Skill must support.
• RequestContext's producer changes by branch (Orchestrator inline vs. Context Builder) rather than always being the same component — a small addition to hold in mind, mitigated by both paths producing the identical typed shape (domain-model-design.md §3, Revision 4).

Revisit Trigger

If a high-confidence Skill develops a genuine, concrete need for retrieved context (Option D above), or if a high-confidence Skill's own internal logic needs dynamic runtime-decided planning independent of the Router's confidence in routing to it — revisit then, using the same containment pattern already established for FallbackAgentSkill, rather than generalizing the injection pattern speculatively now.

Interview Talking Point

The interesting decision here isn't "should we use LangChain" — it's "where does a general-purpose framework's blast radius stop." Confining it to three call sites behind pre-existing typed contracts, rather than letting it become the control-flow substrate, is what keeps the Router/Orchestrator's cost, latency, and testability guarantees intact even as the parts of the system that need agentic reasoning (retrieval, dynamic tool selection) grow around them.