MarketCompass — LangChain & Conditional RAG Decision (Context Summary)

Project: MarketCompass (CompassFoundry Labs) — AI equity research platform Existing architecture: UI → API → Agent Orchestrator → YAML Confidence Router → Skills → Tools, with a separate LLM Service (not a Tool) and a Models layer of Pydantic contracts between every layer. Full details in architecture-overview.md and phase1-implementation-plan.md (not attached here — this doc is a standalone decision summary, not a replacement for them).

This is a discussion-and-decision summary. The binding decisions are also recorded in architecture-overview.md §8, ADR-007-rag-strategy.md, ADR-009-langchain-scope.md, and the invariants in docs/project-context.md §2.

Question this answers

Should LangChain be adopted, and if so, where — without disrupting the existing Router/Skills/Orchestrator design?

Decision

LangChain is scoped to three isolated implementation pockets. It is never adopted as the orchestration framework — the YAML Router and Orchestrator control flow are unchanged.

1. LLM Service (backend/src/llm/providers/*) — LangChain's provider wrappers (langchain-anthropic, langchain-openai, etc.) replace hand-rolled vendor SDK calls. BaseProvider.complete()/.stream() contract is unchanged; nothing above LLMService is affected.
2. Context Builder (Phase 2, RAG) — LangChain's retriever/vectorstore/embedding components used internally. Output is still mapped to the existing typed RequestContext.retrieved_context field before leaving the component.
3. Fallback Agent Skill (new, Phase 2+) — a Skill invoked only when RoutingDecision.low_confidence == true (or no route matched). Its execute() internally uses a LangChain agent to reason over which Tools to call, since this is the one case where the sequence of steps genuinely isn't known in advance. Still returns a typed SkillResult, like every other Skill — indistinguishable to the Orchestrator from a normal Skill.

Rule of thumb applied throughout: LangChain is used only inside a component whose external contract is already a typed Pydantic model. Nothing upstream (Router, Orchestrator, Skill Factory) ever imports or is aware of LangChain.

Where RAG retrieval runs (performance-driven ordering)

• Router runs first, on the raw query — not on a RequestContext that already has retrieval populated. Router's YAML pattern-matching is cheap; it shouldn't pay for embedding + vector search on every request, including trivial high-confidence ones.
• Context Builder (retrieval) is called only on the fallback branch, after the Router has already decided it can't confidently route. Orchestrator owns this: it checks RoutingDecision, and only then conditionally calls ContextBuilder.build(...) before invoking the Fallback Agent Skill.
• High-confidence Skills never see Context Builder — not injected, not optional, simply absent as a dependency.

Why (reasoning, not just the rule)

Current Tools (CompanyProfileTool, FinancialMetricsTool, NewsTool, SECFilingsTool) are live, structured lookups — a vector search doesn't improve on a direct API call for something like "AAPL's P/E ratio," and could introduce noise from stale embedded snapshots. High-confidence routing exists specifically because the query maps cleanly to one of these deterministic lookups. RAG earns its cost on the opposite case: queries that don't map to a clean Tool call and need synthesis across unstructured documents — which is why they're low-confidence in the first place.

Two design options considered for "what if a high-confidence Skill someday needs retrieved context" — and the choice

Chosen: Option A. No Skill today has a genuine need for retrieved context (see reasoning above), so building the more general injection pattern now would be solving a problem that doesn't exist yet. Deferring is cheap; the retrofit path is small and local if a real need shows up later.

One open edge case, explicitly deferred (not decided, not urgent)

Could a high-confidence Skill's own internal logic ever require dynamic, runtime-decided planning (e.g., variable number/order of sub-steps depending on an intermediate result), independent of the Router's confidence in routing to it? If so, that Skill could be internally agentic (LangChain agent inside its own execute()), same containment pattern as the Fallback Agent Skill — this would be a property of that specific Skill's complexity, not a change to Router/Orchestrator. Flagged as hypothetical / not a concrete near-term need; no action taken.

Flow (text form, in place of the diagram)

Orchestrator
    │
    ▼
YAML Router  (scores the RAW query — retrieval not required to route)
    │
    ├── HIGH CONFIDENCE ─────────────────────────┐
    │                                             ▼
    │                                           Skill  (fixed, hand-written execute())
    │                                             │
    │                                    ┌────────┴────────┐
    │                                    ▼                 ▼
    │                                  Tools          LLM Service
    │                                                 (LangChain provider
    │                                                  wrappers inside)
    │                                             │
    └── LOW CONFIDENCE / NO ROUTE ─────────────────┼──────────────────┐
                                                    │                  ▼
                                          Context Builder        SkillResult
                                          (LangChain retrieval/       ▲
                                           embeddings inside,         │
                                           called ONLY here)          │
                                                    │                  │
                                                    ▼                  │
                                          Fallback Agent Skill         │
                                          (LangChain AGENT inside      │
                                           — reasons over which        │
                                           Tools to call, since        │
                                           sequence isn't fixed)       │
                                                    │                  │
                                          ┌─────────┴────────┐         │
                                          ▼                  ▼         │
                                        Tools          LLM Service     │
                                                                       │
                                          └──────────────────┴─────────┘
                                                    (both branches return SkillResult)

Three LangChain pockets, marked above: LLM Service (provider mechanics), Context Builder (retrieval, fallback-only), Fallback Agent Skill (agent reasoning). Router and Orchestrator never import or know about LangChain.

Non-goals reaffirmed (unchanged from base architecture)

No LCEL chains, LangChain agents, or LangGraph in Router, Orchestrator, or Skill Factory. No Skill is expressed as a LangChain chain by default — only the Fallback Agent Skill (and, hypothetically, a future dynamically-planning Skill) use LangChain's agent framework internally, and only because their own logic genuinely requires runtime-decided sequencing.