Skip to content
← How we build

architecture

MarketCompass — Architecture Overview

Status: Design for implementation Owner: CompassFoundry Labs Project: AI-powered equity research platform. This is not a greenfield project — an existing skeletal frontend + backend already works; this document describes the target architecture to implement into it.


1. Why This Architecture

MarketCompass's eventual Analyst Agent (Phase 4) needs a stable seam between four concerns: what should handle this request (Router), how execution is carried out (Orchestrator), what business capability is invoked (Skills), and what does the work (Tools + LLM Service). Cutting that seam once makes Phases 2–4 additive rather than a redesign.

2. Package Map

Every package below is real as of Phase 1; §12's diagram and §13's contract table are the authoritative description of how they fit together. chat.py is the API entry point behind which the Orchestrator sits; /api/chat/stream still runs the pre-Orchestrator ChatAgent/legacy route_request() path (phase1-implementation-plan.md Milestone 8), a known parallel path, not a gap.

backend/src/
├── api/                # FastAPI app factory, routes (health, chat, agents, providers)
├── agents/              # Orchestrator, AgentContext, ContextBuilder, legacy ChatAgent (/api/chat/stream only)
├── router/                # Router — formalized by routing-design.md
├── skills/                 # Skill classes + Skill Factory
├── tools/                    # Tool classes + Tool Registry
├── services/                   # LLMService (llm/ package underneath — factory, provider_catalog, LangChain provider wrappers)
├── models/                       # Pydantic contracts — domain-model-design.md
├── observability/                  # log_event / record_metric / start_span facade
├── core/                              # logging, tracing primitives — wrapped by observability/, not modified
├── streaming/                           # SSE / token streaming, unaffected by any layer above
└── config/                                # settings, validation, paths
config/
├── routing/    # skills_routing.yaml, thresholds.yaml
├── skills/      # registry.yaml
└── tools/         # registry.yaml
evaluations/        # placeholder — §7

Phase 2+ additions (not yet built — phases/phase2-implementation-plan.md): backend/src/rag/, backend/src/memory/, backend/src/skills/fallback_agent_skill.py, config/rag/, config/memory/, top-level data/.

3. LLM as a Platform Service, Not a Tool

The LLM is not modeled as LLMTool, a peer of NewsTool/SECFilingsTool. It's the reasoning engine every Skill depends on, not an optional external integration — so LLM-specific concerns (retries, failover, cost tracking, prompt versioning) live in a dedicated LLMService, not a generic Tool contract.

Skill
 ├── Tool Dependencies (0..n)      -- external data: CompanyProfileTool, NewsTool, etc.
 └── LLM Service Dependency (1)    -- the platform's reasoning engine
Skill Tool Dependencies LLM Service Dependency
CompanySkill CompanyProfileTool LLMService
FinancialSkill FinancialMetricsTool, CompanyProfileTool LLMService
NewsSkill NewsTool LLMService
GenericChatSkill (none) LLMService
FallbackAgentSkill (Phase 2+, §8) any, chosen at runtime by an internal LangChain agent LLMService

LLMService is injected into every Skill the same way — not optional, not looked up through the Tool Registry. Resolved once at Orchestrator/Skill-construction time from a single, application-scoped instance (llm-service-design.md).

Internally, LLMService uses LangChain's provider wrappers (langchain-anthropic, langchain-openai, etc.) instead of hand-rolled vendor SDK calls. LLMService's external contract (generate()/stream()) is unchanged — nothing above it is affected. See §8 for the full scope of where LangChain is (and isn't) used.

4. Centralized Configuration Hierarchy

Registries live in config/, not inside source packages — consistent with routing config and the project's "no code deployment for config changes" principle.

config/
├── routing/
│   ├── skills_routing.yaml
│   └── thresholds.yaml
├── skills/
│   └── registry.yaml
├── tools/
│   └── registry.yaml
├── rag/                    # Phase 2+ — rag-design.md §2
│   ├── ingestion.yaml
│   ├── vectorstore.yaml
│   └── embeddings.yaml
└── memory/                 # Phase 2+ — memory-design.md §2
    ├── short_term.yaml
    └── long_term.yaml

backend/src/skills/, backend/src/tools/, backend/src/rag/, and backend/src/memory/ contain only code (base contracts, concrete implementations, registry/config loader logic) — never data. backend/src/config/settings.py holds explicit paths to each registry/config file (SKILLS_REGISTRY_PATH, TOOLS_REGISTRY_PATH, and — Phase 2+ — the config/rag//config/memory/ paths).

5. LLM Service Layer

Skills
   ↓
LLM Service   (backend/src/services/llm_service.py)
   ↓
llm/ package (factory.py, provider_catalog.py, providers/* — LangChain wrappers inside)
   ↓
Provider SDKs (Anthropic, OpenAI, Ollama, OpenRouter)

LLMService is the only thing Skills are allowed to call for inference — no Skill imports llm/factory.py directly. Responsibilities owned here (not by Skills, not by llm/ alone): model/provider selection policy, retry handling, provider failover, token accounting, cost tracking, and prompt versioning. llm/ remains the low-level, provider-specific mechanics; LLMService is the policy layer on top.

6. Observability as a First-Class Layer

Named as an explicit layer, not just "logging exists" — the system's pitch is explainable, confidence-scored routing, which needs to be provable in telemetry, not just in a reasoning string returned once per request.

Category Examples
Structured logging Every request logged with trace_id, route_name, confidence, skills_invoked, tools_invoked
Tracing trace_context.py/tracing.py, spanning Router → Skill → Tool/LLM Service boundaries
Metrics Route selection frequency, average route confidence, tool latency (p50/p95), tool failure rate, LLM token consumption, request cost, response latency
Routing confidence analytics Distribution of confidence scores per route over time
Cost tracking Token usage × provider pricing, surfaced per request and aggregated

Observability sits beside every layer — every layer emits into it; nothing depends on it to function correctly.

7. Evaluation Framework Placeholder

A top-level evaluations/ package, structural only in Phase 1 — not implemented beyond a stub.

evaluations/
├── datasets/          # golden query/route/answer sets, empty in Phase 1
├── routing_eval.py     # will score router accuracy against datasets/
└── response_eval.py    # will score response quality/hallucination in later phases

Distinct from the unit/integration test suite (§14): evaluation answers "is the behavior still good," not "does the code work as written."

8. Router-First Flow, Context Builder, and LangChain Scope

This is the architecture's core control-flow decision. Full reasoning: langchain-rag-router-decision.md.

The Router runs first, on the raw query

The Router's YAML pattern-matching is cheap and shouldn't pay for embedding + vector search on every request, including trivial high-confidence ones. Current Tools (CompanyProfileTool, FinancialMetricsTool, NewsTool, SECFilingsTool) are live, structured lookups — a vector search doesn't improve on a direct API call for something like "AAPL's P/E ratio," and could introduce noise from stale embedded snapshots. High-confidence routing exists specifically because the query maps cleanly to a deterministic lookup. RAG earns its cost on the opposite case: queries that don't map to a clean Tool call and need synthesis across unstructured documents — which is why they're low-confidence in the first place.

So Router.route(query, AgentContext) -> RoutingDecision takes the raw query, not a RequestContext with retrieval already populated. The Context Builder is not an unconditional pre-Router step.

Flow

API → Orchestrator → Router (scores the RAW query)
                          │
        ┌─────────────────┴─────────────────┐
        ▼                                    ▼
  HIGH CONFIDENCE                    LOW CONFIDENCE / NO ROUTE
        │                                    │
        ▼                                    ▼
      Skill                          Context Builder (RAG)
 (fixed, hand-written execute())     (LangChain retrieval/embeddings
        │                             inside, called ONLY here;
   ┌────┴────┐                        output mapped to typed
   ▼         ▼                        RequestContext.retrieved_context)
 Tools   LLM Service                          │
                                               ▼
                                    Fallback Agent Skill
                                    (LangChain AGENT inside —
                                     reasons over which Tools to
                                     call, since sequence isn't fixed)
                                               │
                                          ┌────┴────┐
                                          ▼         ▼
                                        Tools   LLM Service

Both branches return a typed SkillResult to the Orchestrator.

How SkillRequest.context gets populated on each branch

BaseSkill.execute(request: SkillRequest) (skills-design.md §2) is one contract for every Skill — there is no separate signature for high-confidence vs. fallback. What differs is how RequestContext gets built, not its shape:

  • High confidence: the Orchestrator constructs a trivial RequestContext itself, inline — user_query + session_id/trace_id from AgentContext, empty conversation_history/retrieved_context. This is data-wrapping, not a call to the Context Builder component; it costs nothing and requires no dependency injection. This is what "Context Builder absent as a dependency" (§8 below) actually means: the component that can do retrieval is never invoked, not that RequestContext goes unpopulated.
  • Low confidence / no route: the Orchestrator calls ContextBuilder.build(query, agent_context) -> RequestContext instead of building it inline. Phase 1 this returns the identical trivial shape (PassthroughContextBuilder); Phase 2+ it performs real retrieval and populates retrieved_context.

This keeps SkillRequest{context: RequestContext, routing: RoutingDecision} (domain-model-design.md §3) and the BaseSkill.execute() signature unchanged and identical for every Skill, including FallbackAgentSkill — only the construction path is conditional, never the contract.

Context Builder: no-op in Phase 1, real RAG + memory from Phase 2

ContextBuilder.build(query, AgentContext) -> RequestContext is a no-op passthrough in Phase 1 (PassthroughContextBuilder, RequestContext wraps the raw query plus empty retrieved_context/conversation_history). The interface and its conditional (fallback-only) wiring were established in Phase 1 specifically so Phase 2 only changes internals, not callers, in two steps: RagContextBuilder (Milestone 1, rag-design.md §4 — real LangChain-based RAG retrieval only) is later replaced by RagMemoryContextBuilder (Milestone 4, rag-design.md §4, memory-design.md §5 — adds short-/long-term memory), invoked by the Orchestrator only on the low-confidence/no-route branch. Output is mapped to the existing typed RequestContext fields before leaving the component — LangChain never leaks past that boundary. High-confidence Skills never see the Context Builder — not injected, not optional, simply absent as a dependency, except for user_id: whichever code path constructs RequestContext on a given branch populates that field directly from AgentContext.user_id (Phase 2's Auth Foundation milestone) — the Orchestrator's own inline construction on the high-confidence branch, the Context Builder's build() on the fallback branch (the same way it already sets session_id/trace_id) — since it's a zero-cost identity fact, not a retrieval result.

Which queries actually reach this branch widens in Phase 2: routing-design.md §5b adds a reference/comparison-cue gate to the Router — a query that would otherwise route high-confidence but implicitly depends on prior context (e.g. "compare AAPL's P/E to the one I asked about earlier") gets low_confidence=true forced, specifically so it reaches the Context Builder's memory instead of silently losing that half of the question on a Skill that never sees it. See ADR-006-memory-strategy.md for the full reasoning and the accepted residual risk this doesn't catch (mitigated by a backstop instruction in every Skill's system prompt, skills-design.md §2).

LangChain scope

LangChain is scoped to three isolated pockets, each inside a component whose external contract is already a typed Pydantic model. It is never the orchestration framework — the YAML Router and Orchestrator control flow are unchanged, and Router/Orchestrator/Skill Factory never import or know about LangChain.

  1. LLM Service (§5) — provider wrappers, replacing hand-rolled SDK calls.
  2. Context Builder (above) — retriever/vectorstore/embedding components, fallback-only.
  3. Fallback Agent Skill (below) — agent reasoning over which Tools to call.

Fallback Agent Skill

A Skill invoked only when RoutingDecision.low_confidence == true or no route matched. Constructed by the Skill Factory from a config/skills/registry.yaml entry like any other Skill — declares Tool dependencies and an LLMService dependency the normal way, returns SkillResult. The only difference is internal to execute(): instead of a hand-written Tool call sequence, it uses a LangChain agent (create_react_agent + AgentExecutor, text-based ReAct prompting) to decide which Tools to call and in what order, because this is the one case where that sequence genuinely isn't known in advance. Indistinguishable to the Orchestrator from a normal Skill.

AgentExecutor needs a LangChain BaseChatModel to reason with — this is satisfied by a thin adapter (_LLMServiceChatModel, skills-design.md §3a) that wraps the Skill's injected LLMService.generate() rather than a raw provider client, so the agent's internal reasoning calls still go through LLMService's retry/failover/cost-tracking pipeline like every other Skill's LLM calls. Because ReAct reasons entirely in text (no native tool binding), the adapter only needs plain prompt-in/text-out — it never has to bridge LLMRequest/LLMResponse to LangChain's structured tool_calls, so those contracts stay exactly as domain-model-design.md §3 defines them. The adapter lives inside fallback_agent_skill.py, the same "map/adapt at the boundary" placement already used for the BaseToolStructuredTool adapter.

Skill → Tools → LLM Service → SkillResult              (ordinary Skill, e.g. FinancialSkill)
Skill → [LangChain agent decides which Tools, in what order] → LLM Service → SkillResult   (Fallback Agent Skill)

What if a high-confidence Skill someday needs retrieved context?

Deferred, not built speculatively. Orchestrator-owned conditional retrieval (the design above) is chosen over making ContextBuilder an injectable dependency any Skill could request, because no Skill today has a genuine need for retrieved context. If one arises, the retrofit is small and local: promote that one Skill to receive ContextBuilder via injection so it can call build() itself before executing — note this is not the pattern FallbackAgentSkill uses today (it never receives ContextBuilder; the Orchestrator calls it and passes the resulting RequestContext via the ordinary SkillRequest, skills-design.md §3a) — without forcing the more general injection pattern onto Skills that don't need it.

A related, explicitly deferred question: could a high-confidence Skill's own internal logic ever need dynamic, runtime-decided planning (variable steps depending on an intermediate result), independent of the Router's confidence in routing to it? If so, that Skill could be internally agentic using the same containment pattern as the Fallback Agent Skill. Hypothetical, not a concrete near-term need — no action taken.

Non-goals

No LCEL chains, LangChain agents, or LangGraph in Router, Orchestrator, or Skill Factory. No Skill other than the Fallback Agent Skill (and, hypothetically, a future dynamically-planning Skill per above) uses LangChain's agent framework internally.

9. Rules Engine as a Platform Service, Not a Tool (Phase 3)

Not implemented in Phase 1 — the Rules Engine itself is a Phase 3 deliverable. This section only decides where it will live, using the same Tool-vs-Service test §3 already established for the LLM: is this an external, optional capability (Tool) or a fixed, always-present capability a Skill's core function depends on (Service)?

Question LLM Service (§3) Rules Engine
External integration MarketCompass doesn't own? No — internal reasoning capability No — internal, declarative business logic owned by CompassFoundry Labs
Optional per-Skill, or structurally required? Required by every Skill Required by every Skill that produces financial analysis (FinancialSkill, CompanySkill)
Cross-cutting policy concerns? Yes — retry, failover, cost tracking Yes — rule versioning, consistent threshold evaluation semantics, explainability output shape
Interprets a Skill's own output, or fetches raw external data? N/A (produces output) Interprets/normalizes a Skill's already-fetched Tool data — not itself a data source

Every answer matches the LLM's, not a Tool's — so this is RulesEngineService, not RulesEngineTool.

Skill
 ├── Tool Dependencies (0..n)                -- external data
 ├── LLM Service Dependency (1)              -- reasoning engine
 └── Rules Engine Service Dependency (0..1)  -- declarative business logic, financial-analysis Skills only

Skill → Tools → LLM Service → Rules Engine Service → SkillResult

RulesEngineService is injected the same way LLMService is — resolved once at Skill-construction time, not looked up through the Tool Registry. Skills that don't produce financial analysis (NewsSkill, GenericChatSkill) simply have no Rules Engine dependency.

Whether SkillResult.metadata: dict is sufficient for rule output or needs to become a typed field is an open Phase 3 design-time call, not resolved here. This section does not design the Rules Engine's internals (rule schema, evaluator implementation, config/rules/ YAML shape) — that's Phase 3 scope.

10. Security as a Cross-Cutting Concern

Security isn't a component with its own single responsibility — like Observability (§6), it's a concern that touches every layer. Controls attach to two existing chokepoints rather than a new security/ package:

  1. API layer boundary — authentication, rate limiting, input validation, resolved once before a request reaches the Orchestrator.
  2. LLM Service boundary — prompt injection handling, resolved once since every Skill's LLM call already funnels through LLMService (§3).
Phase New security surface Approach
1 Public API, secrets, cost-abuse .env for secrets, slowapi for rate limiting, scoped CORS, pip-audit
2 (RAG + Memory + Auth Foundation, Milestone 3) First real user data — long-term memory (memory-design.md); watchlists/reports remain unscheduled, further-future FastAPI Users (self-hosted, MIT-licensed, FastAPI-native); JWT auth at the API boundary; user_id-scoped queries for data isolation (security-design.md §"Phase 2")
3 (Rules Engine) YAML config trust boundary yaml.safe_load() only; reuses Pydantic schema validation
4 (Analyst Agent) Prompt injection via retrieved content Delimited context blocks inside LLMService; citation grounding doubles as an injection-detection aid

No paid identity provider (Okta, Auth0, WorkOS) or SSO/SAML at any phase — ruled out by the free-tooling constraint and by not being warranted for a single-tenant project with no enterprise customers. Total added cost across all four phases: $0.

11. Skill Factory: Construction, Not Behavior

A Skill Factory removes dependency-wiring boilerplate when constructing Skills from config/skills/registry.yaml. It handles construction only — resolving a YAML entry's declared Tools, LLMService config, and optional RulesEngineService dependency, then injecting them into a hand-written BaseSkill subclass. It never interprets YAML as behavior: a Skill's execute() — prompt construction, branching, response shaping (including the Fallback Agent Skill's internal LangChain agent, §8) — remains ordinary Python.

This follows the same declarative-vs-behavioral axis already applied elsewhere: Router thresholds are YAML, pattern-matching logic is code; Rules Engine thresholds are YAML, the evaluator is code.

Adding a Skill still requires writing a Python class — a new Skill always means new behavior. What the factory removes is the hand-written constructor call wiring that class to its dependencies, not the need to write the class. Three-step process: class, class-map entry, YAML entry — an explicit registry, not dynamic discovery.

12. Layered Architecture

flowchart TB
    subgraph client["Client"]
        UI["Frontend (React/Vite)"]
    end

    subgraph api["API Layer — backend/src/api"]
        ROUTES["/api/chat, /api/agents, /api/health, /api/providers"]
    end

    subgraph orch["Agent Orchestrator — backend/src/agents"]
        ORCH["Orchestrator"]
        CTX["AgentContext"]
    end

    subgraph models["Models Layer — backend/src/models"]
        MCTX["context.py\n(RequestContext)"]
        MROUT["routing.py\n(RoutingDecision)"]
        MSKILL["skills.py\n(SkillRequest, SkillResult)"]
        MTOOL["tools.py\n(ToolResult)"]
        MLLM["llm.py\n(LLMRequest, LLMResponse)"]
    end

    subgraph routing["Confidence Router — backend/src/router"]
        ROUTER["Router (scores RAW query)"]
        PM["Pattern Matcher"]
        ENT["Entity Extractor"]
        CFG["Config Loader"]
    end

    subgraph cb["Context Builder — backend/src/agents/context_builder.py"]
        CBUILD["ContextBuilder\n(no-op Phase 1; RagContextBuilder Milestone 1;\nRagMemoryContextBuilder Milestone 4+;\ncalled ONLY on low-confidence/no-route branch)"]
        RAGPKG["rag/ (Phase 2+)\nLangChain retriever/vectorstore/embeddings\nrag-design.md"]
        MEMPKG["memory/ (Phase 2+)\nshort-term (session_id) + long-term (user_id, SQLite)\nmemory-design.md"]
        CBUILD --> RAGPKG
        CBUILD --> MEMPKG
    end

    subgraph skills["Skills Layer — backend/src/skills"]
        SREG["Skill Registry (loader)"]
        S1["FinancialSkill"]
        S2["CompanySkill"]
        S3["NewsSkill"]
        S4["GenericChatSkill"]
        S5["FallbackAgentSkill (Phase 2+)\nLangChain agent inside\nreplaces generic_chat on the fallback branch"]
    end

    subgraph tools["Tools Layer — backend/src/tools"]
        TREG["Tool Registry (loader)"]
        T1["CompanyProfileTool"]
        T2["FinancialMetricsTool"]
        T3["NewsTool"]
        T4["SECFilingsTool"]
    end

    subgraph llmsvc["LLM Service — backend/src/services/llm_service.py"]
        LLMS["LLMService\n(model selection, retry, failover, cost/token accounting)"]
        LLMPKG["llm/ package\n(factory, provider_catalog,\nLangChain provider wrappers in providers/*)"]
    end

    subgraph cfgstore["Centralized Configuration — config/"]
        C1["config/routing/*.yaml"]
        C2["config/skills/registry.yaml"]
        C3["config/tools/registry.yaml"]
    end

    subgraph obs["Observability — backend/src/observability"]
        LOG["Structured Logging"]
        TRACE["Tracing"]
        METRICS["Metrics"]
    end

    subgraph eval["Evaluation (placeholder) — evaluations/"]
        EV["routing_eval.py / response_eval.py"]
    end

    UI --> ROUTES --> ORCH
    ORCH --> CTX
    ORCH -->|raw query| ROUTER
    ROUTER --> PM
    ROUTER --> ENT
    ROUTER --> CFG
    CFG -.reads.-> C1
    ROUTER -->|produces RoutingDecision| MROUT
    MROUT -->|RoutingDecision| ORCH

    ORCH -->|HIGH CONFIDENCE: constructs SkillRequest| MSKILL
    ORCH -->|LOW CONFIDENCE / NO ROUTE| CBUILD
    CBUILD -->|produces RequestContext| MCTX
    MCTX -->|RequestContext, retrieved_context populated| MSKILL

    MSKILL -->|SkillRequest| SREG
    SREG -.reads.-> C2
    SREG --> S1 & S2 & S3 & S4 & S5
    S1 & S2 & S3 & S5 -->|invoke tools| TREG
    TREG -.reads.-> C3
    TREG --> T1 & T2 & T3 & T4
    T1 & T2 & T3 & T4 -->|produces ToolResult| MTOOL
    MTOOL -->|ToolResult| S1 & S2 & S3 & S5
    S1 & S2 & S3 & S4 & S5 -->|constructs LLMRequest| MLLM
    MLLM -->|LLMRequest| LLMS
    LLMS --> LLMPKG
    LLMS -->|produces LLMResponse| MLLM
    MLLM -->|LLMResponse| S1 & S2 & S3 & S4 & S5
    S1 & S2 & S3 & S4 & S5 -->|produces SkillResult| MSKILL
    MSKILL -->|SkillResult| ORCH
    ORCH -.emits.-> obs
    ROUTER -.emits.-> obs
    SREG -.emits.-> obs
    TREG -.emits.-> obs
    LLMS -.emits.-> obs
    obs -.feeds.-> eval

13. Layer Contracts

Layer Responsibility Must NOT do
API Layer Auth, request validation, response shaping, streaming transport Contain routing or business logic
Orchestrator Manage AgentContext, call the Router first, conditionally invoke Context Builder on the fallback branch, invoke Skills, call MemoryService.record() twice (user turn, then assistant turn) after FallbackAgentSkill returns (Phase 2+ Milestone 4, memory-design.md §5), assemble the response Know about specific Tools or the LLM Service directly; retrieve memory/RAG context itself (that's the Context Builder's job); call Context Builder unconditionally; call MemoryService.record() on the high-confidence branch
Router Score confidence per configured route against the raw query, return a routing decision Execute anything; call Tools, the LLM Service, or the Context Builder; hold conversation state
Context Builder Assemble the RequestContext the fallback path operates on (no-op in Phase 1; real LangChain retrieval in Phase 2+); invoked only on low-confidence/no-route Make routing decisions; call Tools or the LLM Service directly; run on the high-confidence path
Skills Represent a business capability; orchestrate Tools and the LLM Service Call another Skill directly; call llm/factory.py directly instead of LLMService; read registry YAML directly
Tools Wrap one external, non-LLM integration with a typed input/output contract Contain business/domain logic or routing logic; perform LLM inference
LLM Service Model/provider selection policy, retries, failover, prompt execution, token/cost accounting (LangChain provider wrappers internal) Contain business logic (that's a Skill's job) or fetch external non-LLM data (that's a Tool's job)
Models (backend/src/models/) Define Pydantic models for all inter-component communication Contain business logic or routing logic; perform I/O or external calls
Configuration (config/) Express routing rules, skill registry, tool registry declaratively Contain secrets (those stay in .env/settings.py)
Observability Structured logs, traces, and metrics across every layer Gate or alter request execution (side-effect-only)
Evaluation (placeholder) Will assess routing accuracy and response quality against curated datasets Run inline in the request path — offline/batch, not per-request

14. Request Execution Flow

High-confidence path

sequenceDiagram
    participant U as User
    participant API as API Layer
    participant O as Orchestrator
    participant R as Router
    participant S as Skill
    participant T as Tool
    participant L as LLM Service
    participant M as Models Layer
    participant Obs as Observability

    U->>API: POST /api/chat {messages, session_id}
    API->>O: dispatch(ChatRequest)
    O->>O: build AgentContext (session, trace_id)
    O->>R: route(raw query, AgentContext)
    R->>R: extract entities, pattern match, score confidence
    R->>M: construct RoutingDecision
    M-->>R: RoutingDecision
    R-->>O: RoutingDecision(skills, confidence, reasoning)
    O-->>Obs: emit routing metrics (route, confidence)
    Note over O: high confidence — Context Builder NOT called;<br/>O builds a trivial RequestContext inline
    O->>M: construct SkillRequest(context, routing)
    M-->>O: SkillRequest
    O->>S: execute(SkillRequest)
    S->>T: call(...)
    T->>M: construct ToolResult
    M-->>T: ToolResult
    T-->>S: ToolResult
    S-->>Obs: emit tool latency/success metrics
    S->>M: construct LLMRequest(prompt, context)
    M-->>S: LLMRequest
    S->>L: generate(LLMRequest)
    L->>L: select model/provider, apply retry/failover policy
    L->>M: construct LLMResponse
    M-->>L: LLMResponse
    L-->>S: LLMResponse(content, model_used, tokens, cost)
    L-->>Obs: emit token/cost/latency metrics
    S->>M: construct SkillResult(content, citations, confidence)
    M-->>S: SkillResult
    S-->>O: SkillResult
    O-->>API: AgentResponse
    API-->>U: streamed / JSON response

Low-confidence / fallback path

sequenceDiagram
    participant O as Orchestrator
    participant R as Router
    participant CB as Context Builder
    participant Mem as Memory Service
    participant FS as Fallback Agent Skill
    participant T as Tool
    participant L as LLM Service
    participant M as Models Layer

    O->>R: route(raw query, AgentContext)
    R-->>O: RoutingDecision(low_confidence=true OR no route)
    Note over O: fallback branch — Context Builder called
    O->>CB: build(query, AgentContext)
    CB->>CB: retrieve (rag/, Phase 2+)
    CB->>Mem: load(AgentContext) [Phase 2+ Milestone 4]
    Mem-->>CB: MemorySnapshot(conversation_history, memory_items)
    CB->>M: construct RequestContext(retrieved_context=[...], conversation_history=[...], memory_items=[...])
    M-->>CB: RequestContext
    CB-->>O: RequestContext
    O->>M: construct SkillRequest(context, routing)
    M-->>O: SkillRequest
    O->>FS: execute(SkillRequest)
    FS->>FS: LangChain agent decides which Tools to call, in what order
    FS->>T: call(...) [one or more, order not fixed]
    T-->>FS: ToolResult(s)
    FS->>L: generate(LLMRequest) [via _LLMServiceChatModel adapter]
    L-->>FS: LLMResponse
    FS->>M: construct SkillResult
    M-->>FS: SkillResult
    FS-->>O: SkillResult
    Note over O: Milestone 4+ — record both turns after the Skill returns
    O->>Mem: record(AgentContext, ConversationTurn(role="user", content=query))
    O->>Mem: record(AgentContext, ConversationTurn(role="assistant", content=SkillResult.content))

15. Configuration-Driven Behavior

File Owns Consumed by
config/routing/skills_routing.yaml Route → required skills, keyword/pattern hints, entity requirements Router
config/routing/thresholds.yaml Per-route confidence thresholds, fallback margin Router
config/skills/registry.yaml Skill name → implementing class, required Tool names Skill Registry
config/tools/registry.yaml Tool name → implementing class, provider config keys Tool Registry
backend/src/config/settings.py LLM Service policy defaults (default provider/model, retry count, failover order), secrets via .env LLM Service
config/rag/*.yaml (Phase 2+) Corpus source paths, chunking parameters, Chroma persist directory/collection name, embedding provider/model rag/ package, via the Context Builder
config/memory/*.yaml (Phase 2+) Short-term TTL/max-turns; long-term SQLite path and retention policy memory/ package, via the Context Builder

16. Testing Strategy

Layer Test Type Key Assertions
Models Unit Every Pydantic model validates correctly; field types enforced; required fields reject missing values
Router Unit Fixed YAML fixtures produce expected routes/confidence, scored on the raw query with no RequestContext dependency
Context Builder Unit Phase 1 no-op returns RequestContext unchanged; interface is stable for Phase 2 substitution; integration test confirms it is invoked only on the low-confidence/no-route branch, never on high-confidence
Skills Unit + contract Skill honors declared Tool dependencies and its single LLMService dependency; both are mocked; Skill accepts SkillRequest and returns SkillResult; Fallback Agent Skill's internal LangChain agent is mocked/stubbed for unit tests
Tools Unit + integration Unit tests mock external clients; tagged integration suite hits sandboxed APIs; Tool returns ToolResult with correct structure
LLM Service Unit + integration Unit tests verify retry/failover/model-selection policy with a mocked llm/ package; integration tests verify at least one real provider round-trip through the LangChain wrapper; accepts LLMRequest and returns LLMResponse
Orchestrator Integration End-to-end high-confidence path: API → Router → Skill → Tool/LLM Service → response; end-to-end fallback path: API → Router → Context Builder → Fallback Agent Skill → response; all model round-trips validate
Config Schema validation All config/ subtrees validated against Pydantic schemas at startup
Observability Smoke test Every layer's emitted log/metric event conforms to a shared schema (trace_id present, etc.)
Evaluation (placeholder) N/A in Phase 1 Stub functions exist and are wired to a (currently empty) dataset location

17. Forward Compatibility

Future Phase What this architecture already provides What gets added, not rewritten
Phase 2 (RAG + Memory + Auth) Architecture fully designed, not yet implementedphases/phase2-implementation-plan.md, ADR-006, ADR-010, rag-design.md, memory-design.md. Context Builder interface already wired to the fallback branch; RequestContext already has retrieved_context (currently always empty) RagContextBuilder (Milestone 1) replaces the no-op and populates retrieved_context; RequestContext.user_id is added (Milestone 3) and RequestContext.memory_items is added (Milestone 4), each alongside AgentContext's matching field; RagMemoryContextBuilder (Milestone 4) replaces RagContextBuilder; FallbackAgentSkill replaces generic_chat on the fallback branch (Milestone 2); Auth Foundation adds user_id end to end (Milestone 3); Router gains a reference/comparison gate (routing-design.md §5b, Milestone 4) — Skills/Orchestrator control flow otherwise untouched
Phase 3 (Rules Engine) Skills already call Tools + LLMService; RulesEngineService placement already decided (§9); SkillResult has metadata: dict escape hatch RulesEngineService added as a third standard Skill dependency; config/rules/*.yaml added to the centralized config hierarchy; rule results land in SkillResult.metadata or a new typed field
Phase 4 (Analyst Agent) Orchestrator already separates "decide" (Router) from "assemble context" (Context Builder, conditional) from "do" (Skills); all inter-layer communication is typed Multi-step planning inside the Orchestrator, reusing the same Context Builder/Router/Skill calls per step; models gain additive fields (e.g. WorkflowState) without breaking existing interfaces
Domain entities (Phase 2–3) backend/src/models/ already exists as internal contracts layer New backend/src/domain/ layer for business entities (Company, FinancialStatement, etc.), added alongside models/, not replacing it
Evaluation maturity evaluations/ package and directory convention already exist Real datasets, real scoring logic — no new top-level structure needed
Observability maturity Metrics categories already named and emission points already wired Dashboards/alerting on top of already-emitted data — no new instrumentation points needed

18. Explicit Non-Goals

  • No dynamic/plugin discovery for Skills or Tools — explicit registries only, under config/.
  • No multi-step planning in the Orchestrator yet.
  • No embedding-based semantic routing for the Router itself (ADR-002-yaml-routing.md) — the Router always scores the raw query via YAML pattern-matching, never via retrieval.
  • Context Builder is a no-op in Phase 1, not real memory; from Phase 2 it runs only on the low-confidence/no-route branch, never unconditionally and never on the high-confidence path.
  • Evaluation framework is a placeholder, not a working pipeline, in Phase 1.
  • No multi-agent systems, autonomous planning loops, or graph orchestration frameworks (LangGraph, etc.) anywhere in Router, Orchestrator, or Skill Factory.
  • LangChain is never the orchestration framework and is confined to three pockets (§8): LLM Service provider wrappers, Context Builder retrieval internals, and the Fallback Agent Skill's internal agent. No other Skill uses a LangChain chain or agent by default.
  • Security is scoped deliberately: no paid identity provider, no enterprise SSO, no dedicated authorization engine, no compliance framework (SOC 2/HIPAA) — appropriate to a single-tenant, no-enterprise-customer project.
  • The Skill Factory (§11) is scoped to construction/dependency-wiring only — YAML never expresses Skill behavior, and class resolution uses an explicit map, not dynamic import or filesystem scanning.

Source: docs/architecture/architecture-overview.md

Follow the work.

Occasional updates on SignalFoundry, MarketCompass, and what we are building at CompassFoundry Labs.

No spam. Unsubscribe anytime.