Nicholas CarterFull stack · Data & AI · 20+ years

Verdict / Architecture

Architecture: harness, Core, and gateway

Verdict keeps three responsibilities apart. The harness is where work starts, Core decides, and the gateway executes. The exact-route worker contract and the HTTP relay have different enforcement boundaries.

Reviewed 2026-09-24 · ff18aa5f2be0ea9b8ab6e32b56e713801b2d8b83Status: ShippedStatus: ExperimentalAbout the labels

Three boundaries

Fig. 01System boundaries
  1. HarnessWhere work startsStatus: Experimental
    • Claude Code
    • Codex
    • Cursor
    • Cline
    • OpenCode
    • Prime

  2. OpenAI-compatible request

    Verdict CoreDecides and recordsStatus: Shipped
    • Offer qualification
    • Execution-path optimizer
    • Context packs
    • Model metadata store
    • Receipts and evidence

  3. Path-specific execution

    Gateway adapterExecution transportStatus: Experimental
    • Inventory
    • Execute
    • Health

  4. Provider call

    ProvidersWhere models run
    • Free tiers
    • Paid APIs
    • Local models
Coding tools send requests to Verdict Core. Core decides and records. The HTTP relay uses a configured upstream; packet workers enforce the authorized route.

Core owns policy, eligibility, context packing, the routing decision, and the receipt. The execution-path optimizer owns strategy selection. Legacy selectors may supply evidence or run in explicit compatibility mode; they are not a second strategy authority.

The gateway is an adapter. The current adapter targets OmniRoute and uses it for three things only: which models exist (inventory), running the chosen route (execute), and whether a route responds (health). OmniRoute is never the source of truth for model capabilities, and it does not own the routing decision.

Model capabilities such as tools, vision, structured output, and context window come from Core's own metadata store. The store is built from models.dev and LiteLLM's published model data, and it records the source and fetch time for every field. An unmapped model or an unknown required field is a named drop.

Harness adapters configure a coding tool to use Verdict as its OpenAI-compatible endpoint, rather than calling a gateway directly. They are reversible: disabling an adapter restores the files it changed.

Three request paths

Status: Shipped
Fig. 02Decision-only request sequence
  1. Request

    POST /v1/route submits a task and requirements for a decision, not execution.

  2. Qualify offers

    The execution-path optimizer checks offers against the candidate pool and hard exclusions. No qualified offer means blocked.

    Can stop here as: blocked
  3. Select strategy

    Compare expected complete cost within policy and any authoritative session constraint. Legacy ranking cannot restore excluded offers.

  4. Record and return

    Return decision JSON and route-only evidence. No provider call or worker launch follows from this endpoint.

The authoritative route path qualifies offers, selects a strategy, and records the decision. A route-only response does not execute inference or launch a worker.

Where this lives in code

Decision-only: /v1/route calls IntelligenceService.route() and returns decision JSON with route_only=True evidence. An execution-path decision returns before the legacy EligibilityGate branch. Production defaults to requiring this authority, with explicit context, configuration, and environment overrides for compatibility.

Completion relay: /v1/chat/completions and /v1/responses replace the model and call UpstreamProxy directly, without a Dispatcher hop. The proxy uses the server-configured upstream URL, not the optimizer-selected gateway endpoint. Gateway and route IDs in the authoritative decision are flags, not transport enforcement.

Context injection is conditional: an eligible compiled pack must be attached and the cheap_path_context_pack flag must be present. Without that flag, injection leaves the payload unchanged. The authoritative decision projection does not itself attach a compiled pack.

Packet worker: SwarmDispatcher binds an authorized runtime as a planning contract; it does not invoke a provider. Packet execution separately checks model, provider, and gateway against the authorized route before calling an executor with compiled context.

A denied route returns HTTP 503 with the denial and its reasons, plus evidence and correlation headers that the explain endpoint accepts. See Routing receipts.

Why the boundaries sit here

  • Gateways stay replaceable. A versioned, provider-neutral adapter contract keeps gateway-specific identifiers, credentials, and headers out of Core.
  • A gateway catalog is optimistic by design: it lists what might run. Treating it as capability truth would admit work the control plane cannot prove.
  • Letting a gateway or LiteLLM own the decision was considered and rejected. Fail-closed, reason-coded behavior would then depend on each provider.
  • The planned UI reads the receipt contract instead of defining its own event model, so the interface cannot drift from the evidence.

These choices are recorded in Engineering decisions.

Sources

Primary material in the public Verdict repository.