Every provider has a different request, stream, and tool-call format; a closed gateway hides which model answered, prices at the ceiling, and blocks the EU. Caduceus runs in your process with your keys, normalizes OpenAI, Anthropic, and Gemini, and routes by cost, quality, or latency with failover, hard USD budgets, and an auditable trace. It is Massive Intelligence (IM) infrastructure for non-human entities (MAIC, HIMs, NHEs), the transport for automation beyond OpenClaw, Hermes Agent, and Claude Code.
Caduceus gives every provider a single shape. You register a pool of providers and models and inject your
own dispatch, and it normalizes OpenAI, Anthropic, and Gemini, orders the candidates by a strategy, and
tries them with failover, returning the answer plus the winning model, the cost, and the full trace. Your
code is two lines:
new Caduceus({ dispatch, providers, models }), then caduceus.complete(request).
A strategy orders the candidate models by cost, quality, or latency. Each request is encoded to the provider's wire shape, dispatched through the injectable transport, and decoded back to one vocabulary. A budget tracker forecasts each call and refuses any that would breach the ceiling, a per-model circuit breaker trips a failing provider open and the router fails over to the next, and every attempt is recorded in an auditable route trace sealed with SHA-256. The core never speaks a provider API.
Take a real production scene: a request you want to send to the best available model, with automatic failover when one provider is down, a hard cost ceiling, and the routing visible.
Open transport is the middle ground between brittle per-provider glue and an opaque closed gateway; it keeps the normalization, the routing, and the cost in your hands, on your infrastructure, with every attempt exposed as an auditable trace.
install to your first auditable run.pnpm add @takk/caduceusnpm install @takk/caduceusyarn add @takk/caduceusbun add @takk/caduceusThe core has no runtime dependencies and a node-free core. Every @takk sibling is an optional
peer; install only what you compose with.
# nothing else is required to start experimenting
# optional: install a sibling you compose with, for example
pnpm add @takk/keymesh @takk/modelchainimport { Caduceus, defineProvider, defineModel } from '@takk/caduceus';
import { createEchoDispatch } from '@takk/caduceus/codec';
const providers = [
defineProvider({ id: 'openai', shape: 'openai' }),
defineProvider({ id: 'google', shape: 'gemini' }),
];
const models = [
defineModel({ id: 'gpt', provider: 'openai', model: 'gpt-5', cost: { inputPerMTok: 5, outputPerMTok: 15 } }),
defineModel({ id: 'flash', provider: 'google', model: 'gemini-flash', cost: { inputPerMTok: 0.3, outputPerMTok: 1.2 } }),
];
// Inject your dispatch; createEchoDispatch is a deterministic offline stand-in.
const caduceus = new Caduceus({ dispatch: createEchoDispatch(), providers, models, strategy: 'cost-first' });
const result = await caduceus.complete({ messages: [{ role: 'user', content: 'Draft a migration plan.' }] });
result.winner; // which model answered, after any failover
result.totalCostUsd; // total USD for the route
result.attempts; // the auditable route traceimport { createTransportTool } from '@takk/caduceus/adapter';
// A framework-agnostic descriptor (name, description, JSON Schema, handler) that
// matches what MCP servers and tool-calling APIs expect. Its name is 'caduceus_route'.
const tool = createTransportTool(caduceus);
const result = await tool.handler({
messages: [{ role: 'user', content: 'Audit the dependency tree for advisories.' }],
strategy: 'quality-first',
});
// the host fixes the model pool; the caller only sends a requestThe codec normalizes the request, response, streaming, and tool-call formats of OpenAI, Anthropic, and Gemini into one typed vocabulary, with an OpenAI-compatible shape for any other gateway.
Write against one API; swap or add a provider without touching your call sites.
Cost-then-quality, cost-first, quality-first, latency-first, weighted, round-robin, and sequential cascade order your pool; the router tries them in order, skipping open breakers and failing over on error.
Pick the trade the request deserves, instead of trusting one opaque router.
A per-model breaker trips a failing provider open and the router fails over; on a 429 it first honors Retry-After (or a capped backoff) and retries the same model, optionally with a rotated key, before failing over.
A provider going down or rate-limiting you does not take your request with it.
The routing runs in your process, on your infrastructure, with your keys. The core never speaks a provider API; no vendor sits in the middle of every call, and the EU and EEA are not blocked.
Route to frontier models where a closed, region-locked gateway cannot follow you.
Each model carries a cost profile. The router forecasts the dollar cost of each candidate before any
call and enforces a hard ceiling, raising a typed BUDGET_EXCEEDED before a breaching call.
Cost is a number you set up front, instead of a surprise at the ceiling of a vendor pool.
OpenAI chunk deltas, Anthropic content-block events, and Gemini partial candidates arrive as one typed
StreamEvent union, with tool-call deltas reassembled the same way across every provider.
Consume any provider's stream through one switch, instead of a parser per vendor.
Every request records which models were tried, each breaker's state, each attempt's cost and latency, and the winner. It renders to DOT or Mermaid and seals with SHA-256 via the Web Crypto API.
Prove which model answered, in Node, edge runtimes, or the browser, when a route is questioned.
Two-line bridges wrap a Vercel AI SDK call or an OpenAI-style chat function as the dispatch, expose the router as an MCP-ready tool, and feed the @takk/coryphaeus orchestrator's agent pool.
Adopt it next to what you already run, instead of rewriting your stack.
Every published version signed with npm publish --provenance through GitHub Actions OIDC.
Lockfile committed, zero required runtime dependencies, node-free core.
Verify in one command that the tarball you installed was built from the source commit you trust.
A strategy orders the candidate models per request, and you pick which one. The router then tries them in order, skipping open breakers and failing over on error. The only knobs are the breaker tuning and the budget.
| Strategy | Orders by | Use it when |
|---|---|---|
cost-then-quality |
Cheapest first, ties broken by higher quality. | The default; you want the cheapest acceptable model. |
cost-first / quality-first |
Strictly cheapest, or strictly highest quality. | Cost or quality is the single thing that matters. |
latency-first |
Lowest latency hint first. | Responsiveness matters more than cost. |
weighted / round-robin |
Highest selection weight, or a rotating start each request. | You want to spread load across the pool. |
sequential-cascade |
Cheapest-and-weakest first, escalating to stronger models. | You expect to try several and escalate on failure. |
The 1.0 strategies are transparent and deterministic; a quality-measured strategy (evaluators and bandits) is a later release behind the same Strategy seam.
| Entry point | Subpath export | Runtime | Use it when |
|---|---|---|---|
| Toolkit barrel | @takk/caduceus |
node-free | You want the router, the codec, the strategies, breakers, budget, the trace, and the adapters. |
| Transport | @takk/caduceus/transport |
node-free | You want the Caduceus router on its own to plan, complete, and stream. |
| Adapters and tool | @takk/caduceus/adapter |
node-free | You want the Vercel or OpenAI dispatch bridge, the @takk/coryphaeus AgentClient, or the routing tool for MCP. |
| Node entry | @takk/caduceus/node |
Node only | You need the real fetch dispatch, or to load a provider roster, a request, or a saved trace from JSON. |
The caduceus binary runs the compiled engine. providers lists the pool;
plan shows the strategy ordering and a cost forecast; route routes a request offline
with the deterministic echo dispatch behind --mock; trace renders a saved trace to
DOT or Mermaid. Real provider routing uses the library API with the Node fetch dispatch.
# show the strategy ordering and a cost forecast, no network
npx @takk/caduceus plan request.json --providers providers.json --models models.json# route with the deterministic echo dispatch and save the trace
npx @takk/caduceus route request.json --providers providers.json --models models.json --mock --out trace.json
npx @takk/caduceus trace trace.json --format mermaid
npx @takk/caduceus providers --providers providers.json --models models.json
npx @takk/caduceus --version
# exit codes: 0 ok, 2 usage or input, 21 no route, 30 budget exceededSeal every route into a tamper-evident SHA-256 digest. And because each route is explicit,
complete returns the answer, the winning model, the per-attempt cost, and every attempt the
router made, so the routing is never a black box. Both are node-free and need no external service.
import { sealTrace, verifyTraceSeal, renderTraceDot } from '@takk/caduceus/trace';
const result = await caduceus.complete({ messages: [{ role: 'user', content: 'Triage the alert.' }] });
const seal = await sealTrace(result); // a SHA-256 integrity seal over the route trace
await verifyTraceSeal(result, seal); // true, until any field of the trace is altered
renderTraceDot(result); // a Graphviz transport graph of every attemptconst result = await caduceus.complete({ messages: [{ role: 'user', content: 'Summarize the incident.' }] });
// {
// request: 'req_1',
// strategy: 'cost-then-quality',
// response: { text: '...', toolCalls: [], finishReason: 'stop', usage: {...} },
// winner: { model: 'gpt', provider: 'openai' }, // which model answered
// attempts: [ ... ], // the auditable route trace
// totalCostUsd: 0.0031
// }The result carries the strategy used, the per-attempt cost, and the winner, so you can see which model answered and what the route cost. Seal each trace with no runtime dependency, the governance seam for a non-human entity routing across a pool of models it operates.
The other approaches solve part of the problem. The contrast clarifies where Caduceus sits.
| Capability | Caduceus | Closed endpoint | Agent framework | Vercel AI SDK | Hand-rolled |
|---|---|---|---|---|---|
| Distribution | npm library | hosted endpoint | npm library | npm library | your repo |
| Transparent routing | auditable trace | black box | hand-wired | thin primitives | varies |
| Failover & breakers | per-model | hidden | manual | basic | manual |
| Cost forecast and ceiling | yes | pool ceiling | manual | no | varies |
| Provider-agnostic | injectable | pool fixed | yes | yes | varies |
| Local-first sovereignty | your infra | region-locked | yes | yes | yes |
| Auditable trace | yes | no | no | no | no |
| Zero deps, node-free | yes | n/a | deps | deps | varies |
| License | Apache-2.0 | proprietary | mixed | Apache-2.0 | your call |
The honest summary: a closed gateway is fine when you do not need to see inside, and a single direct call is fine when one provider always answers. Pick Caduceus when you want a single shape over many providers, with automatic failover, the routing and cost visible, sovereignty over where it runs, and a tamper-evident trace, all in one node-free TypeScript library you embed in production.
Caduceus is transparent routing over an injectable dispatch, and that has boundaries. They are documented, and where reality bends them, there is an explicit choice rather than a silent fudge.
| Assumption or property | What it means | How Caduceus handles it |
|---|---|---|
| Deterministic strategies | The 1.0 strategies order candidates by transparent rules, not a learned router. | The ordering is a pure function of cost, quality, and latency; a measured strategy is a documented later release behind the same seam. |
| Injectable dispatch | The core never speaks a provider API; the answer quality is the dispatch's and the providers'. | You provide the transport; Caduceus routes and audits it, and tests run against a deterministic echo dispatch. |
| Single-answer routing | A route returns the first model that answers, not a composition of several. | Composing several models into one answer is the job of the @takk/coryphaeus orchestrator, which Caduceus feeds. |
| Cost is forecast then measured | The pre-dispatch forecast uses a chars-per-token heuristic. | Actual cost is recomputed from the response's reported token counts and recorded per attempt; the forecast only gates dispatch. |
| Failover on error | The router fails over when a dispatch throws or returns an HTTP error status. | A successful but low-quality answer is not a failure; quality-based rejection is a measured-strategy concern, documented for later. |
| Budget is a hard ceiling | The ceiling is enforced before a breaching call is dispatched. | It raises a typed BUDGET_EXCEEDED rather than reporting an overrun after the fact. |
| Provenance from the trace | Provenance is which model answered. | It is read off the recorded attempts, the winning model, not asserted by the model. |
| Integrity, not identity | The trace seal proves a trace was not altered after sealing. | A SHA-256 digest via Web Crypto, an integrity seal, not a signature. |
159 tests passing under Vitest 4 with 94% coverage, across 13 suites on the Node 20, 22,
and 24 matrix. Five runnable examples and a transport benchmark route against the built artifact
end to end. Run pnpm test on a fresh clone to reproduce.
TypeScript 6 in maximum strict mode (exactOptionalPropertyTypes,
useUnknownInCatchVariables, noUncheckedIndexedAccess,
noImplicitOverride, noImplicitReturns). Zero errors under tsc
--noEmit.
Biome 2.5 clean across src and tests. publint clean and
@arethetypeswrong/cli green across all eleven subpaths. Dual ESM + CJS with separate
.d.ts and .d.cts per subpath.
Each strategy orders the pool as documented, the budget guard trips before a breaching call, an open breaker is skipped and the router fails over to the next candidate, and the trace seal detects a tampered field. Every invariant is a test.
A smoke test exercises the compiled ESM and CJS artifacts and spawns the compiled CLI as a single Node process, run on the Node 20, 22, and 24 matrix.
Committed pnpm lockfile, zero required runtime dependencies, node-free core, and SLSA provenance
attestation on every published version. Verify with npm view @takk/caduceus@1.0.0 --json | jq
.dist.attestations.
Yes. 159 tests pass under Vitest 4 with 94% coverage; TypeScript 6 maximum strict mode is clean; Biome 2.5
lint is clean; publint and @arethetypeswrong/cli are clean across all eleven
subpaths. The suite and the distribution smoke test are run on the Node 20, 22, and 24 matrix. Every
published release carries SLSA provenance produced by GitHub Actions.
Because no single provider is always up, always cheapest, or always best; different providers lead on programming, mathematics, security, and long context, and any of them can rate-limit or go down. Caduceus routes to the best available model by your strategy and fails over automatically when one is down, so a single hard-coded provider is never a single point of failure.
You can, and many do. The cost is per-provider request, stream, and tool-call code that you rewrite on every swap, and failover, breakers, budgets, and a trace you must build yourself. Caduceus normalizes the shapes, orders the pool by a strategy, and builds the cost forecast, the budget ceiling, the circuit breakers, and the auditable trace in, so the transport is one library instead of glue per provider.
No. The core is routing logic over an injectable dispatch. You provide the transport, the bundled fetch dispatch, a Vercel AI SDK call, an OpenAI-style chat function, or a deterministic mock, and the router drives it. That keeps the package provider-agnostic, dependency-free, and testable with no network, and it means your provider keys live in your dispatch, never in Caduceus.
Yes. The core is node-free; the trace seal uses the Web Crypto API, not node:crypto. Import
@takk/caduceus or @takk/caduceus/edge on any runtime with Web Crypto, and pair it
with a dispatch built on the runtime's own fetch. Only @takk/caduceus/node, the bundled fetch
dispatch and the JSON loaders, requires the Node standard library.
Caduceus holds the provider and model descriptors you register, the request you pass, and the route results it builds, nothing else. It makes no outbound network calls of its own; the only model calls are the ones your injected dispatch makes to your chosen providers. With the file loaders, only the roster, request, or trace files you point it at are read from disk, and no provider credentials are ever held by Caduceus.
Per request, the strategy orders the candidate models by cost, quality, or latency; the router then tries them in that order, skipping models whose circuit breaker is open and failing over on error. The 1.0 strategies are transparent and deterministic. You can pick a strategy per request, restrict the candidate models, or read the recorded attempts to see exactly which models were tried and why one won.
No, and the docs are explicit about it. The seal is a SHA-256 digest of the trace's canonical form, so any
later edit makes verifyTraceSeal return false. It is an integrity seal, not a signature: it
does not prove who produced the trace. Signed and timestamped seals for stronger third-party evidence are
on the roadmap.
The codec normalizes OpenAI, Anthropic, and Gemini natively, and an OpenAI-compatible shape covers any other gateway (Ollama, vLLM, Together, Groq, OpenRouter, and the rest). Because you inject the dispatch, you can also route to a local model or a mock. Adapters are included for an OpenAI-style chat function and the Vercel AI SDK, so wrapping your existing call is two lines.
In-process memory by default: the registered provider and model pool and the per-model circuit-breaker
counters. For disk-backed input, load a provider roster, a model roster, a request, or a saved trace from
JSON with @takk/caduceus/node. A route trace is a plain object you can serialize to your own
store whenever you want.
Each model carries a cost profile. Caduceus forecasts the dollar cost of each candidate before any call,
and a budget tracker refuses any call that would breach the per-request ceiling you set, raising a typed
BUDGET_EXCEEDED. A cost-first or sequential-cascade strategy further tries the cheapest model
first and escalates only on failure.
Every release is published with npm publish --provenance. Check the attestations with
npm view @takk/caduceus@<version> --json | jq .dist.attestations. The attestation
links the tarball you installed to the GitHub Actions workflow that built it from a specific source commit.
Strict SemVer 2.0.0, starting from 1.0.0. The binding stability surface is documented in SPEC.md section 5. Major bumps require a deprecation cycle; security fixes follow the disclosure flow in SECURITY.md.