You wire up five models and pick one with a hardcoded rule. The cheap model handles the hard prompts badly; the expensive one burns budget on the easy ones, because the routing was a guess. BayesRoute keeps a Beta posterior over acceptable output and Gamma-Exponential posteriors over latency and cost per model and domain, samples each, and routes to the highest expected utility with bounded regret.
You give BayesRoute a set of models and, for each query, a domain. It picks the model most likely to give
you a good answer for the cost and latency you care about, then learns from what actually happened. A model
that wins twenty hard prompts is not the same bet as one that won two, it is a calibrated choice that
balances trying new models against using the proven one. Your code stays the same shape: you call
router.route(features) and report back with router.observe(...).
Each model and domain arm carries a Beta posterior over acceptable-output probability and two Gamma-Exponential posteriors over latency and cost. On every route, BayesRoute samples each posterior, combines the draws into an expected utility over quality, latency, and cost, and routes to the maximum, Thompson sampling with bounded regret. Per-domain priors, a safe-exploration guard, optional time-decay, and a tamper-evident SHA-256 hash-chained audit trail, all in a node-free core.
Take a real production scene: a new, cheaper model lands that is excellent on easy prompts but weak on hard reasoning, and your traffic mixes both kinds of domain through one routing rule.
router.observe(model, domain, ...) folds quality, latency, and cost
back into the arm.The "With BayesRoute" timeline is the behavior of Thompson sampling as evidence accumulates; the safe-exploration guard keeps a single bad burst from a new model from cratering its traffic prematurely.
install to first route.pnpm add @takk/bayesroutenpm install @takk/bayesrouteyarn add @takk/bayesroutebun add @takk/bayesrouteThe core has no runtime dependencies and a node-free core. Every @takk sibling is an optional
peer; install only what you compose with.
# nothing else is required to start routing
# optional: install a sibling you compose with, for example
pnpm add @takk/bayestruth @takk/modelchainimport { createBayesRoute } from '@takk/bayesroute';
const router = createBayesRoute({ models: ['gpt-fast', 'gpt-strong', 'mini'] });
// Pick a model for this query by Thompson sampling.
const model = router.route({ domain: 'reasoning' });
// ...call that model, then report the realized outcome.
router.observe(model, 'reasoning', { quality: 1, latencyMs: 820, cost: 0.004 });
const ranked = router.rank('reasoning');
console.log(ranked[0].model); // best expected utility for this domain
console.log(ranked[0].utility); // its sampled expected utilityimport { createBayesRoute } from '@takk/bayesroute';
const router = createBayesRoute({ models: ['gpt-fast', 'gpt-strong', 'mini'] });
// `run` routes by Thompson sampling, calls the chosen model, then folds
// the realized quality, latency, and cost back into its posteriors.
const answer = await router.run({ domain: 'classification' }, async (model) => {
const start = performance.now();
const result = await callModel(model, prompt); // your own model call
return {
output: result.text,
quality: result.ok ? 1 : 0,
latencyMs: performance.now() - start,
cost: result.usdCost,
};
});
console.log(answer.model); // the model that handled this queryEach route samples the Beta posterior over acceptable output and the Gamma-Exponential posteriors over latency and cost, combines them with your weights, and picks the maximum expected utility.
Every query goes to the model that is the best bet for the quality, cost, and latency you actually care about, not a fixed rule.
Drawing from each arm's posterior balances trying new models against using the proven one, with regret bounded as evidence accrues. The generator is seeded, so the same seed and posteriors route the same way.
Route between competing models optimally, balancing exploration and exploitation, with reproducible, auditable choices.
Posteriors are keyed by model and domain, so a model that is strong at reasoning but weak at extraction carries an honest, separate belief for each kind of query.
Hard domains converge on the strong model and easy domains on the cheap one, instead of one average that fits neither.
Opt-in exponential decay pulls stale evidence back toward the prior with a configurable half-life. The posterior never decays below the prior, so it stays a valid distribution.
A model that was great last quarter does not outvote what it is doing on today's traffic and today's prices.
Opt-in guard caps how much traffic a cold or recently-bad arm can take while it explores, so one bad burst from a new model cannot dominate the route before it has fair evidence.
A single incident cannot starve a good model or hand all traffic to a bad one, which keeps exploration honest under bursty traffic.
One weighting over the three posteriors decides the trade-off: cheap-and-fast for a cheap domain, quality-at-any-cost for a critical one, tuned per domain rather than globally.
You state the trade-off you want once and the router honors it on every query, with the numbers to back the choice.
An append-only log of every route and observation, with a SHA-256 hash-chain you seal and
verify via the Web Crypto API.
Prove the routing decisions were made on the recorded evidence, in Node, edge runtimes, or the browser.
Seed each domain with a prior so a brand-new model starts with a sensible belief instead of a flat one, and a data-rich arm barely moves while a fresh one explores.
The cold-start problem solved on principle, the concrete advantage a Bayesian treatment has over a static routing table.
Every published version signed with npm publish --provenance through GitHub Actions OIDC.
Lockfile committed, zero required runtime dependencies, node-free core.
Verify in one command that the tarball you installed was built from the source commit you trust.
Pass a prior per domain, or per model and domain. The utility weighting stays yours; the
posterior bookkeeping is the library's.
| Prior | What it encodes | When to use it |
|---|---|---|
UNIFORM_PRIOR |
Beta(1, 1): every acceptable-output rate equally likely before any evidence. | Default. No prior knowledge about a model on a domain yet. |
JEFFREYS_PRIOR |
Beta(0.5, 0.5): the reference objective prior for a Bernoulli rate. | When you want the standard objective prior with minimal influence. |
Beta(a, b) |
Your belief about a model on a domain as pseudo-counts before the first route. | A frontier model starts ahead of a tiny model on a hard reasoning domain. |
domainPrior |
A shared per-domain prior so every model on that domain starts from the same belief. | A brand-new model should explore within a sensible domain prior, not a flat one. |
Hierarchical pooling across families of related domains is planned for a later release.
| Entry point | Subpath export | Runtime | Use it when |
|---|---|---|---|
| Facade | @takk/bayesroute |
node-free | You want the createBayesRoute router and the full toolkit. |
| MCP | @takk/bayesroute/mcp |
node-free | You want to route across MCP servers and feed each callTool outcome back. |
| Utility | @takk/bayesroute/utility |
node-free | You want to build a custom utility over quality, latency, and cost. |
| Node store | @takk/bayesroute/node |
Node only | You need durable, file-backed persistence that survives restarts. |
When you want to route or curate outside a running process, the bayesroute binary reads a store
snapshot from a file or stdin, picks a model, folds new outcomes back in, inspects the arms, and verifies
audit seals.
# sample the posteriors and print the chosen model and its utility
npx @takk/bayesroute route \
--domain reasoning \
--store route-store.json# rank models for a domain, best expected utility first
npx @takk/bayesroute rank --domain reasoning --store route-store.json
# fold one observed outcome in and write the store back
npx @takk/bayesroute observe gpt-strong reasoning --quality 1 --latency-ms 820 --cost 0.004 --store route-store.json
# inspect the posteriors for every model and domain arm
npx @takk/bayesroute inspect --store route-store.json
npx @takk/bayesroute verify --log audit-log.json --seal audit-seal.json
# exit codes: 0 ok, 1 error, 64 usage, 65 data error, 66 missing inputEnable the audit trail to record every route and observation, then seal it into a tamper-evident root. Use a custom utility to set how quality, latency, and cost weigh against each other. Both are node-free and need no external service.
const router = createBayesRoute({ models: ['gpt-fast', 'gpt-strong'], audit: true });
const model = router.route({ domain: 'extraction' });
router.observe(model, 'extraction', { quality: 1, latencyMs: 540, cost: 0.001 });
// Seal the append-only log into a SHA-256 hash-chain root.
const seal = await router.seal(); // { algorithm: 'sha-256', root, count }
const result = await router.verify(seal); // { valid: true }
// Any change to a sealed entry makes verify return { valid: false }.import { createBayesRoute } from '@takk/bayesroute';
// Weigh quality, latency, and cost to taste, per domain.
const router = createBayesRoute({
models: ['gpt-fast', 'gpt-strong', 'mini'],
utility: {
// A critical domain pays for quality; a bulk domain pays for cheap-and-fast.
reasoning: { quality: 1.0, latencyMs: 0.1, cost: 0.2 },
classification: { quality: 0.4, latencyMs: 0.4, cost: 1.0 },
},
});
// Each route maximizes the sampled expected utility for that domain.
router.route({ domain: 'reasoning' });
router.route({ domain: 'classification' });Bounded regret holds while the assumptions hold: roughly stationary rewards per arm. The safe-exploration guard and time-decay mitigate the common violations, and the audit trail is how you confirm the routing on your own traffic.
The other approaches solve part of the problem. The contrast clarifies where BayesRoute sits.
| Capability | BayesRoute | Static rule | Hand-rolled bandit | Gateway / SaaS | LLM-judge router |
|---|---|---|---|---|---|
| Distribution | npm library | a config | your repo | SaaS / proxy | extra model |
| Learns from outcomes | Thompson sampling | no | sometimes | varies | no |
| Explicit priors | yes | no | no | no | no |
| Quality, cost, latency utility | yes | no | rarely | partial | quality only |
| Bounded regret | yes | no | rarely | no | no |
| Tamper-evident audit | yes | no | no | partial | no |
| Zero deps, node-free | yes | varies | varies | no | no, extra call |
| License | Apache-2.0 | your call | your call | varies | varies |
The honest summary: a static rule is fine until prices or models change. A gateway is convenient if you accept a proxy in the path. Pick BayesRoute if you want explicit priors, a quality-cost-latency utility, Thompson sampling with bounded regret, and a tamper-evident trail composed in one node-free TypeScript library you embed in production.
BayesRoute is a Bayesian bandit, and a bandit has assumptions. They are documented, and where real traffic breaks them, there is an explicit mitigation rather than a silent fudge.
| Assumption or property | What it means | How BayesRoute handles it |
|---|---|---|
| Independent rewards | Each route's outcome is one draw from its arm. | The safe-exploration guard caps a bursty arm so it cannot dominate. |
| Stationary rewards | An arm's quality, latency, and cost do not drift over time. | Opt-in time-decay relaxes stale evidence toward the prior. |
| Acceptable-output outcome | Each route is acceptable or not, plus a latency and a cost. | You map any result, including MCP isError, to a quality in [0, 1]. |
| Conjugate posteriors | Beta for quality, Gamma-Exponential for latency and cost. | Closed-form updates per observation, O(1) with no inference latency. |
| Bounded regret | Loss vs the best arm is bounded, not unconditional optimality. | Thompson sampling balances exploration and exploitation as evidence grows. |
| Cold start | A new model and domain arm has only its prior. | Per-domain priors and safe exploration give it a fair, bounded trial. |
| Integrity, not identity | The audit seal proves a log was not altered after sealing. | A SHA-256 hash chain via Web Crypto, an integrity seal, not a signature. |
| Reproducible routing | Bandit choices must be auditable. | A seeded generator makes Thompson selection deterministic. |
511 tests passing across 22 suites under Vitest 4. Coverage: 98% lines, 98%
statements, 99% functions, 96% branches. Run pnpm test on a fresh clone to reproduce.
TypeScript 6 in maximum strict mode (exactOptionalPropertyTypes,
useUnknownInCatchVariables, noUncheckedIndexedAccess,
noImplicitOverride, noImplicitReturns). Zero errors under tsc
--noEmit.
Biome clean across src and tests. publint clean and
@arethetypeswrong/cli green across every subpath. Dual ESM + CJS with separate
.d.ts and .d.cts per subpath.
The samplers reproduce known properties: Beta and Gamma draws match their target means and variances over large samples, and a seeded generator makes every route reproducible. Thompson selection is deterministic for a fixed seed and posteriors.
A smoke test exercises the compiled ESM and CJS artifacts and spawns the compiled CLI as a single Node process, run on the Node 20, 22, and 24 matrix.
Committed pnpm lockfile, zero required runtime dependencies, node-free core, and provenance
attestation on every published version. Verify with npm view @takk/bayesroute@1.0.0 --json | jq
.dist.attestations.
Yes. 511 tests across 22 suites pass under Vitest 4 with coverage of 98% statements, 96% branches, 99%
functions, and 98% lines; TypeScript 6 maximum strict mode is clean; Biome lint is clean;
publint and @arethetypeswrong/cli are clean across every subpath. The suite and the
distribution smoke test are run on the Node 20, 22, and 24 matrix. Every published release carries
provenance produced by GitHub Actions.
Fixed weights never learn and go stale the moment a price or a model changes. BayesRoute keeps a Beta posterior over acceptable output and Gamma-Exponential posteriors over latency and cost per model and domain, samples each, and routes to the highest expected utility. That balances exploration and exploitation with bounded regret, so good models earn traffic and weak ones lose it on evidence.
Those are superb for offline experimentation in Python. BayesRoute is embeddable TypeScript for production: closed-form conjugate updates with zero inference latency, a node-free core, and an API a developer can use in five lines without a statistics background.
Yes. The core is node-free; the audit seal uses the Web Crypto API, not node:crypto. Import
@takk/bayesroute or @takk/bayesroute/edge on any runtime with fetch
and Web Crypto. Only @takk/bayesroute/node requires the Node standard library.
BayesRoute records the model and domain identifiers you choose, the posterior parameters, the observed quality, latency and cost, and timestamps. It never sees your prompts or completions and makes no outbound network calls of its own. With the file store, only that operational state reaches disk; no secrets are involved at any point.
A new model and domain arm starts at its prior, the uniform Beta(1, 1) by default, so its belief is wide
and honest. To do better than a flat prior, set a per-domain prior with domainPrior; the
safe-exploration guard then gives the new arm a fair, bounded share of traffic until it has data of its
own.
In-process memory by default, with portable JSON snapshots via snapshot() and
load(). For durability across restarts, use createFileStore from
@takk/bayesroute/node. For multi-process coordination, implement the PosteriorStore
interface over SQLite, Postgres, or Redis.
No, and the docs are explicit about it. The seal is a SHA-256 hash-chain root that proves a log was not altered after sealing. It is an integrity seal, not a signature: it does not prove who produced the log. Signed and timestamped seals for stronger third-party evidence are on the roadmap.
Under the model, roughly stationary rewards per arm, yes, regret is bounded as evidence accumulates. Real
traffic is bursty and non-stationary, so use the safe-exploration guard and time-decay, and confirm the
routing on your own data via the audit trail and router.rank(domain).
Every release is published with npm publish --provenance. Check the attestations with
npm view @takk/bayesroute@<version> --json | jq .dist.attestations. The attestation links
the tarball you installed to the GitHub Actions workflow that built it from a specific source commit.
Yes. Implement the PosteriorStore interface with get, set,
keys, and snapshot, and pass it as store in the config. Back it with
SQLite, Postgres, Redis, or anything else; the file store is the zero-dependency reference.
Strict SemVer 2.0.0, starting from 1.0.0. The binding stability surface is documented in SPEC.md section 5. Major bumps require a deprecation cycle; security fixes follow the disclosure flow in SECURITY.md.