Your agent calls forty tools and MCP servers. Two start failing. You find out from a user, not your monitoring, because trust was a guess. BayesTruth treats every call as Bernoulli evidence, maintains an exact Beta-Bernoulli posterior per subject, and gives you a calibrated trust score with a credible interval that respects the sample size.
You report whether each tool call succeeded or failed. BayesTruth keeps a running belief about how
reliable each tool, MCP server, skill, or agent is, expressed as a trust score between 0 and 1 with a
confidence range around it. Nineteen successes out of twenty is not a flat 5% failure rate, it is a
calibrated number that tells you how sure you can be. Your code does not change shape: you keep calling
trust.observe(subject, outcome) and reading trust.score(subject).
Each subject carries a Beta posterior over its success rate. Every call is one Bernoulli trial, folded in with the closed-form Beta-Bernoulli conjugate update in O(1) with zero inference latency. The trust score is the posterior mean; the interval is the exact Beta quantile, computed from lgamma, the regularized incomplete beta, and its inverse. Explicit priors, calibrated decision policies, Thompson-sampling routing, optional time-decay and a correlated-failure guard, and a tamper-evident hash-chained audit trail.
Take a real production scene: your agent routes to a search tool that has been reliable for weeks, and its provider quietly ships a regression that starts returning empty results.
trust.decide(subject) flips from trust to
monitor, then distrust.The "With BayesTruth" timeline is the calibrated behavior of the default decision policy as evidence accumulates; the correlated-failure guard keeps a single burst from overstating the change.
install to first score.pnpm add @takk/bayestruthnpm install @takk/bayestruthyarn add @takk/bayestruthbun add @takk/bayestruthThe core has no runtime dependencies and a node-free core. Every @takk sibling is an optional
peer; install only what you compose with.
# nothing else is required to start scoring trust
# optional: install a sibling you compose with, for example
pnpm add @takk/keymesh @takk/modelchainimport { createBayesTruth } from '@takk/bayestruth';
const trust = createBayesTruth();
// Each tool call is one Bernoulli trial.
trust.observe('search-api', 'success');
trust.observe('search-api', 'success');
trust.observe('search-api', 'failure');
const score = trust.score('search-api');
console.log(score.score); // posterior mean
console.log(score.interval.lower); // 95% credible interval lower bound
console.log(trust.decide('search-api')); // 'trust' | 'monitor' | 'distrust'import { createBayesTruth } from '@takk/bayestruth';
import { interceptMcpClient } from '@takk/bayestruth/mcp';
const trust = createBayesTruth();
// `client` is any object with a callTool method.
const monitored = interceptMcpClient(client, { sink: trust.sink });
// A result flagged isError counts as a failure, even when it resolves.
await monitored.callTool({ name: 'web_search', arguments: { query: 'IM 2026' } });
console.log(trust.score('mcp:web_search')); // reputation accrues per toolA Beta posterior per subject, updated in closed form on each Bernoulli outcome. The score is the posterior mean; the interval is the exact Beta quantile, not a normal approximation.
You act on a number that says how reliable a tool is and how sure you can be, instead of a threshold someone guessed.
Draw once from each subject's posterior and pick the maximum. The generator is seeded, so the same seed and posteriors always select the same subject.
Route between competing tools optimally, balancing exploration and exploitation, with reproducible, auditable choices.
A calibrated policy returns trust, monitor, or distrust: trust only when the interval lower bound clears the bar, distrust below the mean floor, monitor otherwise.
A single lucky success never earns trust, and a small sample stays under monitor until it earns a verdict.
Opt-in exponential decay pulls stale evidence back toward the prior with a configurable half-life. The posterior never decays below the prior, so it stays a valid Beta.
Reputation earned a year ago does not outweigh what a tool is doing today.
Opt-in coalescing collapses a run of identical outcomes inside a window into one event, so an outage that produced fifty failures is not counted as fifty independent trials.
A single incident cannot crater trust or fake confidence, which is what keeps the calibration honest under bursty traffic.
Brier score, log loss, expected calibration error, and reliability bins, so you can measure calibration on your own data instead of trusting the label.
The evidence an EU AI Act Article 12 review asks for: not a promise of calibration, a measurement of it.
An append-only log of every observation, decision, and selection, with a SHA-256 hash-chain you
seal and verify via the Web Crypto API.
Prove the trust decisions were made on the recorded evidence, in Node, edge runtimes, or the browser.
Fit a shared prior to a category of similar subjects, so a cold-start tool borrows the reputation of its siblings while a data-rich one barely moves.
The cold-start problem solved on principle, the concrete advantage a Bayesian treatment has over a per-subject interval.
Every published version signed with npm publish --provenance through GitHub Actions OIDC.
Lockfile committed, zero required runtime dependencies, node-free core.
Verify in one command that the tarball you installed was built from the source commit you trust.
Pass a prior per subject, or fit one per category. The decision threshold stays yours; the
calibration is the library's.
| Prior | What it encodes | When to use it |
|---|---|---|
UNIFORM_PRIOR |
Beta(1, 1): every success rate equally likely before any evidence. | Default. No domain knowledge about a subject yet. |
JEFFREYS_PRIOR |
Beta(0.5, 0.5): the reference objective prior for a Bernoulli rate. | When you want the standard objective prior with minimal influence. |
Beta(a, b) |
Your domain belief as pseudo-counts before the first call. | A medical or payment tool starts stricter than a search tool. |
fitCategoryPrior |
A shared prior fit to a group of similar subjects by method of moments. | A cold-start subject should borrow the reputation of its siblings. |
Multi-level hierarchical pooling across categories of categories is planned for a later release.
| Entry point | Subpath export | Runtime | Use it when |
|---|---|---|---|
| Facade | @takk/bayestruth |
node-free | You want the createBayesTruth instance and the full toolkit. |
| MCP | @takk/bayestruth/mcp |
node-free | You want every MCP callTool scored automatically by reliability. |
| Calibration | @takk/bayestruth/calibration |
node-free | You want to measure that the scores are calibrated on your own data. |
| Node store | @takk/bayestruth/node |
Node only | You need durable, file-backed persistence that survives restarts. |
When you want to inspect or curate trust outside a running process, the bayestruth binary reads
a store snapshot from a file or stdin, answers trust questions, folds new outcomes back in, and verifies
audit seals.
# print the posterior mean, credible interval, and counts
npx @takk/bayestruth score search-api \
--store trust-store.json \
--level 0.95# rank best-first, by the conservative lower bound
npx @takk/bayestruth rank --store trust-store.json
# fold one outcome in and write the store back
npx @takk/bayestruth observe search-api success --store trust-store.json
npx @takk/bayestruth verify --log audit-log.json --seal audit-seal.json
# exit codes: 0 ok, 1 error, 64 usage, 65 data error, 66 missing inputEnable the audit trail to record every observation, decision, and selection, then seal it into a tamper-evident root. Use the calibration module to confirm the scores are calibrated on your own data. Both are node-free and need no external service.
const trust = createBayesTruth({ audit: true });
trust.observe('tool-x', 'success');
trust.decide('tool-x');
trust.select(['tool-x', 'tool-y']);
// Seal the append-only log into a SHA-256 hash-chain root.
const seal = await trust.seal(); // { algorithm: 'sha-256', root, count }
const result = await trust.verify(seal); // { valid: true }
// Any change to a sealed entry makes verify return { valid: false }.import { calibrationReport } from '@takk/bayestruth/calibration';
// Record the score you acted on and the outcome that followed.
const report = calibrationReport([
{ p: 0.9, outcome: 'success' },
{ p: 0.9, outcome: 'success' },
{ p: 0.2, outcome: 'failure' },
// ...one entry per decision and its realized outcome
]);
// { count, brier, logLoss, ece, bins } Calibration holds while the assumptions hold: independent Bernoulli trials with a stable rate. The correlated-failure guard and time-decay mitigate the common violations, and this report is how you confirm it on your own traffic.
The other approaches solve part of the problem. The contrast clarifies where BayesTruth sits.
| Capability | BayesTruth | Wilson interval | Hand-rolled | Observability | PyMC / Stan |
|---|---|---|---|---|---|
| Distribution | npm library | a formula | your repo | SaaS / library | pip |
| Calibrated credible interval | exact Beta quantile | approximate | rarely | no | yes |
| Explicit priors | yes | no | no | no | yes |
| Partial pooling (cold start) | yes | no | no | no | yes |
| Thompson-sampling routing | yes | no | rarely | no | build it |
| Tamper-evident audit | yes | no | no | partial | no |
| Zero deps, node-free | yes | n/a | varies | no | no, Python |
| License | Apache-2.0 | n/a | your call | varies | Apache / BSD |
The honest summary: a Wilson interval is a fine substitute for one subject's confidence range. Pick PyMC or Stan if you are doing offline statistical modeling in Python. Pick BayesTruth if you want explicit priors, pooling, calibrated decisions, routing, and a tamper-evident trail composed in one node-free TypeScript library you embed in production.
BayesTruth is exact Bayesian inference, and exactness has assumptions. They are documented, and where real traffic breaks them, there is an explicit mitigation rather than a silent fudge.
| Assumption or property | What it means | How BayesTruth handles it |
|---|---|---|
| Independent trials | Each call is its own Bernoulli trial. | The correlated-failure guard coalesces a burst into one event. |
| Stable success rate | The underlying rate does not drift over time. | Opt-in time-decay relaxes stale evidence toward the prior. |
| Binary outcome | Each call is a success or a failure. | A classifier maps any result, including MCP isError, to an outcome. |
| Exact quantile | The interval is the true Beta quantile, not a normal approximation. | lgamma, the regularized incomplete beta, and a bisection inverse. |
| Calibration is conditional | Scores are calibrated under the model, not unconditionally. | The calibration module measures it on your own data. |
| Cold start | A new subject has only its prior. | Empirical-Bayes pooling borrows strength from its siblings. |
| Integrity, not identity | The audit seal proves a log was not altered after sealing. | A SHA-256 hash chain via Web Crypto, an integrity seal, not a signature. |
| Reproducible routing | Bandit choices must be auditable. | A seeded generator makes Thompson selection deterministic. |
170 tests passing across 20 suites under Vitest 4. Coverage: 97.39% lines, 97.48%
statements, 100% functions, 93.55% branches. Run pnpm test on a fresh clone to reproduce.
TypeScript 6 in maximum strict mode (exactOptionalPropertyTypes,
useUnknownInCatchVariables, noUncheckedIndexedAccess,
noImplicitOverride, noImplicitReturns). Zero errors under tsc
--noEmit.
Biome 2.5 clean across src and tests. publint clean and
@arethetypeswrong/cli green across all fourteen subpaths. Dual ESM + CJS with separate
.d.ts and .d.cts per subpath.
The special functions reproduce known identities: exp(lgamma(5)) equals 24, the Beta(1, 1)
CDF is the identity, and the quantile inverts the CDF. The credible interval is the true Beta quantile, not
an approximation.
A 22-check smoke test exercises the compiled ESM and CJS artifacts and spawns the compiled CLI as a single Node process, run on the Node 20, 22, and 24 matrix.
Committed pnpm lockfile, zero required runtime dependencies, node-free core, and SLSA provenance
attestation on every published version. Verify with npm view @takk/bayestruth@1.0.0 --json | jq
.dist.attestations.
Yes. 170 tests across 20 suites pass under Vitest 4 with 97% coverage; TypeScript 6 maximum strict mode is
clean; Biome 2.5 lint is clean; publint and @arethetypeswrong/cli are clean across
all fourteen subpaths. The suite and the distribution smoke test are run on the Node 20, 22, and 24 matrix.
Every published release carries SLSA provenance produced by GitHub Actions.
For a single subject's confidence range, Wilson is a close substitute. BayesTruth wins where Wilson cannot follow: explicit and per-category priors, partial pooling so a cold-start tool borrows strength from siblings, sequential decisions, and Thompson-sampling routing, all composed in one calibrated system.
Those are superb for offline statistical modeling in Python. BayesTruth is embeddable TypeScript for production: closed-form conjugate updates with zero inference latency, a node-free core, and an API a developer can use in five lines without a statistics background.
Yes. The core is node-free; the audit seal uses the Web Crypto API, not node:crypto. Import
@takk/bayestruth or @takk/bayestruth/edge on any runtime with fetch
and Web Crypto. Only @takk/bayestruth/node requires the Node standard library.
BayesTruth records the subject identifiers you choose, the posterior parameters, the observed counts, and timestamps. It never sees your tool payloads and makes no outbound network calls of its own. With the file store, only that operational state reaches disk; no secrets are involved at any point.
A new subject starts at its prior, the uniform Beta(1, 1) by default, so its interval is wide and honest.
To do better than a flat prior, fit a category prior with fitCategoryPrior and score the new
subject against it; it borrows the reputation of its siblings until it has data of its own.
In-process memory by default, with portable JSON snapshots via snapshot() and
load(). For durability across restarts, use createFileStore from
@takk/bayestruth/node. For multi-process coordination, implement the PosteriorStore
interface over SQLite, Postgres, or Redis.
No, and the docs are explicit about it. The seal is a SHA-256 hash-chain root that proves a log was not altered after sealing. It is an integrity seal, not a signature: it does not prove who produced the log. Signed and timestamped seals for stronger third-party evidence are on the roadmap.
Under the model, independent Bernoulli trials with a stable rate, yes, and the interval is the exact Beta
quantile. Real traffic is bursty and non-stationary, so use the correlated-failure guard and time-decay,
and confirm calibration on your own data with @takk/bayestruth/calibration.
Every release is published with npm publish --provenance. Check the attestations with
npm view @takk/bayestruth@<version> --json | jq .dist.attestations. The attestation links
the tarball you installed to the GitHub Actions workflow that built it from a specific source commit.
Yes. Implement the PosteriorStore interface with get, set,
keys, and snapshot, and pass it as store in the config. Back it with
SQLite, Postgres, Redis, or anything else; the file store is the zero-dependency reference.
Strict SemVer 2.0.0, starting from 1.0.0. The binding stability surface is documented in SPEC.md section 5. Major bumps require a deprecation cycle; security fixes follow the disclosure flow in SECURITY.md.