Majority vote counts every agent equally, so an unreliable bloc outvotes a reliable minority and the answer is a tally, not a probability. BayesConsensus learns each agent's reliability and the true answer from the votes alone, then returns a consensus label with a calibrated confidence.
BayesConsensus takes the votes a fleet of agents, judges, or models cast on shared items and returns a
single defensible answer per item. It does not trust voters equally: it learns each one's reliability from
the pattern of agreement across items, then weights the votes accordingly. A reliable minority can overrule
an unreliable majority, and every verdict carries a calibrated confidence. Your code is two lines:
model.observe(item, agent, label), then model.consensus(item).
The Dawid and Skene (1979) expectation-maximization algorithm estimates, from the votes alone, the latent true label of every item AND a confusion matrix for every agent, the diagonal of which is its reliability. The E-step runs in log-space with log-sum-exp normalization, the M-step re-estimates the matrices with diagonal Dirichlet smoothing, and the labels are initialized from the majority vote for determinism. Calibration is measured by simulation, and a tamper-evident hash-chained audit trail records every verdict.
Take a real scene: a fleet of seven agents labels a stream of items, some agents are reliable and some are near-random, and you need one trustworthy answer per item with a confidence you can act on.
Dawid-Skene is the reliability-weighted middle ground between a blind majority and a single trusted expert; in the benchmark it beats majority by up to 14 percentage points where reliability varies across the fleet, ties when every agent is equally competent, and is honestly weaker only when the whole fleet is uniformly weak.
install to your first consensus verdict.pnpm add @takk/bayesconsensusnpm install @takk/bayesconsensusyarn add @takk/bayesconsensusbun add @takk/bayesconsensusThe core has no runtime dependencies and a node-free core. Every @takk sibling is an optional
peer; install only what you compose with.
# nothing else is required to start experimenting
# optional: install a sibling you compose with, for example
pnpm add @takk/keymesh @takk/modelchainimport { ConsensusModel } from '@takk/bayesconsensus';
// Four agents label whether a snippet is safe; one is a rubber stamp.
const jury = new ConsensusModel({ labels: ['safe', 'unsafe'] });
jury.observe('snippet-1', 'reviewer-a', 'unsafe');
jury.observe('snippet-1', 'reviewer-b', 'unsafe');
jury.observe('snippet-1', 'reviewer-c', 'unsafe');
jury.observe('snippet-1', 'rubber-stamp', 'safe');
const verdict = jury.consensus('snippet-1');
// verdict.label is 'unsafe', verdict.confidence ~0.94, the rubber stamp is down-weighted
// Audit any agent's learned reliability at the same time.
jury.reliability('rubber-stamp').accuracy; // a single reliability score in 0..1import { bayesConsensusTool } from '@takk/bayesconsensus/adapter';
// A framework-agnostic descriptor (name, description, JSON Schema, handler) that
// matches what MCP servers and tool-calling APIs expect. Its name is 'bayes_consensus'.
const result = bayesConsensusTool.handler({
votes: [
{ item: 'claim-1', agent: 'model-a', label: 'true' },
{ item: 'claim-1', agent: 'model-b', label: 'true' },
{ item: 'claim-1', agent: 'model-c', label: 'false' },
],
labels: ['true', 'false'],
query: 'claim-1', // omit to get the full report instead of one item
});
// only votes ever enter; never any other agent stateEvery agent gets a confusion matrix learned from the data, and its votes count in proportion to its estimated reliability while the true labels are inferred, in one coupled estimation.
A reliable minority can overrule an unreliable majority, instead of letting the loudest bloc decide by headcount.
Expectation-maximization recovers each agent's accuracy with no answer key, from the structure of agreement and disagreement across many items, alternating label and reliability estimates until convergence.
You never label which agents are good; the model finds out, so it works on data you have not graded.
The soft mode keeps the full posterior for a calibrated confidence; the hard mode (Fast Dawid-Skene) collapses each step to its argmax for speed when calibration is not needed.
Trade a little calibration for throughput with a single option, instead of swapping libraries.
The estimated confusion matrix and a single accuracy score are first-class outputs, with per-class accuracy read straight off the diagonal.
You can find the rubber stamp, the contrarian, and the silently-degrading model in your fleet, instead of trusting them all equally.
Every verdict carries the full posterior over labels, and the winning probability is a calibrated confidence you can threshold, route on, or escalate from.
You act on a probability you can trust, instead of a raw fraction of votes that does not mean what it looks like.
Calibration is measured by simulation, drawing a fleet from known reliabilities, fitting the model, and scoring accuracy and Brier against the truth, not asserted.
You can check the confidence matches the empirical accuracy under your regime, instead of trusting an unverified promise.
An append-only log of every consensus verdict and decision, with a SHA-256 hash-chain you append
to and verifyChain via the Web Crypto API.
Prove a verdict was reached on the recorded votes, in Node, edge runtimes, or the browser, when a decision is questioned.
Observe votes as agents report them and query at any time; the fit is lazy and the state is just the label
space and the votes, so a snapshot round-trips through JSON with fromSnapshot.
Consensus survives restarts and moves between processes as plain JSON, with no hidden state and no database required.
The built-in diversity check measures the error correlation across the fleet, reports the effective number of independent voters, and warns when the agents fail in lockstep, the assumption Dawid-Skene needs but does not police.
An inflated confidence from correlated, near-identical models is caught and can be discounted, instead of trusted because the votes happened to agree.
A streaming model keeps running sufficient statistics and folds each batch in with a warm-started EM step, with optional exponential forgetting so the model adapts when an agent's accuracy drifts.
Reliability evolves with the data and tracks a degrading agent, instead of averaging its decline away or re-fitting the whole history every time.
Every published version signed with npm publish --provenance through GitHub Actions OIDC.
Lockfile committed, zero required runtime dependencies, node-free core.
Verify in one command that the tarball you installed was built from the source commit you trust.
The Dawid-Skene fit runs with sensible defaults, so the common case needs no tuning. The options that exist are explicit and few: the inference mode, the smoothing strength, the label space, and the convergence limits.
| Option | What it is | When to use it |
|---|---|---|
mode |
"soft" (default) keeps the full posterior for a calibrated confidence; "hard" is Fast Dawid-Skene, a one-hot argmax E-step that is faster and less calibrated. | Keep soft for a trustworthy confidence; switch to hard for throughput on large vote sets. |
diagonalPrior |
The extra Dirichlet pseudocount on each confusion-matrix diagonal, so agents start a little better than random. Default 1. | Raise it when agents are few or sparse and you want stronger regularization toward competence. |
labels |
A fixed label space. If omitted, the labels are discovered from the votes and sorted for determinism. | Set it to pin the column order, or to reject votes outside a known set of labels. |
maxIterations / tolerance |
The EM stopping rule: stop when the marginal log-likelihood change falls below the tolerance (default 1e-5) or the iteration cap (default 100) is reached. | Tighten the tolerance for a more exact fit, or cap iterations for a strict time budget. |
Dawid-Skene is an expectation-maximization point estimate; it converges to a local optimum of the marginal likelihood, made deterministic here by initializing the labels from the majority vote and running the M-step first.
| Entry point | Subpath export | Runtime | Use it when |
|---|---|---|---|
| Toolkit barrel | @takk/bayesconsensus |
node-free | You want the facade, the Dawid-Skene core, calibration, the adapter, and the audit chain. |
| Consensus facade | @takk/bayesconsensus/consensus |
node-free | You want ConsensusModel on its own: observe, consensus, reliability, aggregate. |
| Diversity diagnostic | @takk/bayesconsensus/diversity |
node-free | You want to measure error correlation across the fleet and discount an over-confident verdict. |
| Streaming | @takk/bayesconsensus/online |
node-free | You want OnlineConsensus: incremental updates with optional forgetting. |
| Tool adapter | @takk/bayesconsensus/adapter |
node-free | You want the framework-agnostic tool descriptor for an MCP server or tool-calling API. |
| Node loaders | @takk/bayesconsensus/node |
Node only | You need to load votes from JSON or CSV files. |
The bayesconsensus binary runs the compiled engine, so every number it prints comes from
execution. aggregate fits the model and prints every item's verdict; consensus prints
one item's label, confidence, and posterior; reliability prints each agent's confusion matrix;
audit-verify checks a sealed chain.
# fit the model and print every item's consensus verdict
npx @takk/bayesconsensus aggregate votes.json --labels yes,no# one item's consensus label, confidence, and posterior as JSON
npx @takk/bayesconsensus consensus votes.json item-1 --labels yes,no --json
npx @takk/bayesconsensus reliability votes.json --labels yes,no
npx @takk/bayesconsensus audit-verify chain.json
npx @takk/bayesconsensus --version
# exit codes: 0 ok, 2 usage or input, 20 broken chainRecord every consensus verdict and decision in a tamper-evident, hash-chained log. And because each result is
explicit, consensus returns the label, the calibrated confidence, the full posterior, and the
agreement, so a verdict is never a black box. Both are node-free and need no external service.
import { AuditChain, verifyChain } from '@takk/bayesconsensus/audit';
const chain = new AuditChain();
await chain.append({ kind: 'consensus', item: 'claim-1', label: 'true', confidence: 0.93 });
await chain.append({ kind: 'decision', item: 'claim-2', label: 'false' });
await verifyChain(chain.toArray()); // { valid: true }, until any entry is alteredconst verdict = jury.consensus('claim-1');
// {
// item: 'claim-1',
// label: 'true', // the maximum-a-posteriori consensus label
// confidence: 0.9312, // calibrated posterior probability of the winner
// distribution: { true: 0.9312, false: 0.0688 },
// votes: 3,
// agreement: 0.6667, // raw fraction of votes on the consensus label
// source: 'fitted'
// }The confidence is the posterior probability of the winning label, the agreement is the raw fraction of votes on it, and a wide gap between the two is the signal that the model down-weighted an unreliable bloc. Seal each verdict into the audit chain with no runtime dependency, the governance seam for a non-human entity resolving disagreement across its fleet.
The other approaches solve part of the problem. The contrast clarifies where BayesConsensus sits.
| Capability | BayesConsensus | Majority vote | Single expert | Judge-ensemble SaaS | Hand-rolled |
|---|---|---|---|---|---|
| Distribution | npm library | a one-liner | a one-liner | hosted service | your repo |
| Weights by reliability | learned | equal weight | one voter | varies | rarely |
| Calibrated confidence | posterior | a tally | none | varies | rarely |
| Per-agent reliability audit | yes | no | no | partial | no |
| No ground truth needed | yes | yes | yes | varies | varies |
| Tamper-evident audit | yes | no | no | partial | no |
| Zero deps, node-free | yes | trivial | n/a | no | varies |
| License | Apache-2.0 | n/a | n/a | proprietary | your call |
The honest summary: a plain majority is fine when every agent is equally reliable, and a single trusted expert is fine when you have one you can always trust. Pick BayesConsensus when reliability varies across the fleet, you need a calibrated confidence to route or escalate on, you want a per-agent reliability audit, and you want a tamper-evident trail, composed in one node-free TypeScript library you embed in production.
BayesConsensus is the Dawid-Skene model fit by expectation-maximization, and that has assumptions. They are documented, and where reality bends them, there is an explicit choice rather than a silent fudge.
| Assumption or property | What it means | How BayesConsensus handles it |
|---|---|---|
| Conditional independence | Agents err independently of one another given the true label. | Documented; voters that share training data make correlated errors, so the calibration output and the diverse-voter guidance are first-class, not an afterthought. |
| Reliability spread | There is variation in reliability across the fleet for the model to learn. | Where every agent is equally weak there is no signal; the benchmark measures this and the docs say to use majority vote instead. |
| Discrete labels | The target is a categorical label from a fixed or discovered set. | Categorical labels are first-class; ordinal and continuous targets are a documented non-goal for 1.0. |
| EM point estimate | The fit converges to a local optimum of the marginal likelihood, not a full posterior. | Made deterministic by initializing from the majority vote and running the M-step first; an MCMC full-Bayes mode is a non-goal for 1.0. |
| Label switching | Without care, EM can converge to a permutation of the labels. | The deterministic majority-vote initialization anchors the label identities, so the consensus tracks the intended labels. |
| Measured calibration | A calibrated confidence is an empirical property under a regime, not a theorem. | It is measured by simulation, drawing a fleet from known reliabilities and scoring accuracy and Brier against the truth. |
| Missing votes | Agents need not vote on every item. | The model sums only over the votes actually cast; missing votes are first-class and never imputed or padded. |
| Integrity, not identity | The audit seal proves a log was not altered after sealing. | A SHA-256 hash chain via Web Crypto, an integrity seal, not a signature. |
86 tests passing under Vitest 4 with 97% coverage, including an adversarial hardening
suite. The benchmark shows Dawid-Skene beating majority by up to 14 percentage points where reliability
varies across the fleet, and reports the one regime where it loses. Run pnpm test on a fresh
clone to reproduce.
TypeScript 6 in maximum strict mode (exactOptionalPropertyTypes,
useUnknownInCatchVariables, noUncheckedIndexedAccess,
noPropertyAccessFromIndexSignature, verbatimModuleSyntax). Zero errors under
tsc --noEmit.
Biome 2.5 clean across src and tests. publint clean and
@arethetypeswrong/cli green across all fourteen subpaths. Dual ESM + CJS with separate
.d.ts and .d.cts per subpath.
The E-step runs in log-space and the marginal log-likelihood is asserted non-decreasing across an EM sweep. On a reliable-majority dataset the fit recovers the true labels and the per-agent confusion matrices, and a unanimous item returns a confidence above 0.9.
A smoke test exercises the compiled ESM and CJS artifacts and spawns the compiled CLI as a single Node process, run on the Node 20, 22, and 24 matrix.
Committed pnpm lockfile, zero required runtime dependencies, node-free core, and SLSA provenance
attestation on every published version. Verify with npm view @takk/bayesconsensus@1.0.0 --json | jq
.dist.attestations.
Yes. 86 tests pass under Vitest 4 with 97% coverage; TypeScript 6 maximum strict mode is clean; Biome 2.5
lint is clean; publint and @arethetypeswrong/cli are clean across all fourteen
subpaths. The suite and the distribution smoke test are run on the Node 20, 22, and 24 matrix. Every
published release carries SLSA provenance produced by GitHub Actions.
Because majority vote counts every agent equally, so an unreliable bloc outvotes a reliable minority, and it returns a tally rather than a probability, three of four votes is not a 75 percent chance. BayesConsensus learns each agent's reliability from the pattern of agreement across items and weights votes accordingly. In the benchmark it beats majority by up to 14 percentage points where reliability varies across the fleet.
By expectation-maximization. It alternates between estimating the true labels given the current per-agent reliabilities and re-estimating those reliabilities given the current label estimates, until the likelihood converges. The agreement structure across many items is enough to recover who is reliable, with no answer key supplied.
Yes. It is the posterior probability of the winning label under the fitted model, computed in log-space and
normalized per item, not a fraction of votes. Calibration is measured by simulation with
measureCalibration, and the benchmark reports the Brier score, so you can check the confidence
matches the empirical accuracy under your regime.
Yes. The core is node-free; the audit seal uses the Web Crypto API, not node:crypto. Import
@takk/bayesconsensus or @takk/bayesconsensus/edge on any runtime with Web Crypto.
Only @takk/bayesconsensus/node, the JSON and CSV loaders, requires the Node standard library.
BayesConsensus holds the item identifiers, agent identifiers, and labels you feed it, nothing else. It never sees the prompts, models, or content behind a vote, and makes no outbound network calls of its own. With the file loaders, only the votes files you point it at are read from disk; no secrets are involved at any point.
When you expect every agent to be equally reliable. With no spread in reliability there is no signal to
learn, and on a uniformly weak fleet the model can latch onto noise and underperform plain majority, the
benchmark measures exactly this. Be cautious too when voters are near-identical models that share training
data, since their errors are correlated and the independence assumption is violated; run
model.diversity() to measure that correlation, read the effective number of independent voters,
and discount the confidence with temperDistribution when the fleet is lockstep.
No, and the docs are explicit about it. Each entry is chained to the previous one through a SHA-256 hash of its canonical form, so any later edit breaks the chain. It is an integrity seal, not a signature: it does not prove who produced the log. Signed and timestamped seals for stronger third-party evidence are on the roadmap.
No. Missing votes are first-class. The model sums only over the votes that were actually cast, never imputes or pads, so a sparse voting matrix, the normal case for a real fleet where an agent need not see every item, is handled directly.
In-process memory by default, with portable JSON snapshots via snapshot() and
fromSnapshot(), just the label space and the votes. For disk-backed input, load votes from JSON
or CSV with @takk/bayesconsensus/node. For an edge runtime, snapshot to your own KV store
between invocations.
Soft mode (the default) keeps the full posterior at each E-step, so the confidence is calibrated. Hard mode is Fast Dawid-Skene: it collapses each E-step posterior to its argmax, which is faster but less calibrated. Keep soft when you act on the confidence; switch to hard for throughput when you only need the label.
Every release is published with npm publish --provenance. Check the attestations with
npm view @takk/bayesconsensus@<version> --json | jq .dist.attestations. The attestation
links the tarball you installed to the GitHub Actions workflow that built it from a specific source commit.
Strict SemVer 2.0.0, starting from 1.0.0. The binding stability surface is documented in SPEC.md section 5. Major bumps require a deprecation cycle; security fixes follow the disclosure flow in SECURITY.md.