Leaving verification to ad-hoc regexes and one-off judge prompts has no contract and no audit trail; handing it to a closed endpoint hides the verifier and signs nothing. Tribunal runs in your process with your keys: configurable verifier panels and evaluators decide under a policy you set, and every verdict is signed and sealed so you can prove it.
Tribunal governs the verification of model outputs. You configure evaluators and a panel of judges and
inject your own model-as-judge, and it folds their findings into a governed verdict under your policy, with
an outcome, a severity, a score, and the provenance of every contributor, then signs and seals it. Your
code is two lines:
new Tribunal({ evaluators, panel, policy }), then tribunal.tryCandidate(candidate).
Deterministic evaluators, latency, schema, regex, and exact-match, and a model-as-judge run over an injectable client. A panel deliberates under a configurable quorum; the findings fold into a verdict that fails when any lands at or above the severity you choose, or below your minimum score. The verdict is signed with Ed25519 and sealed into a tamper-evident SHA-256 hash chain over the Web Crypto API. The core never speaks a provider API.
Take a real production scene: an agent's output is about to write to a database or trigger a payment, so you want it verified, with the verdict configurable, signed, and auditable, not decided for you in the dark.
Governed verification is the middle ground between scattered DIY checks and an opaque closed box; it keeps the rubric, the policy, and the keys in your hands, on your infrastructure, with every verdict signed and auditable.
install to your first signed verdict.pnpm add @takk/tribunalnpm install @takk/tribunalyarn add @takk/tribunalbun add @takk/tribunalThe core has no runtime dependencies and a node-free core. Every @takk sibling is an optional
peer; install only what you compose with.
# nothing else is required to start experimenting
# optional: install a sibling you compose with, for example
pnpm add @takk/keymesh @takk/modelchainimport { Tribunal, createHeuristicJudge, defineRubric, regexEvaluator } from '@takk/tribunal';
const rubric = defineRubric({
id: 'quality',
passThreshold: 0.6,
criteria: [{ id: 'mentions-fix', description: 'mentions a fix', weight: 1, mustInclude: ['fix'] }],
});
// Configure evaluators, a judge panel, and a policy; inject your own judge for real models.
const tribunal = new Tribunal({
evaluators: [regexEvaluator({ pattern: /fix/ })],
panel: { rubric, judges: [createHeuristicJudge()] },
policy: { failAtOrAbove: 'major' },
});
const verdict = await tribunal.tryCandidate({ id: 'out-1', content: 'applied the fix' });
verdict.outcome; // 'pass' | 'fail' | 'inconclusive'
verdict.severity; // worst finding on the seven-level ladder
verdict.provenance.contributors; // how many evaluators and judges weighed inimport { createVerifyTool } from '@takk/tribunal/adapter';
// A framework-agnostic descriptor (name, description, JSON Schema, handler) that
// matches what MCP servers and tool-calling APIs expect. Its name is 'tribunal_verify'.
const tool = createVerifyTool(tribunal);
const verdict = await tool.handler({
candidate: 'the model output to verify',
expected: 'the model output to verify',
});
// the host fixes the evaluators and policy; the caller only sends a candidateThinker, Worker, and Verifier are first-class and overridable, with system scaffolds you can see and replace, the opposite of an endpoint that hides who reasoned, who produced, and who checked.
You frame how each role judges, instead of trusting a hidden verifier.
Latency, schema validity, regular expression, and exact match run offline with no network and fold into the same verdict as the model-as-judge.
Cheap, reproducible checks catch the obvious failures before a model is ever called.
Inject an OpenAI-style chat or a Vercel call as a judge; Tribunal builds the Verifier prompt from your rubric and parses a typed reply, recovering per-criterion scores.
Bring any model as a judge in two lines, provider-agnostic and testable.
Several judges deliberate over the same candidate under a quorum you choose, majority, unanimous, any, or at-least-N, and the panel reports passes, mean score, and agreement.
Diverse judges catch failure modes a single judge misses, the Condorcet jury effect.
A seven-level severity ladder plus an optional minimum score. You set the severity at which a verdict fails; the verdict's severity is the worst of its findings.
You decide what passed means, instead of a box deciding it for you and hiding why.
Every verdict can be signed over its canonical bytes with Ed25519 via the Web Crypto API; the signed envelope carries the public key needed to verify it.
Prove a verdict was issued by the holder of the key and was not altered.
Signed verdicts chain into a SHA-256 hash chain; altering any past entry breaks every hash after it, and
verifyChain pinpoints the break, in Node, edge runtimes, or the browser.
An unforgeable log the compliance review asks for when a verdict is questioned.
Verification runs in your process, on your infrastructure, with your keys. The core never speaks a provider API; no vendor sits in the middle of every verdict, and the EU and EEA are not blocked.
Run governed verification where a closed, region-locked endpoint cannot follow you.
Every published version signed with npm publish --provenance through GitHub Actions OIDC.
Lockfile committed, zero required runtime dependencies, node-free core.
Verify in one command that the tarball you installed was built from the source commit you trust.
Four deterministic evaluators run offline, and a model-as-judge runs over your injected client. They fold into one governed verdict under your policy; the only knobs are the rubric, the panel quorum, and the severity that fails a verdict.
| Check | What it is | Fails when |
|---|---|---|
latency |
The candidate was produced within a time budget you set. | The recorded latency exceeds maxMs. |
schema |
The content parses as JSON and matches a minimal, dependency-free shape spec. | The content is not valid JSON, or violates the shape. |
regex |
The content matches, or with mustMatch: false does not match, a pattern. |
The pattern is absent (or present, when inverted). |
exact-match |
The content equals an expected string, with optional whitespace and case normalization. | The content differs from the expected value. |
model-as-judge |
A panel of judges scores the candidate against your rubric and votes under a quorum. | Fewer judges pass than the quorum requires. |
The quorum is majority, unanimous, any, or { atLeast: n }.
The built-in createHeuristicJudge is deterministic and offline; a real model-as-judge plugs in
through the same JudgeClient seam.
| Entry point | Subpath export | Runtime | Use it when |
|---|---|---|---|
| Toolkit barrel | @takk/tribunal |
node-free | You want the full surface: the facade, evaluators, judge, panel, rubric, verdict, synthesis, signing, audit, roles, and the adapters. |
| Tribunal facade | @takk/tribunal/tribunal |
node-free | You want the Tribunal facade on its own to verify, synthesize, and seal. |
| Judge and panel | @takk/tribunal/judge, /panel |
node-free | You want the model-as-judge, the heuristic judge, or the mixture-of-agents quorum on their own. |
| Signing and audit | @takk/tribunal/signing, /audit |
node-free | You want Ed25519 signing or the tamper-evident hash chain on their own, over Web Crypto. |
| Adapters and tool | @takk/tribunal/adapter |
node-free | You want the OpenAI-style or Vercel judge adapter, or the verify tool for an MCP server. |
| Node loaders | @takk/tribunal/node |
Node only | You need to load a case, a rubric, a key pair, a verdict, or a saved audit chain from JSON files. |
The tribunal binary runs the compiled engine. verify runs a panel over a case;
judge scores one candidate against a rubric; keygen mints an Ed25519 key pair;
audit-verify checks a sealed chain; verdict renders a verdict and verifies its
signature. It runs offline with the deterministic heuristic judge; real provider runs use the library API.
# run a verifier panel over the case; exit 10 if the synthesis fails
npx @takk/tribunal verify case.json --rubric rubric.json --quorum majority --fail-at major# judge one candidate file against a rubric (exit 10 on fail)
npx @takk/tribunal judge answer.txt --rubric rubric.json
# generate an Ed25519 key pair for signing verdicts
npx @takk/tribunal keygen --out keys.json
npx @takk/tribunal audit-verify chain.json
npx @takk/tribunal verdict signed.json
# exit codes: 0 ok/pass, 2 usage or input, 10 verdict failed, 20 broken chain, 21 invalid signatureSign every verdict with Ed25519 and seal it into a tamper-evident, hash-chained log. And because every
verdict is explicit, tryCandidate returns the outcome, the severity, the score, the findings,
and the provenance, so the judgment is never a black box. Both the signature and the seal are node-free and
need no external service.
import { verifyChain } from '@takk/tribunal/audit';
import { generateSigningKeyPair } from '@takk/tribunal/signing';
const keyPair = await generateSigningKeyPair();
// Verify a candidate, sign the verdict with Ed25519, and seal it into the chain.
const sealed = await tribunal.sealedVerdict({ id: 'out-1', content: 'applied the fix' }, keyPair);
await verifyChain(sealed.chain.toArray()); // { valid: true }, until any entry is alteredconst verdict = await tribunal.tryCandidate({ id: 'out-1', content: 'applied the fix' });
// {
// id: 'verdict:out-1',
// outcome: 'pass', // 'pass' | 'fail' | 'inconclusive'
// severity: 'info', // worst finding on the seven-level ladder
// score: 0.86, // aggregate score in [0, 1]
// findings: [ ... ], // one per evaluator, plus the panel
// provenance: { evaluators: [...], judges: [...], contributors: 4 },
// policy: { failAtOrAbove: 'major' }
// }The verdict carries its outcome, severity, score, findings, and the provenance of every contributor, so you can see how the output was judged. Sign it and seal it into the audit chain with no runtime dependency, the governance seam for a non-human entity that must earn trust in its own outputs.
The other approaches solve part of the problem. The contrast clarifies where Tribunal sits.
| Capability | Tribunal | Closed verifier | Agent framework | Vercel AI SDK | Hand-rolled |
|---|---|---|---|---|---|
| Distribution | npm library | hosted endpoint | npm library | npm library | your repo |
| Configurable verifier | you define it | fixed, hidden | DIY | none | varies |
| Cross-verification quorum | panel quorum | single, hidden | none | none | none |
| Signed verdicts (Ed25519) | yes | no | no | no | varies |
| Provider-agnostic | injectable | pool fixed | yes | yes | varies |
| Local-first sovereignty | your infra | region-locked | yes | yes | yes |
| Tamper-evident audit | yes | no | no | no | no |
| Zero deps, node-free | yes | n/a | deps | deps | varies |
| License | Apache-2.0 | proprietary | mixed | Apache-2.0 | your call |
The honest summary: a closed verifier is fine when you do not need to configure or prove anything, and a single regex is fine for an easy check. Pick Tribunal when you want the verifier configurable, the verdict cross-verified by a quorum, signed and tamper-evident, and sovereignty over where it runs, all in one node-free TypeScript library you embed in production.
Tribunal is governed verification over an injectable judge, and that has boundaries. They are documented, and where reality bends them, there is an explicit choice rather than a silent fudge.
| Assumption or property | What it means | How Tribunal handles it |
|---|---|---|
| Heuristic judge | The built-in judge is deterministic, not an LLM; it scores against the rubric's own hints. | It lets tests, examples, and the CLI run offline with reproducible numbers; a real model-as-judge plugs in behind the same JudgeClient seam. |
| Injectable judge | The core never speaks a provider API; the judgment quality is the injected judge's and the provider's. | You provide the judge; Tribunal scores, governs, signs, and seals it, and tests run against the deterministic heuristic judge. |
| Judge calibration | A single model-as-judge is imperfectly calibrated; the literature documents calibration deficits. | Mitigated by mixture-of-agents panels and rubrics; never claimed as a calibrated oracle. |
| Scores are clamped and aggregated | Per-criterion and evaluation scores are coerced into [0, 1]. | The verdict's score is the mean of evaluation and panel scores; the policy gates the outcome on severity and an optional minimum score. |
| Quorum default | A panel without an explicit quorum defaults to majority. | You choose majority, unanimous, any, or at-least-N; an empty panel is rejected with a typed error. |
| Every check runs | In 1.0 every configured evaluator and the panel run on every candidate. | Selective, confidence-gated verification (run the expensive panel only when confidence is low) is a documented later release. |
| Provenance from contributors | Provenance is which evaluators and judges weighed in. | It is recorded from the actual evaluations and panel votes, not asserted by the judges. |
| Integrity and identity | The audit seal proves a log was not altered; the signature proves who issued a verdict. | A SHA-256 hash chain for integrity and an Ed25519 signature for authenticity, two distinct guarantees, both over Web Crypto. |
117 tests passing under Vitest 4 with 96% coverage, across 18 suites on the Node 20, 22,
and 24 matrix. Six runnable examples and a composition benchmark verify against the built artifact end to
end. Run pnpm test on a fresh clone to reproduce.
TypeScript 6 in maximum strict mode (exactOptionalPropertyTypes,
useUnknownInCatchVariables, noUncheckedIndexedAccess,
noImplicitOverride, noImplicitReturns). Zero errors under tsc
--noEmit.
Biome 2.5 clean across src and tests. publint clean and
@arethetypeswrong/cli green across all fourteen subpaths. Dual ESM + CJS with separate
.d.ts and .d.cts per subpath.
A panel meets the documented quorum, a finding at or above the fail severity fails the verdict, a tampered Ed25519 signature is rejected, and the audit chain detects a tampered entry at exactly its index. Every invariant is a test.
A smoke test exercises the compiled ESM and CJS artifacts, a full verification with a panel, Ed25519 sign and seal, tamper detection, and the compiled CLI spawned as a single Node process, on the Node 20, 22, and 24 matrix.
Committed pnpm lockfile, zero required runtime dependencies, node-free core, and SLSA provenance
attestation on every published version. Verify with npm view @takk/tribunal@1.0.0 --json | jq
.dist.attestations.
Yes. 117 tests pass under Vitest 4 with 96% coverage; TypeScript 6 maximum strict mode is clean; Biome 2.5
lint is clean; publint and @arethetypeswrong/cli are clean across all fourteen
subpaths. The suite and the distribution smoke test are run on the Node 20, 22, and 24 matrix. Every
published release carries SLSA provenance produced by GitHub Actions.
You can, and Tribunal lets you: a single evaluator or one heuristic judge is a one-liner. The gap shows up at scale: ad-hoc checks have no shared contract, a single judge is imperfectly calibrated, and nothing is signed or auditable. Tribunal folds deterministic evaluators and a cross-verifying panel into one governed verdict you define, then signs and seals it, so a high-risk output gets a verdict you can prove.
A closed endpoint verifies internally and opaquely: you cannot configure the verifier, cannot inspect the verdict, and cannot sign it. Tribunal puts the rubric, the evaluators, the quorum, and the severity that fails a verdict in your hands, runs in your process with your keys, and hands back a signed, tamper-evident verdict with provenance, so you can prove the judgment to a reviewer or a regulator.
No. The core is verification logic over an injectable judge. You provide the model-as-judge, an OpenAI-style chat function, a Vercel AI SDK call, or the deterministic heuristic judge, and Tribunal scores, governs, signs, and seals it. That keeps the package provider-agnostic, dependency-free, and testable with no network, and it means your provider keys live in your judge, never in Tribunal.
Yes. The core is node-free; the Ed25519 signatures and the audit seal use the Web Crypto API, not
node:crypto. Import @takk/tribunal or @takk/tribunal/edge on any
runtime with Web Crypto. Only @takk/tribunal/node, the JSON loaders, requires the Node
standard library.
Tribunal holds the candidate content you submit, the rubric and policy you configure, and the verdicts it builds, nothing else. It makes no outbound network calls of its own; the only model calls are the ones your injected judge makes to your chosen providers. With the file loaders, only the case, rubric, key-pair, verdict, or chain files you point it at are read from disk, and no provider credentials are ever held by Tribunal.
A panel runs several judges over the same candidate and folds their votes under the quorum you choose, majority, unanimous, any, or at-least-N. It reports how many judges passed, the quorum needed, the mean score, and the agreement fraction. Diverse judges catch failure modes a single judge misses; you set the quorum, and the verdict records which judges weighed in.
Both, and they are distinct. Each chain entry is bound to the previous one through a SHA-256 hash of its
canonical form, so any later edit breaks the chain, that is the integrity seal. Separately, a verdict can be
signed with Ed25519, which proves who issued it, that is authenticity. Use sealedVerdict to get
a signed verdict sealed into a tamper-evident chain in one call.
Any of them. Because you inject the judge, Tribunal scores against OpenAI, Anthropic, Google, a local model, or a mix, treated uniformly. Adapters are included for an OpenAI-style chat function and the Vercel AI SDK, so wrapping your existing call is two lines.
In-process memory by default. Tribunal holds the candidate content, the rubric, and the policy while
it verifies; snapshot() returns a descriptive view of the configured evaluators, judges, and
policy for inspection. For disk-backed input, load a case, a rubric, a key pair, a verdict, or a chain from
JSON with @takk/tribunal/node. For an edge runtime, persist verdicts and the audit chain to
your own KV store.
Run the cheap deterministic evaluators first; they make no network calls. Reserve the model-as-judge panel for outputs that need it, and keep the panel small. Selective, confidence-gated verification, running the expensive panel only when confidence is low, is on the roadmap. Cross-verification trades more judge calls for fewer escaped errors, the Condorcet effect, so size the panel to the risk.
Every release is published with npm publish --provenance. Check the attestations with
npm view @takk/tribunal@<version> --json | jq .dist.attestations. The attestation
links the tarball you installed to the GitHub Actions workflow that built it from a specific source commit.
Strict SemVer 2.0.0, starting from 1.0.0. The binding stability surface is documented in SPEC.md section 5. Major bumps require a deprecation cycle; security fixes follow the disclosure flow in SECURITY.md.