Your application calls OpenAI, Anthropic, or Gemini and one key hits 429. The request fails. The retry hits the same key. The user waits. Your on-call gets paged. keymesh rotates a pool of keys, opens a circuit on the failing one, and retries on the next eligible key with backoff. The user never sees the 429.
You give keymesh a list of API keys for the same service (OpenAI, Anthropic, Gemini, or any HTTP
endpoint). It hands each request to a key from the pool. When a key fails or rate-limits, keymesh moves
the next request to another key. After a few consecutive failures, it stops using that key for a while.
When the cooldown expires, it tries the key again. Your code does not change shape: you keep calling
client.chat.completions.create() exactly as before.
A generic orchestration layer that wraps any provider client through a deep proxy mirroring the wrapped
SDK shape, attaches eight telemetry events, and runs a per-request loop of pick,
dispatch, classify error, retry or rotate. Pluggable selection strategies,
per-key circuit breaker with closed -> open -> half-open state machine, AWS full-jitter exponential
backoff with total time budget, respect for upstream Retry-After. State persistence is
pluggable too.
Take a real production scene: your job sends 240 requests per minute to OpenAI, you hold three keys provisioned across three projects, and one of them hits its per-minute quota.
429 Too Many Requests.Request timeout.429. keymesh classifies it as transient.key.rotated event fires. Your logger records from=#1
to=#2.half-open.
circuit.closed fires. Key #1 is
back in the pool.The "With keymesh" timeline is the literal event trace from the live integration test in the keymesh repository, run against the real Google Gemini API with seven keys.
install to first rotation.pnpm add @takk/keymeshnpm install @takk/keymeshyarn add @takk/keymeshbun add @takk/keymeshAdapters use optional peer dependencies. Skip this if you only need the generic HTTP adapter.
pnpm add openai # for
@takk/keymesh/openai
pnpm add @anthropic-ai/sdk # for
@takk/keymesh/anthropic
pnpm add @google/genai # for
@takk/keymesh/geminiimport { createKeymesh } from '@takk/keymesh';
import { openaiAdapter } from '@takk/keymesh/openai';
const client = createKeymesh({
provider: openaiAdapter,
keys: process.env.OPENAI_API_KEYS?.split(',')
?? [],
strategy: 'least-used',
circuitBreaker: { threshold: 3, cooldownMs: 30_000
},
retry: { max: 5, baseMs: 200, jitter: true },
telemetry: { enabled: true },
});
// Drop-in: identical to the underlying SDK shape.
const response = await
client.chat.completions.create({
model: 'gpt-4.1',
messages: [{ role: 'user', content: 'Hello.'
}],
});import { createKeymesh } from '@takk/keymesh';
import { httpAdapter } from '@takk/keymesh/http';
const tavily = createKeymesh({
provider: httpAdapter({
baseUrl: 'https://api.tavily.com',
authHeader: (key) => ({ Authorization: `Bearer ${key}` }),
}),
keys: process.env.TAVILY_API_KEYS?.split(',')
?? [],
strategy: 'round-robin',
});
const result = await tavily.post('/search', { query: 'AI infrastructure 2026' });Pluggable selection across a pool, with four built-in strategies and a typed
SelectorStrategy interface for custom logic.
Spread the same workload across N keys and lift the effective rate-limit ceiling by roughly the size of the pool.
Transient errors (408, 425, 429, 500, 502, 503, 504, plus Anthropic 529 and network resets) trigger an instant rotation to the next eligible key.
User-facing failures from a single upstream outage drop to zero as long as one key in the pool is healthy.
Three-state machine (closed, open, half-open) with configurable threshold and cooldown; opens after consecutive failures, recovers on first half-open success.
A failing key stops eating retry budget after a handful of attempts and rejoins the pool on its own once it recovers.
AWS full-jitter exponential backoff with a total time budget, honouring upstream Retry-After
in both numeric-seconds and HTTP-date form.
Predictable tail latency under load instead of synchronised retry storms hammering the same upstream window.
Each key carries a 0-to-100 health score that decays on failure with a configurable half-life and recovers on success.
The pool quietly favours keys that are actually working, without you maintaining a curated whitelist.
Eight in-process events (request.start/success/fail, key.rotated,
circuit.open/closed/half-open, all.exhausted), zero OpenTelemetry dependency.
Drop events into the logger or metrics pipeline you already run; no new agent, no new vendor.
A key that returns 401 Unauthorized is cooled for 24 hours on the assumption that the
credential itself is invalid.
A revoked or rotated key never burns retry budget; you fix it on your schedule, not the request loop's.
Memory backend by default; opt-in file backend persists only the hashed id and operational counters. Raw key value never reaches disk.
Counter durability across restarts without trading away credential hygiene.
Every published version signed with npm publish --provenance through GitHub Actions OIDC.
Lockfile committed; supply-chain policy enforces minimum release age on new dependency versions.
Verify in one command that the tarball you installed was built from the source commit you trust.
Pass strategy: '<name>' in the config, or implement SelectorStrategy for
custom logic.
| Strategy | How it picks | When to use it |
|---|---|---|
round-robin |
Cycles through eligible keys in registration order. | Default baseline. Even distribution across keys with similar quotas. |
weighted |
Pick probability proportional to the weight declared on each
KeyConfig.
|
Some keys have higher quotas than others (paid tier alongside free tier). |
least-used |
Lowest in-flight count; ties broken by total usage then last-used timestamp. | Keys share a quota window. Maximises throughput without overloading any one key. |
sequential-then-rotate |
Always tries the first eligible key in registration order; rotates only when ineligible. | One paid key plus several free fallbacks. Burn the cheap one until it fails. |
cost-aware strategy is planned for 1.1; it requires per-key cost-tier metadata.
| Adapter | Subpath export | Peer dependency | Use it when |
|---|---|---|---|
| OpenAI | @takk/keymesh/openai |
openai >= 4 |
You import openai directly and want rotation without touching call
sites. |
| Anthropic | @takk/keymesh/anthropic |
@anthropic-ai/sdk >= 0.30 |
Same, for the official Anthropic SDK. Recognises HTTP 529 Overloaded as transient. |
| Gemini | @takk/keymesh/gemini |
@google/genai >= 0.7 |
Same, for Google's generative AI SDK. |
| HTTP (generic) | @takk/keymesh/http |
none | Any REST endpoint (Tavily, Serper, Stripe, GitHub API, internal microservice). |
When you cannot or do not want to embed keymesh in your code, run it as a local HTTP proxy. The proxy strips inbound auth headers, dispatches through the same orchestrator the library exposes, and returns the upstream response unchanged.
# With keys in the environment
OPENAI_API_KEYS=key1,key2,key3 npx @takk/keymesh start \
--port 8787 \
--adapter openai \
--strategy round-robincurl http://localhost:8787/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{"model":"gpt-4.1","messages":[{"role":"user","content":"Hi"}]}'
# from another terminal while the proxy is running
curl http://localhost:8787/__keymesh_inspect |
jq
# or via the CLI, against a persisted state file
npx @takk/keymesh inspect --state-file .keymesh-state.jsonlSubscribe directly on the wrapped client. Every event is a typed object; the union
TelemetryEvent is exported so you can branch on event.type with full narrowing in
TypeScript.
client.on('request.start', (e) =>
log.info({ keyId: e.keyId, path: e.path }));
client.on('request.success', (e) =>
metrics.histogram('keymesh.latency',
e.elapsedMs));
client.on('request.fail', (e) => log.warn({ keyId: e.keyId, status: e.status, error: e.error }));
client.on('key.rotated', (e) => log.info({ from: e.from, to: e.to, reason: e.reason }));
client.on('circuit.open', (e) =>
alerts.notify(`Key ${e.keyId} circuit OPEN until
${new Date(e.cooldownUntil).toISOString()}`));
client.on('circuit.closed', (e) =>
log.info({ keyId: e.keyId }));
client.on('circuit.half-open', (e) =>
log.info({ keyId: e.keyId }));
client.on('all.exhausted', (e) =>
alerts.notify(`Pool exhausted:
${e.reason}`));const snapshot = client.inspect();
// {
// strategy: 'least-used',
// totalRequests: 1284,
// totalFailures: 3,
// keys: [
// { id: 'a1b2c3d4', label: 'key-a1b2c3d4',
// circuitState: 'closed', healthScore: 100,
// inFlight: 0, successCount: 642, failureCount: 1,
// consecutiveFailures: 0, cooldownUntil: 0,
// lastUsedAt: 1748449183210 },
// ...
// ]
// }
The id is the first 8 hex characters of the key's SHA-256 hash. The raw key value never appears
in a snapshot, never appears in a telemetry event, never reaches the optional file state backend.
The other tools in this space solve adjacent problems well. The contrast clarifies where keymesh sits.
| Capability | keymesh | LiteLLM | Bifrost | Portkey | Hand-rolled |
|---|---|---|---|---|---|
| Distribution | npm library | pip + sidecar | go binary | SaaS / hosted | your repo |
| Runtime hop | in-process | network hop | network hop | network hop | in-process |
| Runtime dependencies | 0 | many | many | n/a (hosted) | varies |
| TypeScript-first API | yes, strict | partial | no | client typings | varies |
| Embeddable in Edge runtimes | core + HTTP yes | no | no | via fetch only | varies |
| Per-key circuit breaker | yes | partial | yes | yes | rarely |
| SaaS lock-in | none | none | none | yes | none |
| SLSA provenance on releases | yes | partial | partial | n/a | no |
| License | Apache-2.0 | MIT | Apache-2.0 | commercial | your call |
The honest summary: pick LiteLLM if your stack is Python and a sidecar is fine. Pick Bifrost or Portkey if you genuinely want a gateway. Pick keymesh if you write Node.js or TypeScript and prefer a library you embed over a service you operate.
The classifier is deterministic and documented; if a code or condition is not listed here, keymesh treats it as a programmer error or upstream-permanent failure and propagates it untouched.
| Signal | Where | keymesh response |
|---|---|---|
| HTTP 408 Request Timeout | any adapter | retry + rotate |
| HTTP 425 Too Early | any adapter | retry + rotate |
| HTTP 429 Too Many Requests | any adapter, honours Retry-After |
retry + rotate |
| HTTP 500 / 502 / 503 / 504 | any adapter | retry + rotate |
| HTTP 529 Overloaded | Anthropic-specific | retry + rotate |
| HTTP 401 Unauthorized | any adapter | cool the key for 24h |
ECONNRESET / ETIMEDOUT / ECONNREFUSED /
EAI_AGAIN / fetch failed
|
HTTP adapter | retry + rotate |
| Every other 4xx | any adapter | propagate to caller untouched |
| Pool exhausted (every eligible key tried) | orchestrator | throw AllKeysExhaustedError |
| Retry budget exceeded | orchestrator | throw TotalBudgetExceededError |
145 tests passing across 21 suites under Vitest 4. Coverage: 93% lines, 91.6%
statements, 89.1% functions, 79.4% branches. Run pnpm test on a fresh clone to reproduce.
TypeScript 6 in maximum strict mode (exactOptionalPropertyTypes,
useUnknownInCatchVariables, noUncheckedIndexedAccess,
noImplicitOverride, noImplicitReturns). Zero errors under tsc
--noEmit.
Biome 2 clean across src, tests, and examples.
publint clean. Exports map is dual ESM + CJS with separate .d.ts and
.d.cts per subpath.
Validated against the real Gemini API: 429 quota exhaustion triggers rotation; the per-key
circuit completes the full closed -> open -> half-open -> closed lifecycle on real
upstream traffic.
Functional CLI smoke test that spawns the published binary against a fake upstream and asserts end-to-end rotation, headers stripped, response body unchanged.
Committed pnpm lockfile, supply-chain policy with minimum release age, SLSA provenance attestation on
every published version. Verify with npm view @takk/keymesh@1.0.0 --json | jq
.dist.attestations.
cost-aware selection strategyYes. 145 tests across 21 suites pass under Vitest 4 with 93% line coverage; TypeScript 6 maximum strict
mode is clean; Biome 2 lint is clean; publint is clean. Every published release carries SLSA
provenance produced by GitHub Actions. The library has been validated live against the Gemini API: real
429 quota exhaustion triggers rotation, and the per-key circuit breaker completes the full
closed -> open -> half-open -> closed recovery lifecycle on real upstream traffic.
LiteLLM is excellent but Python-first, gateway-style, and requires you to run a sidecar service. keymesh is TypeScript-native, embeddable in any Node, Bun, or Deno process, and runs as either a library or a CLI proxy from the same code path.
Same answer in different words. Those are services; keymesh is a library you pnpm add. No
extra container, no extra hop, no SaaS lock-in. When you do want a proxy, npx @takk/keymesh
start runs the same code path inside a local HTTP server.
The core package and the http adapter run on any modern JavaScript runtime with
fetch. The OpenAI, Anthropic, and Gemini adapters inherit the runtime compatibility of their
respective official SDKs. Edge-optimised adapters are tracked for a later release.
Keys are held in process memory for the lifetime of the client and never transmitted anywhere except to
the upstream you configured. In every telemetry event and in client.inspect(), a key is
identified only by the first 8 hex characters of its SHA-256 hash. The optional file state backend
persists only the hashed id and operational counters; the raw key value never reaches disk.
keymesh throws an AllKeysExhaustedError with the full pool state attached (counts, circuit
states, cooldown timestamps). Catch it at the boundary of your application and decide what to surface to
the user. The library never silently drops the request.
In-process memory by default. For multi-process or restart-survival use cases, opt in to the
file backend by setting state: 'file' in the config. Redis, SQLite, and Postgres
backends are planned for 1.1.
Yes, planned for 1.1. It requires per-key cost-tier metadata and a small economic model; that earns its own release rather than being shoe-horned into 1.0.
Streaming is not wrapped by the rotation layer in 1.0; the wrapper hands streaming calls through to the underlying SDK unchanged. First-class streaming support is on the 1.1 roadmap.
Every release is published with npm publish --provenance. Check the attestations with
npm view @takk/keymesh@<version> --json | jq .dist.attestations. The attestation links
the tarball you installed to the GitHub Actions workflow that built it from a specific source commit.
Yes. Implement SelectorStrategy with a pick(eligible, all) method and pass an
instance as strategy in the config. Anything from "follow a custom priority queue" to
"consult an external service" is two methods of code.
Strict SemVer 2.0.0, starting from 1.0.0. The binding stability surface is documented in SPEC.md section 5. Major bumps require a deprecation cycle; security fixes follow the disclosure flow in SECURITY.md.