Measured,
not promised.
Reproducible, offline measurements of every package — headline claims and runtime cost. No network and no API key anywhere in the suite: model calls use fake, provider-shaped clients.
$ uv run python benchmarks/run_all.py # all tables below $ uv run --with tiktoken python benchmarks/run_all.py # adds exact-token accuracy
Five numbers we stand behind.
Token counting, Decimal pricing, and the event bus every tool rides — measured for accuracy first, speed second.
| Metric | Result | Notes |
|---|---|---|
| Offline heuristic error vs tiktoken — prose | 35.8% | heuristic 163 vs exact 120 tokens |
| Offline heuristic error vs tiktoken — code | 8.4% | heuristic 103 vs exact 95 tokens |
| Offline heuristic error vs tiktoken — json | 18.4% | heuristic 62 vs exact 76 tokens |
| Exact mode error (default) | 0.0% | OpenAI counts are exact out of the box — tiktoken is a required dependency |
| Offline subword fallback vs o200k (Claude/Gemini) | 33.2% | the defensive no-tiktoken fallback; by default Claude/Gemini use o200k directly |
| tokens.count throughput — OpenAI heuristic | 1.13M ops/s | on a 1.4 KB string |
| tokens.count throughput — tiktoken exact | 10.1K ops/s | on a 1.4 KB string |
| instrument() overhead per call | 14.51 µs | bus emit + usage extraction + Decimal pricing; over a no-op client |
| bus dispatch (3 subscribers) | 1.97M emits/s | synchronous fan-out to subscribed tools |
Context assembly under a hard token budget — the claim isn't speed, it's that the budget is never exceeded and the output is deterministic.
| Metric | Result | Notes |
|---|---|---|
| Budget utilization | 100% | used 3500/3500 tokens (reserve 500); never overflows |
| Overflow safety | 0 over budget | 3/25 blocks kept/shrunk, rest dropped by priority |
| Determinism | exact ✓ | identical inputs → byte-identical messages |
| assemble() latency (25 blocks) | 29.31 ms | includes per-block token counting + eviction + ordering |
Reversible payload compression — ratios depend heavily on input shape, which is why the logs row appears twice.
| Metric | Result | Notes |
|---|---|---|
| JSON compression | 48.9% | 90.1 KB → 46.0 KB; 50.1% fewer tokens |
| Logs (repetitive) compression | 99.7% | 70.1 KB → 0.2 KB; 99.8% fewer tokens |
| Logs (mixed-entropy) compression | 30.1% | 80.9 KB → 56.5 KB; 35.9% fewer tokens |
| Code compression | 16.8% | 17.2 KB → 14.3 KB; 14.3% fewer tokens — on representative comment-sparse code; the ratio tracks comment/whitespace density (a comment-heavy file can exceed 50%) |
| Prose compression | 49.1% | 8.6 KB → 4.4 KB; 46.6% fewer tokens |
| Reversibility (expand() == original) | 5/5 exact | every kind restores byte-for-byte |
| compress() throughput (JSON) | 52 MB/s | 90 KB payload, 1.70 ms/call |
Budgets and spend tracking layered on instrument() — the question is what @budget adds to every call.
| Metric | Result | Notes |
|---|---|---|
| Added overhead per call (@budget + track) | 1.64 µs | records spend by tags + checks the active budget(s) |
| report() over 5000 spend rows | 7.39 ms | group-by aggregation into per-tag cost rows |
A deterministic gate at four stages — every built-in is regex/arithmetic, so the story is microseconds and $0. Per-check latency and the per-call overhead the interceptor adds. (measured 2026-07-11, tiktoken installed)
| Metric | Result | Notes |
|---|---|---|
| keyword_deny check latency | 4.58 µs | substring scan of the flattened message text |
| regex_rule check latency | 4.62 µs | one compiled-regex search over the payload |
| url_allowlist check latency | 4.88 µs | extract URLs + host allowlist match |
| length_bounds check latency (chars) | 712 ns | len() of the flattened text |
| length_bounds check latency (tokens) | 21.33 µs | exact token count via cendor.core.tokens (tiktoken) |
| json_schema check latency | 4.39 µs | json.loads + minimal type/required/properties validation |
| apply() 4-rule input gate (pass-through) | 52.3K calls/s | four deterministic checks, nothing trips |
| install() interceptor overhead per call | 26.20 µs | input gate over an instrumented no-op client (bus emit excluded — nothing trips) |
Record once, replay forever — the speedup is measured against a fake client sleeping 4 ms/call, a deliberate floor.
| Metric | Result | Notes |
|---|---|---|
| 25-call run: replayed vs live | 912.03 µs vs 114.18 ms | live = fake client sleeping 4 ms/call (real LLMs are far slower) |
| Replay speedup | 125× | at the modeled 4 ms/call; scales with real latency |
| Replay overhead per call | 36.48 µs | hash the request, look up the recorded response, reconstruct it |
A tamper-evident hash chain with no server — append/verify throughput, signing cost, and that a single edited byte is caught.
| Metric | Result | Notes |
|---|---|---|
| Append throughput (in-memory) | 15.5K entries/s | sha256 chain + default PII redaction per entry |
| HMAC signing overhead | +8% | per-entry HMAC-SHA256 on top of the chain hash |
| Append throughput (file-backed) | 3.5K entries/s | flush + fsync a JSONL line per entry on a kept-open handle |
| verify() throughput | 54.4K entries/s | re-walks a 2001-entry chain in 36.8 ms |
| Tamper detection | ✓ detected | one edited byte → chain hash mismatch → verify() returns False |
How the numbers were made — read before quoting.
Every caveat below is part of the result, not fine print. If a number above looks too good, its honest bound is here.
Every model call is a fake client matching the provider's shape.
The offline heuristic is compared to tiktoken's exact counts; the Claude/Gemini rows are a cross-tokenizer ballpark, not ground truth.
Production LLM calls are 100–1000× slower, so real-world speedup is far larger — the number shown is a floor of the mechanism, not a promise.
The log rows report both a repetition-heavy sample (~55% identical heartbeat lines) and a mixed-entropy one (~15%) — the mixed row is the honest lower bound. The 97% figure elsewhere on the site is the interactive demo's sample (a typical app log), which lands between these two rows.
Re-run locally for your own figures: uv run python benchmarks/run_all.py.
Wiring Cendor into an AI coding assistant? See For AI assistants or the MCP server.