Your OpenAI bill is a black box. Here is the seam that opens it

Here is a thing that happened while writing this post.

A recipe in the Cendor cookbook governs a Microsoft 365 agent: budgets, guardrails, a tamper-evident audit chain, a streamed reply with a mid-stream budget breaker. It had been green in CI for weeks. Its README says, in bold, point the client at instrument(AsyncOpenAI()) and this file is unchanged.

So I did that, with a real key. The first streamed turn died:

IndexError: list index out of range
    if piece := (chunk.choices[0].delta.content or ""):

stream_options={"include_usage": True} — which is how a streamed call reports real usage at all — makes OpenAI send a final chunk with an empty choices list, carrying only the usage. Measured against openai-python 2.48.0: 9 chunks with include_usage, the ninth choices=[]; 8 chunks without it, none empty. A fake stream never emits that chunk. So the offline test was green forever and the documented production swap was broken on its very first streamed turn.

That is the shape of the problem this whole layer exists for. Not “AI is hard” — something much more boring: the difference between what your tests see and what the provider actually sends you, and the fact that nothing tells you. Cost is the same class of problem. Your OpenAI bill arrives at the end of the month as one number, and nothing in your code knows which feature earned it.

What you’ll build

One wrap, four properties: exact token counts, a Decimal cost, a budget that refuses an over-budget call before it is sent, and a redaction that rewrites the prompt before it leaves your process. Then a test suite that replays the whole thing offline for free.

flowchart LR
  A["your code<br/>OpenAI()"] --> B["instrument()<br/>one wrap"]
  B --> C["tokenguard<br/>cap, pre-flight"]
  C --> D["guardrails<br/>gate + redact"]
  D --> E["the provider"]
  D --> F["cassette<br/>replay for $0"]
  D --> G["acttrace<br/>hash-chained evidence"]

Five steps follow, in that order. Each one is a few lines of code, a command you can run right now, and the output it actually prints.

Step 0 — where to start

You need Python 3.11+ or Node 20+. You do not need an API key: every recipe below drives a fake client with the real OpenAI response shape, because instrument() identifies a client by its structure, not by its class name. The fake and the real OpenAI() are recognised identically, so the code you read is the code you would ship.

git clone https://github.com/cendorhq/cendor-cookbook && cd cendor-cookbook && uv sync
uv run python recipes/providers/openai-chat/main.py

git clone https://github.com/cendorhq/cendor-cookbook-js && cd cendor-cookbook-js
cd recipes/providers/openai-chat && npm install && node index.mjs

To wire Cendor into a repo you already have, uvx cendor-init / npx @cendor/init writes the assistant rules file and scaffolds a correct instrument() call. It is offline and idempotent, and no library depends on it.

Step 1 — one wrap

from cendor.core import instrument
from openai import OpenAI

client = instrument(OpenAI())     # the one line that changes

import { instrument } from '@cendor/core';
import OpenAI from 'openai';

const client = instrument(new OpenAI());   // the one line that changes

instrument() wraps the client in place and emits a normalized LLMCall on a shared in-process bus: provider, model, usage, a Decimal cost with an honest label saying whether the provider reported it or Cendor estimated it, and which token-counting method it used. It monkey-patches nothing you own, and it does not import any of the other libraries — they subscribe to the bus.

Run the core quickstart and that is what you get:

LLMCall on the bus:
  provider : openai
  model    : gpt-4o
  usage    : 8 in + 9 out = 17 tokens
  cost     : $0.000110000 (cost_estimated)
  tokens   : counted via 'exact' for gpt-4o

exact, not len(text) / 4. tiktoken is a required dependency, not an extra, because a token count that is approximately right is a cost number that is wrong — that is where the 0% token-count error in the benchmark suite comes from. The overhead for all of this is ~15 µs per call, which is four orders of magnitude below the network round-trip you are already paying.

Step 2 — a cap that actually refuses

The cheapest possible LLM call is the one that never happens. tokenguard’s budget is pre-flight: it prices the request from the assembled prompt plus the output reserve and compares that to what is left, before anything is sent.

from cendor.tokenguard import BudgetExceeded, budget, report, track

@budget(usd=0.50, on_exceed="block", output_reserve=6_000)
def support_bot(client) -> None:
    for _ in range(50):                        # a loop that would happily run forever
        with track(feature="support_bot", user_id="user-42"):
            client.chat.completions.create(model="gpt-4o", messages=[...])

import { BudgetExceeded, budget, report, track } from '@cendor/tokenguard';

// budget(cfg) returns a wrapper — `budget(cfg, fn)` is a compile error on purpose.
const supportBot = budget({ usd: 0.5, onExceed: 'block', outputReserve: 6_000 })(
  async (client) => {
    for (let i = 0; i < 50; i++) {
      await track({ feature: 'support_bot', user_id: 'user-42' }, () =>
        client.chat.completions.create({ model: 'gpt-4o', messages: [/* … */] }),
      );
    }
  },
);

Fifty turns of a context-stuffed support bot, capped at fifty cents:

gate      : BLOCKED by keyword_deny (input) - denied keyword: 'ignore previous instructions'
            provider saw 0 call(s) => $0 spent on it
budget    : BudgetExceeded - blocked pre-flight, no call ran
spend     : by feature/user
            {'feature': 'support_bot', 'user_id': 'user-42'} 5 calls  $0.450000000
            TOTAL 5 calls  $0.450000000

Five turns ran. The sixth was refused, and the model never saw it — $0.00 spent on the refusal. And because track() was in scope, the spend is already attributed by feature and by user, which is the report your finance conversation actually needs.

Those 12,000-in / 6,000-out figures are the fixture’s, chosen so the block lands on the sixth call in a recipe you can run in two seconds. A real gpt-4o reply is nearer 50 output tokens, so the same cap survives far longer against a live key. The mechanism is the point, not the arithmetic.

An honest note on block. It is pre-flight, so it works from an estimate, and an estimate can be under while the real completion is over. That is a real edge, not a hidden one: since cendor-tokenguard 1.6.2 the cumulative post-flight check says so in as many words, names the cap in the dimension you set it in, and points at output_reserve or on_exceed="clamp" — which caps the call server-side instead of guessing.

Where that projection’s rate comes from

A pre-flight cap is arithmetic on a per-token rate, so the rate is the part worth being able to defend. prices.explain(model) reports it:

from cendor.core import prices

print(prices.explain("gpt-4o").summary())
# gpt-4o: cached=0.00000125 input=0.0000025 output=0.00001 — exact, from azure as of 2026-07-01

import { prices } from '@cendor/core';

console.log(prices.explain('gpt-4o').summary());
// gpt-4o: cached=0.00000125 input=0.0000025 output=0.00001 — exact, from azure as of 2026-07-01

from azure as of 2026-07-01 on an OpenAI model is not a mistake — OpenAI publishes no pricing API, and neither does Anthropic. Their model-list endpoints carry ids only. So the rates are reconciled from the sources that do publish machine-readable prices: Microsoft’s and Amazon’s own billing catalogs plus the MIT aggregators, per row, each with the date that source published it.

That table ships generated, not hand-typed — the cendor-prices feed (opens in a new tab) is a static file on GitHub’s CDN with no account and no server behind it, so there is no Cendor outage that can break your cost estimation. A bare prices.refresh() pulls a newer one, in memory, for this process only; there is deliberately no implicit cache, because a hidden cache is exactly how prices go invisibly stale. And if you cap in dollars against a table older than 45 days, tokenguard says so once per process: after a price cut a stale table over-estimates and the cap binds early, which is merely conservative — after a price rise it under-estimates and the cap binds late, so you overspend.

Hand-typing was not just tedious. The previous snapshot’s 44 hand-fed rows had drifted against every other source, one of them by 5×. Generation is why that class of error is gone.

Step 3 — a gate that fires before the request leaves

Blocking a bad prompt after you have paid for it is not much of a control. guardrails gates at the seam, so a decision happens before the HTTP request:

from cendor.guardrails import install, rules

install([
    rules.keyword_deny(["ignore previous instructions"], action="block"),
    rules.regex_rule(r"\bsk-[A-Za-z0-9]{16,}\b", action="redact", stage="input"),
])

import { install, rules } from '@cendor/guardrails';

install([
  rules.keywordDeny(['ignore previous instructions'], { action: 'block' }),
  rules.regexRule(/\bsk-[A-Za-z0-9]{16,}\b/, { action: 'redact', stage: 'input' }),
]);

Driven against the real OpenAI API, with the recipe printing exactly what the provider received:

BLOCKED by keyword_deny (input): denied keyword: 'ignore previous instructions'
  provider calls so far: 0  =>  $0.00 spent on it

REDACTED before send: provider received 'my key is [redacted]'

guardrail_decision entries in the audit chain:
  block  input  keyword_deny
  redact input  regex_rule

chain verifies: True

The second line is the one worth staring at. The leaked key never left the process. And both decisions are in a hash-chained acttrace file that verify() re-walks offline — edit one byte of it and verification fails at the exact entry.

See it happen, without installing anything

The same libraries run in your browser at cendor.ai/try — real tokenguard arithmetic, a real guardrails gate, a real acttrace chain, compiled to WebAssembly. No key, no signup. Send the injection above and the gate refuses it pre-flight:

The Gate panel on the left and the Audit panel on the right are reading the same event bus your own process would. Nothing was sent to a model, and the refusal is already chained as evidence.

Step 4 — reasoning and cached tokens, without a second accounting path

New OpenAI apps are on responses.create, which reports usage differently: input_tokens / output_tokens, cached tokens nested under input_tokens_details.cached_tokens, reasoning under output_tokens_details.reasoning_tokens. Three rates, one response.

You do not write a second code path for it:

client.responses.create(model="gpt-4o", input="Reason briefly, then greet me.")

await client.responses.create({ model: 'gpt-4o', input: 'Reason briefly, then greet me.' });

usage: 1,204 in (200 cached) -> 850 out (620 reasoning) · cost $0.011260000 (cost_estimated)

The reasoning tokens are the ones that surprise people. They are output tokens you are billed for and never see, and on a reasoning model they can consume the entire output allowance — a 48-token cap on one such deployment returned 37 in / 48 out with an empty answer. All of the accounting was correct; there was simply no text. Better to see that number than to discover it on an invoice.

One cross-language wrinkle worth knowing, because it is the kind of thing that silently doubles your numbers: in Python, responses.parse posts its own request and needs its own instrumentation target. In TypeScript it is a helper built on create, so making it a target would count the same call twice. Both ports do the right thing; you only notice if you go looking.

Step 5 — replay the whole thing for free

The last property is the one that changes how you work. cassette records the exchange once and replays it forever:

uv run pytest recipes/testing/pytest-cassette -n auto     # parallel, zero API calls

cd recipes/testing/vitest-cassette && npm install && npx vitest run

Driven live, then replayed:

record leg : 1 real provider call
replay leg : 0 calls, byte-identical output
strict replay on an unrecorded call: CassetteError — drift cannot pass silently

mode="replay" is strict on purpose. If your agent starts making a call the cassette does not know about, the test fails rather than quietly reaching the network.

Watch it while you build

Everything above is in-process and needs no server. But while you are working it helps to see the calls, and Cendor emits standard OpenTelemetry, so any OTLP backend will do. If you would rather not wire one up to watch a loop you are debugging, one docker run gives you Cendor Monitor — self-hosted, optional, never in the data path:

docker run --rm -p 3000:3000 -p 4318:4318 -v cendor-monitor-data:/data \
  ghcr.io/cendorhq/cendor-monitor:0.15.0
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318

⚠️ The environment variable on its own is not enough, and the failure is silent. Cendor’s telemetry is mode=auto: it attaches once your app has configured a global OpenTelemetry provider, and emits nothing before that. Measured — with only OTEL_EXPORTER_OTLP_ENDPOINT set, the recipe runs green, prints every number, and the console stays at 0 runs. Add the standard five lines of OTel bootstrap and the same run lands 7 calls and 3 blocks. There is no Cendor-specific telemetry code in either case.

CENDOR_DEBUG_TELEMETRY=1 tells you which state you are in, in one line: armed (mode=auto); waiting for a provider versus provider=detected, emitter=attached.

from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter

provider = TracerProvider()
provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter()))
trace.set_tracer_provider(provider)      # the ONE global setup — your app owns it

import { NodeSDK } from '@opentelemetry/sdk-node';
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-http';

new NodeSDK({ traceExporter: new OTLPTraceExporter() }).start();  // the ONE global setup

Then run the two recipes from this post. Their governed calls arrive in the Libraries door:

The first tile is the one I find myself looking at: $0.5401075 of projected spend that a budget refused pre-flight — money the caps stopped rather than money spent. Alongside it, 2 guardrail blocks and 2 replayed calls that cost nothing. Every call is also listed individually, priced:

Two things to be clear about. The container is optional dev tooling — no library depends on it, your production default stays your own backend, and Cendor never operates a telemetry endpoint. And what you see here is an operational copy: the hash-chained audit file on your own host is the only verifiable evidence, and verify() runs on that file, never on this screen.

It rides under your framework, not instead of it

None of this asks you to adopt an agent framework, or to leave the one you have. The seam is the client, so whatever drives the loop above it is untouched. A LangChain chain, run live on gpt-4o with the chain code unchanged:

LangChain answer : 'Refunds typically take 5-10 business days to process...'
tokenguard spend : $0.000310000 over 1 model call(s) (budget $0.000310000 of $0.10)
acttrace entries : 5 (chain wrote spend + audit, code unchanged)

And a governed agent on cendor-sdk — the second door, built on the same seven libraries — one live tool-calling turn on gpt-4o-mini:

output      : The weather in Paris is sunny.
cost        : 0.0000334500 USD  (budget $0.25, enforced pre-flight)
usage       : 135 in / 22 out
tools called: ['get_weather']
audit chain : True — ok: 7 entries, head e0a31a757ad4…

Three and a third millionths of a dollar, known to the cent-of-a-cent, with a verifiable record of what the agent did. That is the number your cap can bind to. (The entry count and the verify() result reproduce on every run; the chain head does not — entries carry timestamps, so your hash will differ from mine. That is the chain working, not drifting.)

Honest limits

• A second, un-instrumented client is invisible. Budgets, gates and evidence only see calls through the client you wrapped. uvx cendor-init doctor static-checks for exactly that.
• on_exceed="block" is pre-flight, so a call whose estimate fits and whose real completion does not can still land over the cap. Reserve more output, or use clamp to cap it server-side.
• acttrace produces evidence to support a compliance case. It is not a compliance guarantee and it is not legal advice.
• A guardrail is only as good as its rules. A keyword denylist catches the phrases it lists and misses the ones it does not — the cookbook’s red-team recipe deliberately scores 50% on its own corpus, because a demo that scores 100% has been overfitted to its test set.
• The screenshots above are one seeded session on one machine. Trace ids, timestamps and latency figures will differ on yours; the tile labels and the mechanism are what to compare against.
• Not every model id is priced. An Azure deployment name is a string you invented, and a Bedrock marketplace id prices only when the model inside it has a row — so a USD budget silently cannot bind to those, while a token budget still can. One line fixes it, and which line depends on what you know: prices.register_deployment(dep, like="gpt-4o") when you know the model your deployment serves, prices.register_model_price(id, input=…, output=…) when you hold the exact rates. That is the whole subject of the next post.

Run it yourself

• Python: providers/openai-chat (opens in a new tab) · providers/openai-responses (opens in a new tab) · testing/pytest-cassette (opens in a new tab)
• TypeScript: providers/openai-chat (opens in a new tab) · providers/openai-responses (opens in a new tab) · testing/vitest-cassette (opens in a new tab)
• In your browser, nothing to install: cendor.ai/try
• All the recipes: cendor.ai/cookbook — every one runs offline, no key.
• Docs: cendor.ai/docs · the numbers: cendor.ai/benchmarks

And the bug at the top is fixed, in both language ports, with the measurement in the commit message.