Skip to content
Trazum

1.53.1

The band stays inside the family it was measured in

Fixed#

Claude's band had leaked out of Claude — 1.54's second chapter, and three defects with one root. ±10% is the estimator's error measured against Claude's tokenizer over twenty-one samples, and --exact-tokens counts with Anthropic's endpoint. Both facts were true, neither was enforced, and three claims escaped the family they were measured in.

optimize --exact-tokens handed the user's own model id to Anthropic. countTokensAnthropic({ apiKey, model: result.usage.model }) — correct on a Claude model, and on gpt-5 either a confusing upstream error or a number counted with the wrong tokenizer and labelled exact, which is the strongest word this tool uses about a count. It refuses by name now, naming the model and its family and pointing at that provider's own tooling. The family check runs before the key check, because telling somebody on another family to go find an ANTHROPIC_API_KEY sends them after a credential that could not have helped them.

The context-overflow advisory said "The call will fail" to every family. The most absolute sentence this product produces, with no dollar figure to soften it — and the margin behind it was Claude's. On a family nobody has measured, an estimate over the window is always uncertain however far over it looks, because the margin that would settle it is the unknown. The fix is not a second, wider band: inventing a number would be the same mistake with worse arithmetic.

And both context advisories sent every reader to --exact-tokens. "Settle it with --exact-tokens; the counting endpoint is free" — pointing most of the seven priced families at a counter for a different tokenizer, and, since this release, at a command that refuses them. The advice is bounded rather than deleted: the family it works for still gets it.

BAND_CALIBRATED_PROVIDER and bandGoverns() are exported so the fact has a name. A missing provider reads as not covered, never as covered — the flattering reading of missing information is the one this project does not take.

Two things this found in itself. The provider-less refusal said the model "is not a model in the price catalogue at all", which is false in the only case that reaches it: provider is optional on a priced model and --pricing lets anybody supply one, so the model is in the catalogue and its family is what is missing. Found by running the branch rather than reading it. And the new guard's first draft asked for a prompt 1.01x a 1,000,000-token window but grew the text by doubling, producing 1,966,080 tokens — landing in the certain branch and failing a correct implementation over a badly built input.

threshold-honesty.test.js legitimately failed nine models, because it recognises hedging by vocabulary and the new sentences hedge in words its list did not know. may be became \bmay\b, plus "not knowable" and "cannot be said". Widening a guard's accept-list to make a change pass is how a guard becomes decoration, so the pattern is now handed the flat sentences it exists to refuse and must reject all three.

Added#

The band harness measures four families, not two — 1.54's first chapter. Trazum prices seven providers with an estimator calibrated on one, and ROADMAP.md has called the per-family error "the one number that settles" whether to take a real tokenizer as a dependency. Nobody had measured it, and until 1.53 nobody honestly could: the harness can only reach an endpoint this repository already trusts with a credential.

The 1.53 arc made two more of them facts. OpenAI measures through https://api.openai.com/v1/chat/completions and Google through https://generativelanguage.googleapis.com/v1beta/models/{model}:generateContent — both the gateway's own upstreams, both reading a count this repository already knows how to read. Google deliberately does not use a counting endpoint. One may well exist; nobody here has ever sent a key to it, and an endpoint recalled rather than committed is exactly what the previous arc spent itself refusing. So it measures with :generateContent at one output token and pays for it, rather than guessing and saving pennies.

Neither has been run against the real service. Nothing in this environment has a key. What ships is the shape; the numbers arrive when somebody runs it.

The measuring script is tied to the gateway's allowlist. It sends a real API key to every family it measures — the same act the gateway performs, from a file nobody thinks of as security-sensitive, and in fact the file where DeepSeek's endpoint sat while the gateway was still refusing DeepSeek as unsupported. Its origins must now be ones the gateway forwards to, so adding a family requires the same deliberate edit to security.test.js that adding an upstream does. A measuring script is not a side door.

And exactly one family may claim the published band, counted from the harness source rather than from the fixtures — which do not exist on a clean checkout, the same absence that once let this suite report "0 failures" for a directory that was not there.

Fixed#

"Nothing else has been measured" told a reader on GPT nothing about their own figures. The suite emitted one skip naming deepseek, because that was the only other provider the day the sentence was written — so two families arriving at 1.53 would have gone unmentioned by a message bounded by what happened to exist when it was typed. Each measurable family now gets its own named skip carrying its own command, derived from the harness, and drops out of the list the moment its fixture appears. The twelfth time this project has bounded a message by its neighbour rather than by its subject.

A check that could never have fired. The new "the harness reaches nothing the gateway does not front" assertion would never have run against anything it should reject: a brand-new host fails the decision check above it first. The case it actually exists for is subtler — a host already decided about, and decided to be something other than an upstream, which would satisfy every other check in the file. It is handed exactly that now. Third occurrence this session of a filter over today's correct values proving nothing.