A minor rather than a patch, because it changes every report. The published
error band drops from ±15% to ±10%, which moves the number printed beside every
token count Trazum has ever shown, and a fourteenth finding joins the list.
The band moved because kana and han do not cost the same. Charging one token per CJK character put Japanese at +11.2%, the worst error in the corpus, while Chinese sat at -3.2% under the identical rule. No new measurements were needed — the finding was in the twenty-one samples already committed.
Then three advisories turned out to share one fault: an estimate with a ±10% band compared against an absolute threshold, and the answer stated as a fact. One offered $48.67 a month that could not be collected. One promised money on a prefix that might not clear the cache minimum. One said "the call will fail" as a certainty, and said nothing at all in the case where a prompt that seemed to fit really might not. All three are fixed, and a guard derived from the pricing catalogue now covers the pattern across eighteen models so it cannot ship a fourth time.
1.9.1 was prepared and never published. Its tag failed three times against a trusted-publisher configuration that npm kept refusing, and everything in it is contained here. This release supersedes it.
Added#
A guard so the threshold fault cannot be shipped a fourth time. The same
mistake was found and fixed three times this release — an estimate with a ±10% band
compared against an absolute threshold, and the answer stated as a fact.
threshold-honesty.test.js asserts the property instead of the three instances.
It is derived from the pricing catalogue, not from a list of thresholds typed into a test: eighteen models, four distinct cacheable minimums, six distinct context windows, and a model added later with a new window is covered without anybody remembering to. Weak about wording, strong about presence — it does not care what the caveat says, only that a report facing a line its own error band straddles admits it somewhere. Pinning the phrasing would make it a copy test.
Silence is a failure, not a skip, and getting that wrong is how the first
version missed the bug it was written for. It skipped when no relevant finding
existed — which is precisely the quiet failure mode — so deleting context-near-limit
left it green. The one legitimate silence is a threshold the model does not have.
Coverage stated exactly in the file: reintroducing faults 2 and 3 fails it, both
halves of 3 included. Fault 1 does not, because its property is different — "offers
a saving the tool would refuse to deliver" rather than "admits uncertainty" — and
that one is guarded by the advice-matches-action sweep in cache-minimum.test.js.
Two properties, two tests, said out loud rather than implied.
The first version also fed the cache advisories a bare token count with a placeholder prompt, and those advisories reason about the stable prefix of the real text. A two-token prompt labelled 486 tokens is nowhere near any minimum, so it reported eighteen failures against correct code — a test measuring its own fixture.
Added#
context-near-limit, for the prompt that fits by estimate and might not fit at
all. The third place a ±10% number was compared against a hard threshold and the
answer stated as fact, after cache-prefix-reorder and prompt-caching. This one
carries no dollar figure and was the most absolute of the three: "The call will
fail."
It was wrong in both directions. An estimated 205,000 tokens against a 200,000 window can truly be 184,500 — the call succeeds, and the reader was sent to split a prompt that fitted. An estimated 199,000 can truly be 218,900, which does not fit, and nothing warned at all.
The silent direction is the worse one and it is the new advisory. A prompt over the window fails outright rather than degrading, so there is no partial result to notice and no other finding covers it.
Fixed#
context-overflow no longer states a prediction as a fact. Barely over the
line it says the call will probably fail and names --exact-tokens; far over, it
still says the call will fail, because it does. Hedging there would be its own
dishonesty.
Neither fires on a number the caller measured. An exact count near the edge is not uncertain, and telling somebody their measurement might be wrong pushes them toward a check they have already done — the same rule the other two advisories follow.
Three advisories, one fault, found by asking twice whether it had a twin. The pattern is worth naming because it will recur: any comparison of an estimate against a hard threshold has two failure modes, and the quiet one is usually worse than the loud one. The sweep tests now cover the seam around each threshold rather than sampling either side of it.
Adding the id made the typed union fail the web app's catalogues and a derived guard fail the README's count of findings. Both are the mechanisms working.
Fixed#
prompt-caching hedged in one direction and promised money in the other. Found
by asking whether the bug just fixed in cache-prefix-reorder had a twin. It did.
below-cache-minimum hedges when an estimated prefix lands just under the
threshold — the real one may already be over, and withholding the largest saving
Trazum offers on the strength of a ±10% figure is wrong advice. prompt-caching
did not hedge when an estimate landed just over it. With a ±10% band an
estimated 528-token prefix can truly be 475, and then nothing caches at all and the
dollar figure printed beside the advisory is uncollectable.
Same fault as the reorder advisory, opposite direction, and this is the direction
with money attached. The hedge qualifies the figure rather than withdrawing it —
the prefix probably does clear the line — and names --exact-tokens, which settles
it for free.
Only on an estimate. A caller who supplied their own counter has an authoritative prefix, and telling them it might be wrong pushes them toward a check they have already done.
Added#
A test that no size around the threshold makes an unqualified claim. Whichever advisory fires, if the band around the estimated prefix straddles the minimum, the text has to say so. Swept across the window rather than asserted on two samples, because the fault was an asymmetry between two code paths and only a sweep can show the seam between them is closed.
It reads the prefix from analyzeCachePrefix rather than scraping ~528 out of the
sentence. The first version did scrape it, defaulted to "straddles" when the regex
missed, and reported two failures against correct behaviour — a test asserting its
own parsing instead of the property.
Fixed#
cache-prefix-reorder was offering money that could not be collected. It fired
whenever enough stable content sat after the first placeholder and priced moving it
forward at 90% off — without asking whether the prefix that rearrangement would
build actually clears the model's cacheable minimum.
On a 306-token support prompt against Claude Opus 5's 512-token minimum, the best
prefix any ordering can produce is 302. Nothing caches at any ordering. The
advisory offered $48.67 a month, in the same report as below-cache-minimum
telling the reader caching would not work here at all — two findings contradicting
each other, with the dollar sign winning the argument.
reorderForCache had refused these prompts from the start, for precisely this
reason, so Trazum's advice and Trazum's action disagreed: take the advice, run
--reorder, watch nothing happen. A money figure in the flattering direction is
the one fault this file exists to catch.
The gate is a strict comparison against the minimum. No band hedge, and that was
tried first — widening it by ±10%, on the reasoning that makes
below-cache-minimum hedge near the line, opened a window between 466 and 512
tokens where the advisory offered and the command refused. The same fault one layer
up, and the test caught it. The near-the-line case is already handled in the right
place: below-cache-minimum names --exact-tokens, and once the number is certain
both work from the same certainty.
Changed#
The advisory names the command that does it. It described the rearrangement in
prose and left the reader to perform it by hand, while reorderForCache sat in the
same package able to attempt it — whole blocks only, refusing any block that refers
back to earlier text, and everything after one. It now prints
trazum optimize <file> --reorder and still says to read the diff, because this is
the one transformation that moves text rather than deleting it.
An advisory that withholds the command is the shape of the whole product problem: Trazum knowing something worth more than what it does about it. On a 1,355-token prompt the command takes the cacheable prefix from 13 tokens to 1,350.
Added#
A test that the advice and the action cannot disagree. For every size in a
sweep: if the report offers the saving, reorderForCache must deliver movement.
One-directional on purpose, and the direction took two attempts to get right. The
first version asserted that cache-prefix-reorder and below-cache-minimum never
appear together — wrong about the product, not the code: on a prompt with plenty of
movable content both are true and both useful, a diagnosis followed by its fix. The
second asserted the converse as well, and failed against working code because below
200 movable tokens the advisory stays deliberately quiet while the command remains
available. Silence about a small win is not the same fault as a promise about an
impossible one.
Changed#
The published error band drops from ±15% to ±10%, and the worst measured error from 11.2% to 6.4%. No new API calls were needed for this — the finding was sitting in the twenty-one measurements already committed.
Kana and han do not cost the same. Every CJK character was charged one token, and measured against the counting endpoint that put Japanese at +11.2% — the worst figure anywhere in the corpus — while Chinese sat at −3.2% under the identical rule. One constant cannot be right for both, and the samples say why: the Japanese file is 58% kana and the Chinese one is 0%.
Kana are a small syllabary that appears in every sentence, so a merge table covers runs of them and several characters share a token. Han are tens of thousands of rare characters a merge table cannot cover, and they cost about one each. Measured: kana 0.75 tokens per character, han 1.05.
cjk-japanese +11.2% → -1.5%
cjk-chinese -3.2% → +1.3%
worst in corpus 11.2% → 6.4% (code-heavy, which nothing is fitted to)The signal needs no detector, which is what separates this from language.ts.
A character is kana or it is not; the two samples separate perfectly at 58.3%
against 0.00%. No refusal case, no margin rule, nothing to get wrong on a
three-line prompt.
Rounding up per run was the first attempt and it was wrong by five points.
Ordinary Japanese alternates kana and han inside every sentence, so the runs are
short and numerous, and a Math.ceil per run charges most of a token for each
boundary — an artefact of where the loop breaks rather than of what the text costs.
CJK accumulates as a fraction now and is rounded once, over the whole document. A
test builds the same characters blocked and alternating and requires the two
estimates to agree within one token.
The band is 10 rather than 7, and the margin is deliberate. 6.4 rounded up is 7, and publishing 7 would be a tighter claim than twenty-one samples across six text types can support: there is no Korean here, no Cyrillic prose, no mixed-script document, and a seventh type could easily land at eight. A band that becomes false the first time somebody measures something new is the exact fault this whole exercise was fixing, so the uncertainty is overstated rather than understated.
Added#
Accuracy is ratcheted per text type, separately from the published band. The
band is deliberately loose and that has a cost: a change taking CJK from 1.5% back
to 3.6% passes every band assertion, because both are inside ten. Found while
mutation-testing this very change — setting HAN_TOKENS_PER_CHAR back to a round 1
doubles the CJK error and nothing failed.
Each type now carries a floor set to what it has actually reached. They tighten, never slacken: an improvement lowers its floor in the same commit, and a deliberate trade raises it with a changelog line, which is a different act from not noticing. A type added to the corpus without a floor fails the suite rather than going ungated.
Same idea as trazum baseline turned on this repository's own numbers — publish a
ceiling, gate on drift away from what you had.
RELEASES.md and ROADMAP.md join CHANGELOG.md as records exempt from the
band-consistency guard. The guard requires every file stating a band to match the
code, which is right for a README and wrong for a release note: "the band is still
±15%" is a true statement about 1.9.0, and rewriting it to say 10 would be
falsifying a record to satisfy a test. The other twenty files it flagged were live
claims and were updated.
Fixed#
The preflight told the reader to disbelieve it, and it was right. The caveat closed with "believe your settings over this check", on the reasoning that the exchange endpoint is undocumented and a refusal could be the request being wrong.
Then it was tested. v1.9.1 was tagged against settings that had just been filled
in on all three packages; the check said rejected; the publish failed with the
same E404 it predicted, twice. The check has been right in the only case that
has ever tested it, and the caveat argued the reader out of a true finding — which
is worse than no caveat.
It still says the endpoint is undocumented, because that is true and matters. It now also says it has been right once, and asks to be believed until it is not.
And the diagnosis was unreachable for a third time. The auth check runs before
verify, which then prints thousands of lines, and GitHub's logs API returns the
tail of a job — so the block naming what npm must match could not be retrieved
while a release was actually failing, twice during v1.9.1. Writing it to the job
summary fixed that for anyone on the run page and not for anyone reading the log,
which is where a failure gets read.
The failure step repeats it now, at the end, where the tail always reaches. Four lines, and it is the difference between an error and a diagnosis.
Changed#
The report was leading with its smallest number. Measured rather than
assumed: on an ordinary customer-support prompt — already reasonably written,
which is what a real one is — the rules recover three tokens of 306, worth
$0.75 a month. The advisories listed below them on the same prompt are worth
$345 and $48. The report opened with -1.0% and closed with the comparison.
Start here: is the first thing in the report now:
Start here:
"This task may not need Claude Opus 5" — $345.45/month, 461× what the
rules saved.
Input tokens
306 → 303 -1.0% (estimated, ±15%)That ordering was not a presentation quibble. It taught the reader that shortening the prompt is what this tool is for, and on any prompt somebody competent wrote, shortening it is the smallest thing available. The rules earn their keep on genuine bloat — a duplicated paragraph, "due to the fact that", where they measure -23.5% — and recover close to nothing once it is gone, because they recover waste rather than creating savings. Both figures are from the same build, minutes apart, on two prompts.
Nothing about what Trazum computes changed. It already knew the advisory was worth 461 times more and said so at the bottom of the screen.
One thing got simpler on the way. The line's guard was duplicated — an early return and a filter, both keyed on the same condition — which made it untestable: removing the guard left the filter still suppressing the line, so a mutation that priced a flat plan passed. Two checks for one condition is one check and one place for a bug.