Skip to content
Trazum

1.10.0

Every hard edge, both sides

A minor rather than a patch, because it changes every report. The published error band drops from ±15% to ±10%, which moves the number printed beside every token count Trazum has ever shown, and a fourteenth finding joins the list.

The band moved because kana and han do not cost the same. Charging one token per CJK character put Japanese at +11.2%, the worst error in the corpus, while Chinese sat at -3.2% under the identical rule. No new measurements were needed — the finding was in the twenty-one samples already committed.

Then three advisories turned out to share one fault: an estimate with a ±10% band compared against an absolute threshold, and the answer stated as a fact. One offered $48.67 a month that could not be collected. One promised money on a prefix that might not clear the cache minimum. One said "the call will fail" as a certainty, and said nothing at all in the case where a prompt that seemed to fit really might not. All three are fixed, and a guard derived from the pricing catalogue now covers the pattern across eighteen models so it cannot ship a fourth time.

1.9.1 was prepared and never published. Its tag failed three times against a trusted-publisher configuration that npm kept refusing, and everything in it is contained here. This release supersedes it.

Added#

A guard so the threshold fault cannot be shipped a fourth time. The same mistake was found and fixed three times this release — an estimate with a ±10% band compared against an absolute threshold, and the answer stated as a fact. threshold-honesty.test.js asserts the property instead of the three instances.

It is derived from the pricing catalogue, not from a list of thresholds typed into a test: eighteen models, four distinct cacheable minimums, six distinct context windows, and a model added later with a new window is covered without anybody remembering to. Weak about wording, strong about presence — it does not care what the caveat says, only that a report facing a line its own error band straddles admits it somewhere. Pinning the phrasing would make it a copy test.

Silence is a failure, not a skip, and getting that wrong is how the first version missed the bug it was written for. It skipped when no relevant finding existed — which is precisely the quiet failure mode — so deleting context-near-limit left it green. The one legitimate silence is a threshold the model does not have.

Coverage stated exactly in the file: reintroducing faults 2 and 3 fails it, both halves of 3 included. Fault 1 does not, because its property is different — "offers a saving the tool would refuse to deliver" rather than "admits uncertainty" — and that one is guarded by the advice-matches-action sweep in cache-minimum.test.js. Two properties, two tests, said out loud rather than implied.

The first version also fed the cache advisories a bare token count with a placeholder prompt, and those advisories reason about the stable prefix of the real text. A two-token prompt labelled 486 tokens is nowhere near any minimum, so it reported eighteen failures against correct code — a test measuring its own fixture.

Added#

context-near-limit, for the prompt that fits by estimate and might not fit at all. The third place a ±10% number was compared against a hard threshold and the answer stated as fact, after cache-prefix-reorder and prompt-caching. This one carries no dollar figure and was the most absolute of the three: "The call will fail."

It was wrong in both directions. An estimated 205,000 tokens against a 200,000 window can truly be 184,500 — the call succeeds, and the reader was sent to split a prompt that fitted. An estimated 199,000 can truly be 218,900, which does not fit, and nothing warned at all.

The silent direction is the worse one and it is the new advisory. A prompt over the window fails outright rather than degrading, so there is no partial result to notice and no other finding covers it.

Fixed#

context-overflow no longer states a prediction as a fact. Barely over the line it says the call will probably fail and names --exact-tokens; far over, it still says the call will fail, because it does. Hedging there would be its own dishonesty.

Neither fires on a number the caller measured. An exact count near the edge is not uncertain, and telling somebody their measurement might be wrong pushes them toward a check they have already done — the same rule the other two advisories follow.

Three advisories, one fault, found by asking twice whether it had a twin. The pattern is worth naming because it will recur: any comparison of an estimate against a hard threshold has two failure modes, and the quiet one is usually worse than the loud one. The sweep tests now cover the seam around each threshold rather than sampling either side of it.

Adding the id made the typed union fail the web app's catalogues and a derived guard fail the README's count of findings. Both are the mechanisms working.

Fixed#

prompt-caching hedged in one direction and promised money in the other. Found by asking whether the bug just fixed in cache-prefix-reorder had a twin. It did.

below-cache-minimum hedges when an estimated prefix lands just under the threshold — the real one may already be over, and withholding the largest saving Trazum offers on the strength of a ±10% figure is wrong advice. prompt-caching did not hedge when an estimate landed just over it. With a ±10% band an estimated 528-token prefix can truly be 475, and then nothing caches at all and the dollar figure printed beside the advisory is uncollectable.

Same fault as the reorder advisory, opposite direction, and this is the direction with money attached. The hedge qualifies the figure rather than withdrawing it — the prefix probably does clear the line — and names --exact-tokens, which settles it for free.

Only on an estimate. A caller who supplied their own counter has an authoritative prefix, and telling them it might be wrong pushes them toward a check they have already done.

Added#

A test that no size around the threshold makes an unqualified claim. Whichever advisory fires, if the band around the estimated prefix straddles the minimum, the text has to say so. Swept across the window rather than asserted on two samples, because the fault was an asymmetry between two code paths and only a sweep can show the seam between them is closed.

It reads the prefix from analyzeCachePrefix rather than scraping ~528 out of the sentence. The first version did scrape it, defaulted to "straddles" when the regex missed, and reported two failures against correct behaviour — a test asserting its own parsing instead of the property.

Fixed#

cache-prefix-reorder was offering money that could not be collected. It fired whenever enough stable content sat after the first placeholder and priced moving it forward at 90% off — without asking whether the prefix that rearrangement would build actually clears the model's cacheable minimum.

On a 306-token support prompt against Claude Opus 5's 512-token minimum, the best prefix any ordering can produce is 302. Nothing caches at any ordering. The advisory offered $48.67 a month, in the same report as below-cache-minimum telling the reader caching would not work here at all — two findings contradicting each other, with the dollar sign winning the argument.

reorderForCache had refused these prompts from the start, for precisely this reason, so Trazum's advice and Trazum's action disagreed: take the advice, run --reorder, watch nothing happen. A money figure in the flattering direction is the one fault this file exists to catch.

The gate is a strict comparison against the minimum. No band hedge, and that was tried first — widening it by ±10%, on the reasoning that makes below-cache-minimum hedge near the line, opened a window between 466 and 512 tokens where the advisory offered and the command refused. The same fault one layer up, and the test caught it. The near-the-line case is already handled in the right place: below-cache-minimum names --exact-tokens, and once the number is certain both work from the same certainty.

Changed#

The advisory names the command that does it. It described the rearrangement in prose and left the reader to perform it by hand, while reorderForCache sat in the same package able to attempt it — whole blocks only, refusing any block that refers back to earlier text, and everything after one. It now prints trazum optimize <file> --reorder and still says to read the diff, because this is the one transformation that moves text rather than deleting it.

An advisory that withholds the command is the shape of the whole product problem: Trazum knowing something worth more than what it does about it. On a 1,355-token prompt the command takes the cacheable prefix from 13 tokens to 1,350.

Added#

A test that the advice and the action cannot disagree. For every size in a sweep: if the report offers the saving, reorderForCache must deliver movement.

One-directional on purpose, and the direction took two attempts to get right. The first version asserted that cache-prefix-reorder and below-cache-minimum never appear together — wrong about the product, not the code: on a prompt with plenty of movable content both are true and both useful, a diagnosis followed by its fix. The second asserted the converse as well, and failed against working code because below 200 movable tokens the advisory stays deliberately quiet while the command remains available. Silence about a small win is not the same fault as a promise about an impossible one.

Changed#

The published error band drops from ±15% to ±10%, and the worst measured error from 11.2% to 6.4%. No new API calls were needed for this — the finding was sitting in the twenty-one measurements already committed.

Kana and han do not cost the same. Every CJK character was charged one token, and measured against the counting endpoint that put Japanese at +11.2% — the worst figure anywhere in the corpus — while Chinese sat at −3.2% under the identical rule. One constant cannot be right for both, and the samples say why: the Japanese file is 58% kana and the Chinese one is 0%.

Kana are a small syllabary that appears in every sentence, so a merge table covers runs of them and several characters share a token. Han are tens of thousands of rare characters a merge table cannot cover, and they cost about one each. Measured: kana 0.75 tokens per character, han 1.05.

cjk-japanese   +11.2%  →  -1.5%
cjk-chinese     -3.2%  →  +1.3%
worst in corpus 11.2%  →   6.4%   (code-heavy, which nothing is fitted to)

The signal needs no detector, which is what separates this from language.ts. A character is kana or it is not; the two samples separate perfectly at 58.3% against 0.00%. No refusal case, no margin rule, nothing to get wrong on a three-line prompt.

Rounding up per run was the first attempt and it was wrong by five points. Ordinary Japanese alternates kana and han inside every sentence, so the runs are short and numerous, and a Math.ceil per run charges most of a token for each boundary — an artefact of where the loop breaks rather than of what the text costs. CJK accumulates as a fraction now and is rounded once, over the whole document. A test builds the same characters blocked and alternating and requires the two estimates to agree within one token.

The band is 10 rather than 7, and the margin is deliberate. 6.4 rounded up is 7, and publishing 7 would be a tighter claim than twenty-one samples across six text types can support: there is no Korean here, no Cyrillic prose, no mixed-script document, and a seventh type could easily land at eight. A band that becomes false the first time somebody measures something new is the exact fault this whole exercise was fixing, so the uncertainty is overstated rather than understated.

Added#

Accuracy is ratcheted per text type, separately from the published band. The band is deliberately loose and that has a cost: a change taking CJK from 1.5% back to 3.6% passes every band assertion, because both are inside ten. Found while mutation-testing this very change — setting HAN_TOKENS_PER_CHAR back to a round 1 doubles the CJK error and nothing failed.

Each type now carries a floor set to what it has actually reached. They tighten, never slacken: an improvement lowers its floor in the same commit, and a deliberate trade raises it with a changelog line, which is a different act from not noticing. A type added to the corpus without a floor fails the suite rather than going ungated.

Same idea as trazum baseline turned on this repository's own numbers — publish a ceiling, gate on drift away from what you had.

RELEASES.md and ROADMAP.md join CHANGELOG.md as records exempt from the band-consistency guard. The guard requires every file stating a band to match the code, which is right for a README and wrong for a release note: "the band is still ±15%" is a true statement about 1.9.0, and rewriting it to say 10 would be falsifying a record to satisfy a test. The other twenty files it flagged were live claims and were updated.

Fixed#

The preflight told the reader to disbelieve it, and it was right. The caveat closed with "believe your settings over this check", on the reasoning that the exchange endpoint is undocumented and a refusal could be the request being wrong.

Then it was tested. v1.9.1 was tagged against settings that had just been filled in on all three packages; the check said rejected; the publish failed with the same E404 it predicted, twice. The check has been right in the only case that has ever tested it, and the caveat argued the reader out of a true finding — which is worse than no caveat.

It still says the endpoint is undocumented, because that is true and matters. It now also says it has been right once, and asks to be believed until it is not.

And the diagnosis was unreachable for a third time. The auth check runs before verify, which then prints thousands of lines, and GitHub's logs API returns the tail of a job — so the block naming what npm must match could not be retrieved while a release was actually failing, twice during v1.9.1. Writing it to the job summary fixed that for anyone on the run page and not for anyone reading the log, which is where a failure gets read.

The failure step repeats it now, at the end, where the tail always reaches. Four lines, and it is the difference between an error and a diagnosis.

Changed#

The report was leading with its smallest number. Measured rather than assumed: on an ordinary customer-support prompt — already reasonably written, which is what a real one is — the rules recover three tokens of 306, worth $0.75 a month. The advisories listed below them on the same prompt are worth $345 and $48. The report opened with -1.0% and closed with the comparison.

Start here: is the first thing in the report now:

Start here:
  "This task may not need Claude Opus 5" — $345.45/month, 461× what the
  rules saved.

Input tokens
  306 → 303   -1.0% (estimated, ±15%)

That ordering was not a presentation quibble. It taught the reader that shortening the prompt is what this tool is for, and on any prompt somebody competent wrote, shortening it is the smallest thing available. The rules earn their keep on genuine bloat — a duplicated paragraph, "due to the fact that", where they measure -23.5% — and recover close to nothing once it is gone, because they recover waste rather than creating savings. Both figures are from the same build, minutes apart, on two prompts.

Nothing about what Trazum computes changed. It already knew the advisory was worth 461 times more and said so at the bottom of the screen.

One thing got simpler on the way. The line's guard was duplicated — an early return and a filter, both keyed on the same condition — which made it untestable: removing the guard left the filter still suppressing the line, so a mutation that priced a flat plan passed. Two checks for one condition is one check and one place for a bug.