Skip to content
Trazum

1.56.1

What the rules actually do

A patch, and the number is the honest one. A minor closes an arc, and the arc in progress is 1.57 — the optimiser earns its name again — whose thesis is what belongs on the model's side of the line. Nothing here answers that. What is here is the groundwork the arc could not proceed without: the deterministic side measured, exercised and pinned, so a decision about what to add is made against figures instead of against a headline percentage nobody can reproduce.

Added#

trazum rules --measure <dir> — and what does each rule actually recover? The first chapter of arc 1.57, and it starts by measuring the number the whole arc is about. The README says plainly that the deterministic rules recover about one per cent, which is the fair complaint about this tool — and it is an aggregate, consistent with every rule pulling its weight and equally consistent with two rules doing all of it beside five that have never changed a byte of anybody's prompt. Nobody here had measured which, and deciding what belongs on the model's side of the line is a question about what the dictionary side already covers.

The rule order decides what the reader is told, and now something says so. The measurement showed three deletion rules each recovering the same tokens, which looked like a defect worth chasing. It is not: a repeated stanza is a repeated block and also a set of repeated lines, all three can find it, and whichever runs first takes it. The applied run credits exactly one rule — the leave-one-out measurement credits all three because each would have caught it alone, and those are different questions.

What had no guard was the consequence. Coarsest-first means the same saving is reported as one repeated paragraph instead of three repeated lines, and the only reason written down for that order was that block deletions leave less text for the rest to walk. A reorder would have changed what users read while every number stayed identical and no test noticed. The order is pinned now, adjacency included, and the comment says what it actually decides.

A corpus that exercises every rule. rules --measure could only ever answer "inert here", because nothing in this repository contained what most rules look for. Twelve fixtures now do — one per rule, each a short realistic prompt carrying exactly what that rule is written to find — and the guard is derived from the rule catalogue, so a rule added without a fixture fails the build rather than joining a list nobody reads.

And building it split one field into two. inertHere was "saved nothing", which quietly merged never fired with fired and recovered nothing. Those look identical in a saving column and mean opposite things: the first is a fact about the corpus, the second is a finding about the rule. emphasis is the case — it lowercases shouted words, so the prompt changes, the instruction changes, and the token count does not move. It now lands in firedWithoutSavingHere instead of being filed beside rules nothing exercised.

Two figures per rule, and they are never added together. alone is what a rule saves as the only rule running; marginal is what the whole set loses when it is removed. They diverge exactly where rules overlap — on a prompt with a repeated stanza, duplicate-blocks, near-duplicate-blocks and duplicate-lines each recover forty tokens alone and nothing at the margin, because the other two still find it. Reporting alone alone makes an overlapping rule look load-bearing; reporting marginal alone makes it look inert. sumOfAlone sits beside tokensSaved and the gap between them is stated as the overlap rather than resolved into a total, because a single figure there is the one number that cannot be true.

The floor is separated out, and finding that was the point. The optimiser normalises whitespace whether or not any rule is enabled. The first version of this measurement credited that to the rules — and run over this repository's tokenizer corpus it reported that the optimiser saved twenty-one tokens and that every rule was redundant, a sentence assembled entirely out of somebody else's arithmetic. floor is now its own field and tokensSaved is the rules' work alone.

"Inert" is always said about the corpus. A rule that finds nothing in these files has not been shown to find nothing anywhere, and the difference is the whole distance between "delete this rule" and "measure it on something else".

And a path defect found by running it: --measure <dir> read the walk's relative names without putting the root back on, so it worked only from a run whose working directory happened to be the root — which is the one case a first probe uses.

Recorded rather than asserted: on the two sample prompts this repository ships, the deterministic rules recover nothing at all — every rule lands in inertHere. That is not a defect in the rules or in the samples. It is the reason the measurement exists, because an aggregate quoted from elsewhere is not a measurement of what is here.