request_model values like qwen-plus matched no registered family, so
resolve_model_family fell through to its "glm5" default and the formatter
rendered canonical_prompt with the GLM-5 chat template and tokenized release
blocks with the GLM-5 tokenizer. The fallback is silent, so nothing surfaced it.
Register the family, ship Qwen3-plus tokenizer/chat_template (the template was
embedded in tokenizer_config.json upstream and is extracted here), and add a
QwenPlusTraceAdapter ahead of the GLM catch-all in get_raw_adapter -- the
registry entry alone is not enough, since the adapter is what picks the
template. The shared message/tool normalization is factored out of
build_glm5_canonical_prompt so both builders use one copy.
Measured against provider-reported usage.input_tokens on a 400-request sample,
median tokenized/reported ratio goes 0.9449 -> 1.0000, and unique release block
count on a 2190-request sample moves 5886 -> 5701.
Note deepseek-v4-flash is still unregistered and still resolves to glm5.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Local tokenization does not always agree with the input_length the provider
reports, which left the release row carrying a block count derived from our own
count. Truncate or pad digest_hexes to the block count implied by the provider
value. Padding digests are derived per request, so a padded block can never
collide with another request and be scored as shared prefix reuse.
Collapse chunk_*.jsonl in rounds with a bounded merge fan-in so the number of
simultaneously open chunk iterators stays flat regardless of how many chunks a
trace produces, and write a resume manifest so --resume-tmp-dir can continue
from an existing formatter temp directory without rescanning the raw trace.
