Files
ali-trace-tools/trace_model_meta
Gahow Wang 7cb4cac56d feat(model-meta): register qwen-plus family with its own tokenizer and template
request_model values like qwen-plus matched no registered family, so
resolve_model_family fell through to its "glm5" default and the formatter
rendered canonical_prompt with the GLM-5 chat template and tokenized release
blocks with the GLM-5 tokenizer. The fallback is silent, so nothing surfaced it.

Register the family, ship Qwen3-plus tokenizer/chat_template (the template was
embedded in tokenizer_config.json upstream and is extracted here), and add a
QwenPlusTraceAdapter ahead of the GLM catch-all in get_raw_adapter -- the
registry entry alone is not enough, since the adapter is what picks the
template. The shared message/tool normalization is factored out of
build_glm5_canonical_prompt so both builders use one copy.

Measured against provider-reported usage.input_tokens on a 400-request sample,
median tokenized/reported ratio goes 0.9449 -> 1.0000, and unique release block
count on a 2190-request sample moves 5886 -> 5701.

Note deepseek-v4-flash is still unregistered and still resolves to glm5.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 06:41:50 +00:00
..
2026-04-21 15:44:47 +00:00
2026-04-21 15:44:47 +00:00