feat(model-meta): register qwen-plus family with its own tokenizer and template

request_model values like qwen-plus matched no registered family, so
resolve_model_family fell through to its "glm5" default and the formatter
rendered canonical_prompt with the GLM-5 chat template and tokenized release
blocks with the GLM-5 tokenizer. The fallback is silent, so nothing surfaced it.

Register the family, ship Qwen3-plus tokenizer/chat_template (the template was
embedded in tokenizer_config.json upstream and is extracted here), and add a
QwenPlusTraceAdapter ahead of the GLM catch-all in get_raw_adapter -- the
registry entry alone is not enough, since the adapter is what picks the
template. The shared message/tool normalization is factored out of
build_glm5_canonical_prompt so both builders use one copy.

Measured against provider-reported usage.input_tokens on a 400-request sample,
median tokenized/reported ratio goes 0.9449 -> 1.0000, and unique release block
count on a 2190-request sample moves 5886 -> 5701.

Note deepseek-v4-flash is still unregistered and still resolves to glm5.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-09-11 06:31:34 +00:00
parent 8b536cab2e
commit 7cb4cac56d
7 changed files with 303751 additions and 7 deletions

View File

@@ -42,6 +42,12 @@ MODEL_REGISTRY = {
model_name="Qwen3-Coder-480B-A35B-Instruct",
request_model_hints=("qwen3-coder", "qwen3 coder", "qwen3_coder"),
),
"qwen-plus": ModelMeta(
family="qwen-plus",
provider="Qwen",
model_name="Qwen3-plus",
request_model_hints=("qwen-plus", "qwen3-plus", "qwen_plus"),
),
}
MODEL_ALIASES = {
@@ -54,6 +60,10 @@ MODEL_ALIASES = {
"qwen3-coder": "qwen3-coder",
"qwen3_coder": "qwen3-coder",
"coder": "qwen3-coder",
"qwen-plus": "qwen-plus",
"qwen3-plus": "qwen-plus",
"qwen_plus": "qwen-plus",
"plus": "qwen-plus",
}
@@ -73,6 +83,8 @@ def _infer_model_family_from_path(input_path: str | Path | None) -> str | None:
return None
if "qwen3-coder" in text or "qwen3_coder" in text:
return "qwen3-coder"
if "qwen-plus" in text or "qwen3-plus" in text or "qwen_plus" in text:
return "qwen-plus"
if "glm5" in text or "trace-glm" in text:
return "glm5"
return None