feat(model-meta): register qwen-plus family with its own tokenizer and template
request_model values like qwen-plus matched no registered family, so resolve_model_family fell through to its "glm5" default and the formatter rendered canonical_prompt with the GLM-5 chat template and tokenized release blocks with the GLM-5 tokenizer. The fallback is silent, so nothing surfaced it. Register the family, ship Qwen3-plus tokenizer/chat_template (the template was embedded in tokenizer_config.json upstream and is extracted here), and add a QwenPlusTraceAdapter ahead of the GLM catch-all in get_raw_adapter -- the registry entry alone is not enough, since the adapter is what picks the template. The shared message/tool normalization is factored out of build_glm5_canonical_prompt so both builders use one copy. Measured against provider-reported usage.input_tokens on a 400-request sample, median tokenized/reported ratio goes 0.9449 -> 1.0000, and unique release block count on a 2190-request sample moves 5886 -> 5701. Note deepseek-v4-flash is still unregistered and still resolves to glm5. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -42,6 +42,12 @@ MODEL_REGISTRY = {
|
||||
model_name="Qwen3-Coder-480B-A35B-Instruct",
|
||||
request_model_hints=("qwen3-coder", "qwen3 coder", "qwen3_coder"),
|
||||
),
|
||||
"qwen-plus": ModelMeta(
|
||||
family="qwen-plus",
|
||||
provider="Qwen",
|
||||
model_name="Qwen3-plus",
|
||||
request_model_hints=("qwen-plus", "qwen3-plus", "qwen_plus"),
|
||||
),
|
||||
}
|
||||
|
||||
MODEL_ALIASES = {
|
||||
@@ -54,6 +60,10 @@ MODEL_ALIASES = {
|
||||
"qwen3-coder": "qwen3-coder",
|
||||
"qwen3_coder": "qwen3-coder",
|
||||
"coder": "qwen3-coder",
|
||||
"qwen-plus": "qwen-plus",
|
||||
"qwen3-plus": "qwen-plus",
|
||||
"qwen_plus": "qwen-plus",
|
||||
"plus": "qwen-plus",
|
||||
}
|
||||
|
||||
|
||||
@@ -73,6 +83,8 @@ def _infer_model_family_from_path(input_path: str | Path | None) -> str | None:
|
||||
return None
|
||||
if "qwen3-coder" in text or "qwen3_coder" in text:
|
||||
return "qwen3-coder"
|
||||
if "qwen-plus" in text or "qwen3-plus" in text or "qwen_plus" in text:
|
||||
return "qwen-plus"
|
||||
if "glm5" in text or "trace-glm" in text:
|
||||
return "glm5"
|
||||
return None
|
||||
|
||||
Reference in New Issue
Block a user