13 KiB
13 KiB
Qwen3.6-27B non-MoE/hybrid preflight and topology-smoke record
Status (2026-07-17):
TOPOLOGY-SMOKE PASS; FRONTIER COVERAGE BLOCKED. The exact snapshot and official community-vLLM release are accepted through a fresh-server TP1/TP2/TP4 GPU smoke. No Frontier Qwen3.6 contract/profile, trace replay, or latency-matrix cell exists. The smoke is compatibility evidence only, not simulator-fidelity or latency-selection evidence.
Decision enabled by this preflight
The latency-matrix experiment may add its dense branch only after the exact
model snapshot, a runnable open-source vLLM environment, and a Qwen3.6
Frontier contract/profile root exist on the same stack. Until then, all dense
matrix cells are NOT RUN. They are neither simulator failures nor
evidence for dense+MoE generalization.
Executed staging record (after explicit approval)
| Step | Evidence | Result |
|---|---|---|
| Immutable snapshot staging | ModelScope Qwen/Qwen3.6-27B@cea40373b9214dd387123e68841890af30dcd469 was downloaded to /home/admin/cpfs/wjh/models/Qwen/Qwen3.6-27B. The downloader recorded download_start and download_complete at 2026-07-17T07:42:00Z and 2026-07-17T07:44:45Z, respectively, in /home/admin/cpfs/wjh/models/.staging/qwen36-27b-20260717/download.log. |
PASS; 29 snapshot files, approximately 52 GiB. |
| Snapshot integrity | Sorted per-file SHA-256 inventory: /home/admin/cpfs/wjh/models/.staging/qwen36-27b-20260717/files.sha256; inventory digest 031cea103726cbf27732b6971e898e210aee77f0c32e4bc703c947e0372349c4. It includes config.json, tokenizer files, and all 15 model-*.safetensors shards; no *.incomplete, *.lock, or *.tmp files were found. |
PASS. |
| CPU config/tokenizer parser | The preinstalled system transformers==4.57.5 rejects model_type=qwen3_5; this environment is excluded. A dedicated CPU-only environment with transformers==5.5.3 successfully parsed the snapshot with CUDA_VISIBLE_DEVICES="": Qwen3_5ForConditionalGeneration, text model type qwen3_5_text, 64 layers, hidden size 5120, mamba_ssm_dtype=float32, tokenizer length 248077. |
PASS, but only for checkpoint/tokenizer parsing; it is not the serving runtime. |
| Community vLLM release runtime | The initial source-build attempt was stopped before acceptance and is excluded. Official PyPI wheel vllm==0.20.2 was staged at wheel-probe-proxy/vllm-0.20.2-cp38-abi3-manylinux_2_35_x86_64.whl, SHA-256 22a7dd06eb03371298e13d6100f3dedbf307352342aaf08e87c929c60aae9b4d, then installed into the clean environment /home/admin/cpfs/wjh/venvs/qwen36-vllm-0.20.2. pip check reports no broken requirements. |
PASS (CPU-only): vLLM 0.20.2, Torch 2.11.0, Transformers 5.14.1; with CUDA_VISIBLE_DEVICES="", vllm.transformers_utils.config.get_config(..., trust_remote_code=False) parses the exact snapshot as Qwen3_5ForConditionalGeneration / qwen3_5. |
| Fresh GPU topology smoke | Fresh text-only servers on H20 TP1, TP2, TP4 used the exact release runtime with requested MNS=8, MBT=8192, BF16, chunked prefill on, prefix cache off, and one 2048 → 1 request per topology. Every response reported exactly 2048 prompt and 1 completion token; all four used GPUs were zero-memory after cleanup. Evidence root: artifacts/qwen36/smoke-v4-20260717/; artifact inventory SHA-256 5b09c94e61801bfb7085257cfeea88049dfa81001f50810d85b9f87468088be0. |
PASS (GPU compatibility). It establishes that TP1/2/4 can serve this model on the intended release, not that any topology has lower latency. |
Confirmed model identity and architecture
| Item | Confirmed fact | Consequence for the experiment |
|---|---|---|
| Canonical source | ModelScope Qwen/Qwen3.6-27B |
Use this repository, not an inferred Hugging Face mirror or an old Qwen3.5 checkpoint. |
| Immutable revision | Read-only git ls-remote --symref on 2026-07-17 resolved master/HEAD to cea40373b9214dd387123e68841890af30dcd469. |
Pin this commit hash in the download record. Do not record only the mutable name master. Re-resolve it immediately before an approved download; a changed hash requires a new review. |
| Model metadata | ModelScope API reports Architectures=["Qwen3_5ForConditionalGeneration"] and ModelType=["qwen3_5"]. The official card says the artifact is a causal language model with a vision encoder. |
This is dense with respect to MoE routing, but it is not a conventional all-attention dense transformer. The text benchmark must send no media; the model runner must still be the architecture selected from the exact checkpoint config. |
| Text architecture (official card) | 27B parameters; hidden size 5120; padded token embedding / LM output 248320; 64 layers; 16 × (3 × (Gated DeltaNet → FFN) → 1 × (Gated Attention → FFN)); DeltaNet has V=48/QK=16 heads of 128; gated attention has Q=24/KV=4 heads of 256 and RoPE dimension 64; FFN intermediate size 17408; native context 262144. |
A Frontier profile must cover both DeltaNet/linear-state and gated-attention paths, in prefill, decode, and true mixed execution. An all-attention Qwen3.5 or Qwen3-30B/MoE profile is not a valid substitute. |
| Tokenizer | The staged snapshot contains the tokenizer artifacts in the immutable file inventory. The earlier dedicated CPU parser reported tokenizer length 248077; 248320 above is still only the padded embedding/output size, not the tokenizer contract. |
Request input/output lengths must be tokenized with this staged tokenizer. Before trace materialization, freeze a tokenizer/config subset digest and do not borrow counts from a Qwen3.5 tokenizer. |
The official ModelScope card states that these artifacts are compatible with
Transformers, vLLM, SGLang, and KTransformers. That is useful compatibility
evidence, not a successful run on the selected dash0 stack.
Runtime provenance and excluded alternatives
| Evidence | Read-only observation | Verdict |
|---|---|---|
| System vLLM | /usr/local/bin/vllm reports 0.13.0rc2.dev2111+gb44b43f43.d20260309, with Torch 2.8.0a0+5228986c39.nv25.6 and Transformers 4.57.5. Its installed source contains qwen3_5.py, qwen3_5_mtp.py, registry entries for Qwen3_5ForConditionalGeneration, and branches for qwen3_5_text. |
Architecture-family support is present, but this development build is not yet the frozen community experiment runtime. |
| Uninstalled local source | /home/admin/cpfs/wjh/agentic-kv/third_party/vllm_v20_build is v0.20.0 source with build by-products; /home/admin/cpfs/wjh/venvs/vllm-0.20.0 does not contain a runnable vLLM/Transformers pair. |
Excluded. It is not the experiment runtime and must not be cited as an alternative same-stack result. |
| Qwen-specific state/page rule | The v0.20 Qwen3.5 implementation obtains Mamba/DeltaNet state settings from the checkpoint when cache dtype is auto; the actual v0.20.2 GPU smoke then resolved its attention page to 784 tokens. | The state/page rule is a model-state contract, not an optional benchmark flag. The profile and simulator must consume the observed physical page setting. |
Therefore the defensible statement now is: the exact Qwen3.6-27B snapshot
is CPU-parsed and GPU-served by the frozen official community vLLM release on
dash0 at TP1/2/4. It is a non-MoE hybrid model, not a conventional
all-attention dense-transformer evidence point.
GPU topology-smoke observations (not a latency experiment)
The runner is a one-request topology/usage check, with a fresh server for each TP. Its E2E values are retained only to audit that a request completed; they are not steady-state samples, are not TTFT/TPOT measurements, and must not be ranked.
| TP | Exact usage | Smoke E2E (audit only) | Resolved attention page/block size | GPU KV cache at max_model_len=4096 |
|---|---|---|---|---|
| 1 | 2048 prompt, 1 completion | 4581.977 ms | 784 tokens | 306,289 tokens |
| 2 | 2048 prompt, 1 completion | 5606.756 ms | 784 tokens | 1,068,600 tokens |
| 4 | 2048 prompt, 1 completion | 6252.274 ms | 784 tokens | 2,630,087 tokens |
vLLM accepted the requested --block-size 16 but then logged, at every TP,
Setting attention block size to 784 tokens to ensure that attention page size is >= mamba page size. Thus block size 16 is not the actual engine state
for this model. A later real/simulator comparison must freeze and expose the
resolved 784-token page/state contract (or deliberately select a different
legal engine mode before profiling); it may not label this stack as a
block-16 comparison. First TP1 startup also compiled FlashInfer GDN kernels
and performed a cold profile/warmup; these one-time costs remain outside all
request-latency metrics.
Initial read-only dash0 inventory (historical; superseded where noted)
All paths below were inspected without mutation on 2026-07-17.
| Area | Observation | Consequence |
|---|---|---|
| Disk | /home/admin root filesystem: 245 GiB available. Shared CPFS mounted at /home/admin/cpfs: 1.1 TiB available, already 95% used. Existing /home/admin/cpfs/wjh/models is 1.2 TiB and /home/admin/resource/model is 1.7 TiB. |
Stage only after an explicit capacity check; do not rely on the root filesystem or an unconstrained default cache. |
| Checked model/cache locations | Before staging, the usual locations were absent. | Superseded: the immutable snapshot is now at /home/admin/cpfs/wjh/models/Qwen/Qwen3.6-27B and is bound by the inventory above. |
| Main checkout | /home/admin/cpfs/wjh/aituner/aituner had tracked and untracked user changes. |
The smoke wrote only new artifacts and did not modify its source. A future profile implementation must use an explicitly versioned source/patch root rather than treating this dirty checkout as code provenance. |
| Frontier | Historical Frontier source exists at /home/admin/cpfs/wjh/frontier-qwen30-vllm020-profile-v1/Frontier, commit d9cfeb6d8791fbf2f295dd9744c56a666171776e (pre-release-v0.2), with a clean tracked tree. A source search found no Qwen3.6, Qwen3_5, or qwen3_5 model-contract hit. |
It is not an existing Qwen3.6 profile/model contract. A new dense adapter/spec must be versioned and accepted before it may be used. |
| Existing profiler contract | The old Qwen3-30B profile launcher hard-codes model=qwen3-a3b-30b-moe, --is_moe, vLLM 0.20 API adapters, and Frontier commit d9cf…. |
It cannot be repurposed by only changing a model path; its MoE assumptions and compatibility adapter are excluded from the dense baseline. |
| Qwen3.6 artifacts | A read-only search of the checkout's runs subtree found no Qwen3.6-named run/profile/model path. |
No dense Frontier profile or real/simulator output is admissible today. |
Remaining gates before any latency-matrix cell
- Trace/tokenizer lock: materialize the Qwen3.6 tokenized Fixed/Trace-P/ Trace-PD vectors and hash the exact tokenizer/config subset. Trace-PD is not trace-faithful without the input, output, arrival and prefix-state vector.
- New Frontier contract: introduce a versioned Qwen3.6 non-MoE hybrid model spec
that maps the exact
config.jsonto simulator operators. It must represent Gated DeltaNet/linear-state, gated attention, FFN, TP collectives and KV/ state-cache capacity, including the resolved 784-token page rule. It may not import the Qwen3-30B MoE routing profile, an all-attention Qwen3.5 profile, or an unrecorded additive timing scale. Any compatibility adapter is a separate, hashed intervention. - Profile manifest: collect and freeze per-TP (1/2/4) profile rows for the above stages over shapes covering Fixed-P, Fixed-PD, Trace-P and Trace-PD. Include prefill, decode and genuine mixed batches, plus TP2/TP4 all-reduce. The manifest must bind model revision/config hash, tokenizer hash, vLLM/environment hash, Frontier commit/adapter hash, hardware, precision, profiler command, CSV hashes, and shape coverage.
- CPU simulator acceptance: reject the profile root unless every required profile family has finite rows for all legal TP values and requested shapes; all identity hashes match the real-server lock; profile/model manifests name Qwen3.6; and the simulator emits request-level TTFT/E2E/TPOT where defined for a no-GPU fixture. A missing simulator operator is a coverage blocker, not a license to borrow a Qwen30 profile.
Only after all four gates may a separately approved real/simulator 600-s latency surface begin. The GPU smoke closed the runtime/topology gate only; it does not close any of these simulator or evaluation gates.
Evidence locations
- This record:
runs/simulator-tuning-latency-matrix-v0/dense-preflight.md. - Matrix decision/protocol:
runs/simulator-tuning-latency-matrix-v0/experiment-card.md. - Existing (excluded) MoE-v0.20 profile launcher and its hard-coded contract:
runs/frontier-qwen30-vllm020-profile-v1/run_frontier_linear_smoke.shandruns/frontier-qwen30-vllm020-profile-v1/frontier_vllm020_compat.py. - Existing Frontier surface runner shows the required external source,
ReplayServe builder, profile root, request-level metric, and hash contract:
runs/frontier-phase-factorial-v0/run_frontier_qwen30_prefill_surface.py.
Not completed
- No Qwen3.6 Frontier model spec or profile exists.
- No Qwen3.6 fixed/trace workload vector is frozen.
- No real or simulator latency measurement exists, so this record cannot change the simulator-selection verdict.