# EXP-SIM-TUNING-LATENCY-MATRIX-V0：现有 simulator 能否选择真机低延迟 serving config？

> **状态：** protocol 已冻结；Qwen3.6 release-runtime topology smoke 已通过（2026-07-17）。
> 没有 Frontier Qwen3.6 contract/profile、trace replay 或 latency-selection 结果；本 card
> 不授权 600-s matrix launch。

## Claim 与决策

- **研究问题：** 对冻结的 `(H20 hardware, model, serving engine, request trace, config
  candidate surface)`，现有 simulator 能否为一个明确的 latency objective 选出真机最优、或
  统计上等价的 serving-engine config？
- **Parent claim：** “simulator 已经解决 config tuning”只有在完整 candidate surface 上产生
  正确的真机 latency selection 时才成立；绝对 latency error 或单个吞吐/SLO 结果均不充分。
- **Hypothesis：** simulator 在 fixed-shape / prefill-only 条件下可能足够，但在 decode、真实
  arrival 与 prefix/KV state 同时存在时会因 state-conditioned residual 或 coverage failure 而
  失去 selection fidelity。
- **Competing hypothesis：** 经冻结、非 evaluation 的 operator/collective calibration 后，
  simulator 能跨 non-MoE hybrid/MoE 与四种 workload condition 稳定选出真机低延迟 config。
- **判定：** 每个 `model × workload × metric` 是独立 tuning task，且 config 越低 latency 越好。
  只有所有预注册且 simulator 声称支持的 task 均通过下面的 coverage 与 selection gates，才能
  写“该 simulator 已解决其声明范围内的 tuning”；任一失败 cell 是明确 boundary，不被平均值
  掩盖。

## 固定系统与模型覆盖

| 角色 | 状态 | 运行时契约 | 当前可复用 artifact / 含义 |
|---|---|---|---|
| MoE | **可开始准备** | Qwen3-30B-A3B，BF16，community vLLM 0.20.0，dash0 H20；Frontier/Vidur-class fork | `frontier-phase-factorial-v0` 有 `ISL=2048, OSL=1, r=4` 的 request-level real/sim pilot；`frontier-fidelity-envelope-v1` 有 fixed-PD 与 exact Trace-PD materializers。历史 pilot 不是本 card 的最终 600-s evaluation。 |
| Non-MoE hybrid (not all-attention dense) | **TP1/2/4 smoke passed; profile pending** | `Qwen/Qwen3.6-27B@cea40373b9214dd387123e68841890af30dcd469`，community release `vLLM 0.20.2` / Torch `2.11.0` / Transformers `5.14.1`，dash0 H20；Frontier profile 必须在此 exact stack 采集 | Exact release serves text-only 2048→1 at TP1/2/4. vLLM resolves a 784-token attention page for its GDN/Mamba state, so requested block=16 is not the physical engine state. Qwen3.6 trace, Frontier hybrid model contract, and profile remain missing. |

所有公开结果固定在单机 `dash0`（8×H20）；每个 model 使用相同 engine build 的 real
server 与 simulator contract。Qwen3.6 bring-up 已确认真实 runtime/page legality；其余最低成本
gate 是冻结 trace/tokenizer 与完成 Frontier hybrid profile-schema audit。若不能做到，论文结论
必须限于 MoE，不得写跨模型 generalization。

### Qwen3.6 prerequisite audit (runtime + smoke, 2026-07-17)

| Prerequisite | Evidence | Verdict | Decision |
|---|---|---|---|
| Exact model snapshot | Immutable ModelScope snapshot `Qwen/Qwen3.6-27B@cea40373b9214dd387123e68841890af30dcd469` is staged at `/home/admin/cpfs/wjh/models/Qwen/Qwen3.6-27B`; its sorted-file inventory digest is `031cea103726cbf27732b6971e898e210aee77f0c32e4bc703c947e0372349c4`. | **PASS** | Bind every Qwen3.6 trace, real server, and profile manifest to this revision and inventory. |
| Community vLLM runtime | Clean environment `/home/admin/cpfs/wjh/venvs/qwen36-vllm-0.20.2` contains official wheel `vLLM 0.20.2`, Torch `2.11.0`, and Transformers `5.14.1`; `pip check` passes and CPU-only `get_config(..., trust_remote_code=false)` resolves `Qwen3_5ForConditionalGeneration`. | **PASS (CPU)** | Use exactly this release environment for real serving and profile collection. |
| Topology compatibility | Fresh TP1/2/4 servers each completed a text-only 2048-prompt/1-completion request with exact usage; all GPUs were released after cleanup. Every topology resolved the attention page from requested 16 to 784 tokens for Mamba/GDN state. | **PASS (GPU smoke)** | This is an engine-configuration legality result, not a latency rank or simulator result. Profile/simulator input must expose the 784-token state/page contract. |
| Qwen3.6 Frontier model/profile | No local or remote Qwen3.6-named Frontier profile/model directory was found | **MISSING** | New dense profile root is required; Qwen30/MoE profile CSVs are forbidden inputs. |
| Old dense artifacts | Existing Qwen3.5-27B trace is internal-vLLM/hybrid/speculative | **N/A as baseline** | Keep only as historical context; never feed it to this matrix or profile calibration. |

This audit is not a failed simulator result. The topology gate is closed, but the model contract,
profile root, and frozen trace are still reproducibility blockers: the Qwen3.6 half is `NOT RUN`,
not `FAIL`. For a paper claim about a conventional all-attention dense transformer, Qwen3.6 is not
admissible evidence; it is only a non-MoE hybrid case.

## Workload matrix

每个 model 都运行四个固定 trace。`P` 是 PD disaggregation 的 P-node workload，不是缺失
decode 的坏数据；它的 `OSL=1` 是设计变量。固定 cases 与 trace cases都使用 open-loop replay，
600 秒 observation window；`Fixed` 的 request count 为 `ceil(QPS × 600)`。

| ID | trace contract | output contract | prefix/KV contract | 可评 latency objectives |
|---|---|---|---|---|
| Fixed-P | 固定 `ISL=2048`、uniform fixed QPS=`4 req/s`，token vector deterministic | `OSL=1` | no shared logical prefix；prefix cache off | mean/p90 TTFT；mean/p90 E2E；TPOT=`N/A` |
| Fixed-PD | 固定 `ISL=2048`、uniform fixed QPS=`4 req/s` | `OSL=128`，`min=max=128`，`ignore_eos=true` | no shared logical prefix；prefix cache off | mean/p90 TTFT、TPOT、E2E |
| Trace-P | held-out production trace 的逐 request prompt-token vector、arrival time、request order、session root 与 prefix block relation 原样保留 | **只将 replay output 强制为 `OSL=1`** | source prefix equivalence、warm/cold state、cache policy 原样保留 | mean/p90 TTFT；mean/p90 E2E；TPOT=`N/A` |
| Trace-PD | held-out production trace 的逐 request input/output lengths、arrival time、order、session root 与 prefix block relation 原样保留 | 原始 `min=max=source OSL`，`ignore_eos=true` | source prefix equivalence、warm/cold state、cache policy 原样保留 | mean/p90 TTFT、TPOT、E2E |

`Trace-P` 应明确称为 **trace-derived prefill-only**：它忠实保留输入、到达、会话与 cache
state，但不忠实保留原始 output-length distribution；不能被描述成完整 trace-faithful replay。
`Trace-PD` 才是完整 input/output/arrival/prefix-KV joint replay。

### Trace、correctness 与 cache 的不可变契约

- 每个 trace manifest 列出：request ID、arrival timestamp、input/output token counts、session
  ID、runtime block identities、trace vector SHA-256、source trace SHA-256、requested and
  resolved physical page/block size、cache policy、initial cache state、warmup trace digest 与
  model tokenizer/version。
- 当 simulator 要模拟 vLLM prefix caching 时，`block_hash_ids` 只能描述可复用的**完整**
  runtime blocks：adapter 必须导出 `floor(ISL / block_size)` 个 prompt identities，不能将
  `ceil` 得到的 partial final block 当作 cache hit。该 simulator-facing projection 有独立
  digest；private real replay 仍保留完整 prompt 与原始 token vector。
- `Trace-P` 产生单独 manifest：source OSL 和 replay OSL 都写入 digest；只允许 source
  OSL → 1 这一个变换。`Trace-PD`、real client 与 simulator CSV 的 row-vector digest 必须完全
  相同。
- 真实端逐 request 核验 prompt/completion usage 与 manifest 一致；simulator 核验 request ID、
  input/output shapes、完成数与 trace digest。一项不符即该 cell 的 metric 无效。
- 每个 `(config, workload, trial)` fresh server；server warmup 用独立 manifest，warmup request
  不进入指标，也不得改变 evaluation cache 初态。prefix-on trace 以同一明示 initial cache
  state重放，不能用随机前史替代。

## Candidate-surface 与 baseline 规则

1. 对每个 model 冻结 `C = TP∈{1,2,4} × MNS∈{8,16,32,64}`（12 个 configs），并固定
   `DP=PP=EP=1`、MBT=8192、chunked prefill on。每个 candidate 还必须冻结 **真实 engine
   resolved** page/block and state-cache contract；requested CLI `--block-size 16` 不能替代它。
   Qwen3.6 smoke 在所有 TP 都将其改写为 784 tokens。任何不被某 model/engine 静态支持的
   config 要在运行前从该 model 的 `C` 整体删除，并记录理由；不能看结果后删除。
2. `C_real-valid` 是 server launch 成功、所有三个 fresh trials 全部请求完成且 usage/hash 合法的
   configs。真机 OOM、timeout 或 request error 保留为 real infeasible/safety outcome，而非给它
   赋无限 latency。
3. 对每个 simulator，selection coverage 必须涵盖全部 `C_real-valid`。crash、scheduler stall、
   trace mismatch、缺 request metrics 都是 simulator **coverage failure**，不可当成高 latency 或
   infeasible 来产生排序。只有完整 coverage 才能形成 tuning verdict。
4. Frontier、官方 Vidur 与 APEX 分别独立报告版本、patch、profile 与 capability surface。APEX
   若只支持 DP/PP/TP parallel plan，则它只评 `parallel-plan tuning` 交集；不得用该子问题结果
   宣称解决 MNS/MBT/cache 等完整 engine tuning。

## Metrics、统计与 selection score

每个完整 request 使用同一 client-boundary 定义：

- `TTFT`：客户端发出请求至收到第一个 generated token；包含 serving queue。
- `E2E`：客户端发出请求至完成 `[DONE]`。
- `TPOT`：`(E2E - TTFT) / (completion_tokens - 1)`；只对 `completion_tokens > 1` 的请求定义。
- 每 trial 的 `mean` 是 request-level arithmetic mean；`p90` 是 nearest-rank
  `ceil(0.90 × N)`。P-only 的 TPOT 是 JSON `null` / table `N/A`，绝不写 0。

同一 config 的 real 运行三次 fresh-server trial，随机化 config order；每个 trial 都对整个
trace 计算六项或四项指标。real ranking point estimate 是三个 trial metric 的 median，且报告
pooled request statistic、每-trial statistic 和 95% hierarchical bootstrap interval（先重采样
trial，再在 trial 内重采样 request）。simulator 的 seed、determinism 和 repeat policy同样记录。

对每一个 objective，输出：absolute real/sim surface、real bootstrap top set、simulator exact
top set、top-set hit、sim-chosen config 的 best/worst tie-break regret、Kendall τ-b、non-tied
pair direction accuracy、coverage 与 error/infeasible table。Latency regret 定义为
`(real_metric(sim_choice) - real_metric(real_best)) / real_metric(real_best)`。

## 预注册 decision gates

| Gate | Pass | Failure handling |
|---|---|---|
| Trace/metric validity | 100% planned requests have matching usage/shape and finite applicable metrics | cell invalid；诊断 harness，不作 performance conclusion |
| Simulator coverage | 100% of `C_real-valid` output legal request metrics | `FAIL: coverage`；不把 missing cell 排成最差 |
| Selection | simulator top set 与 bootstrap real top set 有交集 | `FAIL: selection` |
| Risk | simulator top-set 的 **worst** real tie-break regret `≤5%` | `FAIL: regret`，即使 best tie-break 碰巧为 0 |
| Global rank diagnostic | τ-b `≥0.8`；若没有非并列 pair 则写 `N/A`，不当通过 | `NEEDS EVIDENCE` / `FAIL` 与 top-set gate 分开报告 |

这里不使用 SLO、capacity 或 throughput 作 selection objective；它们最多作为安全/诊断附录指标。
P-only 共 4 tasks，PD 共 6 tasks，所以完整 `non-MoE-hybrid/MoE × 4 workloads` matrix 共有 40 个
独立 latency-selection tasks。

## Calibration 与 information boundary

- 允许 simulator 使用同硬件/同 model/同 runtime 的 offline operator、collective 或 memory
  profiles，但 profile command、shape grid、cost 和 hash 必须在 evaluation 前冻结。
- calibration 不得读取 evaluation 的 real request latency、top config 或 E2E scalar；禁止 per-TP
  或 per-config serving E2E scale。若有 trace-driven calibration，必须只使用按 session root 切分
  的 calibration fold；Trace-P/PD 使用完全不相交的 held-out root sessions。
- Fixed traces 与 trace evaluation 都不能同时作为 calibration workload。任何 required simulator
  patch 在查看该 cell real result 前冻结；patched baseline 与 original release 分开作图。

## Minimal Qwen3.6-27B bring-up and profile plan

This is the only permitted path to add the dense case. It intentionally separates cheap provenance
work from GPU work, and it does not authorize either stage by itself.

1. **Preflight and topology smoke (complete):** the snapshot/runtime are locked; fresh TP1/2/4
   servers completed exact-usage text requests. The actual vLLM state/page setting is 784 tokens,
   not the requested 16. This establishes the profile input contract only.
2. **Hybrid Frontier contract (not run):** verify Frontier can represent Gated DeltaNet/linear-state,
   gated attention, FFN, true mixed execution, TP collectives, the 784-token page rule and capacity,
   without substituting Qwen3.5/Qwen30 fields.
3. **Minimal profile set (GPU, separately approved):** on the exact Qwen3.6/community-vLLM/dash0
   stack, collect per-TP Gated-DeltaNet, gated-attention prefill/decode, state/KV and TP2/TP4 all-reduce
   rows. The shape grid must cover the pre-registered Fixed-P/PD and held-out Trace-P/PD ranges, but
   must not contain evaluation E2E latency or a fitted time scale. Freeze CSV/schema/command hashes
   before running any evaluation trace.
4. **Profile acceptance (CPU):** verify model/runtime identity in every profile manifest; require
   requested shape coverage, finite values, and no fallback to a MoE or Qwen3.5 profile. Only then may
   a parent-approved real/simulator latency surface be scheduled.

## 预期产物、复现与成本

- **Figure prototype：** [latency-selection-matrix-schematic.svg](latency-selection-matrix-schematic.svg)
  （明确标注 `SCHEMATIC — NOT MEASURED DATA`）。最终图替换为每个 task 的 top-set hit / regret /
  coverage heatmap，并附 per-config latency surfaces。
- **Raw layout：** `artifacts/<model>/<workload>/<real|simulator>/<config>/trial-*/`；每个目录含
  command、environment/machine fingerprint、server flags、trace/profile hash、raw request records、
  stdout/stderr、duration 与 SHA-256 inventory。
- **Existing reuse audit：** MoE F0 可检验 P-only raw metric reader；现有 F2 可检验 fixed-PD
  trace materialization；T1 可检验 Trace-PD digest and simulator coverage reporting。它们并不构成
这个 600-s no-SLO matrix 的完整 real surface。Qwen3.6 requires a new same-stack trace,
  model contract and profile root; old Qwen3.5 internal-vLLM artifacts are excluded.
- **Cost estimate, not authorization：** 单 config×trial 假定 `10 min replay + 5 min fresh-server
  startup/warmup = 15 min`。完整 12-config sweep 的 GPU allocation 为
  `4 × (TP1 + TP2 + TP4) = 28 GPU`; 因而每个 `model×workload` 的三 trial 约
  `28 × 0.25 × 3 = 21 H20-GPUh`。8 cells 的 real serving core is **168 H20-GPUh**. The Qwen3.6
  bring-up/profile set adds at most **6 H20-GPUh** (three topology smokes plus dense/attention/
  collective rows), giving a **174 H20-GPUh** core estimate and a **190–215 H20-GPUh** planning
  envelope with retries. Every launch must recompute this from the staged trace duration and actual
  server preflight; this paragraph is not authorization.

### Required remote launch echo (not executed)

```text
NOT EXECUTED — awaiting explicit approval after Qwen3.6 model/runtime/profile gates
host=dash0; remote_root=/home/admin/cpfs/wjh/aituner/aituner
models=non-moe-hybrid:Qwen/Qwen3.6-27B@<FROZEN_SNAPSHOT>,moe:Qwen3-30B-A3B-BF16
engine=community-open-source-vLLM@<FROZEN_SOURCE_OR_WHEEL>; simulator=Frontier/Vidur-class,<official baselines if installed>
workloads=Fixed-P(ISL2048,OSL1,QPS4,600s),Fixed-PD(ISL2048,OSL128,QPS4,600s),
          Trace-P(heldout exact input/arrival/prefix,OSL1),Trace-PD(heldout exact joint trace)
surface=TP{1,2,4}×MNS{8,16,32,64}; trials=3; fresh_server=true
real_paths=runs/simulator-tuning-latency-matrix-v0/artifacts/<model>/<workload>/real/
expected_cost=174 H20-GPUh core; 190–215 H20-GPUh planning envelope; expected_wall=multiple queued hours
gate=Qwen3.6 snapshot + community-vLLM fingerprint + resolved-784 hybrid Frontier profile completeness + trace/profile hashes + user/parent approval
```

## Expected outcomes and stop/pivot rules

- If a simulator passes every supported task, stop diagnosing that simulator and look for a new problem
  outside its evaluated scope (e.g., unsupported engine knobs or alignment cost).
- If it fails coverage, repair/liveness is a separate intervention; original capability result remains
  frozen. First rerun simulator only; do not spend GPU-hours until it can produce a complete surface.
- If it covers but selects incorrectly, retain the counterexample and run one minimal discriminative
  mechanism experiment (scheduler state, batch composition, graph/kernel family, or MoE/collective),
  changing one factor at a time.
- If Qwen3.6 cannot be profiled on its exact community-vLLM stack, stop the non-MoE-hybrid half rather
  than treating any internal-vLLM or Qwen3.5 result as a substitute. If the headline requires a pure
  all-attention dense-transformer case, add that model explicitly rather than relabelling Qwen3.6.

## Benchmark-design audit

| Risk | Pre-registered control |
|---|---|
| SLO/capacity result misrepresented as tuning | selection uses only per-request mean/p90 TTFT, TPOT, E2E on the same trace |
| P-only judged by nonexistent TPOT | OSL=1 and TPOT=`N/A` are explicit workload/metric contracts |
| Trace simplification hidden | Trace-P is labelled output-normalized; only Trace-PD claims full joint replay |
| calibration equals evaluation | microprofile/trace calibration folds are disjoint; no E2E correction on evaluation |
| simulator crash treated as a bad config | coverage gate separates missing metrics from latency ranking |
| cross-stack or wrong-architecture reuse | only Qwen3.6-27B hybrid profiles collected on its exact community-vLLM stack are admissible; old Qwen3.5/internal-vLLM or Qwen30/MoE artifacts are excluded |
| selected winning cells only | all 8 cells and all legal configs are frozen before real results |
| tail/noise omitted | p90, three fresh trials, per-trial values and bootstrap top sets are mandatory |