Record Qwen235 state diagnosis results
This commit is contained in:
@@ -82,3 +82,43 @@ T1 保持原 600 秒 arrival window。先在 simulator 上冻结 `sampling_u` br
|
|||||||
| simulator-guided real sampling | 使用共同 guard anchors;未闭合 bracket 不能宣布 top match |
|
| simulator-guided real sampling | 使用共同 guard anchors;未闭合 bracket 不能宣布 top match |
|
||||||
| absolute-vs-rank metric | 同时报绝对 capacity/latency、rank、regret、tau-b、pair direction和 SLO confusion |
|
| absolute-vs-rank metric | 同时报绝对 capacity/latency、rank、regret、tau-b、pair direction和 SLO confusion |
|
||||||
| hidden fallback | A1/A2 强制计数 measured-model hit/fallback,并写入 frozen manifest |
|
| hidden fallback | A1/A2 强制计数 measured-model hit/fallback,并写入 frozen manifest |
|
||||||
|
|
||||||
|
## Qwen235 Fixed-PD TP4/TP8 state diagnosis(2026-07-19)
|
||||||
|
|
||||||
|
**结论:** measured TP4/TP8 collective 已被 simulator 真实消费且 analytical fallback 为 0,但
|
||||||
|
Frontier 在自身闭环状态上仍给出 `TP8-TP4=-20.0663 ms`,与冻结真机
|
||||||
|
`+6.9500 ms` 方向相反。使用 vLLM iteration details 采集真机精确
|
||||||
|
`(context_tokens, generation_requests)`,并对相同 Frontier stage records 做 100% exact joint-state
|
||||||
|
reweight 后,对比翻转为 `+10.9028 ms`。因此该 ranking error 的主因是 closed-loop
|
||||||
|
state/composition mismatch,不是同一真实 state 下的 conditional stage reversal。
|
||||||
|
|
||||||
|
| 量 | TP4/EP1 | TP8/EP8 | TP8-TP4 |
|
||||||
|
|---|---:|---:|---:|
|
||||||
|
| frozen real TPOT (ms) | 21.0442 | 27.9942 | +6.9500 |
|
||||||
|
| exact real decode-token-weighted iteration (ms) | 18.4710 | 27.5887 | +9.1177 |
|
||||||
|
| Frontier own-composition critical path (ms) | 74.2229 | 54.1567 | -20.0663 |
|
||||||
|
| exact-state-matched Frontier critical path (ms) | 34.2575 | 45.1603 | +10.9028 |
|
||||||
|
| exact real decode batch mean | 3.890 | 10.921 | — |
|
||||||
|
| Frontier decode batch mean | 13.537 | 18.938 | — |
|
||||||
|
|
||||||
|
State composition 将 config contrast 推动 `+30.9690 ms`,已经足以解释并反转原错误;
|
||||||
|
exact-state-matched contrast 相对 frozen TPOT 还剩 `+3.9528 ms`,相对同次 exact iteration
|
||||||
|
实验还剩 `+1.7851 ms`。后者是 conditional operator/graph/measurement residual 的上界,而不是
|
||||||
|
当前 ranking error 的主因。
|
||||||
|
|
||||||
|
同 batch 扫描显示 Frontier 从 B1 起就预测 TP8 更快,但 B4→B5 是 TP4 误差被显著放大的拐点:
|
||||||
|
TP4 模拟 critical path 从 `30.4058` 跳至 `54.4194 ms`,其中 MoE work estimate 从
|
||||||
|
`9.9788` 跳至 `34.6845 ms`;真机 pure-decode iteration 只从 `14.2337` 增至
|
||||||
|
`16.1614 ms`。TP4 grouped-MoE 的 rounded effective tokens 在此从 4 变 8,跨 94 层放大 profile
|
||||||
|
cliff;TP8/EP8 的对应曲线更平滑。被高估的 TP4 step latency 又在 simulator 闭环中积累更多
|
||||||
|
active requests,使其大部分时间落入更高 token bucket,形成正反馈。
|
||||||
|
|
||||||
|
Component trace 只作为 non-additive work estimate:在 simulator 自身状态上,MoE compute
|
||||||
|
给 TP8 `-32.4544 ms` 优势,attention `-7.4619 ms`,TP/DP communication `-6.5711 ms`;
|
||||||
|
EP communication 反而给 TP8 `+6.2354 ms` penalty。Frontier 没有单独暴露 graph bucket、padding
|
||||||
|
或 launch-overhead 字段,因此 graph 影响仍折叠在 predictor 中,不能从本实验单独归因。
|
||||||
|
|
||||||
|
真机 exact-state TP4 为单 server 运行,而冻结 TP4 TPOT 来自同机两个四卡 server;TP8 两者均为
|
||||||
|
单 server。因此 exact-state run 只用于机制诊断,冻结三轮仍是主 E2E 对比。分析产物位于
|
||||||
|
`/home/admin/cpfs/wjh/aituner/qwen235-fixed-pd-state-diagnosis-20260719-r1/analysis/state-diagnosis-exact.{json,md}`;
|
||||||
|
两配置 replay scorer 与 frozen A1 request metrics byte-identical。
|
||||||
|
|||||||
Reference in New Issue
Block a user