41 lines
4.2 KiB
Markdown
41 lines
4.2 KiB
Markdown
# EXP-SIMFID-Q30-GRAPH-PIECEWISE：graph-compatible kernel-only profile 是否修正 Frontier trace replay？
|
||
|
||
> **状态：** approved and running（2026-07-17）。本卡是已纠正 prefix-trace contract 后的最小判别实验；不复用此前 `decode_cuda_graph_mode=none` 的数值作 fidelity verdict。
|
||
|
||
## Purpose and hypotheses
|
||
|
||
- **Parent claim：** Frontier 是否已经足以为 Qwen3-30B-A3B 的真实 trace serving surface 选择 config。
|
||
- **Question：** 旧 Frontier replay 低估 decode service rate，是否主要是 simulator 使用 `none` 而真机使用 `FULL_AND_PIECEWISE`、并且没有向 Frontier 提供独立 `KERNEL_ONLY` profile family？
|
||
- **G1 (graph-family omission)：** 用同一 vLLM 0.20/FA3/FlashInfer-CUTLASS stack 的 `RecordFunctionTracer` kernel-only measurements，加真实 capture buckets 和 Frontier `piecewise`，会显著缩小 TP2/MNS16 的 TPOT/service-rate gap，并至少改变一个 config 的 latency ranking。
|
||
- **G2 (remaining composition error)：** 即使 graph family 对齐，TPOT、TTFT 或 E2E ranking 仍与真机不一致；则 graph omission 只是必要修正，不是 simulator 已解决 tuning 的证据。
|
||
|
||
## Controlled setup
|
||
|
||
| Item | Frozen choice |
|
||
|---|---|
|
||
| model/runtime/hardware | Qwen3-30B-A3B BF16; community vLLM 0.20.0 (`88d34c…`); dash0 NVIDIA H20 |
|
||
| simulator | Frontier `deadc4a321f0baaa534c6ebd17f974123733cdc2`; no local source patch |
|
||
| workload | exact 129-request Trace-PD public projection; exact ISL/OSL/arrival order; TP-normalized arrival time and complete 16-token prefix blocks |
|
||
| surface | TP in {1,2,4}; MNS in {8,16,32,64}; MBT=8192; prefix/chunked prefill on |
|
||
| real graph contract | observed vLLM capture sizes: MNS8=[1,2,4,8,16], MNS16=[1,2,4,8,16,24,32], MNS32=[1,2,4,8,16,24,32,40,48,56,64], MNS64=[1,2,4,8,16,24,32,40,48,56,64,72,80,88,96,104,112,120,128] |
|
||
| profile intervention | CUDA-event profile stays frozen for prefill/mixed batches. New `KERNEL_ONLY` linear, FA3 decode + KV-update, MoE, and router rows use Frontier's actual `RecordFunctionTracer` semantics; no relabeling of CUDA-event numbers. |
|
||
| exact capacity | per-cell real observed KV block count and capture list; Frontier CPU-overhead model remains disabled on both old/new simulator runs because the intervention is GPU-kernel family only. |
|
||
|
||
Frontier source inspection fixes the semantic boundary: `piecewise` emits `PIECEWISE` whenever a capture hits, but the MONOLITHIC predictor selects `KERNEL_ONLY` only when `num_prefill_tokens == 0`. Hence new profile coverage is pure decode only; captured mixed/prefill work continues to consume the existing CUDA-event family.
|
||
|
||
## Measurement and decision rule
|
||
|
||
- **Primary outputs：** per-config mean/p90 TTFT, TPOT, E2E; ranking for each metric; TP2/MNS16 per-request TPOT gap against the already frozen three-trial real audit.
|
||
- **Validity gates：** every kernel CSV hash matches its manifest; every row says `KERNEL_ONLY`; every TP/capture-bucket/KV-context required by the runner is present; command records `piecewise`, per-cell blocks and capture sizes; each simulator cell completes all 129 requests.
|
||
- **Decision:** G1 is supported only if the graph-aligned TP2/MNS16 TPOT median moves toward real **and** full-surface rank/error evidence improves. A single-cell timing improvement does not establish tuning sufficiency. If G2 holds, update the research claim to “Frontier has not solved tuning under trace-faithful MoE serving after graph-family alignment,” then profile stage/state composition rather than add arbitrary kernel rows.
|
||
|
||
## Expected figure
|
||
|
||
`graph-piecewise-profile-prototype.svg` is deliberately schematic. The final figure uses the same axes and adds real data only after the profile and replay validity gates pass.
|
||
|
||
## Cost and provenance
|
||
|
||
- **GPU cost:** three 1-GPU FA3 decode profile shards, plus one 1-GPU linear shard and one 1-GPU MoE/router shard; expected 1.5--3.0 H20-GPU-hours, hard cap 4.0 GPU-hours.
|
||
- **CPU cost:** 12 exact-trace simulations, expected 20--45 CPU minutes; a one-cell TP2/MNS16 smoke precedes the full surface.
|
||
- **Calibration separation:** kernel microprofiles are independent measurements, never fitted to trace E2E latency. The frozen real trace audit is evaluation only.
|