Align Frontier piecewise graph profiles
This commit is contained in:
@@ -0,0 +1,40 @@
|
||||
# EXP-SIMFID-Q30-GRAPH-PIECEWISE:graph-compatible kernel-only profile 是否修正 Frontier trace replay?
|
||||
|
||||
> **状态:** approved and running(2026-07-17)。本卡是已纠正 prefix-trace contract 后的最小判别实验;不复用此前 `decode_cuda_graph_mode=none` 的数值作 fidelity verdict。
|
||||
|
||||
## Purpose and hypotheses
|
||||
|
||||
- **Parent claim:** Frontier 是否已经足以为 Qwen3-30B-A3B 的真实 trace serving surface 选择 config。
|
||||
- **Question:** 旧 Frontier replay 低估 decode service rate,是否主要是 simulator 使用 `none` 而真机使用 `FULL_AND_PIECEWISE`、并且没有向 Frontier 提供独立 `KERNEL_ONLY` profile family?
|
||||
- **G1 (graph-family omission):** 用同一 vLLM 0.20/FA3/FlashInfer-CUTLASS stack 的 `RecordFunctionTracer` kernel-only measurements,加真实 capture buckets 和 Frontier `piecewise`,会显著缩小 TP2/MNS16 的 TPOT/service-rate gap,并至少改变一个 config 的 latency ranking。
|
||||
- **G2 (remaining composition error):** 即使 graph family 对齐,TPOT、TTFT 或 E2E ranking 仍与真机不一致;则 graph omission 只是必要修正,不是 simulator 已解决 tuning 的证据。
|
||||
|
||||
## Controlled setup
|
||||
|
||||
| Item | Frozen choice |
|
||||
|---|---|
|
||||
| model/runtime/hardware | Qwen3-30B-A3B BF16; community vLLM 0.20.0 (`88d34c…`); dash0 NVIDIA H20 |
|
||||
| simulator | Frontier `deadc4a321f0baaa534c6ebd17f974123733cdc2`; no local source patch |
|
||||
| workload | exact 129-request Trace-PD public projection; exact ISL/OSL/arrival order; TP-normalized arrival time and complete 16-token prefix blocks |
|
||||
| surface | TP in {1,2,4}; MNS in {8,16,32,64}; MBT=8192; prefix/chunked prefill on |
|
||||
| real graph contract | observed vLLM capture sizes: MNS8=[1,2,4,8,16], MNS16=[1,2,4,8,16,24,32], MNS32=[1,2,4,8,16,24,32,40,48,56,64], MNS64=[1,2,4,8,16,24,32,40,48,56,64,72,80,88,96,104,112,120,128] |
|
||||
| profile intervention | CUDA-event profile stays frozen for prefill/mixed batches. New `KERNEL_ONLY` linear, FA3 decode + KV-update, MoE, and router rows use Frontier's actual `RecordFunctionTracer` semantics; no relabeling of CUDA-event numbers. |
|
||||
| exact capacity | per-cell real observed KV block count and capture list; Frontier CPU-overhead model remains disabled on both old/new simulator runs because the intervention is GPU-kernel family only. |
|
||||
|
||||
Frontier source inspection fixes the semantic boundary: `piecewise` emits `PIECEWISE` whenever a capture hits, but the MONOLITHIC predictor selects `KERNEL_ONLY` only when `num_prefill_tokens == 0`. Hence new profile coverage is pure decode only; captured mixed/prefill work continues to consume the existing CUDA-event family.
|
||||
|
||||
## Measurement and decision rule
|
||||
|
||||
- **Primary outputs:** per-config mean/p90 TTFT, TPOT, E2E; ranking for each metric; TP2/MNS16 per-request TPOT gap against the already frozen three-trial real audit.
|
||||
- **Validity gates:** every kernel CSV hash matches its manifest; every row says `KERNEL_ONLY`; every TP/capture-bucket/KV-context required by the runner is present; command records `piecewise`, per-cell blocks and capture sizes; each simulator cell completes all 129 requests.
|
||||
- **Decision:** G1 is supported only if the graph-aligned TP2/MNS16 TPOT median moves toward real **and** full-surface rank/error evidence improves. A single-cell timing improvement does not establish tuning sufficiency. If G2 holds, update the research claim to “Frontier has not solved tuning under trace-faithful MoE serving after graph-family alignment,” then profile stage/state composition rather than add arbitrary kernel rows.
|
||||
|
||||
## Expected figure
|
||||
|
||||
`graph-piecewise-profile-prototype.svg` is deliberately schematic. The final figure uses the same axes and adds real data only after the profile and replay validity gates pass.
|
||||
|
||||
## Cost and provenance
|
||||
|
||||
- **GPU cost:** three 1-GPU FA3 decode profile shards, plus one 1-GPU linear shard and one 1-GPU MoE/router shard; expected 1.5--3.0 H20-GPU-hours, hard cap 4.0 GPU-hours.
|
||||
- **CPU cost:** 12 exact-trace simulations, expected 20--45 CPU minutes; a one-cell TP2/MNS16 smoke precedes the full surface.
|
||||
- **Calibration separation:** kernel microprofiles are independent measurements, never fitted to trace E2E latency. The frozen real trace audit is evaluation only.
|
||||
Reference in New Issue
Block a user