7.4 KiB
EXP-SIMFID-Q30-GRAPH-PIECEWISE：graph-compatible kernel-only profile 是否修正 Frontier trace replay？
状态： completed（2026-07-17）。本卡是已纠正 prefix-trace contract 后的最小判别实验；不复用此前
decode_cuda_graph_mode=none的数值作 fidelity verdict。
Purpose and hypotheses
- Parent claim： Frontier 是否已经足以为 Qwen3-30B-A3B 的真实 trace serving surface 选择 config。
- Question： 旧 Frontier replay 低估 decode service rate，是否主要是 simulator 使用
none而真机使用FULL_AND_PIECEWISE、并且没有向 Frontier 提供独立KERNEL_ONLYprofile family？ - G1 (graph-family omission)： 用同一 vLLM 0.20/FA3/FlashInfer-CUTLASS stack 的
RecordFunctionTracerkernel-only measurements，加真实 capture buckets 和 Frontierpiecewise，会显著缩小 TP2/MNS16 的 TPOT/service-rate gap，并至少改变一个 config 的 latency ranking。 - G2 (remaining composition error)： 即使 graph family 对齐，TPOT、TTFT 或 E2E ranking 仍与真机不一致；则 graph omission 只是必要修正，不是 simulator 已解决 tuning 的证据。
Controlled setup
| Item | Frozen choice |
|---|---|
| model/runtime/hardware | Qwen3-30B-A3B BF16; community vLLM 0.20.0 (88d34c…); dash0 NVIDIA H20 |
| simulator | Frontier deadc4a321f0baaa534c6ebd17f974123733cdc2; no local source patch |
| workload | exact 129-request Trace-PD public projection; exact ISL/OSL/arrival order; TP-normalized arrival time and complete 16-token prefix blocks |
| surface | TP in {1,2,4}; MNS in {8,16,32,64}; MBT=8192; prefix/chunked prefill on |
| real graph contract | observed vLLM capture sizes: MNS8=[1,2,4,8,16], MNS16=[1,2,4,8,16,24,32], MNS32=[1,2,4,8,16,24,32,40,48,56,64], MNS64=[1,2,4,8,16,24,32,40,48,56,64,72,80,88,96,104,112,120,128] |
| profile intervention | CUDA-event profile stays frozen for prefill/mixed batches. New KERNEL_ONLY linear, FA3 decode + KV-update, MoE, and router rows use Frontier's actual RecordFunctionTracer semantics; no relabeling of CUDA-event numbers. |
| exact capacity | per-cell real observed KV block count and capture list; Frontier CPU-overhead model remains disabled on both old/new simulator runs because the intervention is GPU-kernel family only. |
Frontier source inspection fixes the semantic boundary: piecewise emits PIECEWISE whenever a capture hits, but the MONOLITHIC predictor selects KERNEL_ONLY only when num_prefill_tokens == 0. Hence new profile coverage is pure decode only; captured mixed/prefill work continues to consume the existing CUDA-event family.
Measurement and decision rule
- Primary outputs： per-config mean/p90 TTFT, TPOT, E2E; ranking for each metric; TP2/MNS16 per-request TPOT gap against the already frozen three-trial real audit.
- Validity gates： every kernel CSV hash matches its manifest; every row says
KERNEL_ONLY; every TP/capture-bucket/KV-context required by the runner is present; command recordspiecewise, per-cell blocks and capture sizes; each simulator cell completes all 129 requests. - Decision: G1 is supported only if the graph-aligned TP2/MNS16 TPOT median moves toward real and full-surface rank/error evidence improves. A single-cell timing improvement does not establish tuning sufficiency. If G2 holds, update the research claim to “Frontier has not solved tuning under trace-faithful MoE serving after graph-family alignment,” then profile stage/state composition rather than add arbitrary kernel rows.
Result (completed 2026-07-17)
Validity gates
- The fresh profile family is exclusive
KERNEL_ONLY: linear=57, attention=456, MoE=57 rows. Attention covers every TP in {1,2,4}, all MNS64 capture buckets, and the eight decode KV contexts; the final manifest records Frontierdeadc4a…. - All three TP traces contain the same 129 ISL/OSL requests and prompt semantic vector. Their offered rates are 0.215/0.430/0.860 req/s for TP=1/2/4, while the per-GPU rate is fixed. Prefix IDs use
floor(ISL/16): 36,321 legal complete blocks, not the invalid 36,443ceilprojection. - All 12 Frontier cells completed 129/129 requests. No result has a Frontier error or scheduler stall. Commands record
piecewise, the observed capture list, the real KV-block count, frozen CUDA-event prefill/mixed CSVs, and the new kernel-only CSVs.
Frontier versus real vLLM selection
| Target | Frontier winner | Real winner | Pairwise order agreement |
|---|---|---|---|
| TTFT mean / p90 | TP4/MNS64 / TP4/MNS64 | TP4/MNS64 / TP4/MNS64 | 56/66 (84.8%) / 56/66 (84.8%) |
| TPOT mean / p90 | TP4/MNS8 / TP4/MNS8 | TP4/MNS8 / TP4/MNS8 | 64/66 (97.0%) / 64/66 (97.0%) |
| E2E mean / p90 | TP4/MNS64 / TP4/MNS64 | TP4/MNS64 / TP4/MNS64 | 61/66 (92.4%) / 55/66 (83.3%) |
Thus, for this one Qwen3-30B-A3B Trace-PD surface, graph-aligned Frontier selects the same argmin config for all six reported latency objectives. This is evidence for selection fidelity on this surface, not a general claim about dense models, fixed-shape traffic, prefill-only traffic, another serving engine, or SLO capacity.
Absolute latency is still unusable
The same simulator is not calibrated for absolute latency. Across all 12 cells, sim/real ratios are: TTFT mean 8.13--510.88x (median 38.29x), TPOT mean 4.06--5.72x (median 4.86x), and E2E mean 4.66--13.99x (median 8.24x). At the TP2/MNS16 anchor, Frontier reports mean/p90 TTFT=606,885/1,194,098 ms, TPOT=64.42/66.54 ms, E2E=828,082/1,404,520 ms; real vLLM reports 34,443/77,922 ms, 14.10/16.15 ms, and 82,937/137,726 ms, respectively.
The TPOT error is service-time error, not merely queueing: Frontier's tpot_computation equals its TPOT at the anchor. The large TTFT/E2E error then amplifies through the simulated queue. The profile rows themselves are measured kernel rows, so the next diagnosis must inspect Frontier's layer/operation composition and batching-state model rather than relabeling or adding arbitrary graph kernels.
Decision
G1 is only partially supported: piecewise plus a true kernel-only decode family improves the prior none smoke's 96.37 ms TPOT median to 65.60 ms, but leaves a large absolute gap. G2 is rejected for the argmin selection question on this surface because all six winners match. The defensible claim is therefore:
Frontier has solved config selection for this evaluated MoE trace surface, despite failing absolute latency prediction; it has not yet been shown to solve the broader tuning problem.
The next discriminating tests are the remaining planned dense/MoE × fixed-shape/trace-faithful × prefill+decode/prefill-only surfaces. In parallel, diagnose the decode service-time composition error; do not use Frontier's current absolute values for SLO, cost, or capacity claims.
Artifacts: /home/admin/cpfs/wjh/aituner/graph-piecewise-qwen30-20260717/full/frozen-kernel-only/manifest.json and /home/admin/cpfs/wjh/aituner/graph-piecewise-qwen30-20260717/simulator-piecewise-surface-v2/analysis/{comparison.json,comparison.md}.
Cost and provenance
- GPU cost: three 1-GPU FA3 decode profile shards, plus one 1-GPU linear shard and one 1-GPU MoE/router shard; expected 1.5--3.0 H20-GPU-hours, hard cap 4.0 GPU-hours.
- CPU cost: 12 exact-trace simulations, expected 20--45 CPU minutes; a one-cell TP2/MNS16 smoke precedes the full surface.
- Calibration separation: kernel microprofiles are independent measurements, never fitted to trace E2E latency. The frozen real trace audit is evaluation only.