diff --git a/runs/frontier-fidelity-envelope-v1/graph-piecewise-experiment-card.md b/runs/frontier-fidelity-envelope-v1/graph-piecewise-experiment-card.md index 272fe9c..6a1e0e3 100644 --- a/runs/frontier-fidelity-envelope-v1/graph-piecewise-experiment-card.md +++ b/runs/frontier-fidelity-envelope-v1/graph-piecewise-experiment-card.md @@ -1,6 +1,6 @@ # EXP-SIMFID-Q30-GRAPH-PIECEWISE:graph-compatible kernel-only profile 是否修正 Frontier trace replay? -> **状态:** approved and running(2026-07-17)。本卡是已纠正 prefix-trace contract 后的最小判别实验;不复用此前 `decode_cuda_graph_mode=none` 的数值作 fidelity verdict。 +> **状态:** completed(2026-07-17)。本卡是已纠正 prefix-trace contract 后的最小判别实验;不复用此前 `decode_cuda_graph_mode=none` 的数值作 fidelity verdict。 ## Purpose and hypotheses @@ -29,9 +29,39 @@ Frontier source inspection fixes the semantic boundary: `piecewise` emits `PIECE - **Validity gates:** every kernel CSV hash matches its manifest; every row says `KERNEL_ONLY`; every TP/capture-bucket/KV-context required by the runner is present; command records `piecewise`, per-cell blocks and capture sizes; each simulator cell completes all 129 requests. - **Decision:** G1 is supported only if the graph-aligned TP2/MNS16 TPOT median moves toward real **and** full-surface rank/error evidence improves. A single-cell timing improvement does not establish tuning sufficiency. If G2 holds, update the research claim to “Frontier has not solved tuning under trace-faithful MoE serving after graph-family alignment,” then profile stage/state composition rather than add arbitrary kernel rows. -## Expected figure +## Result (completed 2026-07-17) -`graph-piecewise-profile-prototype.svg` is deliberately schematic. The final figure uses the same axes and adds real data only after the profile and replay validity gates pass. +### Validity gates + +- The fresh profile family is exclusive `KERNEL_ONLY`: linear=57, attention=456, MoE=57 rows. Attention covers every TP in {1,2,4}, all MNS64 capture buckets, and the eight decode KV contexts; the final manifest records Frontier `deadc4a…`. +- All three TP traces contain the same 129 ISL/OSL requests and prompt semantic vector. Their offered rates are 0.215/0.430/0.860 req/s for TP=1/2/4, while the per-GPU rate is fixed. Prefix IDs use `floor(ISL/16)`: 36,321 legal complete blocks, not the invalid 36,443 `ceil` projection. +- All 12 Frontier cells completed 129/129 requests. No result has a Frontier error or scheduler stall. Commands record `piecewise`, the observed capture list, the real KV-block count, frozen CUDA-event prefill/mixed CSVs, and the new kernel-only CSVs. + +### Frontier versus real vLLM selection + +| Target | Frontier winner | Real winner | Pairwise order agreement | +|---|---|---|---:| +| TTFT mean / p90 | TP4/MNS64 / TP4/MNS64 | TP4/MNS64 / TP4/MNS64 | 56/66 (84.8%) / 56/66 (84.8%) | +| TPOT mean / p90 | TP4/MNS8 / TP4/MNS8 | TP4/MNS8 / TP4/MNS8 | 64/66 (97.0%) / 64/66 (97.0%) | +| E2E mean / p90 | TP4/MNS64 / TP4/MNS64 | TP4/MNS64 / TP4/MNS64 | 61/66 (92.4%) / 55/66 (83.3%) | + +Thus, for this one Qwen3-30B-A3B Trace-PD surface, graph-aligned Frontier selects the same argmin config for all six reported latency objectives. This is evidence for *selection fidelity on this surface*, not a general claim about dense models, fixed-shape traffic, prefill-only traffic, another serving engine, or SLO capacity. + +### Absolute latency is still unusable + +The same simulator is not calibrated for absolute latency. Across all 12 cells, sim/real ratios are: TTFT mean 8.13--510.88x (median 38.29x), TPOT mean 4.06--5.72x (median 4.86x), and E2E mean 4.66--13.99x (median 8.24x). At the TP2/MNS16 anchor, Frontier reports mean/p90 TTFT=606,885/1,194,098 ms, TPOT=64.42/66.54 ms, E2E=828,082/1,404,520 ms; real vLLM reports 34,443/77,922 ms, 14.10/16.15 ms, and 82,937/137,726 ms, respectively. + +The TPOT error is service-time error, not merely queueing: Frontier's `tpot_computation` equals its TPOT at the anchor. The large TTFT/E2E error then amplifies through the simulated queue. The profile rows themselves are measured kernel rows, so the next diagnosis must inspect Frontier's layer/operation composition and batching-state model rather than relabeling or adding arbitrary graph kernels. + +### Decision + +G1 is only partially supported: `piecewise` plus a true kernel-only decode family improves the prior `none` smoke's 96.37 ms TPOT median to 65.60 ms, but leaves a large absolute gap. G2 is rejected for the *argmin selection* question on this surface because all six winners match. The defensible claim is therefore: + +> Frontier has solved config selection for this evaluated MoE trace surface, despite failing absolute latency prediction; it has **not** yet been shown to solve the broader tuning problem. + +The next discriminating tests are the remaining planned dense/MoE × fixed-shape/trace-faithful × prefill+decode/prefill-only surfaces. In parallel, diagnose the decode service-time composition error; do not use Frontier's current absolute values for SLO, cost, or capacity claims. + +Artifacts: `/home/admin/cpfs/wjh/aituner/graph-piecewise-qwen30-20260717/full/frozen-kernel-only/manifest.json` and `/home/admin/cpfs/wjh/aituner/graph-piecewise-qwen30-20260717/simulator-piecewise-surface-v2/analysis/{comparison.json,comparison.md}`. ## Cost and provenance