aituner/runs/simulator-tuning-latency-matrix-v0/qwen30-tracepd-u0p01-card.md

Qwen3-30B-A3B Trace-PD u0p01 latency-selection comparison

Status: the 36-run real surface is valid. The initially reported Frontier surface is invalidated for simulator fidelity/selection by a post-hoc prefix-cache trace-contract audit (2026-07-17): its simulator-facing block metadata includes non-cacheable partial final blocks. The raw CPU artifacts are retained for diagnosis, but are not Frontier evidence. This remains the first MoE case of EXP-SIM-TUNING-LATENCY-MATRIX-V0, not a claim about the remaining three workload classes.

Question and decision

Can the frozen Frontier surface select the same low-latency Qwen3-30B-A3B configuration as a fresh community-vLLM deployment when both replay exactly the same trace-derived prefill+decode request vector? The objective contains no SLO or capacity threshold: lower mean/p90 TTFT, TPOT, and E2E is better.

The result is a bounded Trace-PD result only. It cannot establish a claim about Fixed-P, Fixed-PD, Trace-P, other trace rates, another model, or a modified Frontier.

Frozen contract

Item	Value
Hardware	dash0, H20, one replica; fresh vLLM server per config/trial
Model / engine	/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B; community vLLM 0.20.0+cu129, BF16
Config surface	TP ∈ {1,2,4} × MNS ∈ {8,16,32,64}; DP=PP=EP=1, MBT=8192, GPU-memory utilization 0.92, chunked prefill and prefix caching on
Trace	trace-exact-v1, anchor u0p01; 129 requests, original input/output length, order and session are preserved. The first Frontier materialization also copied every 16-token runtime identity, including a final partial block; that is not a legal vLLM prefix-cache-hit representation and invalidates this simulator attempt. For a config of tensor parallelism TP, every arrival is transformed as t′ = t / TP; the TP1/2/4 traces therefore end at 597.037/298.519/149.259 s.
Offered load	Base rate is 129 / 600 = 0.215 req/s. TP1/2/4 replay at 0.215/0.430/0.860 req/s globally, respectively, so all three topologies receive 0.215 req/s/GPU. This is a per-GPU-normalized trace, not an unchanged-wall-clock trace.
Simulator baseline	The existing same-global-rate T1 surface is excluded: it evaluates TP2/TP4 at lower per-GPU load. A new Frontier surface must use the identical TP-normalized arrival vector for each topology before coverage or latency can be reported.
Trials	Three fresh-server trials per config; independent warmup excluded from the measured trace

Validity gates

1. Each real trial must complete all 129 requests, preserve exact prompt and completion-token usage, and have the TP-normalized row-vector digest.
2. A simulator metric is valid only if its cell has all 129 finite request metrics with matching request ID, prompt/output lengths and transformed arrival vector. A simulator scheduler_stall is a coverage result, never an infeasible-latency label.
3. TTFT, TPOT and E2E are kept in milliseconds. TPOT is recorded only for outputs longer than one token (all requests in this Trace-PD anchor qualify).
4. A failed real run remains an infeasible/safety observation; it is not silently converted to infinite latency or omitted from ranking.

Analysis and decision rule

For every config × trial, retain raw per-request records and compute mean and nearest-rank p90 for TTFT, TPOT and E2E. Report pooled 387-request values alongside the three per-trial values. A selection-stability bootstrap is deferred; it is not needed to evaluate Frontier's coverage gate.

The primary gate is Frontier decision-valid coverage on all 12 cells. If any real-valid candidate lacks a legal simulator metric, Frontier receives no full-surface selection, top-set overlap, regret, or Kendall-tau score. We will still show all valid per-cell latency comparisons and the complete real surface, so any coverage failure and its ground truth remain auditable.

The simulator's pre-existing SLO fields do not influence request generation, scheduling, or this no-SLO analysis; they are excluded from scoring.

Post-hoc Frontier trace-contract audit (2026-07-17)

The CPU attempt initially passed only a syntactic trace check (row count, IDs, lengths, and SHA). It did not validate the semantic invariant needed by vLLM prefix caching: only completed 16-token blocks may be cache hits.

• The trace exporter emitted ceil(ISL / 16) identities. Thus 122/129 prompts contained one partial final block; the public trace has 36,443 identities, while a full-block Frontier/vLLM cache adapter must expose 36,321.
• Exact community-vLLM 0.20 source (vllm.v1.core.kv_cache_manager) documents that computed prefix blocks "must be full" and bounds hits by prompt_length - 1. Frontier's fixture contract only requires a non-empty block_hash_ids field and does not validate this full-block invariant.
• Frontier's prefix manager converts every hit to len(blocks) * 16. For the final TP2/MNS16 state, requests 89/111/117 had (ISL, IDs) of (722,46), (678,43), and (706,45). If all supplied identities hit, it obtains 736/688/720 computed tokens and hence num_new_tokens of -14/-10/-14. The scheduler preserves zero-or-negative-token requests in its waiting queue; no event remains, so the simulator reports Sequential simulation ended with non-empty scheduler state.

This proves a trace-adapter/Frontier prefix-cache semantic mismatch, not a GPU capacity failure: a targeted CPU hook found no failed KV allocation and no PP-admission deferral, only the three non-admitted requests with a full 8192-token batch budget. The correct next input is a separate Frontier adapter that exports only floor(ISL / 16) complete prompt-block identities; it must not modify the private real trace or its input/output/arrival vector.

Invalid raw Frontier attempt (2026-07-17; retained for diagnosis only)

The TP-normalized surface completed in 156 seconds of CPU-only wall time on dash0, with Frontier commit deadc4a321f0baaa534c6ebd17f974123733cdc2. All twelve input traces passed the 129-request, ID/shape, and TP-specific trace-SHA checks. Only two cells reached valid request metrics; the other ten ended in scheduler_stall.

Config	Status	mean / p90 TTFT (ms)	mean / p90 TPOT (ms)	mean / p90 E2E (ms)
TP1/MNS32	complete	440032 / 972628	145.8 / 151.5	936301 / 1645895
TP1/MNS64	complete	174335 / 510962	163.6 / 178.9	724328 / 1285374
remaining 10 configs	scheduler_stall	N/A	N/A	N/A

The apparent 2/12 (16.7%) coverage is not a Frontier coverage result: both the ten stalls and the two completed cells consumed the invalid partial block metadata. In particular, the two completed cells may receive false prefix hits. No latency, ranking, coverage, top-set overlap, regret, or Kendall-tau statement about Frontier may be derived from this attempt.

The legacy surface runner writes an offered-rate summary assuming a fixed 600-second window. That field is not used here because TP2/TP4 have compressed arrival clocks. The materialized trace manifests are authoritative: they record global 0.215/0.430/0.860 req/s and the matched 0.215 req/s/GPU rate.

Corrected-adapter CPU liveness probe (2026-07-17)

One deliberately narrow CPU-only rerun tested the causal fix before launching another surface: TP2/MNS16, the same 129-request Trace-PD input, MBT=8192, prefix caching and chunked prefill enabled, and the original Frontier commit. It changed only the simulator-facing prefix projection from ceil(ISL / 16) to floor(ISL / 16); the private real replay remained bytewise unchanged. The materialized trace records 36,321 complete blocks (36,443 raw runtime identities; 122 prompts with a discarded partial final block), global arrival rate 0.430 req/s, and the required 0.215 req/s/GPU.

The run completed 129/129 requests in 105.8 CPU wall-clock seconds with no stall. This is a direct liveness validation of the diagnosis above: the partial-block adapter, rather than KV capacity, caused the old deadlock.

It is not a valid latency or selection result. Even after the repair, Frontier reports p50 TTFT=946,973 ms, p50 TPOT=96.37 ms, p50 E2E=1,305,128 ms, and 112.2 generated decode tokens/s. The trace supplies 460,490 output tokens over a 298.5-s TP2 arrival horizon (about 1,543 output tokens/s), so the simulated service rate is below the offered decode work and its queue must grow. The corresponding real TP2/MNS16 replay has mean/p90 TPOT 14.1/16.1 ms and mean/p90 TTFT 34,443/77,922 ms. Thus the post-fix discrepancy is a decode-service-model / runtime-semantics question, not residual evidence from the invalid cache metadata. The probe artifact is /home/admin/cpfs/wjh/aituner/simulator-tuning-latency-q30-tp-normalized-u0p01-20260717/frontier-contract-probe-v1.

Real result (2026-07-17)

All 36/36 config × trial replays completed with exit code zero. Each trial passed all validity gates: 129/129 successful requests, exact requested and observed input/output token usage, the TP-specific private-trace SHA and normalized arrival-vector SHA, and HTTP routing to the explicit alias qwen3-30b-exact-trace. The cache-populating smoke and earlier failed launch attempts are excluded from this table. Values pool 387 requests/config (three trials); p90 is nearest-rank.

Config	mean / p90 TTFT (ms)	mean / p90 TPOT (ms)	mean / p90 E2E (ms)
TP1/MNS8	88479.9 / 168185.5	14.3 / 16.3	137996.4 / 222136.0
TP1/MNS16	17378.2 / 42739.8	20.6 / 25.3	87355.2 / 156177.2
TP1/MNS32	620.5 / 1931.5	23.8 / 30.0	81051.2 / 162908.6
TP1/MNS64	619.8 / 1956.7	24.1 / 30.4	82253.7 / 165355.3
TP2/MNS8	95920.1 / 183878.8	9.2 / 10.2	128296.9 / 215848.3
TP2/MNS16	34443.0 / 77922.4	14.1 / 16.1	82936.7 / 137726.4
TP2/MNS32	1050.6 / 2521.8	18.0 / 21.3	62267.5 / 117239.3
TP2/MNS64	375.8 / 1148.6	18.2 / 21.6	62353.6 / 118392.3
TP4/MNS8	102605.9 / 193359.7	6.8 / 7.2	126405.4 / 209092.4
TP4/MNS16	35042.6 / 74281.3	8.8 / 9.6	65779.1 / 110374.3
TP4/MNS32	6296.2 / 19159.9	12.2 / 13.8	47944.5 / 83825.7
TP4/MNS64	246.0 / 685.5	13.2 / 15.4	44985.1 / 83763.6

Thus real-vLLM chooses TP4/MNS64 for mean/p90 TTFT and E2E, and TP4/MNS8 for mean/p90 TPOT. These are different single-metric objectives, not a claim that one configuration simultaneously optimizes all three.

Correct conclusion and next gate

This run establishes two bounded findings, neither of which is a full Frontier selection verdict: (1) the initial simulator-facing Trace-PD prefix contract was invalid, and (2) its corrected TP2/MNS16 liveness probe estimates a much smaller decode service rate than the real system. The valid real surface remains a frozen ground truth, but all raw Frontier numbers above are excluded from the research claim. We therefore retract the prior bounded counterexample and do not yet know whether Frontier selects the correct config on this case.

Before a CPU-only Frontier rerun, the comparison must additionally record these remaining alignment gaps:

1. real vLLM uses FULL_AND_PIECEWISE CUDA graphs, whereas the raw Frontier command explicitly used --decode_cuda_graph_mode none; Frontier exposes full_decode_only and piecewise, not the identical combined mode. More importantly, Frontier chooses kernel-only predictor families whenever this mode is non-none, so a graph-compatible rerun also requires corresponding kernel-only profile coverage rather than merely changing the CLI flag;
2. Frontier used one explicit KV-block count per TP, while real vLLM's graph capture changes the count with MNS (the mismatch is small but measurable);
3. Frontier skipped CPU-overhead modeling, and its MoE CSV has only standalone_legacy gating rows, so the code warned and fell back rather than training the requested prefill_hot pseudo-model;
4. the runner's rate metadata incorrectly fixed the TP2/TP4 numerator to a 600-second window. Arrival timestamps supplied to the simulator were correct, so this did not alter the raw execution, but the metadata must be fixed before reporting a rerun.

The next admissible Frontier result uses complete-block prefix metadata, per-TP×MNS real KV capacity, correct TP-normalized rate metadata, and an explicit CUDA-graph compatibility decision. Only a complete, semantically valid surface may receive the selection metrics. Any remaining mismatch after that CPU rerun is then evidence about Frontier's scheduler/profile fidelity, not this adapter.

Cost and output

The measured replay horizons are 597/299/149 s at TP1/2/4. Including five minutes of per-launch server start/warmup, 36 launches consume about 13 H20-GPUh nominally; applying the launcher's historical 12--35 minute fresh-server envelope yields an upper bound of 41 H20-GPUh. A separate CPU-only simulator rerun is mandatory before this GPU stage; no profile collection is included.

The remote, experiment-specific output root is /home/admin/cpfs/wjh/aituner/simulator-tuning-latency-q30-tp-normalized-u0p01-20260717. It contains the prompt-free real audit JSON/Markdown, commands, environment, GPU/trace/model hashes, raw prompt-free request records, and cache inventory. Private prompt text remains on dash0 and is never copied into the repository.