Audit Frontier prefix-cache trace contract
This commit is contained in:
@@ -0,0 +1,191 @@
|
||||
# Qwen3-30B-A3B Trace-PD `u0p01` latency-selection comparison
|
||||
|
||||
> **Status:** the 36-run real surface is valid. The initially reported Frontier
|
||||
> surface is **invalidated for simulator fidelity/selection** by a post-hoc
|
||||
> prefix-cache trace-contract audit (2026-07-17): its simulator-facing block
|
||||
> metadata includes non-cacheable partial final blocks. The raw CPU artifacts
|
||||
> are retained for diagnosis, but are not Frontier evidence. This remains the
|
||||
> first MoE case of `EXP-SIM-TUNING-LATENCY-MATRIX-V0`, not a claim about the
|
||||
> remaining three workload classes.
|
||||
|
||||
## Question and decision
|
||||
|
||||
Can the frozen Frontier surface select the same low-latency Qwen3-30B-A3B
|
||||
configuration as a fresh community-vLLM deployment when both replay exactly
|
||||
the same trace-derived prefill+decode request vector? The objective contains
|
||||
no SLO or capacity threshold: lower mean/p90 TTFT, TPOT, and E2E is better.
|
||||
|
||||
The result is a bounded Trace-PD result only. It cannot establish a claim
|
||||
about Fixed-P, Fixed-PD, Trace-P, other trace rates, another model, or a
|
||||
modified Frontier.
|
||||
|
||||
## Frozen contract
|
||||
|
||||
| Item | Value |
|
||||
|---|---|
|
||||
| Hardware | `dash0`, H20, one replica; fresh vLLM server per config/trial |
|
||||
| Model / engine | `/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B`; community `vLLM 0.20.0+cu129`, BF16 |
|
||||
| Config surface | `TP ∈ {1,2,4} × MNS ∈ {8,16,32,64}`; `DP=PP=EP=1`, `MBT=8192`, GPU-memory utilization `0.92`, chunked prefill and prefix caching on |
|
||||
| Trace | `trace-exact-v1`, anchor `u0p01`; **129 requests**, original input/output length, order and session are preserved. The first Frontier materialization also copied every 16-token runtime identity, including a final partial block; that is not a legal vLLM prefix-cache-hit representation and invalidates this simulator attempt. For a config of tensor parallelism `TP`, every arrival is transformed as `t′ = t / TP`; the TP1/2/4 traces therefore end at 597.037/298.519/149.259 s. |
|
||||
| Offered load | Base rate is `129 / 600 = 0.215 req/s`. TP1/2/4 replay at 0.215/0.430/0.860 req/s globally, respectively, so all three topologies receive 0.215 req/s/GPU. This is a **per-GPU-normalized trace**, not an unchanged-wall-clock trace. |
|
||||
| Simulator baseline | The existing same-global-rate T1 surface is excluded: it evaluates TP2/TP4 at lower per-GPU load. A new Frontier surface must use the identical TP-normalized arrival vector for each topology before coverage or latency can be reported. |
|
||||
| Trials | Three fresh-server trials per config; independent warmup excluded from the measured trace |
|
||||
|
||||
## Validity gates
|
||||
|
||||
1. Each real trial must complete all 129 requests, preserve exact prompt and
|
||||
completion-token usage, and have the TP-normalized row-vector digest.
|
||||
2. A simulator metric is valid only if its cell has all 129 finite request
|
||||
metrics with matching request ID, prompt/output lengths and transformed
|
||||
arrival vector. A simulator `scheduler_stall` is a coverage result, never
|
||||
an infeasible-latency label.
|
||||
3. TTFT, TPOT and E2E are kept in milliseconds. `TPOT` is recorded only for
|
||||
outputs longer than one token (all requests in this Trace-PD anchor
|
||||
qualify).
|
||||
4. A failed real run remains an infeasible/safety observation; it is not
|
||||
silently converted to infinite latency or omitted from ranking.
|
||||
|
||||
## Analysis and decision rule
|
||||
|
||||
For every `config × trial`, retain raw per-request records and compute mean
|
||||
and nearest-rank p90 for TTFT, TPOT and E2E. Report pooled 387-request values
|
||||
alongside the three per-trial values. A selection-stability bootstrap is
|
||||
deferred; it is not needed to evaluate Frontier's coverage gate.
|
||||
|
||||
The primary gate is Frontier decision-valid coverage on all 12 cells. If any
|
||||
real-valid candidate lacks a legal simulator metric, Frontier receives no
|
||||
full-surface selection, top-set overlap, regret, or Kendall-tau score. We
|
||||
will still show all valid per-cell latency comparisons and the complete real
|
||||
surface, so any coverage failure and its ground truth remain auditable.
|
||||
|
||||
The simulator's pre-existing SLO fields do not influence request generation,
|
||||
scheduling, or this no-SLO analysis; they are excluded from scoring.
|
||||
|
||||
## Post-hoc Frontier trace-contract audit (2026-07-17)
|
||||
|
||||
The CPU attempt initially passed only a syntactic trace check (row count,
|
||||
IDs, lengths, and SHA). It did **not** validate the semantic invariant needed
|
||||
by vLLM prefix caching: only completed 16-token blocks may be cache hits.
|
||||
|
||||
- The trace exporter emitted `ceil(ISL / 16)` identities. Thus 122/129 prompts
|
||||
contained one partial final block; the public trace has 36,443 identities,
|
||||
while a full-block Frontier/vLLM cache adapter must expose 36,321.
|
||||
- Exact community-vLLM 0.20 source (`vllm.v1.core.kv_cache_manager`) documents
|
||||
that computed prefix blocks "must be full" and bounds hits by
|
||||
`prompt_length - 1`. Frontier's fixture contract only requires a non-empty
|
||||
`block_hash_ids` field and does not validate this full-block invariant.
|
||||
- Frontier's prefix manager converts every hit to `len(blocks) * 16`. For the
|
||||
final TP2/MNS16 state, requests 89/111/117 had `(ISL, IDs)` of
|
||||
`(722,46)`, `(678,43)`, and `(706,45)`. If all supplied identities hit, it
|
||||
obtains 736/688/720 computed tokens and hence `num_new_tokens` of
|
||||
-14/-10/-14. The scheduler preserves zero-or-negative-token requests in its
|
||||
waiting queue; no event remains, so the simulator reports
|
||||
`Sequential simulation ended with non-empty scheduler state`.
|
||||
|
||||
This proves a trace-adapter/Frontier prefix-cache semantic mismatch, not a
|
||||
GPU capacity failure: a targeted CPU hook found no failed KV allocation and
|
||||
no PP-admission deferral, only the three non-admitted requests with a full
|
||||
8192-token batch budget. The correct next input is a **separate Frontier
|
||||
adapter** that exports only `floor(ISL / 16)` complete prompt-block identities;
|
||||
it must not modify the private real trace or its input/output/arrival vector.
|
||||
|
||||
## Invalid raw Frontier attempt (2026-07-17; retained for diagnosis only)
|
||||
|
||||
The TP-normalized surface completed in 156 seconds of CPU-only wall time on
|
||||
`dash0`, with Frontier commit `deadc4a321f0baaa534c6ebd17f974123733cdc2`.
|
||||
All twelve input traces passed the 129-request, ID/shape, and TP-specific
|
||||
trace-SHA checks. Only two cells reached valid request metrics; the other ten
|
||||
ended in `scheduler_stall`.
|
||||
|
||||
| Config | Status | mean / p90 TTFT (ms) | mean / p90 TPOT (ms) | mean / p90 E2E (ms) |
|
||||
|---|---|---:|---:|---:|
|
||||
| TP1/MNS32 | complete | 440032 / 972628 | 145.8 / 151.5 | 936301 / 1645895 |
|
||||
| TP1/MNS64 | complete | 174335 / 510962 | 163.6 / 178.9 | 724328 / 1285374 |
|
||||
| remaining 10 configs | `scheduler_stall` | N/A | N/A | N/A |
|
||||
|
||||
The apparent **2/12 (16.7%)** coverage is not a Frontier coverage result:
|
||||
both the ten stalls and the two completed cells consumed the invalid partial
|
||||
block metadata. In particular, the two completed cells may receive false
|
||||
prefix hits. No latency, ranking, coverage, top-set overlap, regret, or
|
||||
Kendall-tau statement about Frontier may be derived from this attempt.
|
||||
|
||||
The legacy surface runner writes an offered-rate summary assuming a fixed
|
||||
600-second window. That field is not used here because TP2/TP4 have compressed
|
||||
arrival clocks. The materialized trace manifests are authoritative: they
|
||||
record global 0.215/0.430/0.860 req/s and the matched 0.215 req/s/GPU rate.
|
||||
|
||||
## Real result (2026-07-17)
|
||||
|
||||
All **36/36** `config × trial` replays completed with exit code zero. Each
|
||||
trial passed all validity gates: 129/129 successful requests, exact requested
|
||||
and observed input/output token usage, the TP-specific private-trace SHA and
|
||||
normalized arrival-vector SHA, and HTTP routing to the explicit alias
|
||||
`qwen3-30b-exact-trace`. The cache-populating smoke and earlier failed launch
|
||||
attempts are excluded from this table. Values pool 387 requests/config (three
|
||||
trials); p90 is nearest-rank.
|
||||
|
||||
| Config | mean / p90 TTFT (ms) | mean / p90 TPOT (ms) | mean / p90 E2E (ms) |
|
||||
|---|---:|---:|---:|
|
||||
| TP1/MNS8 | 88479.9 / 168185.5 | 14.3 / 16.3 | 137996.4 / 222136.0 |
|
||||
| TP1/MNS16 | 17378.2 / 42739.8 | 20.6 / 25.3 | 87355.2 / 156177.2 |
|
||||
| TP1/MNS32 | 620.5 / 1931.5 | 23.8 / 30.0 | 81051.2 / 162908.6 |
|
||||
| TP1/MNS64 | 619.8 / 1956.7 | 24.1 / 30.4 | 82253.7 / 165355.3 |
|
||||
| TP2/MNS8 | 95920.1 / 183878.8 | 9.2 / 10.2 | 128296.9 / 215848.3 |
|
||||
| TP2/MNS16 | 34443.0 / 77922.4 | 14.1 / 16.1 | 82936.7 / 137726.4 |
|
||||
| TP2/MNS32 | 1050.6 / 2521.8 | 18.0 / 21.3 | 62267.5 / 117239.3 |
|
||||
| TP2/MNS64 | 375.8 / 1148.6 | 18.2 / 21.6 | 62353.6 / 118392.3 |
|
||||
| TP4/MNS8 | 102605.9 / 193359.7 | **6.8 / 7.2** | 126405.4 / 209092.4 |
|
||||
| TP4/MNS16 | 35042.6 / 74281.3 | 8.8 / 9.6 | 65779.1 / 110374.3 |
|
||||
| TP4/MNS32 | 6296.2 / 19159.9 | 12.2 / 13.8 | 47944.5 / 83825.7 |
|
||||
| TP4/MNS64 | **246.0 / 685.5** | 13.2 / 15.4 | **44985.1 / 83763.6** |
|
||||
|
||||
Thus real-vLLM chooses TP4/MNS64 for mean/p90 TTFT and E2E, and TP4/MNS8
|
||||
for mean/p90 TPOT. These are different single-metric objectives, not a
|
||||
claim that one configuration simultaneously optimizes all three.
|
||||
|
||||
## Correct conclusion and next gate
|
||||
|
||||
This run establishes a **harness finding**, not a Frontier capability finding:
|
||||
the initial simulator-facing Trace-PD prefix contract was invalid. The valid
|
||||
real surface remains a frozen ground truth, but all raw Frontier numbers above
|
||||
are excluded from the research claim. We therefore retract the prior bounded
|
||||
counterexample and do not yet know whether Frontier selects the correct config
|
||||
on this case.
|
||||
|
||||
Before a CPU-only Frontier rerun, the comparison must additionally record
|
||||
these remaining alignment gaps:
|
||||
|
||||
1. real vLLM uses `FULL_AND_PIECEWISE` CUDA graphs, whereas the raw Frontier
|
||||
command explicitly used `--decode_cuda_graph_mode none`; Frontier exposes
|
||||
`full_decode_only` and `piecewise`, not the identical combined mode;
|
||||
2. Frontier used one explicit KV-block count per TP, while real vLLM's graph
|
||||
capture changes the count with MNS (the mismatch is small but measurable);
|
||||
3. Frontier skipped CPU-overhead modeling, and its MoE CSV has only
|
||||
`standalone_legacy` gating rows, so the code warned and fell back rather
|
||||
than training the requested `prefill_hot` pseudo-model;
|
||||
4. the runner's rate *metadata* incorrectly fixed the TP2/TP4 numerator to a
|
||||
600-second window. Arrival timestamps supplied to the simulator were
|
||||
correct, so this did not alter the raw execution, but the metadata must be
|
||||
fixed before reporting a rerun.
|
||||
|
||||
The next admissible Frontier result uses complete-block prefix metadata,
|
||||
per-`TP×MNS` real KV capacity, correct TP-normalized rate metadata, and an
|
||||
explicit CUDA-graph compatibility decision. Only a complete, semantically
|
||||
valid surface may receive the selection metrics. Any remaining mismatch after
|
||||
that CPU rerun is then evidence about Frontier's scheduler/profile fidelity,
|
||||
not this adapter.
|
||||
|
||||
## Cost and output
|
||||
|
||||
The measured replay horizons are 597/299/149 s at TP1/2/4. Including five
|
||||
minutes of per-launch server start/warmup, 36 launches consume about **13
|
||||
H20-GPUh nominally**; applying the launcher's historical 12--35 minute
|
||||
fresh-server envelope yields an upper bound of **41 H20-GPUh**. A separate
|
||||
CPU-only simulator rerun is mandatory before this GPU stage; no profile
|
||||
collection is included.
|
||||
|
||||
The remote, experiment-specific output root is
|
||||
`/home/admin/cpfs/wjh/aituner/simulator-tuning-latency-q30-tp-normalized-u0p01-20260717`.
|
||||
It contains the prompt-free real audit JSON/Markdown, commands, environment,
|
||||
GPU/trace/model hashes, raw prompt-free request records, and cache inventory.
|
||||
Private prompt text remains on `dash0` and is never copied into the repository.
|
||||
Reference in New Issue
Block a user