90 lines
4.6 KiB
Markdown
90 lines
4.6 KiB
Markdown
# EXP-SIMFID-Q30-FIXED-PD-PRESSURE: calibrate a nontrivial fixed-shape workload
|
||
|
||
> Status: complete (2026-07-19). This is a real-only workload calibration; its
|
||
> measurements are not part of a Frontier-versus-vLLM winner comparison.
|
||
|
||
## Question
|
||
|
||
The former Fixed-PD workload (`2048 -> 128`, 0.215 request/s/GPU) was nearly
|
||
single-request at TP4: its real mean E2E was 604 ms at a TP4 cluster rate of
|
||
0.86 request/s, or about 0.52 in-flight requests by Little's law. It therefore
|
||
does not test the decode batching regime of Trace-PD.
|
||
|
||
Can a no-prefix, uniform `4096 -> 256` workload be assigned a fixed offered
|
||
rate that reaches the healthy Trace-PD operating point without using a
|
||
simulator result for calibration?
|
||
|
||
## Controlled calibration
|
||
|
||
The reference is the real Trace-PD `TP4/MNS64` cell, chosen before this probe
|
||
because it is the real TTFT/E2E winner on that surface:
|
||
|
||
| Reference metric | Target |
|
||
|---|---:|
|
||
| mean TTFT | 245.95 ms |
|
||
| mean TPOT | 13.18 ms |
|
||
| mean E2E | 44.99 s |
|
||
| offered rate | 0.215 request/s/GPU; 0.86 request/s at TP4 |
|
||
| in-flight proxy | 38.69 requests |
|
||
|
||
The probe fixes Qwen3-30B-A3B BF16, community vLLM 0.20.0, H20, TP4, MNS64,
|
||
MBT=8192, chunked prefill, and prefix caching off. An incomplete one-trial
|
||
range-finding pilot showed global 4 rps stable and global 8 rps already badly
|
||
overloaded; it is not used in the decision. The formal probe therefore compares
|
||
per-GPU rates `{1, 1.125, 1.25, 1.5}` (global TP4 rates `{4, 4.5, 5, 6}`
|
||
request/s). Every rate has 257 exact `4096 -> 256` requests in each of three
|
||
fresh-server trials; rate orders are rotated across trials. The launcher
|
||
requires a validated vLLM 0.20 FlashInfer kernel cache; this avoids including
|
||
one-off custom-kernel JIT in server startup and does not alter request latency
|
||
measurement.
|
||
|
||
## Decision rule
|
||
|
||
Pool the three trials for each rate. Among rates with every request completed,
|
||
choose the rate minimizing the Euclidean distance of the two *relative* errors
|
||
for mean TTFT and mean TPOT from the above targets. Report p90 TTFT/TPOT/E2E,
|
||
trial variation, and `global_rate * mean_E2E` as an in-flight proxy, but do not
|
||
turn any of them into an SLO. The selected rate becomes a frozen workload
|
||
contract. The subsequent 12-cell real/simulator surface must be fresh and is
|
||
not allowed to reuse this calibration data.
|
||
|
||
## Interpretation boundary
|
||
|
||
This selects a workload regime, not a simulator parameter and not a serving
|
||
capacity point. It may make Fixed-PD more comparable to Trace-PD in batching
|
||
pressure, but it deliberately continues to exclude trace-shaped arrivals and
|
||
prefix reuse; those are separately evaluated workload dimensions.
|
||
|
||
## Result
|
||
|
||
All 12 planned cells completed (`257` requests × `3` fresh-server trials per
|
||
rate, zero client failures). The raw artifact is
|
||
`dash0:/home/admin/cpfs/wjh/aituner/qwen30-fixed-pd-pressure-20260719-r5`;
|
||
it records the vLLM version, model/config checksums, H20 inventory, FlashInfer
|
||
workspace, and an artifact checksum manifest. Pooled results are:
|
||
|
||
| Global / per-GPU rps | TTFT mean / p90 (ms) | TPOT mean / p90 (ms) | E2E mean / p90 (ms) | In-flight proxy |
|
||
|---|---:|---:|---:|---:|
|
||
| 4 / 1 | 121.53 / 125.12 | 10.75 / 11.83 | 2862.42 / 3138.92 | 11.45 |
|
||
| 4.5 / 1.125 | 122.76 / 126.29 | 11.76 / 12.55 | 3122.32 / 3322.26 | 14.05 |
|
||
| 5 / 1.25 | 125.09 / 129.66 | 15.16 / 16.58 | 3989.73 / 4354.22 | 19.95 |
|
||
| 6 / 1.5 | 129.27 / 134.69 | 21.33 / 23.03 | 5569.14 / 6005.12 | 33.41 |
|
||
|
||
The predeclared two-metric rule picks `4.5` global rps (`1.125` rps/GPU), but
|
||
that is only the *least mismatched* tested point: TTFT is still 50.1% below
|
||
the Trace-PD target (122.76 vs 245.95 ms), while TPOT is 10.7% below it (11.76
|
||
vs 13.18 ms). Raising load to 5 global rps moves TPOT past target (+15.0%)
|
||
while TTFT barely changes (125.09 ms); at 6 global rps TPOT is +61.9% while
|
||
TTFT remains 47.4% below target. Trial-level mean standard deviations are at
|
||
most 0.78 ms for TTFT and 0.34 ms for TPOT, so this is not trial-order noise.
|
||
|
||
**Conclusion:** `4096 -> 256` with uniform no-prefix arrivals cannot be called
|
||
a pressure-matched Fixed-PD control simply by setting RPS. It has much shorter
|
||
request residence time (3.12 s versus 44.99 s) and lower concurrency (14.05
|
||
versus 38.69 by the same Little-law proxy) at its closest point. For the
|
||
paper's fixed-versus-trace comparison, retain this case only as a deliberately
|
||
different fixed-shape workload; do not interpret any simulator gap as being
|
||
caused solely by trace fidelity. A future pressure-matched fixed control must
|
||
also increase output length (or otherwise preserve residence time), then repeat
|
||
this real-only calibration before reopening the Frontier-versus-vLLM surface.
|