2.2 KiB
EXP-SIMFID-Q30-FIXED-PD-PRESSURE: calibrate a nontrivial fixed-shape workload
Status: planned. This is a real-only workload calibration; its measurements are not part of a Frontier-versus-vLLM winner comparison.
Question
The former Fixed-PD workload (2048 -> 128, 0.215 request/s/GPU) was nearly
single-request at TP4: its real mean E2E was 604 ms at a TP4 cluster rate of
0.86 request/s, or about 0.52 in-flight requests by Little's law. It therefore
does not test the decode batching regime of Trace-PD.
Can a no-prefix, uniform 4096 -> 256 workload be assigned a fixed offered
rate that reaches the healthy Trace-PD operating point without using a
simulator result for calibration?
Controlled calibration
The reference is the real Trace-PD TP4/MNS64 cell, chosen before this probe
because it is the real TTFT/E2E winner on that surface:
| Reference metric | Target |
|---|---|
| mean TTFT | 245.95 ms |
| mean TPOT | 13.18 ms |
| mean E2E | 44.99 s |
| offered rate | 0.215 request/s/GPU; 0.86 request/s at TP4 |
| in-flight proxy | 38.69 requests |
The probe fixes Qwen3-30B-A3B BF16, community vLLM 0.20.0, H20, TP4, MNS64,
MBT=8192, chunked prefill, and prefix caching off. It compares per-GPU rates
{1, 2, 3, 4} (global TP4 rates {4, 8, 12, 16} request/s). Every rate has
257 exact 4096 -> 256 requests in each of three fresh-server trials; rate
orders are rotated across trials.
Decision rule
Pool the three trials for each rate. Among rates with every request completed,
choose the rate minimizing the Euclidean distance of the two relative errors
for mean TTFT and mean TPOT from the above targets. Report p90 TTFT/TPOT/E2E,
trial variation, and global_rate * mean_E2E as an in-flight proxy, but do not
turn any of them into an SLO. The selected rate becomes a frozen workload
contract. The subsequent 12-cell real/simulator surface must be fresh and is
not allowed to reuse this calibration data.
Interpretation boundary
This selects a workload regime, not a simulator parameter and not a serving capacity point. It may make Fixed-PD more comparable to Trace-PD in batching pressure, but it deliberately continues to exclude trace-shaped arrivals and prefix reuse; those are separately evaluated workload dimensions.