54 lines
2.4 KiB
Markdown
54 lines
2.4 KiB
Markdown
# EXP-SIMFID-Q30-FIXED-PD-PRESSURE: calibrate a nontrivial fixed-shape workload
|
|
|
|
> Status: planned. This is a real-only workload calibration; its measurements
|
|
> are not part of a Frontier-versus-vLLM winner comparison.
|
|
|
|
## Question
|
|
|
|
The former Fixed-PD workload (`2048 -> 128`, 0.215 request/s/GPU) was nearly
|
|
single-request at TP4: its real mean E2E was 604 ms at a TP4 cluster rate of
|
|
0.86 request/s, or about 0.52 in-flight requests by Little's law. It therefore
|
|
does not test the decode batching regime of Trace-PD.
|
|
|
|
Can a no-prefix, uniform `4096 -> 256` workload be assigned a fixed offered
|
|
rate that reaches the healthy Trace-PD operating point without using a
|
|
simulator result for calibration?
|
|
|
|
## Controlled calibration
|
|
|
|
The reference is the real Trace-PD `TP4/MNS64` cell, chosen before this probe
|
|
because it is the real TTFT/E2E winner on that surface:
|
|
|
|
| Reference metric | Target |
|
|
|---|---:|
|
|
| mean TTFT | 245.95 ms |
|
|
| mean TPOT | 13.18 ms |
|
|
| mean E2E | 44.99 s |
|
|
| offered rate | 0.215 request/s/GPU; 0.86 request/s at TP4 |
|
|
| in-flight proxy | 38.69 requests |
|
|
|
|
The probe fixes Qwen3-30B-A3B BF16, community vLLM 0.20.0, H20, TP4, MNS64,
|
|
MBT=8192, chunked prefill, and prefix caching off. It compares per-GPU rates
|
|
`{1, 2, 3, 4}` (global TP4 rates `{4, 8, 12, 16}` request/s). Every rate has
|
|
257 exact `4096 -> 256` requests in each of three fresh-server trials; rate
|
|
orders are rotated across trials. The launcher reuses the validated vLLM 0.20
|
|
FlashInfer kernel cache; this avoids including one-off custom-kernel JIT in
|
|
server startup and does not alter request latency measurement.
|
|
|
|
## Decision rule
|
|
|
|
Pool the three trials for each rate. Among rates with every request completed,
|
|
choose the rate minimizing the Euclidean distance of the two *relative* errors
|
|
for mean TTFT and mean TPOT from the above targets. Report p90 TTFT/TPOT/E2E,
|
|
trial variation, and `global_rate * mean_E2E` as an in-flight proxy, but do not
|
|
turn any of them into an SLO. The selected rate becomes a frozen workload
|
|
contract. The subsequent 12-cell real/simulator surface must be fresh and is
|
|
not allowed to reuse this calibration data.
|
|
|
|
## Interpretation boundary
|
|
|
|
This selects a workload regime, not a simulator parameter and not a serving
|
|
capacity point. It may make Fixed-PD more comparable to Trace-PD in batching
|
|
pressure, but it deliberately continues to exclude trace-shaped arrivals and
|
|
prefix reuse; those are separately evaluated workload dimensions.
|