Profile fixed PD workload pressure against trace anchor
This commit is contained in:
@@ -0,0 +1,51 @@
|
||||
# EXP-SIMFID-Q30-FIXED-PD-PRESSURE: calibrate a nontrivial fixed-shape workload
|
||||
|
||||
> Status: planned. This is a real-only workload calibration; its measurements
|
||||
> are not part of a Frontier-versus-vLLM winner comparison.
|
||||
|
||||
## Question
|
||||
|
||||
The former Fixed-PD workload (`2048 -> 128`, 0.215 request/s/GPU) was nearly
|
||||
single-request at TP4: its real mean E2E was 604 ms at a TP4 cluster rate of
|
||||
0.86 request/s, or about 0.52 in-flight requests by Little's law. It therefore
|
||||
does not test the decode batching regime of Trace-PD.
|
||||
|
||||
Can a no-prefix, uniform `4096 -> 256` workload be assigned a fixed offered
|
||||
rate that reaches the healthy Trace-PD operating point without using a
|
||||
simulator result for calibration?
|
||||
|
||||
## Controlled calibration
|
||||
|
||||
The reference is the real Trace-PD `TP4/MNS64` cell, chosen before this probe
|
||||
because it is the real TTFT/E2E winner on that surface:
|
||||
|
||||
| Reference metric | Target |
|
||||
|---|---:|
|
||||
| mean TTFT | 245.95 ms |
|
||||
| mean TPOT | 13.18 ms |
|
||||
| mean E2E | 44.99 s |
|
||||
| offered rate | 0.215 request/s/GPU; 0.86 request/s at TP4 |
|
||||
| in-flight proxy | 38.69 requests |
|
||||
|
||||
The probe fixes Qwen3-30B-A3B BF16, community vLLM 0.20.0, H20, TP4, MNS64,
|
||||
MBT=8192, chunked prefill, and prefix caching off. It compares per-GPU rates
|
||||
`{1, 2, 3, 4}` (global TP4 rates `{4, 8, 12, 16}` request/s). Every rate has
|
||||
257 exact `4096 -> 256` requests in each of three fresh-server trials; rate
|
||||
orders are rotated across trials.
|
||||
|
||||
## Decision rule
|
||||
|
||||
Pool the three trials for each rate. Among rates with every request completed,
|
||||
choose the rate minimizing the Euclidean distance of the two *relative* errors
|
||||
for mean TTFT and mean TPOT from the above targets. Report p90 TTFT/TPOT/E2E,
|
||||
trial variation, and `global_rate * mean_E2E` as an in-flight proxy, but do not
|
||||
turn any of them into an SLO. The selected rate becomes a frozen workload
|
||||
contract. The subsequent 12-cell real/simulator surface must be fresh and is
|
||||
not allowed to reuse this calibration data.
|
||||
|
||||
## Interpretation boundary
|
||||
|
||||
This selects a workload regime, not a simulator parameter and not a serving
|
||||
capacity point. It may make Fixed-PD more comparable to Trace-PD in batching
|
||||
pressure, but it deliberately continues to exclude trace-shaped arrivals and
|
||||
prefix reuse; those are separately evaluated workload dimensions.
|
||||
Reference in New Issue
Block a user