gahow/aituner
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Narrow fixed PD pressure sweep around load knee

Browse Source
This commit is contained in:
Gahow Wang
2026-07-19 01:26:09 +08:00
parent 909a80f0a6
commit 33b73afe9b
2 changed files with 26 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files

15
runs/frontier-fidelity-envelope-v1/fixed-pd-pressure-matching-card.md
View File

@@ -28,12 +28,15 @@ because it is the real TTFT/E2E winner on that surface:
| in-flight proxy | 38.69 requests |
The probe fixes Qwen3-30B-A3B BF16, community vLLM 0.20.0, H20, TP4, MNS64,
MBT=8192, chunked prefill, and prefix caching off. It compares per-GPU rates
`{1, 2, 3, 4}` (global TP4 rates `{4, 8, 12, 16}` request/s). Every rate has
257 exact `4096 -> 256` requests in each of three fresh-server trials; rate
orders are rotated across trials. The launcher reuses the validated vLLM 0.20
FlashInfer kernel cache; this avoids including one-off custom-kernel JIT in
server startup and does not alter request latency measurement.
MBT=8192, chunked prefill, and prefix caching off. An incomplete one-trial
range-finding pilot showed global 4 rps stable and global 8 rps already badly
overloaded; it is not used in the decision. The formal probe therefore compares
per-GPU rates `{1, 1.125, 1.25, 1.5}` (global TP4 rates `{4, 4.5, 5, 6}`
request/s). Every rate has 257 exact `4096 -> 256` requests in each of three
fresh-server trials; rate orders are rotated across trials. The launcher
requires a validated vLLM 0.20 FlashInfer kernel cache; this avoids including
one-off custom-kernel JIT in server startup and does not alter request latency
measurement.
## Decision rule