Reuse validated FlashInfer cache for pressure probe
This commit is contained in:
@@ -31,7 +31,9 @@ The probe fixes Qwen3-30B-A3B BF16, community vLLM 0.20.0, H20, TP4, MNS64,
|
||||
MBT=8192, chunked prefill, and prefix caching off. It compares per-GPU rates
|
||||
`{1, 2, 3, 4}` (global TP4 rates `{4, 8, 12, 16}` request/s). Every rate has
|
||||
257 exact `4096 -> 256` requests in each of three fresh-server trials; rate
|
||||
orders are rotated across trials.
|
||||
orders are rotated across trials. The launcher reuses the validated vLLM 0.20
|
||||
FlashInfer kernel cache; this avoids including one-off custom-kernel JIT in
|
||||
server startup and does not alter request latency measurement.
|
||||
|
||||
## Decision rule
|
||||
|
||||
|
||||
Reference in New Issue
Block a user