Reuse validated FlashInfer cache for pressure probe

This commit is contained in:
2026-07-19 01:13:50 +08:00
parent 9837fa5133
commit 909a80f0a6
2 changed files with 8 additions and 2 deletions

View File

@@ -31,7 +31,9 @@ The probe fixes Qwen3-30B-A3B BF16, community vLLM 0.20.0, H20, TP4, MNS64,
MBT=8192, chunked prefill, and prefix caching off. It compares per-GPU rates
`{1, 2, 3, 4}` (global TP4 rates `{4, 8, 12, 16}` request/s). Every rate has
257 exact `4096 -> 256` requests in each of three fresh-server trials; rate
orders are rotated across trials.
orders are rotated across trials. The launcher reuses the validated vLLM 0.20
FlashInfer kernel cache; this avoids including one-off custom-kernel JIT in
server startup and does not alter request latency measurement.
## Decision rule