gahow/aituner
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Reuse validated FlashInfer cache for pressure probe

Browse Source
This commit is contained in:
Gahow Wang
2026-07-19 01:13:50 +08:00
parent 9837fa5133
commit 909a80f0a6
2 changed files with 8 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
runs/frontier-fidelity-envelope-v1/fixed-pd-pressure-matching-card.md
View File

@@ -31,7 +31,9 @@ The probe fixes Qwen3-30B-A3B BF16, community vLLM 0.20.0, H20, TP4, MNS64,
MBT=8192, chunked prefill, and prefix caching off. It compares per-GPU rates
`{1, 2, 3, 4}` (global TP4 rates `{4, 8, 12, 16}` request/s). Every rate has
257 exact `4096 -> 256` requests in each of three fresh-server trials; rate
orders are rotated across trials.
orders are rotated across trials. The launcher reuses the validated vLLM 0.20
FlashInfer kernel cache; this avoids including one-off custom-kernel JIT in
server startup and does not alter request latency measurement.
## Decision rule

6
runs/frontier-fidelity-envelope-v1/run_qwen30_fixed_pd_pressure_probe.sh
View File

@@ -11,7 +11,10 @@ RUNNER_DIR="${RUNNER_DIR:-$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)}"
CLIENT="${CLIENT:-${RUNNER_DIR}/../frontier-phase-factorial-v0/qwen30_prefill_client.py}"
VENV_ROOT="${VENV_ROOT:-/tmp/wjh/venvs/vllm-0.20.0-cu129-profiler-v1}"
MODEL_ROOT="${MODEL_ROOT:-/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B}"
FLASHINFER_WORKSPACE_BASE="${FLASHINFER_WORKSPACE_BASE:-${OUT}/flashinfer-workspace}"
# Reuse the validated vLLM 0.20 H20 kernel cache. A per-output workspace
# starts costly FlashInfer MoE JIT compilation and changes startup behavior
# without changing the serving configuration being profiled.
FLASHINFER_WORKSPACE_BASE="${FLASHINFER_WORKSPACE_BASE:-/tmp/wjh/flashinfer-workspace-vllm020-profiler-v1}"
GPU_IDS="${GPU_IDS:-0,1,2,3}"
TP="${TP:-4}"
MNS="${MNS:-64}"
@@ -211,6 +214,7 @@ PY
"${VENV_ROOT}/bin/vllm" --version > "${OUT}/provenance/vllm.version"
"${VENV_ROOT}/bin/python" -c 'import torch, transformers, vllm; print(f"torch={torch.__version__}"); print(f"transformers={transformers.__version__}"); print(f"vllm={vllm.__version__}")' > "${OUT}/provenance/runtime.versions"
ulimit -n > "${OUT}/provenance/open-file-limit"
readlink -f "${FLASHINFER_WORKSPACE_BASE}" > "${OUT}/provenance/flashinfer-workspace"
nvidia-smi --query-gpu=index,name,uuid,driver_version,memory.total --format=csv,noheader > "${OUT}/provenance/gpus.before.csv"
declare -a ORDERS=("4 8 12 16" "16 12 8 4" "8 16 4 12")