gahow/aituner
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Narrow fixed PD pressure sweep around load knee

Browse Source
This commit is contained in:
Gahow Wang
2026-07-19 01:26:09 +08:00
parent 909a80f0a6
commit 33b73afe9b
2 changed files with 26 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files

15
runs/frontier-fidelity-envelope-v1/fixed-pd-pressure-matching-card.md
View File

@@ -28,12 +28,15 @@ because it is the real TTFT/E2E winner on that surface:
| in-flight proxy | 38.69 requests |
The probe fixes Qwen3-30B-A3B BF16, community vLLM 0.20.0, H20, TP4, MNS64,
MBT=8192, chunked prefill, and prefix caching off. It compares per-GPU rates
`{1, 2, 3, 4}` (global TP4 rates `{4, 8, 12, 16}` request/s). Every rate has
257 exact `4096 -> 256` requests in each of three fresh-server trials; rate
orders are rotated across trials. The launcher reuses the validated vLLM 0.20
FlashInfer kernel cache; this avoids including one-off custom-kernel JIT in
server startup and does not alter request latency measurement.
MBT=8192, chunked prefill, and prefix caching off. An incomplete one-trial
range-finding pilot showed global 4 rps stable and global 8 rps already badly
overloaded; it is not used in the decision. The formal probe therefore compares
per-GPU rates `{1, 1.125, 1.25, 1.5}` (global TP4 rates `{4, 4.5, 5, 6}`
request/s). Every rate has 257 exact `4096 -> 256` requests in each of three
fresh-server trials; rate orders are rotated across trials. The launcher
requires a validated vLLM 0.20 FlashInfer kernel cache; this avoids including
one-off custom-kernel JIT in server startup and does not alter request latency
measurement.
## Decision rule

22
runs/frontier-fidelity-envelope-v1/run_qwen30_fixed_pd_pressure_probe.sh
View File

@@ -14,12 +14,12 @@ MODEL_ROOT="${MODEL_ROOT:-/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B}"
# Reuse the validated vLLM 0.20 H20 kernel cache. A per-output workspace
# starts costly FlashInfer MoE JIT compilation and changes startup behavior
# without changing the serving configuration being profiled.
FLASHINFER_WORKSPACE_BASE="${FLASHINFER_WORKSPACE_BASE:-/tmp/wjh/flashinfer-workspace-vllm020-profiler-v1}"
FLASHINFER_WORKSPACE_BASE="${FLASHINFER_WORKSPACE_BASE:?FLASHINFER_WORKSPACE_BASE is required}"
GPU_IDS="${GPU_IDS:-0,1,2,3}"
TP="${TP:-4}"
MNS="${MNS:-64}"
REQUESTS="${REQUESTS:-257}"
GLOBAL_RATES="${GLOBAL_RATES:-4 8 12 16}"
GLOBAL_RATES="${GLOBAL_RATES:-4 4.5 5 6}"
SERVER_READY_ATTEMPTS="${SERVER_READY_ATTEMPTS:-180}"
PORT="${PORT:-8930}"
SERVED_MODEL="qwen3-30b-fixed-pd-pressure"
@@ -30,6 +30,12 @@ SERVER_PID=""
[[ "${REQUESTS}" =~ ^[1-9][0-9]*$ ]] || { echo 'ERROR: REQUESTS must be positive' >&2; exit 1; }
[[ -f "${CLIENT}" ]] || { echo "ERROR: client missing: ${CLIENT}" >&2; exit 1; }
[[ -f "${MODEL_ROOT}/config.json" ]] || { echo "ERROR: model missing: ${MODEL_ROOT}" >&2; exit 1; }
[[ -d "${FLASHINFER_WORKSPACE_BASE}" ]] || { echo "ERROR: FlashInfer workspace missing: ${FLASHINFER_WORKSPACE_BASE}" >&2; exit 1; }
read -r -a RATE_VALUES <<< "${GLOBAL_RATES}"
[[ "${#RATE_VALUES[@]}" -eq 4 ]] || { echo 'ERROR: GLOBAL_RATES must contain exactly four rates' >&2; exit 1; }
for rate in "${RATE_VALUES[@]}"; do
awk -v value="${rate}" 'BEGIN {exit !(value > 0)}' || { echo "ERROR: invalid rate: ${rate}" >&2; exit 1; }
done
mkdir -p "${OUT}/provenance" "${OUT}/trials" "${FLASHINFER_WORKSPACE_BASE}"
exec > >(tee -a "${OUT}/controller.log") 2>&1
@@ -51,7 +57,9 @@ cleanup_server() {
fi
SERVER_PID=""
}
trap cleanup_server EXIT INT TERM
trap cleanup_server EXIT
trap 'cleanup_server; exit 130' INT
trap 'cleanup_server; exit 143' TERM
assert_idle() {
nvidia-smi --query-gpu=index,memory.used,utilization.gpu --format=csv,noheader
@@ -207,7 +215,7 @@ PY
}
{
echo "FIXED_PD_PRESSURE_PROBE_LAUNCH_ECHO host=$(hostname) model=${MODEL_ROOT} engine=vLLM-0.20.0+cu129 dtype=BF16 config=TP${TP}_MNS${MNS}_MBT8192 gpus=${GPU_IDS} prefix=false shape=4096_to_256 requests_per_rate=${REQUESTS} global_rates={${GLOBAL_RATES}} rate_contract=global_rate_divided_by_TP trials=3 fresh_server=true metric_target=TracePD_TP4_MNS64_meanTTFT245.95ms_meanTPOT13.18ms expected_wall=12-20m expected_cost=0.8-1.4_H20-GPUh output=${OUT}"
echo "FIXED_PD_PRESSURE_PROBE_LAUNCH_ECHO host=$(hostname) model=${MODEL_ROOT} engine=vLLM-0.20.0+cu129 dtype=BF16 config=TP${TP}_MNS${MNS}_MBT8192 gpus=${GPU_IDS} prefix=false shape=4096_to_256 requests_per_rate=${REQUESTS} global_rates={${GLOBAL_RATES}} rate_contract=global_rate_divided_by_TP flashinfer_workspace=${FLASHINFER_WORKSPACE_BASE} trials=3 fresh_server=true metric_target=TracePD_TP4_MNS64_meanTTFT245.95ms_meanTPOT13.18ms expected_wall=12-20m expected_cost=0.8-1.4_H20-GPUh output=${OUT}"
date -u +START_UTC=%Y-%m-%dT%H:%M:%SZ
assert_idle
sha256sum "${BASH_SOURCE[0]}" "${CLIENT}" "${MODEL_ROOT}/config.json" > "${OUT}/provenance/input.sha256"
@@ -217,7 +225,11 @@ PY
readlink -f "${FLASHINFER_WORKSPACE_BASE}" > "${OUT}/provenance/flashinfer-workspace"
nvidia-smi --query-gpu=index,name,uuid,driver_version,memory.total --format=csv,noheader > "${OUT}/provenance/gpus.before.csv"
declare -a ORDERS=("4 8 12 16" "16 12 8 4" "8 16 4 12")
declare -a ORDERS=(
"${RATE_VALUES[*]}"
"${RATE_VALUES[3]} ${RATE_VALUES[2]} ${RATE_VALUES[1]} ${RATE_VALUES[0]}"
"${RATE_VALUES[1]} ${RATE_VALUES[3]} ${RATE_VALUES[0]} ${RATE_VALUES[2]}"
)
for trial in 1 2 3; do
trial_root="${OUT}/trials/trial${trial}"
mkdir -p "${trial_root}"