gahow/agentic-kvc
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Unified bench report: mean+TPS+per-worker GPU util, auto-captured

Browse Source 
scripts/bench_report.py is now the canonical analyzer: per run + per input-
class it emits TTFT/TPOT/E2E mean+p50+p90+p99, decode/prefill TPS (aggregate
and per-worker), APC, per-worker GPU util mean/max, and load-spread ratios.

b3_isolated_policy.sh auto-captures the inputs for every run: gpu_util.csv
(via gpu_monitor.sh, 5s, replay-window only) + bench_config.json (worker->GPU
map); teardown stops the sampler. Future runs populate per-worker GPU util
automatically.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Gahow Wang
2026-05-29 16:08:22 +08:00
parent d9046322c6
commit 160c29133d
2 changed files with 290 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

14
scripts/b3_isolated_policy.sh
View File

@@ -50,6 +50,7 @@ mkdir -p "$RUNDIR/engine_state" "$RUNDIR/logs"
echo "[isolated] policy=$POLICY trace=$(basename $TRACE) rundir=$RUNDIR"
cleanup() {
pkill -f gpu_monitor.sh 2>/dev/null || true
pkill -9 -f cache_aware_proxy 2>/dev/null || true
pkill -9 -f "vllm serve" 2>/dev/null || true
pkill -9 -f "EngineCore" 2>/dev/null || true
@@ -183,6 +184,18 @@ until curl -sf "http://127.0.0.1:$PROXY_PORT/stats" >/dev/null 2>&1; do
sleep 2
done
# Unified bench infra: record worker->GPU mapping + sample per-GPU util during
# the replay so bench_report.py can emit per-worker GPU util / TPS for every run.
python3 - "$RUNDIR" "$BASE_PORT" "$PROXY_PORT" "$GPU_INDICES" "$N_INSTANCES" <<'PY'
import json, sys
rundir, base_port, proxy_port, gpu_indices, n = sys.argv[1:]
json.dump({"base_port": int(base_port), "proxy_port": int(proxy_port),
"gpu_indices": [int(x) for x in gpu_indices.split()],
"n_instances": int(n)}, open(f"{rundir}/bench_config.json", "w"), indent=2)
PY
bash "$ROOT/scripts/gpu_monitor.sh" "$RUNDIR/gpu_util.csv" 5 >/dev/null 2>&1 &
GPU_MON_PID=$!
t_start=$(date +%s.%N)
echo "[isolated] running replayer ..."
PYTHONPATH="$ROOT" "$VENV/python" -m replayer \
@@ -192,6 +205,7 @@ PYTHONPATH="$ROOT" "$VENV/python" -m replayer \
--model "$MODEL" \
2>&1 | tee "$RUNDIR/replayer.log" | tail -3
t_end=$(date +%s.%N)
kill "${GPU_MON_PID:-}" 2>/dev/null || true
python3 - "$RUNDIR" "$POLICY" "$TRACE" "$t_start" "$t_end" <<'PY'
import json, sys