Gahow Wang
9c105cf05a
Reuse and concurrency axes redone with proper controlled variables, plus
the orchestration used to run them on dash0:
- run_reuse_fixed.sh: hold REAL prefill work (delta) constant, vary only
cached prefix -> reuse = C/(C+U). Supersedes old fig1 (which held
input=8192 and sliced prefix out, confounding "more reuse" with "less
prefill").
- run_conc.sh: agentic-corner config (in=32768, delta=512, reuse=0.984,
out=128) that exposes PD's structural KV-transfer tax. Supersedes old fig3.
- run_campaign{,2,3}.sh, backfill_d2048o128.sh: serial campaign drivers
(strictly one driver at a time), out=128 sweeps, PD wall-cap for
collapse-draining high-reuse arms, and flaked-arm backfill.
- mb5_run_gpu.sh: per-config bring-up / replay / teardown orchestrator.
- plot_pd_crossover.py: render the reuse_compare figures from fig_agg dumps.
- fig_agg.py: tolerate null stats from fully-collapsed arms (0 successes
write the stat keys as null; `dict.get(k, {})` returns null, not {}).
Data: fig1_reuse_fixed.json, fig1_reuse_d{1024,2048}_o128.json
Figs: reuse_compare_AB.png, reuse_compare_ABC.png
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
21 lines
1.1 KiB
Bash
21 lines
1.1 KiB
Bash
#!/usr/bin/env bash
|
|
# Campaign 3 (2026-05-31): the uncapped d2048/o128 reuse sweep stalled on a
|
|
# collapse-draining high-reuse PD arm (4P+4D @ reuse 0.90, ~1 req/several-min).
|
|
# Finish it by re-running ONLY the high-reuse points (0.90, 0.95) WITH the PD
|
|
# wall-cap (low-reuse arms already completed and are cap-insensitive). Then run
|
|
# the capped conc sweep. STRICTLY serial. NO set -e.
|
|
cd /home/admin/cpfs/wjh/agentic-kv-fresh
|
|
export MB5_VENV="${MB5_VENV:-/home/admin/cpfs/wjh/agentic-kv-fresh/.venv_dash0}"
|
|
FS=microbench/fresh_setup
|
|
echo "=== CAMPAIGN3 START $(date) ==="
|
|
|
|
echo "=== [1/2] finish reuse d2048/o128: re-run pts pfx=18432,38912 (PD capped 500s) $(date) ==="
|
|
DELTA=2048 OL=128 PFXS="18432 38912" REUSE_PD_MAXDUR=500 bash "$FS/run_reuse_fixed.sh"; rc1=$?
|
|
echo "=== reuse d2048 o128 finish rc=$rc1 $(date) ==="
|
|
sleep 12; nvidia-smi --query-gpu=index,memory.used --format=csv,noheader | head -8
|
|
|
|
echo "=== [2/2] CONC capped (PD wall=600s, colo uncapped), N 8..128 $(date) ==="
|
|
NLIST="8 16 32 48 64 96 128" CONC_PD_MAXDUR=600 bash "$FS/run_conc.sh"; rc2=$?
|
|
echo "=== conc rc=$rc2 $(date) ==="
|
|
echo "=== CAMPAIGN3 DONE reuse_finish=$rc1 conc=$rc2 $(date) ==="
|