MB5 PD ablation: controlled-variable reuse/conc redo + campaign tooling

Reuse and concurrency axes redone with proper controlled variables, plus
the orchestration used to run them on dash0:

- run_reuse_fixed.sh: hold REAL prefill work (delta) constant, vary only
  cached prefix -> reuse = C/(C+U). Supersedes old fig1 (which held
  input=8192 and sliced prefix out, confounding "more reuse" with "less
  prefill").
- run_conc.sh: agentic-corner config (in=32768, delta=512, reuse=0.984,
  out=128) that exposes PD's structural KV-transfer tax. Supersedes old fig3.
- run_campaign{,2,3}.sh, backfill_d2048o128.sh: serial campaign drivers
  (strictly one driver at a time), out=128 sweeps, PD wall-cap for
  collapse-draining high-reuse arms, and flaked-arm backfill.
- mb5_run_gpu.sh: per-config bring-up / replay / teardown orchestrator.
- plot_pd_crossover.py: render the reuse_compare figures from fig_agg dumps.
- fig_agg.py: tolerate null stats from fully-collapsed arms (0 successes
  write the stat keys as null; `dict.get(k, {})` returns null, not {}).

Data: fig1_reuse_fixed.json, fig1_reuse_d{1024,2048}_o128.json
Figs: reuse_compare_AB.png, reuse_compare_ABC.png

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-06-01 01:03:27 +08:00
parent 32f7f55990
commit 9c105cf05a
14 changed files with 586 additions and 4 deletions

View File

@@ -100,11 +100,13 @@ def main():
continue
s = json.load(open(sp))
arm, pg, dg, ports = arm_of(run.name)
lat = s.get("latency_stats_s", {})
ttft = s.get("ttft_stats_s", {})
tpot = s.get("tpot_stats_s", {})
# `or {}` because a fully-collapsed arm (0 successes) writes these as null,
# and dict.get(k, {}) returns null (not {}) when the key exists with value null.
lat = s.get("latency_stats_s") or {}
ttft = s.get("ttft_stats_s") or {}
tpot = s.get("tpot_stats_s") or {}
wall = s.get("wall_clock_s") or 1.0
out = s.get("actual_output_tokens_stats", {})
out = s.get("actual_output_tokens_stats") or {}
n = s.get("success_count", 0); req = s.get("request_count", 0)
tot_out = out.get("count", 0) * out.get("mean", 0)
tps = tot_out / wall