MB5 PD ablation: controlled-variable reuse/conc redo + campaign tooling
Reuse and concurrency axes redone with proper controlled variables, plus
the orchestration used to run them on dash0:
- run_reuse_fixed.sh: hold REAL prefill work (delta) constant, vary only
cached prefix -> reuse = C/(C+U). Supersedes old fig1 (which held
input=8192 and sliced prefix out, confounding "more reuse" with "less
prefill").
- run_conc.sh: agentic-corner config (in=32768, delta=512, reuse=0.984,
out=128) that exposes PD's structural KV-transfer tax. Supersedes old fig3.
- run_campaign{,2,3}.sh, backfill_d2048o128.sh: serial campaign drivers
(strictly one driver at a time), out=128 sweeps, PD wall-cap for
collapse-draining high-reuse arms, and flaked-arm backfill.
- mb5_run_gpu.sh: per-config bring-up / replay / teardown orchestrator.
- plot_pd_crossover.py: render the reuse_compare figures from fig_agg dumps.
- fig_agg.py: tolerate null stats from fully-collapsed arms (0 successes
write the stat keys as null; `dict.get(k, {})` returns null, not {}).
Data: fig1_reuse_fixed.json, fig1_reuse_d{1024,2048}_o128.json
Figs: reuse_compare_AB.png, reuse_compare_ABC.png
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -100,11 +100,13 @@ def main():
|
||||
continue
|
||||
s = json.load(open(sp))
|
||||
arm, pg, dg, ports = arm_of(run.name)
|
||||
lat = s.get("latency_stats_s", {})
|
||||
ttft = s.get("ttft_stats_s", {})
|
||||
tpot = s.get("tpot_stats_s", {})
|
||||
# `or {}` because a fully-collapsed arm (0 successes) writes these as null,
|
||||
# and dict.get(k, {}) returns null (not {}) when the key exists with value null.
|
||||
lat = s.get("latency_stats_s") or {}
|
||||
ttft = s.get("ttft_stats_s") or {}
|
||||
tpot = s.get("tpot_stats_s") or {}
|
||||
wall = s.get("wall_clock_s") or 1.0
|
||||
out = s.get("actual_output_tokens_stats", {})
|
||||
out = s.get("actual_output_tokens_stats") or {}
|
||||
n = s.get("success_count", 0); req = s.get("request_count", 0)
|
||||
tot_out = out.get("count", 0) * out.get("mean", 0)
|
||||
tps = tot_out / wall
|
||||
|
||||
Reference in New Issue
Block a user