Characterization plan: progress snapshot + Claude work plan

- Add Progress Snapshot table to the intern TODO so per-batch status
  (DONE / partial / blocked-on-instrumentation) is visible at a glance.
- New analysis/claude_characterization_work_plan.md scopes the Phase A
  instrumentation tasks (A1-A5) plus Window 1 (B1'+B2+B3) and Window 2
  (B4+B5) on dash0, with locked decisions for model, topology, trace,
  SLO style, and GPU phasing.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

analysis/characterization_todo_for_interns.md

@@ -2,6 +2,35 @@
 
 Status: execution checklist for interns
 Date: 2026-05-25
+Last progress audit: 2026-05-25
+
+## Progress Snapshot (2026-05-25)
+
+| Batch | State | Evidence |
+|---|---|---|
+| B0 Substrate audit | tool DONE, legacy runs partial | `analysis/characterization/analyze.py` implements per-session concurrency/arrival/inter-turn analyzer; legacy `metrics.jsonl` lacks dispatch/finish timestamps so actual sequentiality cannot be proven on old runs (correctly labeled in `current_results/`) |
+| B1 Workload characterization | trace-shape DONE, reuse pending | `current_results/full_trace_summary.json` covers 2.11M req / 1.31M sessions from `051315-051317.jsonl`; KV-footprint and reuse decomposition still require `--kv-bytes-per-token` rerun and cached_tokens+hash_ids joined records |
+| B2 PD interference | protocol DONE, run pending | `analysis/characterization/protocols.md` Batch 2 section ready; needs fresh GPU run with decode-step + prefill-chunk timestamps |
+| B3 Hot-spot imbalance | partial; needs new instrumentation | Legacy `gpu_util.csv` shows imbalance but lacks per-worker queue delay and session→worker map |
+| B4 SRR sweep | NOT DONE | No arrival-rate sweep artifacts; depends on session-causal open-loop loadgen |
+| B5 Failure attribution | NOT DONE | Depends on B2/B4 outputs |
+| B6 Audit package | scaffold DONE | `current_results/{characterization_claim_matrix.md, all_figures_index.md, reviewer_risk_register.md, main_claim_allowed_runs.md, reproduction_commands.sh}` + 6 figures committed |
+
+Reusable assets already in repo:
+
+- `analysis/characterization/analyze.py` — B0+B1 CPU-only analyzer
+- `analysis/characterization/summarize_runs.py` — existing-run audit producing the B6 scaffold
+- `analysis/characterization/plot_current_results.py` — figure regeneration script
+- `analysis/characterization/protocols.md` — B2–B6 protocol with required instrumentation, sweep, pass condition
+- `analysis/characterization/current_results/` — current audit package (claim matrix + risk register + allowed-runs gate + 6 PNG figures)
+
+Hard gates still blocking main claims:
+
+1. Replayer/proxy must emit per-request dispatch + finish/error wall-clock timestamps (blocks B0 actual sequentiality, B4 SRR validity).
+2. Per-request record must carry `session_id` + `hash_ids` + `cached_tokens` jointly (blocks B1 reuse decomposition).
+3. Engine/proxy must log decode-step and prefill-chunk timestamps with worker id (blocks B2 interference index).
+4. Proxy must log route decision, chosen worker, candidate scores, per-worker queue/KV/APC snapshot (blocks B3 hot-spot proof).
+
 
 ## 0. Purpose
 
@@ -84,6 +113,8 @@ saved alongside the artifact.
 
 ## 2. Batch 0: Benchmark Substrate Audit
 
+Status: analyzer DONE (`analyze.py`); legacy-run sequentiality claim BLOCKED by missing dispatch/finish timestamps in `metrics.jsonl`. New replayer must add those fields before any `online_realistic` classification is allowed.
+
 ### Goal
 
 Prove the load generator and trace replay are valid before trusting any
@@ -147,6 +178,8 @@ The `audit.md` must answer:
 
 ## 3. Batch 1: Workload Characterization
 
+Status: trace-shape items (1, 2, 3, 6, 8) DONE on full 7200 s GLM-5.1 trace; recorded in `current_results/full_trace_summary.json`. Items 4 (KV footprint), 5 (reuse decomposition), 7 (uncached append delta) are PENDING because they need `--kv-bytes-per-token` for the production model and joinable `cached_tokens`+`hash_ids` per request.
+
 ### Goal
 
 Establish agentic workload facts independent of any proposed system.
@@ -225,6 +258,8 @@ links and plotted figures.
 
 ## 4. Batch 2: PD-Colo Prefill-Decode Interference Proof
 
+Status: protocol DONE (`analysis/characterization/protocols.md` §"Batch 2 Protocol"); execution NOT STARTED — needs new engine instrumentation for decode-step and prefill-chunk timestamps.
+
 ### Goal
 
 Prove that PD-colocation can suffer from prefill-decode interference under
@@ -302,6 +337,8 @@ The `audit.md` must answer:
 
 ## 5. Batch 3: Session Hot-Spot Residual Imbalance Proof
 
+Status: protocol DONE; partial signal from legacy `gpu_util.csv` (GPU-util imbalance visible) but causal proof NOT STARTED — needs per-worker queue/KV/APC and session→worker map from instrumented proxy.
+
 ### Goal
 
 Prove that cache-aware/LMetric is a strong baseline but still leaves residual
@@ -398,6 +435,8 @@ The `audit.md` must answer:
 
 ## 6. Batch 4: Sustainable Request Rate Sweep
 
+Status: protocol DONE; execution NOT STARTED — requires open-loop session-causal loadgen and policy-comparable arrival process.
+
 ### Goal
 
 Connect interference and hot-spot mechanisms to the final metric:
@@ -476,6 +515,8 @@ The `audit.md` must answer:
 
 ## 7. Batch 5: Failure Attribution Near SRR Boundary
 
+Status: protocol DONE; execution NOT STARTED — depends on B2 instrumentation and B4 SRR boundary.
+
 ### Goal
 
 At and around the PD-colo/LMetric failure point, determine whether SLO
@@ -542,6 +583,8 @@ The batch must answer:
 
 ## 8. Batch 6: Audit Package
 
+Status: scaffold DONE — all five final artifacts exist under `analysis/characterization/current_results/` and are regenerated by `summarize_runs.py` + `plot_current_results.py`. Future B2–B5 outputs must be merged into the same package by re-running `summarize_runs.py` after new runs.
+
 ### Goal
 
 Make the whole characterization package reviewable by a strict systems