MB5 PD ablation v2 results: concurrency axis + reuse 3-way writeup

- fig3_conc32k.json + fig3_concurrency_axis.png: agentic-corner concurrency
  sweep (in=32768, reuse=0.984, out=128), N 8->128, PD capped 600s / colo
  uncapped. colo completes 100% at every N (graceful, E2E 2.4s->81s); every
  static PD split collapses, earlier as N rises (viable only N<=16; <27% by N=32).
- analysis/mb5_pd_ablation/README.md: self-contained v2 writeup. Reuse axis
  3-way (A=d2048/o256, C=d2048/o128, B=d1024/o128) decomposes shape: output
  ~negligible, delta (real prefill/turn) dominant; crossover to colo at reuse
  ~90-95% robust. Run on dash2 (dash0 NICs faulted for Mooncake).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

.gitignore

@@ -3,7 +3,14 @@ __pycache__/
 .venv/
 *.egg-info/
 outputs/
-traces/
+traces/*
+# ship the anonymized sampled trace + its provenance (metadata only, no cleartext)
+!traces/w600_r0.0015_st30.jsonl
+!traces/README.md
 *.log
 .claude/
 # third_party/vllm tracked in git for patch management
+!traces/w600_r0.0015_st30_first600s.jsonl
+# + time_to_parent_chat annotation (for --dispatch-mode thinktime); same anon data
+!traces/w600_r0.0015_st30_ttp.jsonl
+!traces/w600_r0.0015_st30_first600s_ttp.jsonl

MEETING.md

@@ -0,0 +1,137 @@
+# EAR — Agentic Serving Scheduler 汇报
+
+**One-liner**：Agentic workload 的 KV reuse 93% 在 session 内，turn 间 tool-call 反馈耦合把单 request 延迟差放大成 throughput 差距 —— locality 因此是主导调度杠杆；现有 load-balance 丢 locality、static PD-disagg 撞 D 侧 KV 墙、pure sticky 造 hot pin；我们提 EAR (Elastic Affinity Router) = session-affinity routing + hot-instance 触发 session migration。
+
+---
+
+## 1. 关键洞察：Dispatch Coupling
+
+每个 turn 间有一段外部 gap `T_external`（chatbot 是人类读+想+打字；agentic 是 tool 执行）。**Little's Law `L = Λ · N · (W_turn + T_external)`** 在两种 workload 下都成立 —— 差异在于 `T_external` 的分布相对于 `W_turn` 的位置：
+- `T_external ≫ W_turn` → 开环 regime：scheduler 退一步不动 L
+- `T_external ≲ W_turn` → 闭环 regime：`W_turn(L)` 因 KV 竞争耦合到 L，反馈环把 scheduler 的 ε 退步放大几倍
+
+**Production trace 实测 `T_external` 分布**（next.start − prev.end，formatted session 链作 ground truth）：
+
+![](figs/f3a_inter_turn_gap.png)
+
+| | Agentic | Chatbot |
+|---|---:|---:|
+| p50 | **1.6s** | **7.2s** |
+| gap < 1s | **39%** | 4% |
+| gap < 5s | 67% | 29% |
+| p99 | 738s | 43s |
+
+两个分布形状完全不同：chatbot unimodal 集中在 5–15s（人类节奏）；agentic bimodal —— **39% 的 gap 在 sub-second 里（autonomous tool-call mode）**，外加 13% > 30s 的长尾。**Agentic 的 sub-second mass 是 chatbot 没有的**，正是 dispatch coupling 激活的来源。
+
+**实测 regime**：在 unified（TTFT p90 = 7.3s）下，**73% 的 agentic turn 把系统推进闭环**（W_turn > T_external），chatbot 仅 32%。在 lmetric（15.7s）下 agentic 80%、**chatbot 也到 88%** —— lmetric 把 chatbot 自己也拖进闭环，这就是它在两种 workload 都 underperform 的根因。
+
+**结果**：lmetric 跑 600s trace 用 49 min wall-clock = **8x amplification**。**per-turn W 的小差异被放大成 wall-clock 数量级差距** —— locality 不是 nice-to-have，是 dominant lever。
+
+---
+
+## 2. Workload 实证（三件事）
+
+| | 数据 | 图 |
+|---|---|---|
+| KV reuse 几乎只在 session 内 | intra 93.2% / cross 5.7% / shared 1.1% | ![](figs/f2a_reuse_topology.png) |
+| Session 极度偏斜 | production trace 上 top 1% / 5% / 10% / 25% / 50% = **46.5% / 66.5% / 74.6% / 87.5% / 96.0%** input mass | ![](figs/f2b_session_skew.png) |
+| 单请求 KV footprint 大，单 instance KV pool 很快被占满 | per-instance KV pool ≈ **38 GiB**（0.4 × 96 GiB H20，剩 50% params + 10% activation）；p99 req 11.5 GiB → 一个 instance 只装 **3 个 p99 decode**；4P+4D 让系统 decode 容量直接减半 | ![](figs/f2c_kv_footprint_cdf.png) |
+
+理论 APC 上界 = intra-session 79.6% / any-session 80.3%，差 <1pp。**任何不 affinity 的调度都丢绝大部分 reuse。**
+
+---
+
+## 3. 现有调度的三种失败模式
+
+### Load-balanced (LMetric / round-robin / kv-aware)：丢 locality
+
+![](figs/f4a_apc_loss.png)
+
+LMetric 56.9%、load_only 54.1% APC，远低于 79.6% 上界。23pp 缺口直接来自跨 instance 路由丢的 intra-session hit。
+
+注意 LMetric 比 load_only 只好 **+3.3pp**：LMetric 的 score = `(pending_prefill + input − cache_hit) × num_requests`，cache_hit 只作 cost-model 减项，但 score 是**乘性**的 —— 一个有 affinity 的 instance 因 num_requests 高被乘式吃掉 cache 收益，LMetric 仍然会选冷 instance。sticky 把 cache 作硬约束直接拉到 77.2%。**结论：cache-aware-load routing 不够 —— affinity 必须是独立路由路径，不能折叠进 load cost 里**。
+
+### 静态 PD-disagg：D 侧 KV 容量墙
+
+![](figs/f4b_pdsep_kv_wall.png)
+
+agentic 平均请求 33.6k token 需 3.3GB KV；4P+4D / 6P+2D 在 agentic regime 都穿过 90% 内存墙。**TTFT p50 暴涨 62-72x，成功率 99.5% → 52-68%**。
+
+### Pure sticky：全员被 hot session 拖累
+
+![](figs/f4c_per_worker_ttft.png)
+
+我们刻意 **用 (median, max) 两个绝对数**衡量 worker 不平衡，不用 `max/median` 单一比值 —— 比值会把 unified（一个 worker 牺牲、其他 7 个快）算成比 sticky（全员一起慢）更不平衡，与系统 e2e p90 实际排序反向。下面是绝对数：
+
+| | median worker TTFT p90 | max worker | system e2e p90 |
+|---|---:|---:|---:|
+| sticky | **20.3s** | 55.4s | **34.6s** |
+| unified | **10.3s** | 37.7s | **18.0s** |
+
+机制：production trace 上 top 1% 的 session 占 46.5% input 量、且 hot session 数量远多于 instance 数（8 个），sticky 的 hash 绑定让 **每个 worker 都自己承接一份 hot session**，median worker 也被拖慢。Unified 用 LMetric fallback 把 cold/new session 重路由到非 hot worker，保留 7/8 worker 的速度。系统 p90 由大多数请求决定，所以 unified 几乎 2x 快。
+
+---
+
+## 4. EAR 设计
+
+两个 pillar，所有 instance 对称 PD-colocated（无静态 P/D 分区）：
+
+**Pillar 1 — Affinity-default routing（已实现）**
+新 session 用 load-balance 分配 host；后续 turn 按 session→host binding 路由。
+→ 这就是当前 `unified` 算法（hybrid LMetric + high-cache affinity），APC 79.4%，达到上界 97%。
+
+**Pillar 2 — Hot-triggered session migration（end-to-end 实证待补，substrate 已验证）**
+当 host 的 `pending_prefill_tokens > T_hot`，把整个 session 的 KV 通过 mooncake `kv_connector` migrate 到更轻 instance；session binding 更新；后续 turn 路由到新 host。
+
+> 🆕 **2026-05-27 数据**（commit `ef9e010`）：之前认为是 migration blocker 的 `kv_both` substrate overhead 已经不存在。在 8×TP1 trace replay 上 A/B/C 对比：
+> - plain unified: TTFT p90 = 11.97s
+> - unified + `kv_both`（未 DR-fix）: 9.74s（**−18.6%** vs plain）
+> - unified + `kv_both` + DR-fix: 7.58s（**−36.6%** vs plain）
+>
+> 即原 elastic_migration_v2 论文里 "+45% kv_both penalty" 已 obsolete；当前 substrate 是 **net positive**（connector mode 的 `delay_free_blocks=True` 在 93% intra-session-reuse trace 上把跨 turn cache hit 窗口拉长）。Migration 之前 4 次 revert 的主因消失。
+
+关键 design：
+- Target 选择用 **observable pending prefill tokens**，**不用** cost-model prediction（实测 mooncake cost model 误差 10-21x，绕过）
+- Per-session cooldown 防 thrashing
+- 若无候选 instance 能装下 session context → 保留当前 binding，opportunistic 不 mandatory
+
+---
+
+## 5. 进展 & TODO
+
+### ✅ 已完成
+
+- Workload characterization 三件事的实证齐全（`f2a/b/c`）
+- 三类 baseline 失败的实证齐全（`f4a/b/c/d`）
+- Anchor + paper outline（`PAPER_OUTLINE.md`）
+- Pillar 1 affinity routing 已实现并测过（current `unified` 算法）
+- Dispatch coupling 的 Little's Law 形式化推导
+- `replayer/replay.py` patched 输出 `amplification`
+- 🆕 **kv_both substrate validation**（commit `ef9e010`）：trace replay A/B/C 证明 substrate 已经是 net positive（TTFT p90 −18.6% / DR-fix 后 −36.6% vs plain），原 +45% penalty obsolete
+
+### 🟢 不依赖 migration 可以现在做
+
+1. **5 baseline × 3 runs wall-clock sweep**（patched replayer 直接出 amplification 字段）— §2.3 的实证 closure，**最高优先级**，一晚能跑完
+2. Static PD-disagg 补进 end-to-end 表
+3. λ / skew / KV pool 三轴 sensitivity
+4. Draft §1-§4 正文（数据已齐）
+
+### 🚧 待 migration end-to-end validation
+
+- §4.3 migration mechanism 的 e2e trigger + target selection 实验（substrate 已通，只缺策略层）
+- Full ablation（migration-only + both）
+- §5.6 migration microbench
+
+### 风险
+
+- Migration 之前 4 次尝试（`6b255fa`, `e991960/5772149`, `cc6e562`, `4c583f2`）都被 transfer overhead 吞掉而 revert ——  **该 overhead 已在 2026-05-27 验证不再存在**（substrate net positive）
+- 仍未直接验证 e2e migration 策略层（trigger + target 选择）能在反馈环里产生正收益；中间还有"决策错误 + cooldown thrashing"两类风险，独立于 substrate
+- 即便 migration e2e 仍 marginal，affinity-only pillar 的实证已经独立成立，paper 至少有 strong-affinity 的 storyline 可写
+
+---
+
+## 6. 一句话总结要 sell 的事
+
+> **Agentic 让 locality 从 nice-to-have 变成 dominant lever（dispatch coupling 论证）；EAR 用 affinity-default + hot-triggered migration 单一方案同时拿到 locality 和 balance。Pillar 1 已实证（APC 79.4%）；Pillar 2 design 完整、validation pending in DR-fix 之上的重测。**
+
+下一步主战场：跑 wall-clock sweep 把 §2.3 dispatch coupling 论证钉死。

PAPER_OUTLINE.md

@@ -0,0 +1,362 @@
+# GPU-Hit-First: Serving Agentic LLM Workloads by Keeping the Working Set in HBM
+
+> **Thesis (one-liner)**: 对 agentic LLM 负载，用户感受到的端到端 metric 是 **request latency / TPS / GPU
+> utilization**，而它们由一件事主导 —— **KV cache 命中是否发生在 GPU HBM 上**。Agentic 的 KV reuse 93% 在
+> session 内、且活跃 working set 小到一个节点就能常驻 HBM；命中层级 `GPU ≫ CPU-local > remote-RDMA-store ≫
+> recompute` 的代价差随 context 拉大。由此得到一条统一原则 —— **GPU-hit-first**：把活跃 working set 留在 HBM，
+> 而不是建深的 CPU/storage hierarchy 去追长尾。三个推论分别修复现有系统的三处失配：(3.1) 让 PD-colocation 重新
+> 成为默认；(3.2) 在全局路由里做 biased KV-cache-awareness；(3.3) 用 KV migration 而非 replication 做跨实例
+> GPU 去重。
+
+> **Framing note (2026-05-30)**：本 outline 取代早期的 "EAR: Elastic Affinity Routing" 版本（保留在 git
+> 历史里）。原 EAR 的 dispatch-coupling 形式化在此 **降级为 §2 的 metric 论证**（解释"为什么是 request latency
+> 而不是 TTFT/TPOT"），不再是 headline；headline 升格为 GPU-hit-first 原则，affinity routing / migration
+> 成为该原则的两个推论（§3.2 / §3.3）。
+
+---
+
+## 📊 Validation Status (2026-05-30)
+
+| 章节 | 论点 | 证据 | 状态 |
+|---|---|---|---|
+| §1 | 背景：PD-colo / PD-disagg / KV storage hierarchy | — | 写作 |
+| §2 metric | request latency 而非 TTFT/TPOT；TPS；GPU util | dispatch coupling + amplification；`bench_report.py` | 🟡 论证全，wall-clock sweep 待补 |
+| §2.1 | KV hit 普遍且关键 | `f2a` intra 93.2%、APC 上界 79.6%、C1/C2 | ✅ |
+| **§2.2** | **GPU hit > CPU hit > RDMA-store hit ≫ miss** | **`v2/figs/exp_a_tier_latency.png`（四层实测）** | ✅ **NEW** |
+| **§2.2 Ev#1** | **GPU 足以常驻"有价值的" working set** | **`v2/figs/exp_b_capacity_knee.png`（knee）+ working_set + cluster-scale 校正** | ✅ **NEW** |
+| §3.1 | Make PD-colocation great again | `PD_DISAGG_RESULTS`、`crossover_pd_advantage`、MB1/MB2、C2/C3 | ✅ |
+| §3.2 | Biased KV-aware global routing | LPWL（TTFT p90 −31%）、LMetric 乘性稀释、sticky hot-pin、ES ablation | ✅ |
+| §3.3 | GPU dedup via migration not replication | substrate net-positive（−18.6%/−36.6%）；correctness smoke tests | 🟡 substrate 通，policy e2e 待验证 |
+| §4 | 集成系统端到端 eval | 散落 mb1/mb2/mb5/crossover/lpwl，需统一 | 🚧 |
+| §5 | Related work（含 storage hierarchy 正面回应）| — | 写作 |
+
+---
+
+## §1 Background and System Setup
+
+### §1.1 LLM 与 KV cache
+
+Transformer 自回归推理分两段：**prefill**（一次性算完 prompt 的全部 KV，compute-bound）与 **decode**（逐 token
+生成，memory-bandwidth-bound）。每个 token 的 KV 常驻 GPU HBM 才能被后续 attention 复用。Prefix caching（APC）让
+相同 prompt 前缀直接命中已算好的 KV，省掉重复 prefill —— 这是本文全部优化的物理基础。
+
+> Qwen3-Coder-30B-A3B（GQA, 48 层, 4 KV heads, head_dim 128, bf16）：**KV = 96 KiB/token**，1 GiB = 10,923
+> token，block(16 tok) = 1.573 MB。
+
+### §1.2 Agentic workflow
+
+Agentic 负载 = LLM 通过 tool-call 自驱、多 turn 完成任务。与 chatbot 的本质差异：每个 turn 由上一个 turn 的
+tool-call 结果触发（无人类 think-time），prefill-dominated（input/output ≈ 75×）。
+
+**但它是一个 mixture，不是"全多轮"**（C1, `figs/workload_chars/c1_session_mixture.png`）：
+
+- **90.3%** 的 session 是单轮（mean 1.62 turns）；但多轮 session（9.7%）= **44.2% 的请求**、**66.9% 的
+  prefill 质量**。
+- Continuation hazard（Lindy）：turn1→2 仅 **10.2%**，turn5→6 87%，turn12→13 **94.3%** —— heaviness 在
+  cold-start 几乎不可预测（corr(turn1_input, n_turns) = **0.04**）。
+
+> **Routing 含义（贯穿 §3.2）**：heaviness 在 session 起点不可预测 → 必须 **reactive**（观测累计负载），不能
+> proactive 预判；单轮海洋与深尾的最优策略相反（前者 load-balance、后者 affinity-pin），且 turn-1 无法区分 →
+> 唯一可行的策略是"人人 load-balanced 起步、随 turn 累积变 sticky" —— 正是 LPWL 的 emergent 行为。
+
+### §1.3 Serving agents in the wild
+
+- **PD colocation（8C）**：每个 instance 对称，prefill 与 decode 在同一 GPU 上由 chunked-prefill +
+  continuous batching 交错。弹性 KV 池，无静态分区。
+- **PD disaggregation**：把 instance 静态分成 prefill 池（P）与 decode 池（D），物理隔离两个阶段
+  （DistServe / Splitwise）。
+- **KV cache storage hierarchy**：GPU HBM → CPU DRAM → 远端 pooled store（RDMA/SSD，如 Mooncake Store /
+  LMCache）。把被淘汰/跨实例的 KV 下沉到更慢但更大的层，用传输换重算。
+
+> 三者各被本文一节回应：PD-colo 在 §3.1 被"复活"为默认；PD-disagg 在 §3.1 被证否（agentic regime）；storage
+> hierarchy 在 §2.2 被定量地"限位"（GPU 命中远胜下层，且活跃 working set 本就装得下）。
+
+---
+
+## §2 GPU memory hit is the key to serving agents
+
+### §2.0 正确的 metric：request latency / TPS / GPU utilization（不是 TTFT/TPOT）
+
+**为什么不是 per-request TTFT/TPOT**：agentic 的 turn 之间有反馈环，单 turn 的延迟会跨 turn **复利**成 session
+端到端时间与系统吞吐差距。只有 **request/session latency、tokens-per-second、GPU utilization** 能 capture 这件事。
+
+**Dispatch coupling（降级为本节论证）**。每个 turn 间有外部 gap `T_external`（chatbot 是人在读/想/打字；agentic
+是 tool 执行）。Little's Law：`L = Λ · N · (W_turn(L) + T_external)`。当 `W_turn ≳ T_external` 时
+`W_turn(L)` 经 KV 竞争耦合到并发 L，scheduler 的 ε 退步被反馈环放大成 wall-clock 数倍差。
+
+production trace 实测 `T_external` CDF（`figs/f3a_inter_turn_gap.png`）：
+
+| | Agentic | Chatbot |
+|---|---:|---:|
+| p50 | **1.6 s** | 7.2 s |
+| gap < 1 s | **39%** | 4% |
+| gap < 5 s | 67% | 29% |
+| p99 | 738 s | 43 s |
+
+agentic 有一段 chatbot 没有的 **sub-second tool-call mass（39% vs 4%）**，几乎天然 `W_turn ≫ T_external` → 闭环。
+**实测**：lmetric 跑 600s trace 用 49 min wall-clock = **8× amplification**。**结论：per-turn 延迟的小差被放大成
+端到端数量级差 → 必须用 request latency / TPS / GPU util 衡量。**
+
+- **TPS / GPU util** 的工具：`microbench/fresh_setup/bench_report.py`（TTFT/TPOT/E2E 全分位 + TPS +
+  per-worker GPU util）。PD stall 时 GPU ~0% util vs colo 34%（§3.1）即是 GPU-util 作为 metric 的直接体现。
+
+> **🚧 待补**：5 baseline × ≥3 runs 的 wall-clock amplification sweep（replayer 已输出 `amplification` 字段），
+> 钉死本节实证 closure。优先级高。
+
+### §2.1 KV$ hit is common and critical
+
+Trace 上 KV reuse 的分解（`figs/f2a_reuse_topology.png`）：**intra-session 93.2% / cross-session 5.7% /
+shared-prefix 1.1%**。理论 APC 上界：intra-only **79.6%** vs any-session 80.3%，差 <1pp —— **cache 本质上是
+session-local 的**。
+
+per-turn 视角（C2, `figs/workload_chars/c2_work_amortization.png`）：resident context 11k→56k+ token 增长而
+new-prefill 从 2.7k 坍缩到 ~200 token，per-turn reuse 爬到 **99.6%**，resident/new（"PD tax"）到 turn 12 ≈
+250×、turn 30 ≈ 450×。**绝大部分 prefill 工作是可被命中省掉的**；命中与否直接决定 TTFT。
+
+### §2.2 Hits on GPU is more important than the CPU
+
+**命中层级的代价是实测的，不是断言的**（Qwen3-Coder-30B-A3B / H20）。TTFT(s, p50) 服务一段长 L 的复用前缀，
+来自每个 KV 层：
+
+![四层命中延迟：GPU < CPU-local < remote-RDMA-store ≪ miss，差距随 context 拉大](v2/figs/exp_a_tier_latency.png)
+
+
+| prefix L | miss(recompute) | **remote RDMA store** | CPU-local(DRAM,PCIe) | GPU(HBM) | miss/RDMA | RDMA/CPU | CPU/GPU |
+|---:|---:|---:|---:|---:|---:|---:|---:|
+| 8k  | 0.588 | 0.151 | 0.076 | 0.053 | 3.9× | 2.0× | 1.5× |
+| 16k | 1.547 | 0.262 | 0.105 | 0.063 | 5.9× | 2.5× | 1.7× |
+| 32k | 4.604 | 0.680 | 0.158 | 0.080 | 6.8× | 4.3× | 2.0× |
+| **64k** | **15.23** | **0.97** | **0.27** | **0.11** | **15.8×** | **3.6×** | **2.4×** |
+
+- **GPU hit ~flat**（42→111 ms / 1k→64k）：命中即整段前缀在 HBM，只重算最后一个 token。
+- **CPU-local hit** transfer-bound（PCIe H2D 实测 ~54 GB/s）；CPU-hit ≈ GPU-hit + KV/PCIe + ~0.15s 开销。
+  （native KV offload，命中经 `vllm:external_prefix_cache_hits` 100% 验证。）
+- **remote RDMA-store hit** = Mooncake-Store 机制（实测：两 instance，B 用 `do_remote_prefill` 经 RDMA 从 A 拉取
+  缓存前缀而非重算；`mb2_kv_transfer.py` / `v2/.../run_rdma.sh`）。对 recompute 是大赢（**最高 16×**，与 blog 的
+  46× 同向），但付 **NIC 税**（有效 ~5–7 GB/s，cf. MB2 raw ~9.7 GB/s；multi-NIC pooling 可抬高），故比 CPU-local
+  慢 3.6×、比 GPU 慢 ~9×（64k），**代价差随 context 拉大**。
+- **结论 —— 层级严格且随 context 拉大：`GPU < CPU-local < remote-RDMA-store ≪ miss`**。global KV store 确实
+  有用（这也是该路线存在的理由），但每靠近 GPU 一层就再省 1.4–4× TTFT。**最值钱的复用是 GPU-resident 的那种。**
+
+#### Evidence #1：GPU is sufficient to hold most KV requests
+
+**realized APC 与 latency 在很小的 GPU 容量就饱和**（closed-loop 多轮负载，并发 4，扫 GPU KV 容量）：
+
+![容量 knee：APC 与 TTFT p90 在 3.6 GB（=活跃 working set）饱和，之后 dead-flat](v2/figs/exp_b_capacity_knee.png)
+
+| GPU KV (GB) | realized APC | TTFT p90 |
+|---:|---:|---:|
+| 1.2 | 7.4% | 13.00 s |
+| 2.4 | 36.3% | 4.62 s |
+| **3.6** | **80.3%** | **0.53 s** |
+| 9.7 | 72.9% | 0.65 s |
+| 14.5| 72.9% | 0.65 s |
+
+**Knee 出现在 3.6 GB = 恰好 = 活跃 working set（4 session × 0.91 GB）**：APC 饱和到上界、TTFT p90 从 13.0s 坍缩
+到 0.53s，之后 dead-flat。**超过 working set 的 HBM 买不到额外收益；为追长尾而建的 CPU/storage tier 同理 ≈ 0。**
+
+**Cluster-scale 校正（关键）**：working_set 分析（`analysis/working_set/`，`figs/working_set/`）显示"装下整个
+2h cluster 的全部 reuse 尾巴需 ~14 节点"，**这不构成 CPU offload 的动机** —— 那是用 1 个 replica 容量去装**整个
+cluster** 的 reuse；产出该 trace 的真实 cluster 远不止 14 节点（trace 是 cluster 聚合，见
+`project-trace-is-cluster-level`）。在 per-cluster 的 HBM 总量下，活跃 working set 本就 GPU-resident（live KV
+533–1157 GB ≪ 单节点 1528 GB）。**knee 位置随并发线性增长 = 随 cluster GPU 数增长**，而 cluster 本就提供了它。
+
+> ⚠ **Scope**：本小节"装得下"指的是**活跃 working set 产生的近期高价值复用**，不是"全部 reuse 尾巴"。冷
+> session 长 gap 后回来的深尾命中（既低价值/byte 又贵于 fetch）正是 storage-hierarchy 派追的东西；本文论点是
+> 在 agentic 下这条尾巴不值得为之建深层级。§5 正面回应该派。
+
+### §2.3 Takeaway
+
+正确的 metric（§2.0）+ 命中集中在 GPU 才便宜（§2.2）+ 活跃 working set 装得下 HBM（Ev#1）⇒ **GPU-hit-first**：
+设计目标是最大化活跃 working set 的 **GPU 常驻 + 命中**，而非建深 CPU/storage hierarchy。§3 给出三个推论。
+
+---
+
+## §3 Optimizing agent serving with the GPU-Hit-First Principle
+
+### §3.1 Make PD-colocation great again
+
+静态 PD-disaggregation 对 chatbot 有效，对 agentic 结构性失败 —— colocation 才应是默认。
+
+**端到端证据**（`microbench/fresh_setup/PD_DISAGG_RESULTS.md`，8×H20，trace replay）：**没有任何静态 P/D 比能赢
+8-way colocation（8C）**，且失败模式随比例移动：
+
+| Metric | **8C** | 6P+2D | 4P+4D | 2P+6D |
+|---|---:|---:|---:|---:|
+| completion | **100%** | 100% | 100% | **9%** 💀 |
+| wall-clock | **2994 s** | 3419 | 4171 | 5762 |
+| prefix-cache hit | **19.4%** | 0% | 0% | 0% |
+| TTFT p50 | **7.0 s** | 41.0 | 56.4 | 23.6 |
+| E2E p90 | **83.3 s** | 91.8 | 157.1 | 499 |
+
+- **D-heavy（4P+4D）**：decode 池饱和 **97.5%**、prefill 池 ~30% —— 半个 cluster 的 KV 被困在错的一侧；agentic
+  请求大（p99 KV **11.5 GiB**），4D 让系统 decode 容量直接减半（24→12 并发，`figs/f2c_kv_footprint_cdf.png`、
+  `figs/f4b_pdsep_kv_wall.png`）。
+- **P-heavy（2P+6D）**：prefill 池 jam 99.7%，872 请求堆积，**91% 永不完成**。
+- **更聪明的路由救不了**（§6.x）：给 P 侧加 session-affinity 反而更差（4P+4D completion 100%→36%），GPU ~0%
+  util，cluster 卡在 KV-transfer 协调而非 compute —— 复现 producer hot-pinning。
+
+**为什么 colo 赢（正确论证，C2/C3 支撑）**：
+
+- **时变 P:D 需求**：agentic 同时在 roofline 两侧有实质工作 —— compute-bound prefill（~30% 时间）+
+  memory-bound decode（**~70% 时间**，C3 token≠time 校正，`figs/workload_chars/c3_prefill_decode_balance.png`）。
+  colo 的弹性池吸收当下热的那一相；静态分区让 P-instance 带宽闲、D-instance 算力闲。
+- **resident KV 本地化**（C2）：下一 turn 的 prefix = [prevPrompt+prevAnswer] 横跨 P/D 两侧，disagg 必须
+  gather/transfer，colo 免费本地保留。
+- **transfer 不便宜且拓扑无关**（MB2，`figs/mb2_transfer_time_compare.png`）：Mooncake `batch_transfer_sync_write`
+  恒走 RDMA NIC（~9.7 GB/s），intra ≈ inter；PD-disagg 的 per-request transfer 税无法靠拓扑买回。
+- **phase-isolation 是 disagg 唯一的真赢面但被压倒**（MB1，`figs/mb1_interference.png`：32k prefill 让
+  per-stream TPOT 退化 52×，131k → 183×）—— 但被 D 侧容量天花板压倒（见上）。
+
+**边界（不 overclaim）**：crossover sweep（`analysis/crossover/`，`figs/crossover_pd_advantage.png`）给出 colo
+停止占优的 input 长度 —— colo 在 agentic 工作点赢，且我们知道边界在哪。
+
+### §3.2 Biased KV-cache-awareness in global routing
+
+GPU-hit-first 在路由层 = **把 cache-awareness 作为带偏置的独立路由路径**，而不是折叠进 load cost。
+
+**反例：load-balanced / 朴素 cache-aware-load 丢 locality**（`figs/f4a_apc_loss.png`）。LMetric（OSDI'26）打分
+`P = pending_prefill + (input − cache_hit)`，`score = P × num_requests` —— cache 只作 cost-model **减项**，而 score
+是**乘性**的，有 affinity 的 instance 因 num_requests 高被乘式吃掉 cache 收益：
+
+| 策略 | APC | vs load_only | cache 处理方式 |
+|---|---:|---:|---|
+| load_only | 53.9% | — | 无 |
+| LMetric | 57.2% | **+3.3pp** | cost-model 减项（被稀释）|
+| sticky | 77.7% | **+23.8pp** | 硬约束 |
+| unified | 78.7% | **+24.8pp** | 硬+软混合 |
+
+`load_only→LMetric` 的 +3.3pp 几乎可忽略；**+20.5pp 的回报来自把 cache 作独立路由路径**。
+
+**本文方法：LPWL（least-prefill-work，parameter-free）**（`project-lpwl-policy`、`analysis/lpwl_5policy_600s.md`）：
+按 `new_uncached ≈ input − cache_hit` 路由 —— `new_uncached≈input`（冷/新 session）自动按负载分散，`new_uncached≈0`
+（暖 session）自动 stick。**零旋钮**，在 600s trace 上击败 tuned unified+A+B（TTFT p90 **−31%**），full w600 上打平。
+这正是 C1 mixture 要的形状：无需 classifier 自动切分单轮海洋与深尾。
+
+**两条被否的杠杆（节省读者时间）**：
+
+- **real-time engine state 不是路由杠杆**（ES ablation，`project-es-ablation-sweep`）：只对一次性 placement 有用
+  （sticky −26%），对 per-req load-chasing 有害（load_only +27%）。
+- **derived-κ decode 项 net-negative**：decode-awareness 是错的杠杆。
+
+**Pure sticky 的失败用绝对 per-worker latency 衡量**（`figs/f4c_per_worker_ttft.png`，不能用 max/median 比值）：
+
+| | median worker TTFT p90 | max worker | system e2e p90 |
+|---|---:|---:|---:|
+| sticky | **20.3 s** | 55.4 s | **34.6 s** |
+| unified | **10.3 s** | 37.7 s | **18.0 s** |
+
+hot session 数 ≫ instance 数（8），sticky 的 hash 绑定让**每个 worker 都自接一份 hot session**，median 也被拖慢；
+biased 路由把 cold/new 重路由到非 hot worker，保 7/8 worker 速度 → e2e p90 ~2× 快。**这引出 §3.3：sticky 的残余
+hot-pin 需要 migration 解。**
+
+### §3.3 GPU KV-cache dedup with migration instead of replication
+
+**视角**：多个 instance 各自缓存同一段 prefix = GPU 容量被 **replication** 浪费；GPU-hit-first 要求**全局只留一份 +
+把 session 搬到那一份**（migration/dedup），既保 GPU 命中又均衡负载、修复 §3.2 的残余 hot-pin。
+
+- **Trigger**：host 的 `pending_prefill > T_hot` 且 session 在 `T_cool` 内未迁移过。
+- **Target**：用 **observable pending prefill tokens** 选最轻 instance，**不用** cost-model 预测（mooncake cost
+  model 误差 10–21×，by construction 绕过）。
+- **Mechanism**：当前 request 重定向到 target，session KV 经 Mooncake `kv_connector` 迁移；binding 更新；后续
+  turn 按 affinity 路由到新 host。迁移成本在该 session 剩余 turn 上摊销。
+- **Thrashing prevention**：per-session cooldown `T_cool`。
+
+> **状态**：**substrate 已验证 net-positive**（`kv_both` connector 在 trace replay 上 TTFT p90 **−18.6%**，
+> DR-fix 后 **−36.6%** vs plain；之前认为是 blocker 的 transfer overhead 已不存在，4 次历史 revert 的主因消失）。
+> migration correctness smoke tests 通过。**policy 层 e2e（trigger + target 在反馈环里的真实收益）仍未直接验证**
+> —— 这是全文最弱的一环，独立风险是"决策错误 + cooldown thrashing"。affinity-only pillar（§3.2 LPWL）已独立成立，
+> 即便 migration 仍 marginal，paper 也有 strong-routing 主线。
+
+---
+
+## §4 System Evaluation
+
+> **🚧 关键缺口**：目前证据散落在 mb1/mb2/mb5/crossover/lpwl/v2；§4 需要一个**集成系统**（colocation +
+> biased routing + dedup-migration，统一命名）跑端到端、用 §2.0 的新 metric（request latency / TPS / GPU util）
+> 评测，并把 §3.1/§3.2/§3.3 做成 ablation。
+
+### §4.1 Setup
+- **Trace**：真实 Qwen3-Coder agentic trace（cluster-level，见 `project-trace-is-cluster-level`）；日常迭代
+  `w600_r0.0015_st30`（~850 req / ~13 min / peak QPS ~1.6 / APC headroom ~76%）。
+- **Hardware**：8× H20 (96 GB)；vLLM 0.18.1 (V1, chunked-prefill) + Mooncake。
+- **Baselines**：① round-robin ② LMetric (OSDI'26) ③ kvcache-aware+load 线性混合 ④ sticky ⑤ static PD-disagg
+  (4P/4D) ⑥ **本文系统**（colo + LPWL + dedup-migration）。
+- **Metrics**：request latency (mean/p50/p90/p99)、**TPS**、**GPU util (median/max worker)**、APC、
+  wall-clock amplification（不单看 TTFT/TPOT，§2.0）。
+
+### §4.2 End-to-end
+- `figs/f6_e2e_latency_bars.png` / `f6_e2e_latency_full_grid.png`（现有 4–5 baseline；🚧 补 static PD-disagg
+  列 + 本文系统列）。
+
+### §4.3 Ablation（GPU-hit-first 三推论各关一个）
+- −colocation（→ static PD-disagg，§3.1）
+- −biased routing（→ load-balance / LMetric，§3.2）
+- −dedup-migration（→ pure sticky，§3.3）
+- 🚧 migration-only / full 待 policy e2e 验证。
+
+### §4.4 Microbench 支撑（已有，复用）
+- §2.2 四层命中：`v2/figs/exp_a_tier_latency.png`
+- §2.2 容量 knee：`v2/figs/exp_b_capacity_knee.png`
+- §3.1 PD-disagg 失败：`figs/f4b`、`PD_DISAGG_RESULTS`、`crossover_pd_advantage`
+- transfer cost：`figs/mb2_*`；phase interference：`figs/mb1_interference.png`
+
+### §4.5 Sensitivity
+- 到达率 λ / skew(Zipf α) / KV pool size（不依赖 migration，可做）；`T_hot`/`T_cool`（依赖 migration，deferred）。
+
+---
+
+## §5 Related Work
+
+- **Serving systems**：vLLM, Mooncake, SGLang, DistServe, Splitwise。本文基于 vLLM+Mooncake，与
+  DistServe/Splitwise 不同在于**不做静态 P/D 分区**（§3.1 给出 agentic regime 的证否 + crossover 边界）。
+- **Cache-aware routing**：LMCache, Production-Stack, LMetric (OSDI'26)。本文指出 cache 信号不能折叠进乘性 load
+  score（§3.2 LMetric 稀释分析），须作独立 biased 路由路径。
+- **KV storage hierarchy / offload（主要对手，正面回应）**：Mooncake Store, LMCache, AttentionStore 等把 KV 下沉
+  到 CPU/SSD/远端 pool。**本文不否认其对 recompute 的收益**（§2.2 实测 remote-RDMA 命中比 recompute 快达 16×，
+  与 Mooncake-Store blog 的 46× 同向）；但论证在 **agentic** 下 (i) 命中层级 GPU ≫ CPU ≫ RDMA-store（§2.2），
+  (ii) 活跃 working set 本就 GPU-resident（§2.2 Ev#1）⇒ 深层级的边际收益低，应优先 GPU 常驻而非建深 hierarchy。
+- **Stateful migration**：Pollux, Gandiva（RL training 的 migration-as-rebalancing）。本文把该思路迁到 LLM KV
+  cache 的 dedup（§3.3）。
+
+---
+
+## §6 Conclusion
+
+对 agentic LLM 负载，用户感受的 request latency / TPS / GPU util 由 **KV 命中是否在 GPU HBM 上** 主导。命中层级
+`GPU ≫ CPU > RDMA-store ≫ recompute` 且代价差随 context 拉大（§2.2），而活跃 working set 小到本就 GPU-resident
+（§2.2 Ev#1）。**GPU-hit-first** 原则由此统一三件事：复活 PD-colocation（§3.1）、在路由里做 biased
+KV-awareness（§3.2）、用 migration 而非 replication 做 GPU 去重（§3.3）。
+
+---
+
+## Work Plan
+
+### ✅ Done
+- §1/§2 背景与 metric 论证；dispatch-coupling 数学；inter-turn gap CDF（`f3a`）
+- §2.1 reuse topology / C1–C3（`f2a`、`workload_chars/*`）
+- **§2.2 四层命中实测（`v2/exp_a_tier_latency`：GPU/CPU/RDMA/miss）**
+- **§2.2 Ev#1 容量 knee（`v2/exp_b_capacity_knee`）+ working_set + cluster-scale 校正**
+- §3.1 PD-disagg 证否（`PD_DISAGG_RESULTS`、crossover、MB1/MB2）
+- §3.2 LPWL（−31%）、LMetric 稀释、sticky hot-pin、ES ablation
+- §3.3 migration substrate net-positive + correctness smoke tests
+
+### 🟢 不依赖 migration，现在可做
+- §2.0 wall-clock amplification sweep（5 baseline × ≥3 runs）— **优先级最高**
+- §4 集成系统命名 + 端到端 baseline 矩阵（含 static PD-disagg 列）
+- §4.5 λ / skew / KV pool sensitivity
+- 草拟 §1–§3 正文（证据/图已齐）
+
+### 🚧 Deferred（待 migration policy e2e）
+- §3.3 migration trigger + target 的反馈环收益验证
+- §4.3 full / migration-only ablation
+- §4.5 `T_hot` / `T_cool` sensitivity
+
+### 🎨 待画
+- §1.3 storage-hierarchy 示意（GPU HBM → CPU DRAM → RDMA store）
+- §2.0 dispatch-coupling schematic（chatbot vs agentic timeline + 反馈环）
+- 集成系统 architecture 图
+
+### ❓ Open
+- 集成系统最终命名（GPU-hit-first 是原则；系统名待定）
+- §4 instance 数 / trace 总长定稿

README.md

@@ -0,0 +1,77 @@
+# agentic-kv
+
+Serving agentic LLM workloads by keeping the KV working set in GPU HBM
+(GPU-hit-first). Research outline: [`PAPER_OUTLINE.md`](PAPER_OUTLINE.md).
+Evidence + experiments: [`v2/`](v2/).
+
+---
+
+## ⚠️ Benchmarking methodology — read this first
+
+> **Replay agentic traces with `--dispatch-mode thinktime`, not the default
+> `tracets`.** It is the faithful, more realistic load — and the dispatch mode
+> materially changes the performance you measure.
+
+The replayer offers two ways to time each turn:
+
+| mode | turn-k dispatched at | what it models |
+|---|---|---|
+| `tracets` (default) | `max(prev_turn_finished, trace_ts)` | absolute production schedule |
+| **`thinktime` (use this)** | `prev_turn_finished + time_to_parent_chat` | real closed-loop agent pacing |
+
+**Why it matters.** `tracets` collapses the inter-turn think-time to ~0 whenever
+the system falls behind (it fires the next turn immediately because the trace
+timestamp is already in the past). That manufactures **artificial request
+bursts** — spiking instantaneous concurrency → KV-pool pressure → preemption →
+inflated tail latency and wasted throughput. `thinktime` keeps each turn's real
+gap (tool-exec + agent think), so the offered load is what a real agent produces.
+
+**Measured (w600 first-300s window, 8×H20, round-robin, 100% completion):**
+
+| metric (N=8) | `tracets` (Mode 1) | **`thinktime` (Mode 2)** | Δ |
+|---|---:|---:|---:|
+| E2E p90 | 102.8 s | **73.5 s** | **−28%** |
+| E2E p99 | 245 s | **227 s** | −7% |
+| TTFT p90 | 56.1 s | **39.7 s** | **−29%** |
+| system TPS | 111.8 | **119.3** | **+7%** |
+| wall-clock | 967 s | **787 s** | −19% |
+| TPOT p90 | 0.174 s | 0.188 s | ~flat |
+
+So under realistic capacity, `tracets` makes the system look **~30% worse on
+tail latency** than it actually is. Tell-tale: scaling 6→8 instances barely helped
+`tracets` (975→967 s — its bursts re-saturate regardless of capacity) but helped
+`thinktime` a lot (1125→787 s). Under heavy saturation (N=6) the two converge
+(E2E p90 ≈ 118–120 s), since there is no slack for bursts to harm. Decode (TPOT)
+is dispatch-independent everywhere.
+
+**Recommendation:** benchmark with `--dispatch-mode thinktime`; use `tracets`
+only as an explicit bursty stress case. Full ablation:
+[`v2/exp_c_dispatch_ablation/`](v2/exp_c_dispatch_ablation/).
+
+### How to use it
+
+```bash
+# 1. annotate a trace with the real per-turn gap (one-time; scans the raw trace)
+python scripts/add_time_to_parent.py traces/w600_r0.0015_st30.jsonl traces/w600_ttp.jsonl
+
+# 2. replay closed-loop with faithful think-time
+python -m replayer --trace traces/w600_ttp.jsonl --endpoint <eps> \
+    --model <model> --dispatch-mode thinktime
+```
+
+`time_to_parent_chat = this_turn.request_ready_time_ms − parent_turn.request_end_time_ms`,
+computed from the raw trace and stored per request; turn-1 has none (fires at its
+trace arrival). Traces without the field fall back to `tracets`.
+
+---
+
+## Project map
+
+- [`PAPER_OUTLINE.md`](PAPER_OUTLINE.md) — GPU-hit-first paper outline (the thesis).
+- [`v2/`](v2/) — evidence experiments:
+  - `exp_a_tier_latency/` — KV-hit cost by tier (GPU < CPU-local < remote-RDMA < miss).
+  - `exp_b_capacity_knee/` — realized APC / latency knee vs GPU capacity.
+  - `exp_c_dispatch_ablation/` — the replay-mode study above.
+- `replayer/` — trace replayer (`--dispatch-mode`, closed-loop think-time).
+- `scripts/add_time_to_parent.py` — trace annotation for `thinktime`.
+- `microbench/`, `analysis/` — PD-disagg, routing, workload characterization.

REPORT.md

@@ -27,6 +27,12 @@ For agentic LLM workloads (long input, short output, high KV cache reuse), is pr
 >   was removed when replay moved to trace-driven dispatch. The next-step
 >   experiment requires restoring the flag first (see `FIXES.md` §B2
 >   route A) before any production-concurrency numbers can be produced.
+> - **§3.9 "Final Design" framing**: the single-argmin + PUSH-migration
+>   design was retired after `cc6e562` / `4c583f2` showed forced and
+>   relaxed-gate migration variants both regressed E2E tail. Current
+>   policy is the hybrid LMetric + high-cache affinity landed in
+>   `255c8e6`. See the per-section note in §3.9 and the active algorithm
+>   in `docs/migration-policy-design.md`.
 >
 > The authoritative results are in **§3.6 and §3.7**.
 
@@ -356,7 +362,23 @@ The elastic numbers on dash1 were genuinely fresh. The "improvement" was actuall
 
 **Output**: `outputs/eval_direct_rdma_v*/` on dash0.
 
-### 3.9 Unified Routing (Final Design)
+### 3.9 Unified Routing (Historical — superseded)
+
+> **Superseded by git history.** The "single argmin + PUSH migration" design
+> described here was implemented in `6b255fa`, refined through
+> `5892739` (soft affinity), `2b9eae0` (numbers below), and `4b50c5a`
+> (queue/overload-gate fixes). Follow-on attempts to scale migration —
+> `e991960`/`5772149` (forced session migration) and `bf4469a` (relaxed
+> push gate) — were both reverted (`cc6e562`, `4c583f2`) after they
+> regressed E2E tail (57 migrations → HEAVY TTFT p90 15.9s → 59.1s;
+> 134 offloads → E2E p90 37s → 82s).
+>
+> Current implementation is the **hybrid LMetric + high-cache affinity**
+> direction landed in `255c8e6`. See `docs/migration-policy-design.md`
+> for the active algorithm and `analysis/unified_routing_fix_review.md`
+> for the reasoning. The numbers below remain valid for the
+> `eval_unified_v3` artifact; do not treat them as the current
+> production policy.
 
 Replaced two-phase routing (pick_instance → offload gate) with single `argmin(expected_latency)` per instance:
 

RESULTS_SUMMARY.md

@@ -0,0 +1,134 @@
+# 目前已成立的结论（2026-05-27）
+
+EAR 项目目前能用实测数据支撑的论点汇总。每条都标了对应的图/数据路径。
+
+---
+
+## 1. Workload 性质（§2）
+
+Production trace = Qwen3-Coder agentic，1.3 M sessions / 2.1 M reqs / 7200 s。
+
+| 性质 | 数据 | 实证 |
+|---|---|---|
+| **KV 复用几乎全在 session 内** | intra 93.2% / cross 5.7% / shared 1.1%；理论 APC 上界 79.6% | `figs/f2a_reuse_topology.png` |
+| **Session 极度偏斜** | top 1%/5%/10%/25%/50% = 46.5%/66.5%/74.6%/87.5%/**96.0%** input mass | `figs/f2b_session_skew.png` |
+| **单请求 KV 已经很大** | p50 1.8 GiB / p90 8.0 / p95 9.6 / **p99 11.5 GiB**；KV pool 38 GiB/instance（0.4 × H20 96 GiB）→ p99 req 只能装 **3 个/instance** | `figs/f2c_kv_footprint_cdf.png` |
+
+**结论**：cache 是 session-local 的，scheduling 必须保留 session affinity；单 request KV 接近 pool 上限，**PD-disagg 4P+4D 让系统 decode 容量直接减半**。
+
+## 2. Dispatch Coupling（§2.3）
+
+| 数据 | Agentic (Qwen3-Coder) | Chatbot (qwen3-max) |
+|---|---:|---:|
+| Inter-turn `T_external` p50 | **1.6 s** | 7.2 s |
+| `gap < 1 s` 比例 | **39%** | 4% |
+| `gap < 5 s` 比例 | 67% | 29% |
+| p99 | 738 s | 43 s |
+
+参考图：`figs/f3a_inter_turn_gap.png`。
+
+**结论**：agentic 有一段 chatbot 没有的 **sub-second tool-call mode**（39% vs 4%）。当 `W_turn ≳ T_external`（任何 W_turn > 1 s 的 scheduler 在 agentic 上都满足这条件），Little's Law `L = Λ · N · (W_turn(L) + T_external)` 进入闭环 regime，scheduler 的 ε 退步通过 KV 竞争反馈环被放大成 wall-clock 数倍差距。**实测**：lmetric 跑 600 s trace 用 49 min wall-clock = **8x amplification**。
+
+## 3. 现有调度的三类失败（§3）
+
+| Baseline | 失败模式 | 数据 |
+|---|---|---|
+| **load-balance / LMetric** | 丢 locality | lmetric APC **56.9%**（vs 上界 79.6%）；LMetric 比 load_only 只好 +3.3pp，因为 cache 信号在乘性 score `(pending+input−hit) × num_req` 里被 num_req 吞掉 |
+| **静态 PD-disagg** | D 侧 KV 容量墙 + transfer 成本 | 见 §4 cost-vs-benefit |
+| **Pure sticky** | 全员被 hot session 拖累，不是单一热点 | sticky median worker 20.3 s vs unified 10.3 s；system e2e p90 sticky 34.6 s vs unified 18.0 s（**用 max/median ratio 衡量是误导**，§3.3 用 absolute per-worker latency）|
+
+参考图：`figs/f4a_apc_loss.png`、`figs/f4b_pdsep_kv_wall.png`、`figs/f4c_per_worker_ttft.png`、`figs/f6_e2e_latency_bars.png`、`figs/f6_e2e_latency_full_grid.png`。
+
+## 4. Static PD-disagg 为什么失败（§3.2）—— 容量问题，不是 cost-benefit 问题
+
+⚠ **2026-05-27 纠正**：本节前一版本论证"PD-disagg 因为 transfer cost > phase isolation benefit 而失败"。**这个论证算错了**。正确的 phase-isolation benefit 是**每个 prefill 事件 × D 个 concurrent stream** 的总和（≈ `D × T_prefill`），不是单个 request 的 decode 时长。用正确公式，PD-disagg 在 phase-isolation 这一维上**赢 colo 一两个数量级**。Static PD-disagg 在 agentic 上失败的**真正根因是 D 侧 KV pool 容量**，不是这一维。
+
+### 4.1 真正的失败模式：D 侧 KV 容量天花板
+
+| | 8C colo | 4P+4D PD-disagg |
+|---|---:|---:|
+| Per-D-instance KV pool（0.4 × 96 GiB） | 38 GiB | 38 GiB |
+| 系统总 decode 容量（D 实例数 × 单池） | 8 × 38 = **304 GiB** | 4 × 38 = **152 GiB** |
+| p99 单请求 KV = 11.5 GiB → 最多并发 decode | 24 | **12（减半）** |
+
+Colleague 4P+4D 实测：TTFT p50 0.91 s → **62.8 s（62×）**、success rate **99.5% → 52%**。失败模式：**D 池溢出 + 排队**，不是 transfer 延迟。
+
+参考图：`figs/f4b_pdsep_kv_wall.png`（pdf 版本是高质量 paper figure）。
+
+### 4.2 MB2 — KV transfer cost（per-request 一次性成本，**不**是 dominant cost）
+
+dash1 GPU 0+1（intra）和 dash1 ↔ dash2（inter, 200 Gbps RoCE）扫 9 个 size × 5 reps。
+
+| 路径 | 稳态带宽（≤ 3 GiB） | p99 agentic 请求 11.5 GiB transfer |
+|---|---|---|
+| Intra-node | **9.7 GB/s** | p50 **1.9 s** · max 10 s |
+| Inter-node | **10.0 GB/s**（差 <3%） | p50 **1.7 s** · max 9.2 s |
+
+**新发现**：intra/inter 几乎重合 → **Mooncake `batch_transfer_sync_write` 永远走 RDMA NIC**，不走 NVLink。200 Gbps NIC 是天花板。**PD-disagg transfer cost 与拓扑无关**。
+
+参考图：`figs/mb2_transfer_time_compare.png`、doc `analysis/mb2/README.md`。
+
+### 4.3 MB1 — Phase interference（PD-disagg 的潜在 benefit 上界）
+
+dash1 GPU 0 单 instance（无 kv_connector），chunked-prefill 默认开启，D × P 扫描。D=8 结果：
+
+| Prefill | T_prefill | per-stream TPOT during | penalty |
+|---|---:|---:|---:|
+| 2k tok | 143 ms | 32 ms | 4× |
+| 32k tok | 4.5 s | 388 ms | **52×** |
+| 131k tok | 57 s | 1419 ms | **183×** |
+
+**Decode 在 prefill 期间被几乎完全 halted**，单 stream 损失 ≈ `T_prefill` 的时间。**每个 prefill event 总 decode 损失 ≈ `D × T_prefill`**。
+
+参考图：`figs/mb1_interference.png`、doc `analysis/mb1/README.md`。
+
+### 4.4 联合 cost-benefit（per-prefill event）
+
+| Prefill (KV size) | T_prefill | Cost = T_transfer | Benefit = D × T_prefill (D=8) | Cost / Benefit |
+|---:|---:|---:|---:|---:|
+| 2k tok (192 MiB) | 0.14 s | 8 ms | 1.1 s | **0.7%** |
+| 33k tok (3 GiB, trace mean) | 4.5 s | 0.32 s | 36 s | **0.9%** |
+| 125k tok (12 GiB, ~p99) | 57 s | 1.9 s | 456 s | **0.4%** |
+
+**PD-disagg 在 phase-isolation 这一维赢 100×–250×**。但**这不是 §3.2 该用的论证**，因为 §3.2 真正的 dominant failure 是 §4.1 的 D 池容量天花板（颠覆了上面的全部数学）。
+
+**总结**：
+- D 侧 KV 容量天花板（§4.1）→ PD-disagg 在 agentic 上**结构性失败**。
+- MB1 + MB2 的合计 cost-benefit 在 phase isolation 维度上 PD-disagg 是赢的，**但这件事被容量天花板压倒**。
+- Paper §3.2 论证应该聚焦"D 池装不下"，MB1/MB2 数据用作 supporting context（per-request transfer charge 量化、phase isolation benefit 量化）而不是 main argument。
+
+> ✅ **2026-05-30 更新 — 干净栈三轴 ablation 证实本节、并加 regime 细化。**
+> 本节的容量论点（D 池容量天花板 / decode 减半）在修复 `e13391e` 污染后的 clean stack
+> 上**得到确认**：concurrency 轴 N=64 时 PD 倾覆，**producer APC 从 71% 崩到 1.4%、TPS −30%**，
+> 而 colo 线性 scale（Fig 3）。**细化**：PD 并非"在 agentic 上一律失败"——它在
+> *低负载 / decode-heavy / 低复用* 区间**赢** colo；真正失败的是 agentic corner（高复用 +
+> 短输出 + 大上下文 + 高并发）——静态 P:D split 无法同时给出复用所需的 producer 容量
+> *和* decode 容量，而 colo 的弹性池两者兼得。
+> **另注**：旧 MB5 文档（`PD_DISAGG_RESULTS.md` §6）"session-affinity 救不了 PD / PD 复用=0%"
+> 的结论是 `e13391e`（producer 每次 KV 传输后 evict prefix）的**污染产物，已撤回**；
+> 干净栈上 session 路由的 producer APC 与 colo 持平（71–82%）。
+> 图：[`figs/mb5_pd_ablation/`](figs/mb5_pd_ablation/)。
+
+## 5. EAR 设计的实证状态（§4）
+
+| Pillar | 已实证 | 待实证 |
+|---|---|---|
+| **Affinity-default routing** (Pillar 1) | ✅ Current `unified` 算法 = LMetric + high-cache affinity；APC **79.4%**（达到 79.6% 上界 97%），TTFT p90 **7.3 s**，median worker p90 **10.3 s** | — |
+| **Hot-triggered session migration** (Pillar 2) | substrate 已通：`kv_both` connector 在 trace replay 上 net positive（TTFT p90 −18.6%，DR-fix 后 −36.6%），原 elastic_migration_v2 paper 的 "+45% kv_both penalty" obsolete | e2e 策略层（trigger 阈值 + target selection 在反馈环里）未直接验证 |
+
+## 6. 已经能写的 paper 主张（按 confidence 排序）
+
+1. **Agentic vs chatbot 在调度上是不同 regime**（dispatch coupling + sub-second tool-call mass）—— 实证完整
+2. **三类现有调度 baseline 各自的失败模式**（load-balance / static PD-disagg / pure sticky）—— 实证完整
+3. **Static PD-disagg 在 agentic 下失败的 dominant 根因是 D 侧 KV 容量**（不是 phase-isolation cost-benefit）—— 实证完整（`f4b` + colleague 4P+4D 数据）
+4. **Mooncake transfer cost 拓扑无关**（intra ≈ inter，~9.7 GB/s NIC 上限）—— 实证完整（MB2）
+5. **Phase isolation interference 在 chunked-prefill on 下仍然显著**（per-stream TPOT during prefill 15×–2000× baseline）—— 实证完整（MB1）。**注意**：这条数据本身不直接论证 "PD-disagg 失败"，因为算正确账后 PD-disagg 反而在这一维上赢；它的用途是给 §3.2 提供 phase-isolation benefit 上界的量化。
+6. **Affinity-default 调度（current unified）达到 APC 上界**，per-worker latency 也压倒 sticky —— 实证完整
+7. **Hot-triggered migration 修复 sticky 的 hot pin** —— design 完整、e2e 待验证
+
+## 7. 待做
+
+- **MB3-5**（end-to-end PD-disagg deployment）：D-pool runtime occupancy、cache reuse × PD interaction、PD ratio sweep。这些是 §5 完整实验矩阵的事
+- **EAR Pillar 2 migration e2e validation**（在 connector_tax DR-fix 之上重测）
+- **§5.4 wall-clock amplification sweep**（5 baseline × 3 runs，钉死 dispatch coupling 论证的实证 closure）
+- **Scale-out 验证**（dash1+dash2 = 16 GPU，等 dash0 + 3-node 可用时扩到 80 GPU）

analysis/agentic_pd_unified_story_plan.md

@@ -0,0 +1,552 @@
+# Agentic PD / Unified Routing Story Plan
+
+Status: draft for review
+Date: 2026-05-25
+
+## 0. Goal
+
+This document aligns three threads:
+
+1. `agentic-kv`: vLLM-based PD-colocation, full PD separation, LMetric,
+   Unified routing, and elastic migration experiments.
+2. `dash0:/home/admin/cpfs/wjh/agentic-kv`: run artifacts and the
+   latest PD-separation paper-section scaffold.
+3. `~/phd/agentic-pd-hybrid`: SGLang/PPD/KVC experiments, including
+   retractions and stricter framing around loadgen validity.
+
+The purpose is to converge on a defensible story and a concrete task plan,
+not to force the old Unified routing hypothesis to be true.
+
+## 1. Current Best Framing
+
+### 1.1 Workload premise
+
+Agentic serving is not chatbot serving.
+
+- Requests have long input contexts and short outputs.
+- Most reusable KV is intra-session, not cross-session.
+- Sessions are multi-turn and causally sequential: turn N+1 cannot be
+  faithfully issued before turn N finishes.
+- Long-lived sessions create two competing needs:
+  - keep cache locality for future turns;
+  - avoid pinning all future work to an overloaded worker.
+
+This workload makes cache locality a first-order system objective, but
+also makes naive session pinning dangerous.
+
+### 1.2 System premise
+
+PD separation is not a universally good abstraction. It helps only when:
+
+```
+saved decode interference > KV transfer + P queue + D queue + KV capacity cost
+```
+
+For agentic workloads, that inequality often fails because long-context KV
+is large and decode-side KV residency becomes the limiting resource.
+
+### 1.3 Main thesis candidate
+
+Static PD separation is the wrong default for single-node agentic serving.
+The stronger baseline is PD-colocation with cache-aware routing. The
+interesting open problem is not "separate prefill and decode everywhere",
+but:
+
+> how to preserve session-level KV locality while retaining enough routing
+> freedom to avoid hot-worker queueing and decode interference.
+
+Unified routing should be framed as an attempt at that problem. The current
+experiments show that the migration actuator was too expensive, so the
+story should distinguish the principle from the failed mechanism.
+
+## 2. What We Should Align Across Repos
+
+### 2.1 Naming / architecture mapping
+
+Use one taxonomy consistently:
+
+| Name in paper/story | vLLM repo term | SGLang/KVC repo term | Meaning |
+|---|---|---|---|
+| Replica / PD-colo | combined / PD-colocated | `pd_colo`, SGLang `cache_aware` | all workers do prefill + decode |
+| x=0 PD-disagg | full PD separation | `pd_disagg` | every turn goes P then D |
+| x=1 / append-prefill-on-D | not implemented as such in vLLM experiments | KVC / PPD-style direct-to-D | turn 1 seeds D; later turns prefill locally on D |
+| Elastic migration | Unified PUSH / elastic offload | smart migration / re-pin sessions | move a session or a request away from overloaded worker |
+| Hybrid routing | current Unified baseline | PD-colo + soft pin / kv-aware | cache-aware LB plus explicit affinity only when worth it |
+
+Important distinction: vLLM Unified PUSH migration is not the same as PPD
+x=1. Unified PUSH still pays cross-instance KV movement for migrated
+requests. PPD x=1 tries to avoid P-to-D transfer on later turns by doing
+append-prefill directly on the resident D node.
+
+### 2.2 Results that look stable
+
+These are safe to build around:
+
+1. Full/static PD separation is weak for agentic on one node.
+   - vLLM evidence: decode-side KV memory wall and transfer overhead.
+   - SGLang evidence: x=0 PD-disagg is consistently worse than PD-colo.
+
+2. LMetric/cache-aware routing is a strong baseline.
+   - In vLLM, corrected LMetric nearly matches session-sticky because
+     `new_tokens = input - cached` creates implicit soft affinity.
+   - In SGLang, `cache_aware` is the production-stable baseline and often
+     wins or ties.
+
+3. Explicit session pinning is not automatically good.
+   - It can improve cache hit rate.
+   - It can create head-of-line blocking if sessions grow unevenly.
+   - Initial placement matters; mid-session migration is desirable in
+     principle but hard in practice.
+
+4. Transfer-based migration is currently too expensive.
+   - vLLM experiments: forced/relaxed migration worsened E2E tail.
+   - Mooncake path lacks enough overlap/layerwise benefit in current setup.
+
+5. Loadgen validity must be treated as substrate, not detail.
+   - `agentic-pd-hybrid` explicitly retracted high-concurrency claims
+     where session sequentiality was violated.
+   - Future experiments must enforce per-session causality.
+
+### 2.3 Results that need more careful wording
+
+1. "Unified beats LMetric" should not be stated as a strong result yet.
+   The latest stable implementation is closer to LMetric plus a high-cache
+   affinity gate. Expected gain is small by design.
+
+2. "PD separation is always bad" is too broad.
+   The correct claim is conditional: static/full PD separation is net
+   negative in the long-context, high-KV-footprint, single-node agentic
+   regime we measured.
+
+3. "KVC/PPD wins" is not established for our stack.
+   The SGLang repo contains useful PPD framing, but also several retractions:
+   high-concurrency wins were affected by loadgen issues, and KVC stability
+   was not production-ready in some runs.
+
+4. "Session migration will fix load balance" is still a hypothesis.
+   It is valid as a first-principles goal, but the tested vLLM actuator
+   did not satisfy the cost budget.
+
+## 3. Proposed Storytelling Outline
+
+### Section A: Why agentic serving is different
+
+Claim:
+
+- Agentic workloads combine long contexts, high intra-session reuse, and
+  sequential multi-turn sessions.
+- This makes KV cache lifecycle and routing more important than the classic
+  prefill/decode kernel dichotomy.
+
+Evidence to use:
+
+- Input/output token CDF.
+- KV reuse decomposition: intra-session vs cross-session.
+- Session length / context growth examples.
+- Per-session sequentiality requirement.
+
+Needed cleanup:
+
+- Use one trace definition and report sampling method.
+- Explicitly state whether a trace is online-realistic, synthetic burst,
+  or stress test.
+
+### Section B: Why static PD separation fails
+
+Claim:
+
+- The classic roofline premise is true but insufficient.
+- Prefill can be compute-bound while static PD separation still loses at
+  the system level.
+
+Mechanism:
+
+- PD separation relocates prefill; it does not reduce total prefill work.
+- It adds KV transfer.
+- It concentrates decode KV residency onto fewer D GPUs.
+- Long-context agentic requests hit a decode-side KV memory wall.
+
+Evidence to use:
+
+- `analysis/pd_sep_paper_section/system_analysis.md`
+- C1 workload figures.
+- C6 roofline figure.
+- KV memory wall model.
+- Fresh PD matrix once rerun without forced eager mode.
+
+Task implication:
+
+- Complete C2/C3/C4/C5 matrix before making this a paper-grade section.
+
+### Section C: Why cache-aware PD-colo is hard to beat
+
+Claim:
+
+- Cache-aware routing already captures much of the desired session affinity.
+- LMetric's cache-adjusted prefill cost gives implicit soft affinity without
+  hard pinning.
+
+Mechanism:
+
+- A worker with cached prefix has lower `new_tokens`.
+- This naturally attracts later turns unless the worker is sufficiently
+  loaded.
+- This is exactly the balance we want: preserve locality while retaining
+  routing freedom.
+
+Evidence to use:
+
+- Corrected LMetric vs Linear comparison.
+- APC distribution.
+- PD-colo stability from SGLang/KVC repo.
+
+Task implication:
+
+- Treat LMetric/cache-aware PD-colo as the primary baseline, not round-robin
+  or naive sticky.
+
+### Section D: Why Unified migration did not improve over LMetric
+
+Claim:
+
+- Unified's principle was right, but the migration mechanism failed the
+  cost budget.
+
+Mechanism:
+
+- At conservative gates, too few requests migrate to change load balance.
+- At relaxed gates, migration overhead dominates.
+- Cold/heavy requests often cannot benefit from source cache and remain
+  colocated.
+- Cached migration still pays P-side queue, KV movement, and D admission.
+- The cost model initially underestimated cache-attraction feedback and
+  queue effects.
+
+Evidence to use:
+
+- Git history: single argmin -> soft affinity -> decode load/hard gate ->
+  forced migration -> revert -> hybrid LMetric.
+- Approach B / relaxed gate regressions.
+- 16-session contention: interference exists, but elastic RDMA made TPOT
+  worse and offloaded too few requests.
+
+Task implication:
+
+- Do not revive three-way argmin or aggressive PUSH migration.
+- Frame current Unified as hybrid LMetric plus selective affinity.
+
+### Section E: What remains promising
+
+There are two different future paths. They should not be conflated.
+
+Path 1: Conservative, vLLM-ready.
+
+- Stay PD-colocated.
+- Use corrected LMetric as base.
+- Add only explicit high-cache affinity / tie-break logic where it improves
+  stability.
+- Improve scheduling: adaptive chunked prefill, decode-priority controls,
+  better observability of queue and cache state.
+
+Path 2: Research, PPD-style.
+
+- Turn 1 seeds session on D.
+- Later turns do append-prefill on resident D, avoiding P-to-D transfer.
+- Dynamic x chooses P vs D based on append size, P queue, D load, and SLO.
+- Requires stable implementation and strict loadgen validation.
+
+The paper/story can say: transfer-based migration did not work; append-
+prefill-on-resident-D remains a different and potentially better actuator.
+
+## 4. Design Direction Recommendation
+
+### 4.1 Near-term path
+
+Use PD-colo cache-aware as the production baseline and paper baseline.
+
+Implement/validate only low-risk routing improvements:
+
+1. Pure LMetric baseline must stay separate and reproducible.
+2. Unified hybrid should be LMetric plus:
+   - high-cache explicit affinity;
+   - overload escape;
+   - deterministic non-degenerate tie-break;
+   - route-decision logging.
+3. No Mooncake/PUSH migration on the critical comparison path.
+
+This gives a clean statement:
+
+> The best robust single-node policy we have is cache-aware PD-colocation.
+> Unified hybrid is a small refinement, not a new disaggregation win.
+
+### 4.2 Research path
+
+If we want a stronger contribution beyond "PD-sep loses", the promising
+research direction is:
+
+> session-resident append-prefill with dynamic P/D selection.
+
+This aligns better with PPD than vLLM PUSH migration does.
+
+Key design principle:
+
+- Do not move KV just to run prefill elsewhere unless the future benefit is
+  large enough to amortize the transfer.
+- Prefer using the worker that already owns the session KV, unless decode
+  load or append size makes that choice violate SLO.
+
+## 5. Experiment Plan
+
+### 5.1 Must-have validity checks
+
+For every benchmark:
+
+- Per-session sequentiality enforced.
+- Attempted/completed/error counts reported.
+- Pair by `(session_id, turn_id)` when comparing arms.
+- Report goodput, not only latency of successes.
+- Record git commit, launch flags, trace path, request limit, time scale,
+  session sampling method, and hardware.
+
+### 5.2 PD separation matrix
+
+Goal: make the static PD-sep negative result paper-grade.
+
+Arms:
+
+- PD-colo cache-aware.
+- PD-sep 4P+4D.
+- PD-sep 6P+2D.
+- Optional: round-robin baseline only as sanity, not main comparison.
+- Optional: eager vs cudagraph ablation.
+
+Metrics:
+
+- TTFT/E2E/TPOT p50/p90/p99.
+- Goodput and error rate.
+- APC mean and per-instance distribution.
+- GPU util and decode-side KV occupancy time series.
+- TTFT breakdown: prefill, KV transfer, D wait.
+
+Output:
+
+- C2 headline bar with error bars.
+- C3 KV utilization time series.
+- C4 TTFT stacked breakdown.
+- C5 cuda-graph ablation.
+
+### 5.3 LMetric vs Unified hybrid
+
+Goal: determine whether current Unified has any real gain over LMetric.
+
+Arms:
+
+- Pure corrected LMetric.
+- Current Unified hybrid.
+
+Run:
+
+- 3-5 paired trials on the same trace.
+- No Mooncake/PUSH.
+- Same launch flags.
+
+Additional logging:
+
+- Route reason: `lmetric`, `high_cache_affinity`, `overload_escape`,
+  `tie_break`.
+- Chosen instance load, cache hit, effective new tokens.
+
+Decision rule:
+
+- If gain is within noise, do not oversell Unified as a performance win.
+  Keep it as a policy cleanup / safety improvement.
+
+### 5.4 Interference and scheduler experiments
+
+Goal: test whether scheduling is the right actuator after routing saturates.
+
+Arms:
+
+- Different chunked prefill sizes.
+- Decode-priority / prefill throttling if available.
+- High-concurrency but session-sequential trace.
+
+Metrics:
+
+- TPOT under concurrent heavy prefills.
+- TTFT for heavy turns.
+- Decode queue delay.
+- GPU util timeline.
+
+Expected value:
+
+- If migration is too expensive, reducing prefill interference in-place is
+  the most plausible next improvement.
+
+### 5.5 PPD/KVC-style research validation
+
+Goal: separate PPD x=1 from failed x=0/full PD and failed transfer-based
+migration.
+
+Arms:
+
+- PD-colo cache-aware.
+- x=0 PD-disagg.
+- x=1 append-prefill-on-D if implementation is stable.
+- Dynamic x if available.
+
+Guardrails:
+
+- Do not use old high-concurrency KVC numbers without the loadgen caveat.
+- Do not compare partial successful subsets without goodput.
+- Treat SGLang implementation bugs as system results, not hidden noise.
+
+## 6. Task Breakdown
+
+### Track 1: Documentation alignment
+
+Owner task:
+
+- Update `REPORT.md`, `docs/migration-policy-design.md`, and
+  `analysis/research_findings.md` so they use the taxonomy in section 2.
+
+Concrete edits:
+
+- Mark single-argmin/PUSH Unified as historical.
+- State that current Unified is hybrid LMetric plus high-cache affinity.
+- Add mapping to PPD taxonomy: Replica, x=0 PD, x=1 append-prefill.
+- Add loadgen validity checklist.
+
+Done when:
+
+- A reviewer can no longer confuse vLLM PUSH migration with PPD x=1.
+- LMetric baseline and Unified hybrid are described as separate policies.
+
+### Track 2: Current routing cleanup
+
+Owner task:
+
+- Make current Unified hybrid auditable and minimal.
+
+Concrete edits:
+
+- Remove stale unreachable PUSH code from `scripts/cache_aware_proxy.py`.
+- Keep pure `--policy lmetric` untouched.
+- Add route-decision fields for Unified hybrid.
+- Add tests:
+  - pure LMetric remains pure;
+  - high-cache affinity triggers only under its intended gate;
+  - overload escape works;
+  - empty-batch tie-break does not collapse to instance 0.
+
+Done when:
+
+- `pytest tests/test_proxy_pick.py` covers LMetric and Unified separately.
+- Bench logs can count how often Unified did something beyond LMetric.
+
+### Track 3: PD-sep paper matrix
+
+Owner task:
+
+- Finish the `analysis/pd_sep_paper_section` missing claims.
+
+Concrete work:
+
+- Run `bench_pd_matrix.sh` on dash0.
+- Collect `metrics.summary.json`, `breakdown.json`, `apc.txt`,
+  `gpu_util.csv`, and per-instance KV logs.
+- Add plotters for C2/C3/C4/C5.
+- Replace legacy C7 numbers with matrix outputs.
+
+Done when:
+
+- The PD-sep negative result no longer relies on old `--enforce-eager`
+  methodology or single snapshots.
+
+### Track 4: Benchmark substrate validation
+
+Owner task:
+
+- Audit the vLLM replayer and any dash0 loadgen scripts for session
+  sequentiality and arrival semantics.
+
+Concrete checks:
+
+- Verify no session has more than one in-flight turn unless explicitly
+  configured as a stress test.
+- Add an analyzer that reports max concurrent turns per session.
+- Report sampled session-start distribution.
+- Add goodput and error-rate comparisons to all summary scripts.
+
+Done when:
+
+- We can label each experiment as online-realistic, burst stress, or
+  synthetic microbench.
+
+### Track 5: Scheduler/interference path
+
+Owner task:
+
+- Test whether in-place scheduling beats transfer-based migration.
+
+Concrete experiments:
+
+- Chunk size sweep.
+- Decode-priority or prefill-throttle sweep.
+- 16+ session sequential replay.
+
+Done when:
+
+- We know whether the next performance lever is scheduler policy or routing
+  policy.
+
+### Track 6: PPD-style appendix / related design
+
+Owner task:
+
+- Extract the useful `agentic-pd-hybrid` lessons without importing invalid
+  claims.
+
+Concrete work:
+
+- Summarize:
+  - loadgen bug and retractions;
+  - PD-colo as stable baseline;
+  - x=0 PD-disagg failure;
+  - x=1/append-prefill motivation;
+  - dynamic threshold lessons.
+- Decide whether this is mainline future work or an appendix framing.
+
+Done when:
+
+- The story can cite PPD-style append-prefill as a distinct future actuator,
+  not as evidence that the current Unified migration already works.
+
+## 7. Proposed One-Sentence Story
+
+Agentic serving breaks the classic PD-disaggregation intuition: long-lived
+sessions make KV locality dominant, while long contexts make decode-side KV
+capacity and transfer costs dominate the gains from isolating prefill; the
+robust design is cache-aware PD-colocation with carefully limited session
+affinity, and future disaggregation must be dynamic and session-resident
+rather than static or transfer-heavy.
+
+## 8. Open Decisions For Review
+
+1. Do we want the main paper contribution to be the negative result
+   "static PD separation fails for agentic", or the positive system
+   "cache-aware PD-colo / Unified hybrid"?
+
+2. Is PPD-style x=1 append-prefill a future-work section, or do we need to
+   implement a minimal stable version before finalizing the story?
+
+3. Should current Unified be presented as a named system if its measured
+   improvement over LMetric is small, or should it be framed as an audit of
+   why LMetric/cache-aware is already strong?
+
+4. Which trace is the canonical trace for claims: the vLLM trace in
+   `agentic-kv`, the GLM-5.1 trace in `agentic-pd-hybrid`, or both with
+   explicit regime labels?
+
+5. What is the target venue-style claim: systems negative result,
+   workload characterization, or routing/scheduling algorithm?

analysis/characterization/README.md

@@ -0,0 +1,255 @@
+# Characterization Analyzer Runbook
+
+CPU-only scaffold for Batch 0 and Batch 1 in
+`analysis/characterization_todo_for_interns.md`.
+
+This directory has three components:
+
+- `analyze.py`: Batch 0/1 analyzer for trace and per-request metrics.
+- `summarize_runs.py`: CPU-only audit of already completed benchmark
+  directories.
+- `protocols.md`: exact protocol for Batch 2-6 experiments that require fresh
+  GPU runs or additional instrumentation.
+
+The analyzer reads existing trace and metrics artifacts and writes:
+
+```text
+outputs/characterization/<date>/<task_name>/
+├── manifest.json
+├── raw/
+├── summary.json
+├── summary.md
+├── audit.md
+├── session_concurrency.json
+├── session_arrival_stats.json
+├── turn_interval_stats.json
+├── trace_profile.json
+├── invalid_runs.md
+├── workload_summary.json
+├── kv_footprint_summary.json
+├── reuse_decomposition.json
+├── session_skew.json
+├── append_delta_stats.json
+└── figures/
+```
+
+If `matplotlib` is installed, simple PNG/PDF figures are emitted under
+`figures/`. If it is not installed, all JSON/Markdown data artifacts are still
+written.
+
+## Canonical Data Sources
+
+Canonical full traces live on dash0:
+
+- formatted trace: `~/ali-trace/trace-glm5.1-formatted/`
+- raw unformatted trace: `~/ali-trace/trace-glm5.1/`
+
+For the current GLM-5.1 characterization, prefer the compact formatted file:
+
+```text
+~/ali-trace/trace-glm5.1-formatted/051315-051317.jsonl
+```
+
+Do not pass `051315-051317-raw.jsonl` or the files under
+`~/ali-trace/trace-glm5.1/` directly to this analyzer unless you first convert
+them to the formatted schema. Those raw files are tens to hundreds of GiB and
+contain full prompt payloads rather than the compact characterization schema.
+
+The analyzer is CPU-only. For full trace characterization, either:
+
+- run it on dash0 against the formatted JSONL files without starting any GPU
+  service; or
+- copy/rsync the needed trace files from dash0 to this repository or another
+  local path, then run the analyzer locally.
+
+Only light directory/field inspection is needed on dash0 before choosing which
+trace file to analyze.
+
+The raw unformatted directory is listed as a source option for provenance, but
+this analyzer expects formatted JSONL records. Raw files should be converted to
+the formatted schema before being passed to `--trace`.
+
+## Inputs
+
+Trace JSONL:
+
+- Expected formatted fields: `chat_id`, `parent_chat_id`, `timestamp`,
+  `input_length`, `output_length`, `type`, `turn`, `hash_ids`, optional
+  `session_id`.
+- If `session_id` is absent, sessions are reconstructed from
+  `parent_chat_id` chains.
+- `timestamp` is treated as scheduled trace time, not proof of actual dispatch
+  time.
+
+Metrics JSONL:
+
+- Expected replayer fields: `request_id`, `session_id`, `turn_id`,
+  `trace_timestamp_s`, `input_length`, `output_length`, `cached_tokens`,
+  `latency_s`, `ttft_s`, `tpot_s`, `actual_output_tokens`, `error`.
+- If the metrics file is from the current replayer, it does not include actual
+  dispatch/finish wall-clock timestamps. Batch 0 will therefore mark actual
+  session sequentiality as unavailable and separately report a scheduled
+  estimate from `trace_timestamp_s + latency_s`.
+
+Proxy breakdown:
+
+- Optional JSON/JSONL with fields such as `request_id`, `t_proxy_recv`,
+  `t_first_token`, `t_done`, `cache_hit`, `estimated_new_tokens`,
+  `route_class`, `routed_to`, `policy`.
+- Batch 0 can prove actual per-session in-flight concurrency only when these
+  timing rows can be joined to analyzed requests by `request_id`.
+- Existing proxy breakdown artifacts may not contain `session_id`; without a
+  request-id join to trace/metrics, they can still support append/cache-hit
+  statistics but not per-session concurrency.
+
+Run config:
+
+- Optional JSON, usually `outputs/<run>/config.json`.
+- Used for manifest fields such as `policy`, `time_scale`, and request count
+  when available.
+
+## Commands
+
+Trace-only dry run:
+
+```bash
+python3 analysis/characterization/analyze.py \
+  --trace traces/w600_r0.0015_st30.jsonl \
+  --task-name w600_trace_only \
+  --overwrite
+```
+
+Trace plus replayer metrics:
+
+```bash
+python3 analysis/characterization/analyze.py \
+  --trace traces/w600_r0.0015_st30.jsonl \
+  --metrics outputs/smoke_test/metrics.jsonl \
+  --task-name smoke_trace_metrics \
+  --overwrite
+```
+
+Proxy breakdown append/cache analysis:
+
+```bash
+python3 analysis/characterization/analyze.py \
+  --breakdown outputs/contention_16s_elastic/breakdown.json \
+  --config outputs/contention_16s_elastic/config.json \
+  --task-name contention_breakdown \
+  --overwrite
+```
+
+Full trace on dash0, CPU-only:
+
+```bash
+python3 analysis/characterization/analyze.py \
+  --trace ~/ali-trace/trace-glm5.1-formatted/051315-051317.jsonl \
+  --task-name full_trace_characterization \
+  --overwrite
+```
+
+Local run after copying from dash0:
+
+```bash
+rsync -av dash0:~/ali-trace/trace-glm5.1-formatted/<trace-file>.jsonl traces/
+python3 analysis/characterization/analyze.py \
+  --trace traces/<trace-file>.jsonl \
+  --task-name full_trace_characterization \
+  --overwrite
+```
+
+By default the analyzer records file size and mtime but skips full SHA256
+hashing, because canonical raw trace files can be hundreds of GiB. Add
+`--hash-inputs` only when you intentionally want a full file hash.
+
+KV footprint requires a model-specific value:
+
+```bash
+python3 analysis/characterization/analyze.py \
+  --trace traces/w600_r0.0015_st30.jsonl \
+  --kv-bytes-per-token 98304 \
+  --task-name w600_with_kv_estimate \
+  --overwrite
+```
+
+Summarize existing completed runs:
+
+```bash
+python3 analysis/characterization/summarize_runs.py
+```
+
+This writes:
+
+```text
+analysis/characterization/current_results/
+├── run_summaries.json
+├── comparisons.json
+├── claim_matrix.json
+├── reviewer_risk_register.json
+├── current_results.md
+├── characterization_claim_matrix.md
+├── all_figures_index.md
+├── reviewer_risk_register.md
+└── reproduction_commands.sh
+```
+
+## Batch 0 Semantics
+
+The online-serving invariant is:
+
+```text
+Each session has at most one in-flight turn.
+```
+
+The analyzer reports:
+
+- actual interval status from dispatch and finish/error timestamps;
+- scheduled estimate from trace timestamps plus latency when available;
+- per-session max in-flight;
+- session start-time distribution;
+- turn inter-arrival distribution;
+- attempted/completed/error counts and goodput when metrics exist;
+- run classification.
+
+Important limitation: trace timestamps alone cannot prove actual replay
+sequentiality. A run is only classified as `online_realistic` when actual
+per-request dispatch and finish/error timestamps prove
+`max_inflight_per_session <= 1`.
+
+## Batch 1 Semantics
+
+The analyzer reports:
+
+- input/output CDF stats;
+- input/output ratio;
+- KV footprint CDF stats when `--kv-bytes-per-token` is supplied;
+- session skew and top-session contribution;
+- append/uncached token stats when `cached_tokens` or `cache_hit` exists;
+- reuse decomposition when both cached-token fields and `hash_ids` exist.
+
+Reuse decomposition is conservative:
+
+- `intra_session`: cached hash block was seen earlier in the same session;
+- `cross_session`: cached hash block was seen earlier in another session;
+- `shared/system-prefix`: early-position block appears in many sessions;
+- `unclassified`: cached tokens could not be mapped to a previously seen hash
+  block.
+
+If cached-token/cache-hit fields are absent, reuse and append artifacts are
+written with `status: "unavailable"` and list the required fields.
+
+## Limitations
+
+- The script does not run a benchmark, query a live service, touch GPU state,
+  or start any daemon.
+- Request-id joins are exact. If trace, metrics, and proxy artifacts use
+  different request IDs, the unmatched rows are preserved under `raw/`.
+- Actual Batch 0 sequentiality needs actual dispatch and finish/error
+  timestamps. Current `replayer/metrics.py` metrics are not enough by
+  themselves.
+- `kv_bytes_per_token` depends on model architecture, layer count, KV heads,
+  head dimension, and dtype. The analyzer will not guess it.
+- Shared/system-prefix reuse classification is a heuristic based on trace
+  `hash_ids` positions and cross-session frequency. Adjust
+  `--shared-prefix-min-sessions` and `--system-prefix-blocks` if the formatted
+  trace provides a stronger system-prefix marker.

analysis/characterization/agentic_dispatch_coupling.md

@@ -0,0 +1,187 @@
+# Agentic Dispatch Coupling: Why Session-Sequential Replay is the Realistic Mode
+
+Date: 2026-05-26
+Owner: characterization
+Status: methodology note for paper framing
+
+## The observation
+
+In `replayer/replay.py:282-287`, turn N of a session fires at:
+
+```
+t_fire(N) = max(
+    turn_N_trace_timestamp,         # what the trace asked for
+    turn_{N-1}_finish_wall_clock,   # but turn N-1 must complete first
+)
+```
+
+When the system is fast, the second term loses → turn N fires at its trace
+timestamp → the replay matches the captured trace. When the system is slow,
+the second term dominates → turn N fires *immediately* after turn N-1
+completes, with the trace timestamp ignored. The session's "inter-turn
+think time" collapses to zero.
+
+A first reading flags this as a benchmark concern: under saturation the
+arrival process becomes policy-dependent, so cross-policy latency
+comparisons are confounded by a feedback loop (slow policy → longer
+sessions → more concurrent in-flight → harder system → slower latency).
+
+This note argues the opposite: **the trace-replayer's behavior is the
+correct model of agentic workloads, and the feedback loop is a real
+property of production systems, not a methodology artifact**.
+
+## Why agentic workloads do not have user think-time
+
+In chat workloads, the turn N+1 message is composed by a human reading the
+turn N response. The inter-turn gap is dominated by human reading + typing
+speed, which is independent of how fast the server replied. The trace's
+timestamp captures the human cadence and is a faithful arrival process.
+
+In **agentic workloads**, turn N+1 is generated by:
+- A tool-call response feeding back into the model context
+- An autonomous loop deciding the next action
+- A planner / executor stepping to the next subgoal
+
+None of these wait for a human. Turn N+1 fires as soon as the
+infrastructure can hand the previous turn's output back to the next
+inference step. There is no think-time floor.
+
+This means: in a real agentic system, **the wall-clock time between turn N
+finish and turn N+1 dispatch is essentially zero**. If the model serving
+infrastructure slows down (high TTFT or E2E for turn N), turn N+1's
+dispatch slips by the same amount — exactly the behavior the replayer
+exhibits.
+
+## What B3's session-sequential dispatch is actually measuring
+
+B3's trace replayer drives a workload that:
+- preserves the *causal structure* of the original trace (which turns
+  belong to which session and in what order),
+- uses the *original timestamps as a lower bound* (turn N+1 cannot fire
+  before its trace timestamp),
+- *but* lets turn N+1 fire immediately when the system has fallen behind.
+
+For an agentic workload, this is the right model:
+
+1. The captured trace's timestamps reflect the **production system's
+   actual response speed at capture time** — they already encode the
+   round-trip time the model + tool stack delivered.
+2. When we replay against a *different* policy, what we want to measure
+   is "what wall-clock would this session take under policy X" — which
+   includes the same tool-call-driven cadence: each next turn fires as
+   soon as it can.
+3. The "inter-turn gap" is not a fixed delay we should respect; it is an
+   artifact of the captured system's speed that we are explicitly trying
+   to replace.
+
+So the replayer's behavior is not "broken under saturation"; it is
+modeling the agentic semantic correctly: **no think-time, sequential
+within session, fire-immediately when ready**.
+
+## The feedback loop is a real production phenomenon
+
+Once we accept the agentic semantic, the so-called "dispatch slip
+artifact" is not an artifact — it is a real system behavior:
+
+```
+slow policy
+    → each turn takes longer
+    → each session lives in the system longer
+    → at any moment, more sessions are concurrently in-flight
+    → 8 workers' KV / queue pressure is higher
+    → each request gets less per-worker capacity
+    → each turn takes even longer
+    → ...
+```
+
+By Little's Law: `N_concurrent ≈ session_arrival_rate × mean_session_lifetime`.
+
+In our B3 data:
+- lmetric: mean session lifetime is much longer than the original
+  trace's ~600 s span (lmetric's 1214-request replay took 49 min wall
+  clock — sessions stayed alive ~8× longer than the trace captured).
+- unified: sessions drain ~3× faster than lmetric.
+
+So under unified, the 8-worker pool sees fewer concurrent sessions than
+under lmetric — and this is **what production would also see** if the
+operator switched routing policies on the same incoming agentic load.
+
+**This is not a fairness violation**. It is a faithful reflection of:
+"a faster routing policy is faster both because of its per-request
+behavior AND because it reduces the steady-state concurrent load it
+inflicts on itself".
+
+A user running an agentic system *does* benefit from both effects when
+they pick a better policy. The combined "policy × system-feedback" gain
+is what the user actually experiences.
+
+## What this means for B3 and B4 in the paper
+
+| | B3 trace-driven replay | B4 open-loop Poisson |
+|---|---|---|
+| Arrival process | original trace timestamps with session-sequential "fire-on-finish" | Poisson session inter-arrival at fixed λ |
+| Inter-turn think-time | none (matches agentic) | none (matches agentic) |
+| Session lifetime under load | *grows* with policy slowness (feedback) | *fixed* by trace template plus per-turn latency |
+| What latency at p90 measures | end-user latency under agentic feedback amplification | per-request behavior at the operator-chosen load level |
+| What "fair across policies" means | same trace, same total session set; arrival process is policy-dependent **on purpose** | same λ, decoupled from policy throughput |
+| When to use it | "if we run this real captured load through our system, what does the user see" | "what is the max sustainable rate (SRR) before SLO breaks, per policy" |
+
+The two are **complementary**, not "B3 is unfair and B4 fixes it":
+
+- **B3 answers the "in-production replay" question** — including feedback amplification, which agentic users will actually experience.
+- **B4 answers the "saturation envelope" question** — what's the policy's intrinsic throughput at fixed load.
+
+A paper that drops B3 in favor of B4 would understate how much the
+**combined** effect (policy + feedback) actually helps the user. A paper
+that drops B4 in favor of B3 would conflate the two effects and prevent
+a "policy X sustains higher λ" statement.
+
+## Recommended paper framing
+
+1. **B3 is the production-replay experiment**. Report latency percentiles
+   as "end-to-end under captured agentic load with no-think-time
+   sequencing". Acknowledge that the *combined* gap (e.g. unified TTFT
+   p90 = 7.24 s vs lmetric 15.6 s) reflects both policy and feedback;
+   call this out, do not hide it.
+
+2. **B4 is the controlled-load experiment**. Report `SRR_max` per policy
+   under per-class SLO. This is the experiment that decouples policy
+   from feedback and gives a sustainable-rate ranking.
+
+3. **The feedback amplification itself is a finding to call out**. It is
+   the reason why a "marginally better" routing policy (e.g. unified
+   over lmetric in microbenchmarks) can deliver a much bigger gap in
+   production (here ~2×): the feedback halves the in-flight count which
+   compounds on top of the per-request improvement.
+
+4. **The contrast with chat workloads is a paper section** (or at least a
+   paragraph). Chat workloads have human think-time bounded by reading
+   speed, so the feedback loop is partially broken: even if the server
+   slows down, the user-driven inter-turn delay still puts a floor on
+   how concentrated the load can become. Agentic workloads remove that
+   floor.
+
+## Open questions
+
+- **Is the feedback amplification quantifiable from B3 alone?** We have
+  total wall-clock per policy and per-session lifetime distributions; we
+  can in principle attribute the policy-vs-feedback split by comparing
+  B3's saturated-replay p90 to B4's at-fixed-λ p90 (when B4 runs).
+- **Does it matter that the original trace was captured under one
+  policy's behavior?** The trace's timestamps were the production
+  system's output at capture time. When we replay against a slower
+  policy, we are asking "what if this same set of session+turns ran on
+  a worse policy" — and the answer is "the sessions would live longer".
+  This is precisely the counterfactual we want.
+- **What happens if real tools have variable per-call latency?** Our
+  replayer assumes turn N+1 fires the instant turn N finishes. Real
+  agentic systems have some tool-execution time between turns. This is
+  a quantitative correction (raises the floor on inter-turn gap), not a
+  qualitative one — the feedback loop still applies, just with a higher
+  baseline.
+
+## Cross-reference
+
+- `replayer/replay.py:282-287` — the dispatch rule
+- `analysis/characterization/window_1_results.md` §"What Window 1 does not answer" — current treatment as caveat
+- `analysis/claude_characterization_work_plan.md` §B4 — open-loop Poisson loadgen as the orthogonal measurement

analysis/characterization/b2_sweep_analysis.py

@@ -0,0 +1,126 @@
+"""Aggregate B2 microbench cells: same- vs different-worker prefill overlap.
+
+For each (variant × prefill_size) cell we have:
+- 240 short-prompt decode requests at qps=4
+- 4 large-prompt one-token "prefill injections"
+
+The interesting question is *not* "does any other request's prefill overlap
+this decode" (the answer is always yes — every decode begins with its own
+short prefill, and at qps=4 they overlap each other constantly). The
+interesting question is "does an injected large prefill on the *same* worker
+materially slow this decode down?".
+
+So we:
+1) extract each cell's injection windows = [(t_dispatch, t_finish)
+   for r in metrics if r.workload=="prefill"];
+2) label each decode request as overlap iff its
+   [t_first_token, t_finish] intersects at least one injection window;
+3) compute TPOT p50/p90/p99 for overlap vs clean;
+4) the per-cell interference index = TPOT_p90(overlap) /
+   TPOT_p90(clean). For "different" variant this should hover near 1.0;
+   for "same" it should rise with prefill_size.
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+from collections import defaultdict
+from pathlib import Path
+
+from analysis.characterization.joined_analysis import (
+    _percentile,
+    load_jsonl,
+    write_json,
+)
+
+
+def _overlaps(a_start: float, a_end: float, b_start: float, b_end: float) -> bool:
+    return a_start <= b_end and b_start <= a_end
+
+
+def _analyze_cell(metrics_rows: list[dict]) -> dict:
+    prefills = [r for r in metrics_rows if r.get("workload") == "prefill"
+                 and r.get("error") is None]
+    decodes = [r for r in metrics_rows if r.get("workload") == "decode"
+                and r.get("error") is None]
+
+    inj_windows: list[tuple[float, float]] = []
+    for p in prefills:
+        ts = p.get("t_dispatch_unix")
+        te = p.get("t_finish_unix")
+        if ts is None or te is None:
+            continue
+        inj_windows.append((float(ts), float(te)))
+
+    overlap_tpots: list[float] = []
+    clean_tpots: list[float] = []
+    overlap_ttfts: list[float] = []
+    clean_ttfts: list[float] = []
+    for d in decodes:
+        ts = d.get("t_dispatch_unix")
+        te = d.get("t_finish_unix")
+        if ts is None or te is None:
+            continue
+        is_overlap = any(_overlaps(ts, te, ws, we) for ws, we in inj_windows)
+        tpot = d.get("tpot_s")
+        ttft = d.get("ttft_s")
+        if tpot is not None:
+            (overlap_tpots if is_overlap else clean_tpots).append(float(tpot))
+        if ttft is not None:
+            (overlap_ttfts if is_overlap else clean_ttfts).append(float(ttft))
+
+    p90_overlap = _percentile(overlap_tpots, 0.90) if overlap_tpots else None
+    p90_clean = _percentile(clean_tpots, 0.90) if clean_tpots else None
+    idx = (p90_overlap / p90_clean) if (p90_overlap and p90_clean) else None
+    return {
+        "n_prefill_injections": len(prefills),
+        "n_decode_total": len(decodes),
+        "n_decode_overlap": len(overlap_tpots),
+        "n_decode_clean": len(clean_tpots),
+        "tpot_p50_overlap_s": _percentile(overlap_tpots, 0.50),
+        "tpot_p90_overlap_s": p90_overlap,
+        "tpot_p99_overlap_s": _percentile(overlap_tpots, 0.99),
+        "tpot_p50_clean_s": _percentile(clean_tpots, 0.50),
+        "tpot_p90_clean_s": p90_clean,
+        "tpot_p99_clean_s": _percentile(clean_tpots, 0.99),
+        "ttft_p90_overlap_s": _percentile(overlap_ttfts, 0.90)
+                                if overlap_ttfts else None,
+        "ttft_p90_clean_s": _percentile(clean_ttfts, 0.90)
+                                if clean_ttfts else None,
+        "interference_index": idx,
+    }
+
+
+def main() -> None:
+    p = argparse.ArgumentParser(description="B2 sweep aggregation (window-overlap)")
+    p.add_argument("--sweep-dir", type=Path, required=True)
+    p.add_argument("--out", type=Path, default=None)
+    args = p.parse_args()
+
+    rows: list[dict] = []
+    for variant_dir in sorted(args.sweep_dir.glob("*/")):
+        if variant_dir.name in ("logs",):
+            continue
+        for cell_dir in sorted(variant_dir.glob("p*/")):
+            window_path = cell_dir / "run_window.json"
+            metrics_path = cell_dir / "metrics.jsonl"
+            if not window_path.exists() or not metrics_path.exists():
+                continue
+            window = json.loads(window_path.read_text())
+            metrics_rows = load_jsonl(metrics_path)
+            cell = _analyze_cell(metrics_rows)
+            rows.append({
+                "variant": variant_dir.name,
+                "prefill_size": int(window["prefill_size"]),
+                "decode_endpoint": window["decode_endpoint"],
+                "prefill_endpoint": window["prefill_endpoint"],
+                **cell,
+            })
+    out_path = args.out or args.sweep_dir / "b2_sweep_summary.json"
+    write_json(out_path, {"rows": rows})
+    print(json.dumps(rows, indent=2))
+
+
+if __name__ == "__main__":
+    main()

analysis/characterization/b3_policies_pseudocode.md

@@ -0,0 +1,167 @@
+# B3 Routing Policies — Pseudocode
+
+Reference: `scripts/cache_aware_proxy.py`. All five policies share the
+same per-worker state machine; only the per-request `pick_instance_*`
+function differs.
+
+## Shared per-instance state
+
+```text
+inst.url
+inst.ongoing_tokens        # sum of input_length across in-flight reqs
+inst.pending_prefill_tokens
+inst.ongoing_decode_tokens
+inst.num_requests          # waiting + running
+inst.cached_blocks         # LRU set of 512-token block hashes
+inst.estimate_cache_hit(tokens) -> int
+                           # longest prefix of `tokens` (in BLOCK_SIZE
+                           # chunks) currently in cached_blocks
+```
+
+Each pick is one round-trip on every routing decision; counters are
+mutated when a request starts/finishes, not inside the picker.
+
+## 1. `lmetric` — main baseline
+
+Pure per-request LMetric scoring. No session affinity, no
+overload-break logic.
+
+```text
+def pick_lmetric(instances, token_ids, input_length):
+    best, best_score = None, +inf
+    for inst in instances:
+        cache_hit   = inst.estimate_cache_hit(token_ids)
+        new_prefill = max(0, input_length - cache_hit)
+        p_tokens    = inst.pending_prefill_tokens + new_prefill
+        bs          = inst.num_requests
+        score       = p_tokens * bs
+        if score < best_score:
+            best, best_score = inst, score
+    return best
+```
+
+Intuition: prefer the instance where the expected new prefill cost
+times the running batch size is smallest. Cache hit reduces
+`new_prefill`, so cache-warm workers win at equal load.
+
+## 2. `load_only` — B3 control (no cache, no affinity)
+
+```text
+def pick_load_only(instances):
+    return min(instances, key=lambda inst: inst.num_requests)
+```
+
+Ties: Python `min` returns the first-seen, so when `num_requests` is
+equal across all instances (e.g. fresh start), pick always lands on
+`instances[0]`. This produces unintentional stickiness at low
+concurrency — the B3 lmetric/load_only comparison reads APC=54.1%
+for load_only partly because of that.
+
+## 3. `sticky` — B3 control (hard affinity)
+
+Once a session is bound, never break the binding under any load.
+
+```text
+def pick_sticky(instances, session_id, affinity):
+    if session_id in affinity:
+        return instances[affinity[session_id]]  # unconditional
+    chosen = min(instances, key=lambda i: i.num_requests)
+    affinity[session_id] = index_of(chosen)
+    return chosen
+```
+
+This is the upper bound on locality and the worst case on hot-spot
+behavior — a single heavy session pins one worker forever.
+
+## 4. `unified` — hybrid affinity + LMetric fallback
+
+Sticks to the affinity worker only when the cache is genuinely warm
+and the affinity worker is not overloaded; otherwise falls back to
+LMetric with a 4-key tie-breaker.
+
+```text
+def pick_unified(instances, token_ids, input_length, session_id, affinity):
+    avg_reqs = max(mean(inst.num_requests for inst in instances), 1.0)
+
+    # Affinity gate (both must hold)
+    if session_id in affinity:
+        a = instances[affinity[session_id]]
+        a_hit_ratio = a.estimate_cache_hit(token_ids) / max(input_length, 1)
+        if a_hit_ratio > 0.5 \
+                and a.num_requests <= avg_reqs * OVERLOAD_FACTOR:
+            return a   # stick
+
+    # LMetric fallback with multi-key tie-break
+    keys = []
+    for inst in instances:
+        cache_hit   = inst.estimate_cache_hit(token_ids)
+        new_prefill = max(0, input_length - cache_hit)
+        p_tokens    = inst.pending_prefill_tokens + new_prefill
+        bs          = inst.num_requests
+        score       = p_tokens * bs
+        keys.append((score, new_prefill, bs, idx_of(inst)))
+
+    best_3tuple = min(k[:3] for k in keys)
+    tied = [k for k in keys if k[:3] == best_3tuple]
+    if len(tied) > 1:
+        # Round-robin among ties so brand-new traffic doesn't pin
+        # instance 0 when BS=0 across the board.
+        winner = tied[_rr_counter % len(tied)]
+        _rr_counter += 1
+    else:
+        winner = tied[0]
+    return instances[winner.idx]
+```
+
+Tie-break ordering: `score` (LMetric primary), then `new_prefill`
+(prefer the most cache-warm at equal score), then `num_requests`
+(prefer least-loaded), then a global round-robin counter.
+
+`OVERLOAD_FACTOR` defaults to 2.0; when the affinity worker is
+above 2× average load, the picker treats it as overloaded and steers
+away.
+
+## 5. `capped` — `lmetric` on a session-mass-capped trace
+
+Not a new picker. The picker is the same `pick_lmetric` from §1; the
+input trace is preprocessed.
+
+```text
+def build_capped_trace(input_path, output_path, MAX_TURNS=8):
+    by_session = group_by_session_id(load(input_path))
+    capped = []
+    for sid, turns in by_session.items():
+        turns.sort_by(lambda t: (t.turn_id, t.timestamp))
+        capped.extend(turns[:MAX_TURNS])
+    capped.sort_by(timestamp)        # restore wall-clock order
+    write_jsonl(capped, output_path)
+
+# At run time:
+trace      = build_capped_trace("w600_r0.0015_st30.jsonl")
+picker     = pick_lmetric
+```
+
+For this trace `MAX_TURNS=8` truncates the heavy-tail sessions (full
+trace turns/session p90=1, p99=18, max=3091). The intent is to
+isolate "what does LMetric look like when no session is heavy
+enough to hot-spot a worker?" — comparing capped vs lmetric is the
+session-mass ablation.
+
+## Decision matrix
+
+|  | session affinity | cache aware | load aware | overload break |
+|---|---|---|---|---|
+| `lmetric` | ✗ | ✓ (via `cache_hit` → `new_prefill`) | ✓ (`num_requests` BS factor) | n/a |
+| `load_only` | ✗ | ✗ | ✓ (`num_requests` only) | n/a |
+| `sticky` | ✓ (hard) | ✗ (relies on physical hits, not scoring) | only on first turn | **never** |
+| `unified` | ✓ (gated) | ✓ | ✓ | gate: `cache_ratio>0.5` AND `num_req ≤ 2× avg` |
+| `capped` | same as `lmetric`; the trace itself is truncated | | | |
+
+## What each control isolates
+
+- `lmetric` vs `load_only` → contribution of cache awareness alone.
+- `lmetric` vs `sticky` → contribution of session affinity vs
+  per-request LMetric scoring at the cost of hot-spot.
+- `lmetric` vs `unified` → did adding gated session affinity help?
+- `lmetric` vs `capped` → how much of the residual hot-spot in
+  `lmetric` is driven by heavy-tail sessions specifically?

analysis/characterization/current_results/all_figures_index.md

@@ -0,0 +1,29 @@
+# Figures Index
+
+## Window 0 (pre-Window-1 audit, legacy runs)
+
+| Figure | Intended Claim |
+|---|---|
+| [fig_full_trace_workload.png](figures/fig_full_trace_workload.png) | Full GLM-5.1 trace is long-input, short-output, and high input/output ratio. |
+| [fig_session_skew.png](figures/fig_session_skew.png) | Session input-token mass is highly skewed; top sessions dominate work. |
+| [fig_pdsep_vs_combined.png](figures/fig_pdsep_vs_combined.png) | Static PD-sep regresses TTFT/E2E vs combined (legacy 200-req A/B). |
+| [fig_elastic_vs_baseline.png](figures/fig_elastic_vs_baseline.png) | Existing elastic transfer-based run does not improve TTFT/TPOT over high-contention baseline. |
+| [fig_gpu_balance.png](figures/fig_gpu_balance.png) | Existing runs show GPU-util imbalance; not sufficient for hot-spot causal claim. |
+| [fig_claim_status.png](figures/fig_claim_status.png) | Audit separates supported / partial / unsupported claims. |
+
+## Window 1 (B1' + B3 + B2)
+
+Generated by `analysis/characterization/render_window1_figures.py`.
+Source data: `analysis/characterization/window_1_results/`.
+
+| Figure | Intended Claim |
+|---|---|
+| [fig_kv_footprint_cdf.png](../window_1_results/figures/fig_kv_footprint_cdf.png) | KV per request for Qwen3-Coder-30B-A3B: p50/p90/p99 = 1.83/8.04/11.49 GiB; p99 takes 12% of H20 HBM. |
+| [fig_reuse_decomposition.png](../window_1_results/figures/fig_reuse_decomposition.png) | Cached_tokens decompose 93.2% intra / 5.7% cross / 1.1% shared on w600 lmetric run. |
+| [fig_b3_apc_vs_upper.png](../window_1_results/figures/fig_b3_apc_vs_upper.png) | Per-policy APC achieved vs theoretical intra-session ceiling (79.6%). |
+| [fig_b3_apc_vs_hotspot.png](../window_1_results/figures/fig_b3_apc_vs_hotspot.png) | Locality-vs-hotspot tradeoff across policies; unified dominates the frontier. |
+| [fig_b3_latency_bars.png](../window_1_results/figures/fig_b3_latency_bars.png) | TTFT / TPOT / E2E p90 bars per policy. |
+| [fig_b3_per_worker_ttft_p90.png](../window_1_results/figures/fig_b3_per_worker_ttft_p90.png) | Per-worker TTFT p90 distribution per policy; sticky's engine_3 and unified's engine_4 are the hot workers. |
+| [fig_b3_failure_breakdown.png](../window_1_results/figures/fig_b3_failure_breakdown.png) | Slow-request cause stacked bar per policy. |
+| [fig_b2_tpot_vs_prefill.png](../window_1_results/figures/fig_b2_tpot_vs_prefill.png) | TPOT during decode under same-worker prefill injection scales with prefill size; different-worker control flat. |
+| [fig_b2_ttft_vs_prefill.png](../window_1_results/figures/fig_b2_ttft_vs_prefill.png) | TTFT shows the same monotone same-worker scaling, peaking at 218× for 65k injection. |

analysis/characterization/current_results/characterization_claim_matrix.md

@@ -0,0 +1,21 @@
+# Characterization Claim Matrix
+
+Updated 2026-05-25 after Window 1 (B1' KV-footprint + reuse, B3 5-policy
+sweep, B2 PD-colo interference microbench).
+
+| Claim | Status | Supporting Data | Needed Next | Reviewer Risk |
+|---|---|---|---|---|
+| Per-session sequentiality is enforced when replayer + proxy carry the new join fields. | `supported` | A1 unix timestamps (t_dispatch/t_first_token/t_finish_unix) and X-Request-Id passthrough; smoke validation 2026-05-25 confirmed 30/30 join coverage. | Use this stack for all Window 2 B4/B5 SRR runs. | Legacy outputs/ without these fields still cannot be re-classified as `online_realistic`. |
+| Agentic workload is long-input / short-output / heavy-tail session mass. | `supported` | Full trace CPU summary (full_trace_summary.json): input p50/p90/p99 = 20k/87.9k/125.5k; top 1% sessions hold 46.5% of input mass. Full trace 2.11M requests, 1.31M sessions. | — | Sample trace (w600) percentiles inherit from this full trace but should not be extrapolated. |
+| KV per request for Qwen3-Coder-30B-A3B is 98304 B/token; p50/p90/p99 footprint = 1.83/8.04/11.49 GiB. | `supported` | window_1_results/kv_footprint_summary.json; computed from model config and full trace input lengths. | — | Assumes bf16; would scale for fp8/int8 quant. |
+| Workload reuse is overwhelmingly intra-session. | `supported` | Real reuse decomposition from w600 lmetric run: intra 93.2%, cross 5.7%, shared 1.1% (window_1_results/lmetric_reuse.json). Theoretical any-vs-intra ceiling gap 0.7 pp. | — | Trace-specific; ChatGPT-style workloads with long system prompts would shift toward shared-prefix. |
+| Theoretical APC ceiling on w600 trace is 79.6% (intra) / 80.3% (any-session). | `supported` | window_1_results/apc_upper_w600.json from block-level trie walk on `hash_ids`. | — | Assumes infinite per-worker cache (no eviction); achieved values must be read as fraction of this ceiling. |
+| Cache-aware LMetric leaves a measurable locality gap (22.7 pp). | `supported` | lmetric achieved 56.9% vs intra-session ceiling 79.6%; B3 sweep window_1_results/b3_policy_comparison.json. | — | sticky data shows the gap can be recovered by harder affinity. |
+| Hybrid affinity (`unified`) breaks the locality-vs-latency tradeoff. | `supported` | unified APC 79.4% (97% of intra ceiling) AND TTFT p90 7.24 s (lmetric is 15.6 s). | — | unified concentrates a single very hot worker (engine_4 at 37.7 s p90); hotspot_index 3.35. |
+| Same-worker prefill-decode interference is causal, not correlation. | `supported` | B2 microbench: different-worker control idx 0.92-1.02 across 32× prefill-size variation; same-worker TTFT idx scales 2.15× (2k) → 218× (65k). window_1_results/b2_sweep_summary.json. | — | Synthetic decode load (256-token prompts at 4 req/s) bounds the realism; production behavior is layered on top of B3. |
+| The cost of same-worker prefill interference migrates from TPOT to TTFT as prefill size grows past the chunked-prefill horizon. | `supported` | B2 same-worker TPOT p90 idx peaks at 32k (7.89×) and *drops* at 65k (2.26×), while TTFT idx grows monotonically (94.6× → 218×) and TPOT p99 grows monotonically (59 → 169.5 ms). See window_1_results.md "TPOT idx peaks at 32k, not 65k". | — | SLO thresholds for TTFT and TPOT cannot be the same under PD-colo; this should be reflected in B4 SRR sweep design. |
+| Hard session affinity (`sticky`) inflates same-worker prefill-decode interference. | `supported` | sticky interference_index 13.65 vs lmetric 6.53; sticky's slow-request breakdown 57% same-worker overlap vs lmetric 23%. | — | Confirms the B2 causal claim observed at the system level. |
+| Heavy-tail sessions are a contributor to hot-spot but not the sole cause. | `supported` | Cap-8 trace (37% requests dropped) reduces hotspot_index only ~10% (2.253 → 2.020 after fixing the `joined_analysis.hotspot_index` median bug). | Run capped under unified to see whether unified's hotspot also persists. | Reviewer might counter that cap=8 is too soft; a stricter cap could be tried. |
+| B3 saturated-replay latency gaps include an agentic dispatch-coupling feedback term, which is intentional and matches production. | `supported, framed as feature` | `replayer/replay.py:282-287` fires turn N+1 immediately when turn N is behind schedule (no human think-time). Under saturation, slow policies have longer mean session lifetime, more concurrent in-flight, higher worker pressure — so B3 latency gaps reflect "policy + feedback amplification", which is what a production operator switching policies on agentic workload experiences. See `analysis/characterization/agentic_dispatch_coupling.md`. | Run B4 open-loop Poisson at fixed λ to get the orthogonal "controlled-load" measurement; both are needed, not "B4 fixes B3". | Some reviewers will read "non-Poisson arrivals" as benchmark crime; the rebuttal is the agentic-vs-chat workload distinction. |
+| SRR per policy under SLO is not yet measured. | `not_yet_supported` | B3 was driven by trace timestamps with strict session sequentiality; saturation is reached but not parameterized. | Run B4 with the A4 open-loop Poisson loadgen, per-class SLO, 5 policies × λ binary search. | Without B4 the paper cannot claim "policy X sustains higher load than Y". |
+| Failure attribution near SRR boundary is not yet measured. | `not_yet_supported` | B5 protocol exists; no runs. | After B4, rerun each policy at 0.9× / 1.0× / 1.1× of its SRR_max with the same instrumentation, label slow requests. | The current `joined_analysis.label_slow_requests` is the labeler; needs SRR boundaries to point at. |

analysis/characterization/current_results/claim_matrix.json

@@ -0,0 +1,51 @@
+[
+  {
+    "claim": "Batch 0 substrate audit is only partially complete for existing runs.",
+    "needed_next": "Add request dispatch and finish/error timestamps to future replayer/proxy metrics.",
+    "reviewer_risk": "Cannot use these runs to prove online per-session sequentiality.",
+    "status": "partially_supported",
+    "supporting_data": "metrics.jsonl lacks actual dispatch/finish timestamps in current artifacts."
+  },
+  {
+    "claim": "Batch 1 workload shape can be characterized from formatted traces and metrics.",
+    "needed_next": "Add cache-hit joined records for actual reuse decomposition.",
+    "reviewer_risk": "Actual cache reuse decomposition needs cached_tokens joined with hash_ids.",
+    "status": "supported_for_trace_shape",
+    "supporting_data": "Full compact trace CPU summary in full_trace_summary.json: input p50/p90/p99 = 20k/87.9k/125.5k, output p50/p90/p99 = 80/811/6.6k, top 1% sessions hold 46.5% of input-token mass."
+  },
+  {
+    "claim": "Static PD separation is worse than combined in existing 200-request GPU A/B.",
+    "needed_next": "Refresh with PD matrix, multiple seeds, cudagraph-enabled methodology.",
+    "reviewer_risk": "Legacy run has no per-stage TTFT breakdown and no step-level KV occupancy.",
+    "status": "supported_by_existing_artifact",
+    "supporting_data": "outputs/gpu_ab_combined vs outputs/gpu_ab_pdsep metrics.summary.json."
+  },
+  {
+    "claim": "Elastic transfer-based migration does not improve high-contention 500-request run.",
+    "needed_next": "Attribute whether failure is trigger quality, transfer overhead, or wrong load regime.",
+    "reviewer_risk": "Existing metrics lack actual sequentiality proof and per-request transfer waterfall.",
+    "status": "supported_by_existing_artifact",
+    "supporting_data": "outputs/contention_16s_ts10 vs outputs/contention_16s_elastic metrics.summary.json and gpu_util.csv."
+  },
+  {
+    "claim": "PD-colo prefill/decode interference is not yet directly proven by step-level data in this package.",
+    "needed_next": "Run Batch 2 controlled same-worker/different-worker injection with step timestamps.",
+    "reviewer_risk": "Cannot claim interference as causal without Batch 2.",
+    "status": "not_yet_supported",
+    "supporting_data": "No decode-step and prefill-overlap timestamp artifact found in summarized runs."
+  },
+  {
+    "claim": "Session hot-spot residual imbalance is suggested but not fully attributed.",
+    "needed_next": "Collect per-worker queue delay, session-to-worker map, and per-session token mass per worker.",
+    "reviewer_risk": "GPU util imbalance alone is not enough to prove session hot-spot.",
+    "status": "partially_supported",
+    "supporting_data": "gpu_util.csv shows per-GPU mean-util imbalance in existing runs."
+  },
+  {
+    "claim": "SRR is not measured by existing fixed-request runs.",
+    "needed_next": "Implement Batch 4 Poisson session-arrival SRR sweep.",
+    "reviewer_risk": "Latency-at-one-load cannot support sustainable throughput claim.",
+    "status": "not_yet_supported",
+    "supporting_data": "No arrival-rate sweep artifacts found."
+  }
+]

analysis/characterization/current_results/comparisons.json

@@ -0,0 +1,95 @@
+[
+  {
+    "baseline": "outputs/gpu_ab_combined",
+    "e2e_p50_delta_pct": 40.870329127661,
+    "e2e_p90_delta_pct": 15.206416995091814,
+    "error_count": [
+      2,
+      13
+    ],
+    "gpu_imbalance_ratio": [
+      3.2445157838416265,
+      11.149056603773586
+    ],
+    "gpu_mean_util": [
+      30.541666666666664,
+      12.367081447963802
+    ],
+    "name": "combined_vs_pdsep_200",
+    "request_count": [
+      200,
+      200
+    ],
+    "success_count": [
+      198,
+      187
+    ],
+    "tpot_p90_delta_pct": 1.3481309269699875,
+    "ttft_p50_delta_pct": 98.06752892925572,
+    "ttft_p90_delta_pct": 44.79649177751278,
+    "variant": "outputs/gpu_ab_pdsep",
+    "wall_clock_delta_pct": 142.27736808267244
+  },
+  {
+    "baseline": "outputs/contention_16s_ts10",
+    "e2e_p50_delta_pct": 11.538788125232664,
+    "e2e_p90_delta_pct": -5.080083318118138,
+    "error_count": [
+      2,
+      2
+    ],
+    "gpu_imbalance_ratio": [
+      2.310775410408662,
+      2.600767754318618
+    ],
+    "gpu_mean_util": [
+      23.030492424242425,
+      26.349561403508773
+    ],
+    "name": "contention_baseline_vs_elastic_500",
+    "request_count": [
+      500,
+      500
+    ],
+    "success_count": [
+      498,
+      498
+    ],
+    "tpot_p90_delta_pct": 13.63098996823875,
+    "ttft_p50_delta_pct": 12.433589435386224,
+    "ttft_p90_delta_pct": 13.412576920999959,
+    "variant": "outputs/contention_16s_elastic",
+    "wall_clock_delta_pct": -0.5645626396767849
+  },
+  {
+    "baseline": "outputs/combined_1000req",
+    "e2e_p50_delta_pct": 202.85189980479385,
+    "e2e_p90_delta_pct": 128.274511020719,
+    "error_count": [
+      2,
+      204
+    ],
+    "gpu_imbalance_ratio": [
+      null,
+      null
+    ],
+    "gpu_mean_util": [
+      null,
+      null
+    ],
+    "name": "combined_1000_vs_pdsep_mooncake",
+    "request_count": [
+      1000,
+      1000
+    ],
+    "success_count": [
+      998,
+      796
+    ],
+    "tpot_p90_delta_pct": -34.83638659447109,
+    "ttft_p50_delta_pct": 781.9835547522864,
+    "ttft_p90_delta_pct": 1030.68607857992,
+    "variant": "outputs/exp3_pd_sep_tp1_mooncake",
+    "wall_clock_delta_pct": 119.18997774599991
+  }
+]

analysis/characterization/current_results/current_results.md

@@ -0,0 +1,77 @@
+# Current Characterization Results
+
+Generated: 2026-05-25T06:52:18.096448+00:00
+Git commit: `21ffb3d4f77956d008b1815a3c0d46e0188ac390`
+
+## Canonical Full-Trace CPU Summary
+
+Source: `dash0:/home/admin/cpfs/wjh/ali-trace/trace-glm5.1-formatted/051315-051317.jsonl`.
+This is CPU-only parsing of the compact formatted trace with session IDs
+reconstructed from `parent_chat_id` chains.
+
+| Metric | Value |
+|---|---:|
+| Requests | 2,114,220 |
+| Sessions | 1,307,276 |
+| Trace span | 7,199.975 s |
+| Input tokens p50/p90/p99 | 20,030 / 87,855 / 125,527 |
+| Output tokens p50/p90/p99 | 80 / 811 / 6,615 |
+| Input/output ratio p50/p90/p99 | 217.8 / 1,204.4 / 4,251.6 |
+| Turns/session p50/p90/p99/max | 1 / 1 / 18 / 3,091 |
+| Session input tokens p50/p90/p99/max | 12,486 / 72,676 / 974,934 / 156,756,974 |
+| Top 1% / 5% / 10% sessions by input-token mass | 46.5% / 66.5% / 74.6% |
+
+Immediate reading: the full trace strongly supports long-input/short-output
+and heavy-tailed session token mass. It does **not** by itself prove online
+sequentiality or actual cache-hit reuse; those require runtime timestamps and
+cache-hit fields.
+
+## Existing Run Summaries
+
+| Run | OK/Req | TTFT p50/p90 | E2E p50/p90 | TPOT p90 | GPU mean util | GPU imbalance |
+|---|---:|---:|---:|---:|---:|---:|
+| outputs/gpu_ab_combined | 198/200 | 1.01/9.36 | 5.05/30.2 | 0.0732 | 30.5 | 3.24 |
+| outputs/gpu_ab_pdsep | 187/200 | 1.99/13.5 | 7.11/34.8 | 0.0742 | 12.4 | 11.1 |
+| outputs/contention_16s_ts10 | 498/500 | 0.826/9.71 | 5.8/51 | 0.103 | 23 | 2.31 |
+| outputs/contention_16s_elastic | 498/500 | 0.929/11 | 6.47/48.4 | 0.117 | 26.3 | 2.6 |
+| outputs/combined_1000req | 998/1000 | 0.393/2.57 | 3.22/28 | 0.113 | n/a | n/a |
+| outputs/exp3_pd_sep_tp1_mooncake | 796/1000 | 3.47/29 | 9.75/63.9 | 0.0739 | n/a | n/a |
+
+## Pairwise Comparisons
+
+| Comparison | TTFT p50 Δ | TTFT p90 Δ | E2E p50 Δ | E2E p90 Δ | TPOT p90 Δ | Wall-clock Δ |
+|---|---:|---:|---:|---:|---:|---:|
+| combined_vs_pdsep_200 | +98.1% | +44.8% | +40.9% | +15.2% | +1.3% | +142.3% |
+| contention_baseline_vs_elastic_500 | +12.4% | +13.4% | +11.5% | -5.1% | +13.6% | -0.6% |
+| combined_1000_vs_pdsep_mooncake | +782.0% | +1030.7% | +202.9% | +128.3% | -34.8% | +119.2% |
+
+## What We Can Say Now
+
+- **partially_supported**: Batch 0 substrate audit is only partially complete for existing runs.
+  Supporting data: metrics.jsonl lacks actual dispatch/finish timestamps in current artifacts.
+  Next: Add request dispatch and finish/error timestamps to future replayer/proxy metrics.
+- **supported_for_trace_shape**: Batch 1 workload shape can be characterized from formatted traces and metrics.
+  Supporting data: full compact trace CPU summary in `full_trace_summary.json`: input p50/p90/p99 = 20k/87.9k/125.5k, output p50/p90/p99 = 80/811/6.6k, top 1% sessions hold 46.5% of input-token mass.
+  Next: add cache-hit joined records for actual reuse decomposition.
+- **supported_by_existing_artifact**: Static PD separation is worse than combined in existing 200-request GPU A/B.
+  Supporting data: outputs/gpu_ab_combined vs outputs/gpu_ab_pdsep metrics.summary.json.
+  Next: Refresh with PD matrix, multiple seeds, cudagraph-enabled methodology.
+- **supported_by_existing_artifact**: Elastic transfer-based migration does not improve high-contention 500-request run.
+  Supporting data: outputs/contention_16s_ts10 vs outputs/contention_16s_elastic metrics.summary.json and gpu_util.csv.
+  Next: Attribute whether failure is trigger quality, transfer overhead, or wrong load regime.
+- **not_yet_supported**: PD-colo prefill/decode interference is not yet directly proven by step-level data in this package.
+  Supporting data: No decode-step and prefill-overlap timestamp artifact found in summarized runs.
+  Next: Run Batch 2 controlled same-worker/different-worker injection with step timestamps.
+- **partially_supported**: Session hot-spot residual imbalance is suggested but not fully attributed.
+  Supporting data: gpu_util.csv shows per-GPU mean-util imbalance in existing runs.
+  Next: Collect per-worker queue delay, session-to-worker map, and per-session token mass per worker.
+- **not_yet_supported**: SRR is not measured by existing fixed-request runs.
+  Supporting data: No arrival-rate sweep artifacts found.
+  Next: Implement Batch 4 Poisson session-arrival SRR sweep.
+
+## Main Reviewer Risks
+
+- **high**: Session sequentiality not proven - Add dispatch/finish timestamps and run Batch 0 before SRR claims.
+- **medium**: Legacy PD-sep data may not match final methodology - Use fresh PD matrix for paper-grade claims.
+- **medium**: GPU util is not a sufficient hot-spot proof - Add route-decision and per-worker queue logs for Batch 3.
+- **medium**: Cache reuse decomposition is incomplete without joined hash/cache-hit data - Emit hash_ids/session_id/cached_tokens in the same per-request record.

analysis/characterization/current_results/full_trace_summary.json

@@ -0,0 +1,56 @@
+{
+  "input": {
+    "count": 2114220,
+    "max": 202371,
+    "mean": 33637.38370084476,
+    "p50": 20030.0,
+    "p90": 87855.1000000001,
+    "p95": 104738.0,
+    "p99": 125527.0
+  },
+  "input_output_ratio": {
+    "count": 2108130,
+    "max": 143664.0,
+    "mean": 534.3516074828406,
+    "p50": 217.8,
+    "p90": 1204.3769610389616,
+    "p95": 1814.3478327228322,
+    "p99": 4251.585499999998
+  },
+  "output": {
+    "count": 2114220,
+    "max": 132665,
+    "mean": 444.97059624826176,
+    "p50": 80.0,
+    "p90": 811.0,
+    "p95": 2213.0,
+    "p99": 6614.810000000056
+  },
+  "path": "/home/admin/cpfs/wjh/ali-trace/trace-glm5.1-formatted/051315-051317.jsonl",
+  "records": 2114220,
+  "session_input_tokens": {
+    "count": 1307276,
+    "max": 156756974,
+    "mean": 54400.77639916896,
+    "p50": 12486.0,
+    "p90": 72676.0,
+    "p95": 108523.25,
+    "p99": 974933.75
+  },
+  "sessions": 1307276,
+  "top_session_input_fraction": {
+    "top10pct": 0.7464402483455778,
+    "top1pct": 0.46456810581415175,
+    "top5pct": 0.6651718740752172
+  },
+  "trace_span_s": 7199.975,
+  "turns_per_session": {
+    "count": 1307276,
+    "max": 3091,
+    "mean": 1.6172713336739908,
+    "p50": 1.0,
+    "p90": 1.0,
+    "p95": 2.0,
+    "p99": 18.0
+  }
+}

analysis/characterization/current_results/main_claim_allowed_runs.md

@@ -0,0 +1,76 @@
+# Main-Claim Allowed Runs
+
+Status: post-Window-1 audit gate
+Date: 2026-05-25
+
+## Allowed For Workload-Shape Claims
+
+- `dash0:/home/admin/cpfs/wjh/ali-trace/trace-glm5.1-formatted/051315-051317.jsonl`
+  - Compact formatted full trace (2.11M requests / 1.31M sessions).
+  - CPU summary in `current_results/full_trace_summary.json` and
+    Window 1 KV footprint in `window_1_results/kv_footprint_summary.json`.
+  - Supports: long-input / short-output / heavy-tail token mass /
+    KV per request distribution.
+
+- `traces/w600_r0.0015_st30.jsonl`
+  - 1214 requests / 274 sessions / 53.3 M tokens.
+  - APC theoretical bounds in `window_1_results/apc_upper_w600.json`.
+  - Routing-policy comparison trace used by B3.
+
+## Allowed For Routing-Policy Comparison Claims
+
+These five runs share an identical trace, model, and 8-instance topology;
+they support all per-policy claims about APC, hotspot, interference,
+latency, failure breakdown.
+
+- `outputs/b3_sweep_20260525_095043/lmetric/`  — main baseline
+- `outputs/b3_sweep_20260525_095043/load_only/` — control: no cache / no affinity
+- `outputs/b3_sweep_20260525_095043/sticky/`   — control: hard affinity
+- `outputs/b3_sweep_20260525_095043/unified/`  — hybrid (interference index
+  unavailable; see note in claim matrix)
+- `outputs/b3_sweep_20260525_095043/capped/`   — lmetric on cap-8 trace
+
+Aggregated comparison: `outputs/b3_sweep_20260525_095043/b3_policy_comparison.json`.
+Rendered figures: `analysis/characterization/window_1_results/figures/fig_b3_*.png`.
+
+## Allowed For PD-colo Interference Causal Claims
+
+- `outputs/b2_microbench/sweep/{same,different}/p{2048,8192,16384,32768,65536}/`
+  - Decode-load + prefill-injection microbench.
+  - `b2_sweep_summary.json` aggregates per-cell TPOT and TTFT
+    (overlap vs clean), indexed by `prefill_size × variant`.
+  - Different-worker control idx ≈ 1.0 across 32× variation;
+    same-worker idx scales monotonically.
+
+## Allowed For Legacy Baseline Sanity Claims
+
+These older runs predate Window 1 instrumentation. They can still support
+"static PD-sep was worse than combined on this fixed-request workload"
+type claims, but **not** the new SRR or per-policy comparisons.
+
+- `outputs/gpu_ab_combined`, `outputs/gpu_ab_pdsep`
+- `outputs/contention_16s_ts10`, `outputs/contention_16s_elastic`
+- `outputs/combined_1000req`, `outputs/exp3_pd_sep_tp1_mooncake`
+
+## NOT Allowed For Main Claims
+
+The following need new runs:
+
+- **B4 SRR sweep**: arrival-rate sweep with open-loop Poisson session
+  arrivals and per-class SLO. No data yet.
+- **B5 failure attribution near SRR boundary**: depends on B4.
+- **Production interference under cache_aware proxy**: B2 used direct
+  endpoints; the production routing might shift the same-worker
+  collision profile.
+
+## Required Upgrade Path
+
+For Window 2 (B4 + B5), the existing stack already meets the needs:
+- A1 unix timestamps on every metric row ✓
+- A2 worker_state snapshots ✓
+- A3 step-level engine_state (works in isolated runs since `df32499`) ✓
+- A4 open-loop Poisson loadgen ✓
+- A5 joined_analysis + failure labels ✓
+
+No new instrumentation required. The only software gap is `b3_analyze.sh`
+must use per-policy engine_state when present (fixed at commit `df32499`).

analysis/characterization/current_results/reproduction_commands.sh

@@ -0,0 +1,62 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+# Window 0 audit refresh (legacy run summaries).
+python3 analysis/characterization/summarize_runs.py \
+    --output-dir analysis/characterization/current_results \
+    --runs outputs/gpu_ab_combined outputs/gpu_ab_pdsep \
+           outputs/contention_16s_ts10 outputs/contention_16s_elastic \
+           outputs/combined_1000req outputs/exp3_pd_sep_tp1_mooncake
+
+# B1' Per-request KV footprint on the full trace (runs on dash0 directly,
+# CPU-only; the formatted full trace is hundreds of GiB).
+python3 analysis/characterization/analyze.py \
+    --trace ~/ali-trace/trace-glm5.1-formatted/051315-051317.jsonl \
+    --kv-bytes-per-token 98304 \
+    --task-name full_trace_with_kv \
+    --output-root outputs/characterization \
+    --overwrite
+
+# w600 trace APC theoretical bound.
+python3 scripts/compute_apc_upper_bound.py \
+    --trace traces/w600_r0.0015_st30.jsonl \
+    --out outputs/apc_upper_w600.json
+
+# B3 5-policy routing sweep on dash0 (8 × TP1 instances).
+#   First three policies share one vLLM lifecycle (hot-cache, fast):
+bash scripts/b3_sweep.sh                  # writes outputs/b3_sweep_<TS>/
+
+#   Last two run isolated with cold vLLM:
+bash scripts/b3_isolated_policy.sh unified \
+    traces/w600_r0.0015_st30.jsonl \
+    outputs/b3_sweep_<TS>/unified
+
+python3 scripts/build_capped_trace.py \
+    --input traces/w600_r0.0015_st30.jsonl \
+    --output outputs/b3_sweep_<TS>/capped/trace.jsonl \
+    --max-turns 8
+
+bash scripts/b3_isolated_policy.sh lmetric \
+    outputs/b3_sweep_<TS>/capped/trace.jsonl \
+    outputs/b3_sweep_<TS>/capped
+
+# B3 analysis (joined records + indices) and report.
+bash scripts/b3_analyze.sh outputs/b3_sweep_<TS>
+python3 scripts/render_b3_report.py --sweep-dir outputs/b3_sweep_<TS>
+
+# B2 PD-colo interference microbench. Launch 2 vLLM instances on
+# ports 8100 and 8101 with --enable-prompt-tokens-details first, then:
+python3 scripts/b2_interference.py \
+    --decode-endpoint http://127.0.0.1:8100 \
+    --prefill-endpoint http://127.0.0.1:8101 \
+    --model <model-path> \
+    --out-dir outputs/b2_microbench/sweep \
+    --prefill-sizes 2048,8192,16384,32768,65536 \
+    --variants different,same
+python3 analysis/characterization/b2_sweep_analysis.py \
+    --sweep-dir outputs/b2_microbench/sweep
+
+# Window 1 figure rendering (CPU only).
+python3 analysis/characterization/render_window1_figures.py \
+    --results-dir analysis/characterization/window_1_results \
+    --out-dir analysis/characterization/window_1_results/figures

analysis/characterization/current_results/reviewer_risk_register.json

@@ -0,0 +1,26 @@
+[
+  {
+    "evidence": "Current metrics include trace timestamp and latency but not actual dispatch/finish wall-clock timestamps.",
+    "mitigation": "Add dispatch/finish timestamps and run Batch 0 before SRR claims.",
+    "risk": "Session sequentiality not proven",
+    "severity": "high"
+  },
+  {
+    "evidence": "PD matrix scaffold exists separately; some old runs used earlier flags/methodology.",
+    "mitigation": "Use fresh PD matrix for paper-grade claims.",
+    "risk": "Legacy PD-sep data may not match final methodology",
+    "severity": "medium"
+  },
+  {
+    "evidence": "Existing artifacts have gpu_util.csv but lack per-worker queue and session ownership.",
+    "mitigation": "Add route-decision and per-worker queue logs for Batch 3.",
+    "risk": "GPU util is not a sufficient hot-spot proof",
+    "severity": "medium"
+  },
+  {
+    "evidence": "Trace has hash_ids; metrics have cached_tokens; request IDs may not join across all artifacts.",
+    "mitigation": "Emit hash_ids/session_id/cached_tokens in the same per-request record.",
+    "risk": "Cache reuse decomposition is incomplete without joined hash/cache-hit data",
+    "severity": "medium"
+  }
+]

analysis/characterization/current_results/reviewer_risk_register.md

@@ -0,0 +1,16 @@
+# Reviewer Risk Register
+
+Updated 2026-05-25 after Window 1.
+
+| Risk | Severity | Evidence | Mitigation |
+|---|---|---|---|
+| ~~Session sequentiality not proven~~ | resolved | A1 instrumentation lands per-request t_dispatch/t_first_token/t_finish unix timestamps + proxy_request_id. Smoke validation 2026-05-25 confirms 30/30 join coverage. | All Window 1 runs already use this; Window 2 inherits. |
+| ~~Cache reuse decomposition incomplete~~ | resolved | Real reuse decomposition computed in `window_1_results/lmetric_reuse.json` from joined records carrying session_id + hash_ids + cached_tokens. | — |
+| APC across hot-sweep policies may be contaminated by prior policy runs | low | First-turn cached_tokens distribution shows < 1% empirical contamination; load_only and sticky vLLMs were not restarted between policies. `unified` and `capped` are isolated cold-start. | Window 2 will isolate each policy launch by default; document in paper that lmetric/load_only/sticky reflect "warm-cache" condition. |
+| Unified missing `interference_index` due to analyzer truncate-write bug | medium | The original `b3_analyze.sh` unconditionally `slice_engine_state.py`'d each policy and used `open("w")`, overwriting unified's correctly-written engine_state with the empty-window slice from the (hot-sweep) shared dir. | Fixed in commit `df32499`. B2 microbench provides the cleaner same-vs-different interference proof, so we do not need to rerun unified. |
+| GPU 0 ghost memory after vLLM crash | low | EngineCore subprocess name is `VLLM::EngineCor`; `pkill -f "vllm serve"` misses it. Killed manually on 2026-05-25; cleanup logic in `b3_sweep.sh` and `b3_isolated_policy.sh` now also targets `EngineCore`. | — |
+| w600 trace is a 1k-request sample, not the full GLM-5.1 trace | low | All B3 + B2 percentiles are on this sample. Full-trace KV-footprint and reuse claims use the 2.11M-request full trace. | Window 2 SRR sweep uses w600; full-trace SRR would need a larger sample and more GPU budget. |
+| Trace-timestamp dispatch with strict session sequentiality stretches replay wall time | medium | lmetric's 600s trace dispatched over 49 min; system over-saturates and the dispatch window expands. | Window 2 uses A4 open-loop Poisson loadgen with explicit arrival rate, decoupling load level from trace structure. |
+| Capped cap=8 may be too soft | low | Reviewer might prefer cap=2 or cap=4 to test "no multi-turn" extreme. Cap=8 was chosen to sit between turns/session p90 (1) and p99 (18). | Re-run with a stricter cap if reviewer pushes back; underlying capped script is parameterized. |
+| B2 microbench uses synthetic short-prompt decode load (256 tokens) | low | This bounds the realism of the "decode" workload. Production decode tokens come from prior turns of long context. | The signal magnitude is robust enough that prompt length shouldn't qualitatively change conclusions; B3 sticky's failure breakdown is the production-trace confirmation. |
+| Reading B2 same-worker interference from TPOT p90 alone gives a non-monotone curve | low | TPOT p90 idx peaks at 32k (7.89×) then drops at 65k (2.26×) even though TTFT idx grows monotonically (94.6× → 218×) and TPOT p99 grows monotonically (59 → 169.5 ms). The drop is regime shift (cost migrates from TPOT to TTFT once prefill blocks first-token long enough), not interference relief. | Reports must lead with TTFT idx; TPOT p99 is the right tail indicator for TPOT. See window_1_results.md §"TPOT idx peaks at 32k, not 65k". |

analysis/characterization/current_results/run_summaries.json

@@ -0,0 +1,720 @@
+[
+  {
+    "apc_summary": {
+      "reason": "apc.txt missing",
+      "status": "unavailable"
+    },
+    "artifact_availability": {
+      "apc_txt": false,
+      "breakdown_json": false,
+      "gpu_util_csv": true,
+      "metrics_jsonl": true,
+      "metrics_summary_json": true
+    },
+    "breakdown_summary": {
+      "reason": "breakdown.json missing",
+      "status": "unavailable"
+    },
+    "error_count": 2,
+    "exists": true,
+    "external_cache_hit_ratio": null,
+    "gpu_summary": {
+      "gpu_count": 8,
+      "max_mean_util_pct": 63.166666666666664,
+      "max_min_ratio": 3.2445157838416265,
+      "mean_util_pct": 30.541666666666664,
+      "min_mean_util_pct": 19.46875,
+      "per_gpu_mean_util_pct": {
+        "0": 29.145833333333332,
+        "1": 20.041666666666668,
+        "2": 25.0,
+        "3": 63.166666666666664,
+        "4": 21.927083333333332,
+        "5": 34.90625,
+        "6": 19.46875,
+        "7": 30.677083333333332
+      },
+      "status": "available",
+      "stddev_across_gpu_mean_util_pct": 13.337857305429534
+    },
+    "latency_stats_s": {
+      "count": 198.0,
+      "mean": 13.01780862723021,
+      "p50": 5.048548387829214,
+      "p90": 30.18109704903327,
+      "p99": 119.01174414204434
+    },
+    "metrics_jsonl_rows": 200,
+    "metrics_summary_available": true,
+    "prefix_cache_hit_ratio": 0.0,
+    "request_count": 200,
+    "run": "outputs/gpu_ab_combined",
+    "session_summary": {
+      "request_cached_tokens": {
+        "count": 200,
+        "max": 0.0,
+        "mean": 0.0,
+        "p50": 0.0,
+        "p90": 0.0,
+        "p95": 0.0,
+        "p99": 0.0
+      },
+      "request_input_tokens": {
+        "count": 200,
+        "max": 111927.0,
+        "mean": 29318.375,
+        "p50": 21376.0,
+        "p90": 81218.19999999998,
+        "p95": 87571.2,
+        "p99": 101619.27999999994
+      },
+      "request_output_tokens": {
+        "count": 200,
+        "max": 5083.0,
+        "mean": 257.675,
+        "p50": 72.0,
+        "p90": 664.3,
+        "p95": 1063.9999999999993,
+        "p99": 3300.1799999999994
+      },
+      "session_count": 145,
+      "session_input_tokens": {
+        "count": 145,
+        "max": 1567423.0,
+        "mean": 40439.137931034486,
+        "p50": 10879.0,
+        "p90": 72438.39999999997,
+        "p95": 106934.39999999988,
+        "p99": 374927.28
+      },
+      "status": "available",
+      "top_session_input_fraction": {
+        "top_10pct": 0.6588294883328288,
+        "top_1pct": 0.33276622595897626,
+        "top_5pct": 0.5534430199490934
+      },
+      "turns_per_session": {
+        "count": 145,
+        "max": 21.0,
+        "mean": 1.3793103448275863,
+        "p50": 1.0,
+        "p90": 1.0,
+        "p95": 2.0,
+        "p99": 10.560000000000002
+      }
+    },
+    "success_count": 198,
+    "tpot_stats_s": {
+      "count": 198.0,
+      "mean": 0.04929455188644857,
+      "p50": 0.03717198147904128,
+      "p90": 0.07317714408040046,
+      "p99": 0.10039294634234945
+    },
+    "ttft_stats_s": {
+      "count": 198.0,
+      "mean": 3.8987488178453984,
+      "p50": 1.0068706551101059,
+      "p90": 9.355209570843726,
+      "p99": 33.855437273858115
+    },
+    "wall_clock_s": 483.543720243033
+  },
+  {
+    "apc_summary": {
+      "reason": "apc.txt missing",
+      "status": "unavailable"
+    },
+    "artifact_availability": {
+      "apc_txt": false,
+      "breakdown_json": false,
+      "gpu_util_csv": true,
+      "metrics_jsonl": true,
+      "metrics_summary_json": true
+    },
+    "breakdown_summary": {
+      "reason": "breakdown.json missing",
+      "status": "unavailable"
+    },
+    "error_count": 13,
+    "exists": true,
+    "external_cache_hit_ratio": null,
+    "gpu_summary": {
+      "gpu_count": 8,
+      "max_mean_util_pct": 26.737556561085974,
+      "max_min_ratio": 11.149056603773586,
+      "mean_util_pct": 12.367081447963802,
+      "min_mean_util_pct": 2.3981900452488687,
+      "per_gpu_mean_util_pct": {
+        "0": 26.737556561085974,
+        "1": 14.44343891402715,
+        "2": 19.036199095022624,
+        "3": 7.4389140271493215,
+        "4": 2.3981900452488687,
+        "5": 8.841628959276019,
+        "6": 11.963800904977376,
+        "7": 8.076923076923077
+      },
+      "status": "available",
+      "stddev_across_gpu_mean_util_pct": 7.15857427072345
+    },
+    "latency_stats_s": {
+      "count": 187.0,
+      "mean": 15.366858840781827,
+      "p50": 7.111906730104238,
+      "p90": 34.77056052000262,
+      "p99": 119.2064151619561
+    },
+    "metrics_jsonl_rows": 200,
+    "metrics_summary_available": true,
+    "prefix_cache_hit_ratio": 0.0,
+    "request_count": 200,
+    "run": "outputs/gpu_ab_pdsep",
+    "session_summary": {
+      "request_cached_tokens": {
+        "count": 200,
+        "max": 0.0,
+        "mean": 0.0,
+        "p50": 0.0,
+        "p90": 0.0,
+        "p95": 0.0,
+        "p99": 0.0
+      },
+      "request_input_tokens": {
+        "count": 200,
+        "max": 111927.0,
+        "mean": 29318.375,
+        "p50": 21376.0,
+        "p90": 81218.19999999998,
+        "p95": 87571.2,
+        "p99": 101619.27999999994
+      },
+      "request_output_tokens": {
+        "count": 200,
+        "max": 5097.0,
+        "mean": 290.18,
+        "p50": 75.5,
+        "p90": 685.8,
+        "p95": 1102.5999999999997,
+        "p99": 3433.6599999999844
+      },
+      "session_count": 145,
+      "session_input_tokens": {
+        "count": 145,
+        "max": 1567423.0,
+        "mean": 40439.137931034486,
+        "p50": 10879.0,
+        "p90": 72438.39999999997,
+        "p95": 106934.39999999988,
+        "p99": 374927.28
+      },
+      "status": "available",
+      "top_session_input_fraction": {
+        "top_10pct": 0.6588294883328288,
+        "top_1pct": 0.33276622595897626,
+        "top_5pct": 0.5534430199490934
+      },
+      "turns_per_session": {
+        "count": 145,
+        "max": 21.0,
+        "mean": 1.3793103448275863,
+        "p50": 1.0,
+        "p90": 1.0,
+        "p95": 2.0,
+        "p99": 10.560000000000002
+      }
+    },
+    "success_count": 187,
+    "tpot_stats_s": {
+      "count": 187.0,
+      "mean": 0.05175559044923285,
+      "p50": 0.04020531923068981,
+      "p90": 0.07416366779122173,
+      "p99": 0.10770365060609463
+    },
+    "ttft_stats_s": {
+      "count": 187.0,
+      "mean": 5.3933139261814524,
+      "p50": 1.9942838260903955,
+      "p90": 13.546015257015824,
+      "p99": 39.30951087013818
+    },
+    "wall_clock_s": 1171.516998933861
+  },
+  {
+    "apc_summary": {
+      "reason": "apc.txt missing",
+      "status": "unavailable"
+    },
+    "artifact_availability": {
+      "apc_txt": false,
+      "breakdown_json": false,
+      "gpu_util_csv": true,
+      "metrics_jsonl": true,
+      "metrics_summary_json": true
+    },
+    "breakdown_summary": {
+      "reason": "breakdown.json missing",
+      "status": "unavailable"
+    },
+    "error_count": 2,
+    "exists": true,
+    "external_cache_hit_ratio": 0.0,
+    "gpu_summary": {
+      "gpu_count": 8,
+      "max_mean_util_pct": 40.095454545454544,
+      "max_min_ratio": 2.310775410408662,
+      "mean_util_pct": 23.030492424242425,
+      "min_mean_util_pct": 17.35151515151515,
+      "per_gpu_mean_util_pct": {
+        "0": 22.243939393939392,
+        "1": 22.798484848484847,
+        "2": 22.09090909090909,
+        "3": 40.095454545454544,
+        "4": 17.484848484848484,
+        "5": 20.225757575757576,
+        "6": 17.35151515151515,
+        "7": 21.953030303030303
+      },
+      "status": "available",
+      "stddev_across_gpu_mean_util_pct": 6.752783259358802
+    },
+    "latency_stats_s": {
+      "count": 498.0,
+      "mean": 21.908606036002105,
+      "p50": 5.798741589998826,
+      "p90": 51.00022796400299,
+      "p99": 201.6967467680006
+    },
+    "metrics_jsonl_rows": 500,
+    "metrics_summary_available": true,
+    "prefix_cache_hit_ratio": 0.0,
+    "request_count": 500,
+    "run": "outputs/contention_16s_ts10",
+    "session_summary": {
+      "request_cached_tokens": {
+        "count": 500,
+        "max": 0.0,
+        "mean": 0.0,
+        "p50": 0.0,
+        "p90": 0.0,
+        "p95": 0.0,
+        "p99": 0.0
+      },
+      "request_input_tokens": {
+        "count": 500,
+        "max": 134865.0,
+        "mean": 29190.97,
+        "p50": 17756.0,
+        "p90": 77365.70000000006,
+        "p95": 87790.89999999997,
+        "p99": 111360.44
+      },
+      "request_output_tokens": {
+        "count": 500,
+        "max": 16427.0,
+        "mean": 404.286,
+        "p50": 75.0,
+        "p90": 863.7000000000004,
+        "p95": 2384.45,
+        "p99": 4928.559999999999
+      },
+      "session_count": 347,
+      "session_input_tokens": {
+        "count": 347,
+        "max": 1929072.0,
+        "mean": 42061.916426512966,
+        "p50": 11342.0,
+        "p90": 68297.20000000004,
+        "p95": 116213.79999999993,
+        "p99": 521898.36000000057
+      },
+      "status": "available",
+      "top_session_input_fraction": {
+        "top_10pct": 0.6824335059780473,
+        "top_1pct": 0.32352388426969025,
+        "top_5pct": 0.579125805000656
+      },
+      "turns_per_session": {
+        "count": 347,
+        "max": 29.0,
+        "mean": 1.440922190201729,
+        "p50": 1.0,
+        "p90": 1.0,
+        "p95": 3.0,
+        "p99": 10.54000000000002
+      }
+    },
+    "success_count": 498,
+    "tpot_stats_s": {
+      "count": 498.0,
+      "mean": 0.07772806444638436,
+      "p50": 0.06774563665097065,
+      "p90": 0.10325221968416468,
+      "p99": 0.503443661631568
+    },
+    "ttft_stats_s": {
+      "count": 498.0,
+      "mean": 3.5127640336366928,
+      "p50": 0.8264882279981975,
+      "p90": 9.71474659699743,
+      "p99": 34.40133859900379
+    },
+    "wall_clock_s": 1472.1331836190002
+  },
+  {
+    "apc_summary": {
+      "line_count": 8,
+      "preview": "inst_0: prefix=45.7% ext=24.7%\ninst_1: prefix=33.1% ext=13.3%\ninst_2: prefix=14.2% ext=0.0%\ninst_3: prefix=50.7% ext=6.3%\ninst_4: prefix=33.6% ext=0.0%\ninst_5: prefix=66.6% ext=9.2%\ninst_6: prefix=42.1% ext=18.8%\ninst_7: prefix=36.7% ext=7.9%",
+      "status": "available"
+    },
+    "artifact_availability": {
+      "apc_txt": true,
+      "breakdown_json": true,
+      "gpu_util_csv": true,
+      "metrics_jsonl": true,
+      "metrics_summary_json": true
+    },
+    "breakdown_summary": {
+      "field_sample": [
+        "cache_hit",
+        "estimated_new_tokens",
+        "input_length",
+        "policy",
+        "request_id",
+        "route_class",
+        "routed_to",
+        "t_done",
+        "t_first_token",
+        "t_proxy_recv"
+      ],
+      "mode_counts": {
+        "HEAVY_COLO": 111,
+        "HEAVY_OFFLOAD": 13,
+        "MEDIUM": 163,
+        "WARM": 213
+      },
+      "route_counts": {
+        "linear": 487
+      },
+      "row_count": 500,
+      "status": "available"
+    },
+    "error_count": 2,
+    "exists": true,
+    "external_cache_hit_ratio": 0.0,
+    "gpu_summary": {
+      "gpu_count": 8,
+      "max_mean_util_pct": 47.54385964912281,
+      "max_min_ratio": 2.600767754318618,
+      "mean_util_pct": 26.349561403508773,
+      "min_mean_util_pct": 18.280701754385966,
+      "per_gpu_mean_util_pct": {
+        "0": 22.410526315789475,
+        "1": 26.57894736842105,
+        "2": 23.54035087719298,
+        "3": 47.54385964912281,
+        "4": 18.96842105263158,
+        "5": 26.403508771929825,
+        "6": 18.280701754385966,
+        "7": 27.07017543859649
+      },
+      "status": "available",
+      "stddev_across_gpu_mean_util_pct": 8.6079068381278
+    },
+    "latency_stats_s": {
+      "count": 498.0,
+      "mean": 21.780466573040204,
+      "p50": 6.467846095998539,
+      "p90": 48.40937389100145,
+      "p99": 196.93401125499804
+    },
+    "metrics_jsonl_rows": 500,
+    "metrics_summary_available": true,
+    "prefix_cache_hit_ratio": 0.0,
+    "request_count": 500,
+    "run": "outputs/contention_16s_elastic",
+    "session_summary": {
+      "request_cached_tokens": {
+        "count": 500,
+        "max": 0.0,
+        "mean": 0.0,
+        "p50": 0.0,
+        "p90": 0.0,
+        "p95": 0.0,
+        "p99": 0.0
+      },
+      "request_input_tokens": {
+        "count": 500,
+        "max": 134865.0,
+        "mean": 29190.97,
+        "p50": 17756.0,
+        "p90": 77365.70000000006,
+        "p95": 87790.89999999997,
+        "p99": 111360.44
+      },
+      "request_output_tokens": {
+        "count": 500,
+        "max": 16427.0,
+        "mean": 395.466,
+        "p50": 75.0,
+        "p90": 809.8000000000013,
+        "p95": 2264.2999999999943,
+        "p99": 4928.559999999999
+      },
+      "session_count": 347,
+      "session_input_tokens": {
+        "count": 347,
+        "max": 1929072.0,
+        "mean": 42061.916426512966,
+        "p50": 11342.0,
+        "p90": 68297.20000000004,
+        "p95": 116213.79999999993,
+        "p99": 521898.36000000057
+      },
+      "status": "available",
+      "top_session_input_fraction": {
+        "top_10pct": 0.6824335059780473,
+        "top_1pct": 0.32352388426969025,
+        "top_5pct": 0.579125805000656
+      },
+      "turns_per_session": {
+        "count": 347,
+        "max": 29.0,
+        "mean": 1.440922190201729,
+        "p50": 1.0,
+        "p90": 1.0,
+        "p95": 3.0,
+        "p99": 10.54000000000002
+      }
+    },
+    "success_count": 498,
+    "tpot_stats_s": {
+      "count": 498.0,
+      "mean": 0.08306021258238128,
+      "p50": 0.06973490711122092,
+      "p90": 0.117326519391297,
+      "p99": 0.5879216773072795
+    },
+    "ttft_stats_s": {
+      "count": 498.0,
+      "mean": 3.757858794331448,
+      "p50": 0.9292503809992922,
+      "p90": 11.017744456999935,
+      "p99": 34.20241540799907
+    },
+    "wall_clock_s": 1463.8220696580029
+  },
+  {
+    "apc_summary": {
+      "reason": "apc.txt missing",
+      "status": "unavailable"
+    },
+    "artifact_availability": {
+      "apc_txt": false,
+      "breakdown_json": false,
+      "gpu_util_csv": false,
+      "metrics_jsonl": true,
+      "metrics_summary_json": true
+    },
+    "breakdown_summary": {
+      "reason": "breakdown.json missing",
+      "status": "unavailable"
+    },
+    "error_count": 2,
+    "exists": true,
+    "external_cache_hit_ratio": null,
+    "gpu_summary": {
+      "reason": "gpu_util.csv missing",
+      "status": "unavailable"
+    },
+    "latency_stats_s": {
+      "count": 998.0,
+      "mean": 15.744418202954922,
+      "p50": 3.220371481962502,
+      "p90": 28.005282410187647,
+      "p99": 204.00479479599744
+    },
+    "metrics_jsonl_rows": 1000,
+    "metrics_summary_available": true,
+    "prefix_cache_hit_ratio": 0.5443149471989938,
+    "request_count": 1000,
+    "run": "outputs/combined_1000req",
+    "session_summary": {
+      "request_cached_tokens": {
+        "count": 1000,
+        "max": 0.0,
+        "mean": 0.0,
+        "p50": 0.0,
+        "p90": 0.0,
+        "p95": 0.0,
+        "p99": 0.0
+      },
+      "request_input_tokens": {
+        "count": 1000,
+        "max": 171427.0,
+        "mean": 31611.14,
+        "p50": 19798.0,
+        "p90": 82584.20000000001,
+        "p95": 93305.64999999997,
+        "p99": 111947.83999999998
+      },
+      "request_output_tokens": {
+        "count": 1000,
+        "max": 41233.0,
+        "mean": 392.579,
+        "p50": 70.0,
+        "p90": 685.8000000000002,
+        "p95": 1834.6999999999975,
+        "p99": 4584.459999999996
+      },
+      "session_count": 671,
+      "session_input_tokens": {
+        "count": 671,
+        "max": 2320064.0,
+        "mean": 47110.49180327869,
+        "p50": 12394.0,
+        "p90": 71875.0,
+        "p95": 112984.5,
+        "p99": 760591.4999999925
+      },
+      "status": "available",
+      "top_session_input_fraction": {
+        "top_10pct": 0.6915149849072194,
+        "top_1pct": 0.36223103627392117,
+        "top_5pct": 0.5945521736957288
+      },
+      "turns_per_session": {
+        "count": 671,
+        "max": 36.0,
+        "mean": 1.4903129657228018,
+        "p50": 1.0,
+        "p90": 1.0,
+        "p95": 3.0,
+        "p99": 12.599999999999909
+      }
+    },
+    "success_count": 998,
+    "tpot_stats_s": {
+      "count": 998.0,
+      "mean": 0.05681003092240363,
+      "p50": 0.04029440155459775,
+      "p90": 0.11343103184021618,
+      "p99": 0.2962149563411783
+    },
+    "ttft_stats_s": {
+      "count": 998.0,
+      "mean": 1.0351265607308877,
+      "p50": 0.39304836583323777,
+      "p90": 2.5655168020166457,
+      "p99": 8.210839052917436
+    },
+    "wall_clock_s": 1340.061015482992
+  },
+  {
+    "apc_summary": {
+      "reason": "apc.txt missing",
+      "status": "unavailable"
+    },
+    "artifact_availability": {
+      "apc_txt": false,
+      "breakdown_json": false,
+      "gpu_util_csv": false,
+      "metrics_jsonl": true,
+      "metrics_summary_json": true
+    },
+    "breakdown_summary": {
+      "reason": "breakdown.json missing",
+      "status": "unavailable"
+    },
+    "error_count": 204,
+    "exists": true,
+    "external_cache_hit_ratio": null,
+    "gpu_summary": {
+      "reason": "gpu_util.csv missing",
+      "status": "unavailable"
+    },
+    "latency_stats_s": {
+      "count": 796.0,
+      "mean": 23.561836449842698,
+      "p50": 9.752956213895231,
+      "p90": 63.928921481827274,
+      "p99": 168.2179710320197
+    },
+    "metrics_jsonl_rows": 1000,
+    "metrics_summary_available": true,
+    "prefix_cache_hit_ratio": 0.0,
+    "request_count": 1000,
+    "run": "outputs/exp3_pd_sep_tp1_mooncake",
+    "session_summary": {
+      "request_cached_tokens": {
+        "count": 1000,
+        "max": 0.0,
+        "mean": 0.0,
+        "p50": 0.0,
+        "p90": 0.0,
+        "p95": 0.0,
+        "p99": 0.0
+      },
+      "request_input_tokens": {
+        "count": 1000,
+        "max": 171427.0,
+        "mean": 31611.14,
+        "p50": 19798.0,
+        "p90": 82584.20000000001,
+        "p95": 93305.64999999997,
+        "p99": 111947.83999999998
+      },
+      "request_output_tokens": {
+        "count": 1000,
+        "max": 41233.0,
+        "mean": 412.176,
+        "p50": 74.0,
+        "p90": 759.7000000000008,
+        "p95": 1834.6999999999975,
+        "p99": 4928.559999999999
+      },
+      "session_count": 671,
+      "session_input_tokens": {
+        "count": 671,
+        "max": 2320064.0,
+        "mean": 47110.49180327869,
+        "p50": 12394.0,
+        "p90": 71875.0,
+        "p95": 112984.5,
+        "p99": 760591.4999999925
+      },
+      "status": "available",
+      "top_session_input_fraction": {
+        "top_10pct": 0.6915149849072194,
+        "top_1pct": 0.36223103627392117,
+        "top_5pct": 0.5945521736957288
+      },
+      "turns_per_session": {
+        "count": 671,
+        "max": 36.0,
+        "mean": 1.4903129657228018,
+        "p50": 1.0,
+        "p90": 1.0,
+        "p95": 3.0,
+        "p99": 12.599999999999909
+      }
+    },
+    "success_count": 796,
+    "tpot_stats_s": {
+      "count": 796.0,
+      "mean": 0.05211825330315642,
+      "p50": 0.06622354774330945,
+      "p90": 0.07391575907026088,
+      "p99": 0.10499966285609896
+    },
+    "ttft_stats_s": {
+      "count": 796.0,
+      "mean": 10.753033336598003,
+      "p50": 3.4666219488717616,
+      "p90": 29.00794132403098,
+      "p99": 81.7531874559354
+    },
+    "wall_clock_s": 2937.2794416199904
+  }
+]

analysis/characterization/elastic_migration_v2/README.md

@@ -0,0 +1,329 @@
+# Elastic Migration v2: Selective PD-Separation via Mooncake
+
+Date: 2026-05-26
+Trace: `traces/w600_r0.0015_st30.jsonl` (1214 reqs, 274 sessions, 53.3 M tokens)
+Model: Qwen3-Coder-30B-A3B-Instruct, 8 × TP1 on H20
+
+## TL;DR
+
+This section explores whether the **B2-confirmed same-worker
+prefill–decode interference** can be relieved by selectively
+migrating prefill to a different worker for the requests where the
+interference cost would dominate the transfer cost. We implement
+two flavors of the routing policy (strict gates, then relaxed
+gates) and **two isolation controls** that use the unified picker
+but launch vLLMs in `kv_role=kv_both` so the connector substrate
+is on but never PD-seps:
+
+- `unified_kv_both`: with **MooncakeConnector**
+- `unified_nixl_both`: with **NixlConnector** (NVIDIA's official
+  v1 connector; isolates connector implementation from policy)
+
+Four findings:
+
+1. **`kv_role=kv_both` imposes a substantial always-on tax even
+   when no PD-sep ever fires**: with Mooncake it's TTFT p90 +45%,
+   TPOT p90 +25%, hotspot +19%; with NIXL it's TTFT p90 +38%,
+   TPOT p90 +16%, hotspot +0.2%.
+2. **About half of the substrate cost is generic v1-connector
+   framework overhead** (proxied by NIXL since it's the leanest
+   implementation): KV buffer GPU memory cut from the model's
+   working budget, `SchedulerOutput.kv_connector_metadata`
+   round-trip, and altered `kv_cache_manager` block-lifecycle
+   semantics. **NIXL is meaningfully better than Mooncake** but
+   still imposes a 16-38% tax vs no connector.
+3. **PD-sep almost never triggers on a real agentic workload**:
+   0.16% with strict gates, 0.41% with relaxed gates. Agentic
+   workloads have 93% intra-session reuse, so most requests land
+   on workers that already hold cache — the uncached tail is too
+   small to be worth migrating.
+4. **When PD-sep does fire, the cost model is wrong by ~10–20×**:
+   the calibrated `0.3s + bytes / 2.7 GB/s` predicts 1–2 s migrate
+   cost; observed TTFT on triggered requests is 12–45 s.
+
+The net latency of `unified_v2` is **not better than plain
+`unified`** under either Mooncake or NIXL substrate. Improving
+agentic PD-sep requires (a) using the leaner connector (NIXL >
+Mooncake by 5-19 pp across metrics), and (b) fixing the underlying
+transfer mechanism (E2 patches 6.1 lazy block reservation and 6.3
+layerwise pipelining), not just the routing decision.
+
+## Substrate
+
+We compare four policies on identical traces:
+
+| policy | picker | vLLM launch mode | what's it for |
+|---|---|---|---|
+| `unified` | hybrid affinity + LMetric | plain (no connector) | the headline baseline |
+| `unified_kv_both` | same as `unified` | `MooncakeConnector` + `kv_both` | substrate control: Mooncake cost without PD-sep |
+| `unified_nixl_both` | same as `unified` | `NixlConnector` + `kv_both` | substrate control: NIXL cost without PD-sep, attributes overhead to "framework vs Mooncake" |
+| `unified_v2` | unified + selective PD-sep | `MooncakeConnector` + `kv_both` + bootstrap | the actual experiment |
+
+All four use the same trace, the same 8-instance topology, the same
+shadow-drift–corrected proxy (`scripts/cache_aware_proxy.py` post-fix
+`95c8ef8`). Plain `unified` was rerun on the patched proxy
+(`b3_sweep_20260525_095043/unified`) under the same conditions.
+
+NIXL required two launch fixes beyond Mooncake:
+- `VLLM_NIXL_SIDE_CHANNEL_PORT` must be unique per instance
+  (default 5600 → 5600..5607); otherwise instances 2..8 silently
+  hang in `zmq.error.ZMQError: Address already in use`.
+- Health-check timeout had to be raised from 180 s to 360 s
+  because NIXL initialization (UCX agent + memory registration)
+  takes ~100-150 s per instance under 8-way concurrent launch.
+
+## Result 1 — kv_both is expensive by itself, and only partly Mooncake's fault
+
+![](figures/fig_kv_both_overhead.png)
+
+Switching the vLLM launch from plain to `kv_role=kv_both` without
+ever triggering PD-sep imposes a substrate tax. We compare the two
+connectors available in vendored vLLM:
+
+| metric | plain `unified` | `unified_nixl_both` | `unified_kv_both` (Mooncake) |
+|---|---:|---:|---:|
+| TTFT p50 | 0.50 s | 0.51 s (+1%) | 0.50 s (+0%) |
+| **TTFT p90** | 7.35 s | **10.13 s (+38%)** | **10.67 s (+45%)** |
+| TTFT p99 | 42.34 s | 44.58 s (+5%) | 45.19 s (+7%) |
+| TPOT p90 | 17.1 ms | **19.8 ms (+16%)** | **21.3 ms (+25%)** |
+| E2E p90 | 18.03 s | **21.18 s (+17%)** | **22.89 s (+27%)** |
+| APC | 79.4% | 79.1% (−0.3 pp) | 78.3% (−1.1 pp) |
+| **hotspot index** | 3.667 | **3.674 (+0.2%)** | **4.363 (+19%)** |
+| interference index | n/a | 5.58 | 8.57 |
+
+![](figures/fig_connector_substrate_attribution.png)
+
+Reading the table from left to right gives a clean attribution:
+
+- **NIXL−plain** = the **v1-connector framework's irreducible cost**
+  (TTFT p90 +38%, TPOT p90 +16%, E2E p90 +17%). This is the cost
+  *any* v1 KV connector imposes:
+  - the 1 GB `kv_buffer_size` carved from `gpu-memory-utilization`,
+    reducing the KV cache budget;
+  - per-step `SchedulerOutput.kv_connector_metadata` serialization
+    and round-trip through the connector worker;
+  - altered block-lifecycle semantics in `kv_cache_manager`
+    (`delay_free_blocks=True` is the default once any connector is
+    loaded, slowing LRU eviction).
+- **Mooncake−NIXL** = the **Mooncake-implementation-specific extra**
+  (TTFT p90 +7 pp, TPOT p90 +9 pp, E2E p90 +10 pp, hotspot +19 pp).
+  This is the cost Mooncake's design choices add on top of the
+  generic framework:
+  - per-scheduler-step `set(self._block_pool.cache.keys())` diff
+    against `_known_hash_keys` (`mooncake_connector.py:432-456`)
+    walks O(|cache|) on every step on every engine, costing ~4 M
+    set operations per second on a 200 k-block cache;
+  - the hash sync runs even when no `direct_read` consumer is
+    present, so the cost is paid unconditionally;
+  - block-lifecycle is further constrained because Mooncake
+    requires `delay_free` until the explicit `finished_sending`
+    arrives, vs NIXL which can release blocks earlier.
+
+The **most striking gap is hotspot**: Mooncake's per-step hash
+sync runs on the scheduler's GIL and disrupts the timeliness of
+routing decisions, amplifying load imbalance by 19%. NIXL has no
+equivalent global-state maintenance and preserves the plain-unified
+hotspot to within 0.2%.
+
+Practical implication: **you don't enable any v1 KV connector for
+free**, but if you have to enable one, NIXL is meaningfully cheaper
+than Mooncake. Even NIXL's 38% TTFT p90 tax is large enough that
+PD-sep needs to recover it on a non-trivial fraction of requests
+before being worth it.
+
+## Result 2 — PD-sep rarely fires on a real agentic trace
+
+![](figures/fig_v2_trigger_funnel.png)
+
+We log every routing decision's `v2_reason` (why we did or did not
+PD-sep). Two runs with different gate thresholds:
+
+| fall-through bucket | v2.0 strict | v2.1 relaxed | what it means |
+|---|---:|---:|---|
+| `new_local < threshold` | 1077 (88.7%) | 924 (76.1%) | uncached tail too small to justify transfer |
+| `chosen_no_active_decode` | 115 (9.5%) | 229 (18.9%) | no decode on chosen to protect |
+| `src_cache_below_threshold` | 14 (1.2%) | 36 (3.0%) | no alt instance holds enough cache |
+| `src_not_meaningfully_more_cache` | 6 (0.5%) | 16 (1.3%) | alt instance doesn't help vs chosen |
+| `cost_benefit not enough margin` | 0 | 4 (0.3%) | model says transfer cost + interference on src ≥ local interference |
+| **PD-sep TRIGGERED** | **2 (0.16%)** | **5 (0.41%)** | passed all gates and cost-benefit favored migrate |
+
+The dominant filter is `new_local < threshold`. Even with the
+threshold dropped from 16 k to 8 k tokens, three out of four requests
+have less than 8 k uncached tokens at the chosen worker. This is
+structural: with intra-session reuse measured at 93% on the same
+trace (window_1_results.md), most turns hit prefix cache on the
+session's previous worker.
+
+The second filter, `chosen_no_active_decode`, kills another fifth.
+This is a snapshot-time phenomenon: at the moment the picker runs,
+the chosen worker often has its previous request still in prefill,
+not yet decoding. The gate's intent ("don't migrate if no decode is
+being hurt by the prefill we're routing") is correct, but it ends up
+suppressing PD-sep for a real situation where decode is *about to*
+start.
+
+Even after these two filters, the cost-benefit step itself rejects
+nearly half of remaining candidates (4 out of 9 in relaxed). So the
+final trigger rate of 0.41% is a structural property, not a
+parameter-tuning problem.
+
+## Result 3 — when PD-sep fires, the cost model is wrong by 10–20×
+
+![](figures/fig_v2_predicted_vs_actual.png)
+
+The 5 PD-sep-triggered requests in v2.1 relaxed:
+
+| input | new_local | new_src | src→dst | cost_local | cost_migrate (model) | actual TTFT | actual E2E |
+|---:|---:|---:|---|---:|---:|---:|---:|
+| 21963 | 21963 |  9163 | 6→5 | 4.39 s | 4.17 s |   3.69 s |   8.48 s |
+|  8706 |  8706 |  2050 | 5→7 | 1.09 s | 0.73 s |  12.48 s |  14.31 s |
+| 13616 | 13616 |  2352 | 4→0 | 1.70 s | 1.03 s |  18.33 s |  19.50 s |
+| 49483 | 49483 |   843 | 3→4 | 11.75 s | 2.16 s | **45.13 s** | **53.55 s** |
+| 19806 | 19806 |   350 | 3→6 | 3.96 s | 1.06 s |  20.06 s |  31.98 s |
+
+The cost model predicts the migrate path will take 0.7–2.2 s; the
+actual TTFT on these requests is 12–45 s. The model's `0.3 s +
+bytes / 2.7 GB/s` calibration captures pure RDMA bandwidth in
+isolation but misses everything else that happens on the
+`decode_sent → first_token` clock: D-side scheduler step latency,
+block reservation before KV arrives (so D's cache pressure
+increases for the entire wait), the per-layer scatter of
+`batch_transfer_sync_write`, and the next-step scheduler promotion
+after `finished_recving`. The E2 audit measured this end-to-end at
+p50 = 1.1 s and **p90 = 6.7 s** on production runs; the v2.1
+triggered requests landed in the p99 tail of that distribution
+because their dst was already loaded.
+
+The first-token clock for the 49 k request is **21× the model's
+prediction**. This is not a small mis-tuning — it's a structurally
+different model.
+
+## Result 4 — four-way comparison
+
+![](figures/fig_three_way_hotspot.png)
+
+The full table:
+
+| metric | unified (plain) | unified_nixl_both | unified_kv_both (Mooncake) | unified_v2 (relaxed) |
+|---|---:|---:|---:|---:|
+| n_ok | 1214 | 1214 | 1214 | 1214 |
+| TTFT p50 | 0.50 s | 0.51 s | 0.50 s | 0.49 s |
+| TTFT p90 | 7.35 s | 10.13 s | 10.67 s | 10.98 s |
+| TTFT p99 | 42.34 s | 44.58 s | 45.19 s | 49.45 s |
+| TPOT p90 | 17.1 ms | 19.8 ms | 21.3 ms | 18.4 ms |
+| E2E p90 | 18.03 s | 21.18 s | 22.89 s | 22.53 s |
+| APC | 79.4% | 79.1% | 78.3% | 77.6% |
+| interference index | n/a | 5.58 | 8.57 | 8.46 |
+| hotspot index | 3.667 | 3.674 | 4.363 | 3.910 |
+| n_slow | 189 | 192 | 198 | 198 |
+
+### v2 vs the kv_both control (the right comparison)
+
+Compared to the kv_both control — same substrate, no PD-sep — the
+5 PD-sep triggers in v2:
+
+- **slightly improve TPOT p90 (−14%) and hotspot (−10%)**
+- **slightly worsen TTFT p90 (+3%) and TTFT p99 (+9%)**, because the
+  triggered requests themselves take ~20× the predicted transfer
+  time
+
+The net effect against the kv_both control is in the noise. The
+hotspot improvement is within the run-to-run stochastic range we saw
+earlier (v2 strict run scored 2.733 hotspot under the same
+substrate; v2 relaxed scored 3.910).
+
+### v2 vs plain unified (the headline question)
+
+`unified_v2` is **27% slower on E2E p90** and **49% slower on TTFT
+p90** than plain `unified`. The 45 pp of TTFT p90 inflation is from
+kv_both substrate, not the routing decision; nothing PD-sep does can
+recover this in our current Mooncake implementation.
+
+## Why v2's PD-sep is fundamentally choked
+
+There are three independent structural problems, each by itself
+enough to make v2 not win:
+
+1. **The kv_both substrate is the wrong default**. It pays a 45%
+   TTFT p90 tax on every request. To make selective PD-sep beat
+   plain `unified`, the saved interference per triggered request
+   times the trigger rate must exceed 45% × average TTFT, on
+   average. With 0.41% trigger rate, even saving 100% of TTFT per
+   triggered request would only save ~0.4%, which can't recover 45%.
+
+2. **Agentic intra-session reuse leaves no headroom for migration**.
+   Most turns hit cache on the worker that handled the previous
+   turn. Migrating prefill to a *different* worker is the *exact*
+   thing intra-session affinity tries to avoid: it forces the new
+   worker to pay for the cached prefix transfer instead of just
+   reusing what's already on the affinity worker. This is a
+   structural mismatch between PD-sep semantics ("send big prefills
+   to a less-busy worker") and agentic workloads ("keep sessions
+   sticky to wherever the cache is").
+
+3. **The Mooncake mechanism is 10–20× slower than the cost model
+   predicts**, primarily due to D-side pre-allocation of KV blocks
+   and the absence of layerwise pipelining (E2 audit §6.1 / §6.3).
+   The cost model can be re-calibrated, but doing so would push the
+   gate even tighter, dropping the already-tiny trigger rate to
+   nearly zero.
+
+The three are stacked: even if any two were fixed, the remaining
+one would still make PD-sep a net loss on this trace.
+
+## What this section claims for the paper
+
+1. **Same-worker prefill–decode interference is a real mechanism**
+   (B2 microbench), but **agentic workloads rarely expose it**: the
+   typical request has high cache hit and small uncached tail, so
+   the interference cost is bounded.
+2. **Routing-only solutions (unified) already capture 79% of the
+   intra-session APC ceiling and recover the latency** by avoiding
+   the heavy-tail sessions through the affinity gate. The remaining
+   23 pp gap to the ceiling is from APC LRU eviction under capacity
+   pressure, not from prefill–decode interference.
+3. **Per-request PD-sep via Mooncake on agentic workloads is not a
+   net win** in our measurements, even with a carefully-gated cost
+   model. The combined effect of kv_both substrate overhead, low
+   trigger rate, and mechanism-vs-model gap is uniformly negative.
+4. **A productive direction is mechanism-level**: fix the Mooncake
+   D-side block reservation (E2 §6.1), implement layerwise transfer
+   pipelining (E2 §6.3), and re-measure. Only if these patches drop
+   the substrate tax to <10% and the realized transfer to ≤2 s p90
+   does PD-sep become competitive with routing on agentic traces.
+
+## What v2 still validates
+
+- **The cost model's *qualitative* shape is correct**: when it says
+  "migrate", that's a request where local interference *would have*
+  been ≥ 4 s and src has ≥ 80% prefix cache. The model picks the
+  right candidate requests.
+- **The gate logic catches the right exclusions**: 88% by uncached
+  tail size, 19% by no-decode-to-protect, the rest by missing
+  source cache. Each is a structurally correct reason.
+- **The proxy shadow-drift fix is necessary infrastructure** for
+  any long-running routing experiment. We observed 3 phantom
+  corrections per ~50-minute run.
+
+## Files
+
+- `data/b3_policy_comparison.json` — the four policies' headline
+  metrics from the same B3 sweep root.
+- `data/breakdown_<policy>.json` — per-request proxy breakdown
+  including v2 gate fields and triggered-event metadata.
+- `data/per_worker_<policy>.json` — per-worker TTFT/latency p90s
+  used in the hotspot figure.
+- `figures/*.png` — the four section figures referenced above.
+- `render_figures.py` — regenerates the figures from data/.
+
+## Cross-references
+
+- `analysis/characterization/window_1_results.md` — B2 microbench
+  (same-worker interference causal proof) and B3 baseline 5-policy
+  sweep
+- `analysis/characterization/agentic_dispatch_coupling.md` — why
+  the saturated-replay setup matches agentic production
+- `analysis/characterization/b3_policies_pseudocode.md` — pickers
+  for the five baseline policies; `unified_v2` extends `unified`
+- E1 / E2 subagent reports (commit `4b833d3` message and the
+  conversation log) — full mechanism audit that informed v2's design

analysis/characterization/elastic_migration_v2/data/b3_policy_comparison.json

@@ -0,0 +1,237 @@
+{
+  "rows": [
+    {
+      "policy": "capped",
+      "n_ok": 770,
+      "n_total": 770,
+      "ttft_p50_s": 1.1989156164927408,
+      "ttft_p90_s": 12.827629912580612,
+      "ttft_p99_s": 46.61752380923125,
+      "tpot_p50_s": 0.007231239004497606,
+      "tpot_p90_s": 0.015998617687440243,
+      "tpot_p99_s": 0.11515370831539476,
+      "e2e_p50_s": 2.598489043477457,
+      "e2e_p90_s": 21.245602010778384,
+      "e2e_p99_s": 74.60736650204846,
+      "apc_ratio": 0.3158312503528108,
+      "interference_index": 6.331064378362814,
+      "hotspot_index_ttft_p90": 2.0204268015410918,
+      "reuse_intra_frac": 0.9192657105586233,
+      "reuse_cross_frac": 0.0602232594931501,
+      "n_slow": 185,
+      "failure_counts": {
+        "cache_miss_large_append": 60,
+        "hot_worker_queue": 66,
+        "same_worker_prefill_overlap": 45,
+        "unknown": 14
+      }
+    },
+    {
+      "policy": "lmetric",
+      "n_ok": 1214,
+      "n_total": 1214,
+      "ttft_p50_s": 0.9387824369769078,
+      "ttft_p90_s": 15.671339168207492,
+      "ttft_p99_s": 53.56683189840049,
+      "tpot_p50_s": 0.008854518407308914,
+      "tpot_p90_s": 0.02122720699121469,
+      "tpot_p99_s": 0.18280341184277568,
+      "e2e_p50_s": 2.754255389008904,
+      "e2e_p90_s": 24.8209177934099,
+      "e2e_p99_s": 80.59924928059091,
+      "apc_ratio": 0.5694312382571595,
+      "interference_index": 6.530231061794441,
+      "hotspot_index_ttft_p90": 2.252837147833725,
+      "reuse_intra_frac": 0.9321238805590836,
+      "reuse_cross_frac": 0.05679481258506571,
+      "n_slow": 295,
+      "failure_counts": {
+        "cache_miss_large_append": 94,
+        "hot_worker_queue": 68,
+        "same_worker_prefill_overlap": 69,
+        "unknown": 64
+      }
+    },
+    {
+      "policy": "load_only",
+      "n_ok": 1214,
+      "n_total": 1214,
+      "ttft_p50_s": 1.2609447415161412,
+      "ttft_p90_s": 20.197147866390882,
+      "ttft_p99_s": 52.84285237012196,
+      "tpot_p50_s": 0.009231464695980247,
+      "tpot_p90_s": 0.026851662550158716,
+      "tpot_p99_s": 0.3211630676943426,
+      "e2e_p50_s": 3.58568156149704,
+      "e2e_p90_s": 33.459180271782685,
+      "e2e_p99_s": 93.95083751494239,
+      "apc_ratio": 0.5412093853102866,
+      "interference_index": 9.16424627504275,
+      "hotspot_index_ttft_p90": 1.2940319990630569,
+      "reuse_intra_frac": 0.9353191550754928,
+      "reuse_cross_frac": 0.053372184678592026,
+      "n_slow": 379,
+      "failure_counts": {
+        "cache_miss_large_append": 151,
+        "hot_worker_queue": 33,
+        "same_worker_prefill_overlap": 108,
+        "unknown": 87
+      }
+    },
+    {
+      "policy": "sticky",
+      "n_ok": 1214,
+      "n_total": 1214,
+      "ttft_p50_s": 0.5415176274836995,
+      "ttft_p90_s": 18.021296651283045,
+      "ttft_p99_s": 74.09429564891524,
+      "tpot_p50_s": 0.008952101894096181,
+      "tpot_p90_s": 0.03641285916619554,
+      "tpot_p99_s": 0.35152006935195085,
+      "e2e_p50_s": 2.081947358994512,
+      "e2e_p90_s": 34.62592205510591,
+      "e2e_p99_s": 139.68334607904353,
+      "apc_ratio": 0.7720092868396378,
+      "interference_index": 13.651718321568111,
+      "hotspot_index_ttft_p90": 2.727756623171119,
+      "reuse_intra_frac": 0.9327723488279339,
+      "reuse_cross_frac": 0.05495149683864246,
+      "n_slow": 234,
+      "failure_counts": {
+        "cache_miss_large_append": 20,
+        "hot_worker_queue": 51,
+        "same_worker_prefill_overlap": 134,
+        "unknown": 29
+      }
+    },
+    {
+      "policy": "unified",
+      "n_ok": 1213,
+      "n_total": 1214,
+      "ttft_p50_s": 0.4997710260213353,
+      "ttft_p90_s": 7.345769894809922,
+      "ttft_p99_s": 42.34170345296613,
+      "tpot_p50_s": 0.008079791456705824,
+      "tpot_p90_s": 0.017110194704198407,
+      "tpot_p99_s": 0.12655874612209597,
+      "e2e_p50_s": 1.7495028690318577,
+      "e2e_p90_s": 18.033410895219994,
+      "e2e_p99_s": 68.80023987947489,
+      "apc_ratio": 0.794261466256467,
+      "interference_index": null,
+      "hotspot_index_ttft_p90": 3.667136528736114,
+      "reuse_intra_frac": 0.9311187350942534,
+      "reuse_cross_frac": 0.056702150437367635,
+      "n_slow": 189,
+      "failure_counts": {
+        "cache_miss_large_append": 18,
+        "hot_worker_queue": 116,
+        "unknown": 55
+      }
+    },
+    {
+      "policy": "unified_kv_both",
+      "n_ok": 1214,
+      "n_total": 1214,
+      "ttft_p50_s": 0.4958424885116983,
+      "ttft_p90_s": 10.671844050800438,
+      "ttft_p99_s": 45.19353310586651,
+      "tpot_p50_s": 0.008573156389059812,
+      "tpot_p90_s": 0.021303916384344358,
+      "tpot_p99_s": 0.21501837408937963,
+      "e2e_p50_s": 1.9310281965008471,
+      "e2e_p90_s": 22.8941433175176,
+      "e2e_p99_s": 76.06128971517893,
+      "apc_ratio": 0.7828397082703908,
+      "interference_index": 8.571603637346875,
+      "hotspot_index_ttft_p90": 4.363145984888287,
+      "reuse_intra_frac": 0.9313000825240145,
+      "reuse_cross_frac": 0.056182260858791105,
+      "n_slow": 198,
+      "failure_counts": {
+        "cache_miss_large_append": 28,
+        "hot_worker_queue": 34,
+        "same_worker_prefill_overlap": 87,
+        "unknown": 49
+      }
+    },
+    {
+      "policy": "unified_nixl_both",
+      "n_ok": 1214,
+      "n_total": 1214,
+      "ttft_p50_s": 0.5138550130068325,
+      "ttft_p90_s": 10.127110345300755,
+      "ttft_p99_s": 44.5789094621703,
+      "tpot_p50_s": 0.008423213202440761,
+      "tpot_p90_s": 0.019759515867947428,
+      "tpot_p99_s": 0.1079433335279151,
+      "e2e_p50_s": 1.866590676479973,
+      "e2e_p90_s": 21.179128799570027,
+      "e2e_p99_s": 96.01196486203865,
+      "apc_ratio": 0.791441828164218,
+      "interference_index": 5.580715970433481,
+      "hotspot_index_ttft_p90": 3.673957447190547,
+      "reuse_intra_frac": 0.930632797070364,
+      "reuse_cross_frac": 0.05718149217603143,
+      "n_slow": 192,
+      "failure_counts": {
+        "cache_miss_large_append": 21,
+        "hot_worker_queue": 75,
+        "same_worker_prefill_overlap": 72,
+        "unknown": 24
+      }
+    },
+    {
+      "policy": "unified_v2",
+      "n_ok": 1214,
+      "n_total": 1214,
+      "ttft_p50_s": 0.4851180294645019,
+      "ttft_p90_s": 10.97665627548705,
+      "ttft_p99_s": 49.44861259821856,
+      "tpot_p50_s": 0.008261419251554481,
+      "tpot_p90_s": 0.018414033703249108,
+      "tpot_p99_s": 0.20999689490980364,
+      "e2e_p50_s": 1.8092182099935599,
+      "e2e_p90_s": 22.528888442111203,
+      "e2e_p99_s": 82.40234094743934,
+      "apc_ratio": 0.7758437361549086,
+      "interference_index": 8.45656745230457,
+      "hotspot_index_ttft_p90": 3.9096187869766164,
+      "reuse_intra_frac": 0.9324663389938368,
+      "reuse_cross_frac": 0.055154184817413764,
+      "n_slow": 198,
+      "failure_counts": {
+        "cache_miss_large_append": 36,
+        "hot_worker_queue": 26,
+        "same_worker_prefill_overlap": 82,
+        "unknown": 54
+      }
+    },
+    {
+      "policy": "unified_v2_strict",
+      "n_ok": 1214,
+      "n_total": 1214,
+      "ttft_p50_s": 0.4849805940175429,
+      "ttft_p90_s": 8.960840504511737,
+      "ttft_p99_s": 44.63598358390898,
+      "tpot_p50_s": 0.008222105788569446,
+      "tpot_p90_s": 0.018078321745916927,
+      "tpot_p99_s": 0.14616439095890604,
+      "e2e_p50_s": 1.8335122870048508,
+      "e2e_p90_s": 22.435233922180526,
+      "e2e_p99_s": 68.254801789901,
+      "apc_ratio": 0.789281361129855,
+      "interference_index": 6.231677388887276,
+      "hotspot_index_ttft_p90": 2.7334230011629197,
+      "reuse_intra_frac": 0.9309082618411778,
+      "reuse_cross_frac": 0.05689887985860397,
+      "n_slow": 186,
+      "failure_counts": {
+        "cache_miss_large_append": 26,
+        "hot_worker_queue": 44,
+        "same_worker_prefill_overlap": 73,
+        "unknown": 43
+      }
+    }
+  ]
+}

analysis/characterization/elastic_migration_v2/data/per_worker_unified.json

@@ -0,0 +1,24 @@
+{
+  "hotspot_index_ttft_p90": 3.667136528736114,
+  "per_worker_latency_p90_s": {
+    "http://127.0.0.1:8000": 41.42001512600109,
+    "http://127.0.0.1:8001": 12.4878579101933,
+    "http://127.0.0.1:8002": 22.462878945574648,
+    "http://127.0.0.1:8003": 15.501050900109117,
+    "http://127.0.0.1:8004": 39.956250199786155,
+    "http://127.0.0.1:8005": 36.69850301651168,
+    "http://127.0.0.1:8006": 10.116177947795954,
+    "http://127.0.0.1:8007": 20.35038618039107
+  },
+  "per_worker_ttft_p90_s": {
+    "http://127.0.0.1:8000": 11.264844838529825,
+    "http://127.0.0.1:8001": 3.6063860427122614,
+    "http://127.0.0.1:8002": 16.175747957825664,
+    "http://127.0.0.1:8003": 9.314684258581842,
+    "http://127.0.0.1:8004": 37.73397144810297,
+    "http://127.0.0.1:8005": 18.328030522551852,
+    "http://127.0.0.1:8006": 3.6328767628350773,
+    "http://127.0.0.1:8007": 7.772977900883419
+  },
+  "status": "supported"
+}

analysis/characterization/elastic_migration_v2/data/per_worker_unified_kv_both.json

@@ -0,0 +1,24 @@
+{
+  "hotspot_index_ttft_p90": 4.363145984888287,
+  "per_worker_latency_p90_s": {
+    "http://127.0.0.1:8000": 7.273825440008658,
+    "http://127.0.0.1:8001": 40.48809068736155,
+    "http://127.0.0.1:8002": 24.491076068370596,
+    "http://127.0.0.1:8003": 18.828550089401002,
+    "http://127.0.0.1:8004": 20.06954986089262,
+    "http://127.0.0.1:8005": 9.634067087399307,
+    "http://127.0.0.1:8006": 35.7432237003348,
+    "http://127.0.0.1:8007": 24.362499430915342
+  },
+  "per_worker_ttft_p90_s": {
+    "http://127.0.0.1:8000": 2.725343641615472,
+    "http://127.0.0.1:8001": 30.449911632167645,
+    "http://127.0.0.1:8002": 16.297463109577073,
+    "http://127.0.0.1:8003": 6.766894554614579,
+    "http://127.0.0.1:8004": 11.146178993489595,
+    "http://127.0.0.1:8005": 4.552643961587455,
+    "http://127.0.0.1:8006": 6.90922680192164,
+    "http://127.0.0.1:8007": 7.048551249800954
+  },
+  "status": "supported"
+}

analysis/characterization/elastic_migration_v2/data/per_worker_unified_nixl_both.json

@@ -0,0 +1,24 @@
+{
+  "hotspot_index_ttft_p90": 3.673957447190547,
+  "per_worker_latency_p90_s": {
+    "http://127.0.0.1:8000": 21.5702620673168,
+    "http://127.0.0.1:8001": 21.44246501957532,
+    "http://127.0.0.1:8002": 7.497513776784784,
+    "http://127.0.0.1:8003": 18.975387462502113,
+    "http://127.0.0.1:8004": 27.733961877820548,
+    "http://127.0.0.1:8005": 14.178356938017535,
+    "http://127.0.0.1:8006": 25.44877168269595,
+    "http://127.0.0.1:8007": 54.500166546402035
+  },
+  "per_worker_ttft_p90_s": {
+    "http://127.0.0.1:8000": 7.380765471985799,
+    "http://127.0.0.1:8001": 14.109222683508415,
+    "http://127.0.0.1:8002": 3.001173847797329,
+    "http://127.0.0.1:8003": 14.087287129514152,
+    "http://127.0.0.1:8004": 14.151121024426537,
+    "http://127.0.0.1:8005": 6.165523712011057,
+    "http://127.0.0.1:8006": 6.314287615299688,
+    "http://127.0.0.1:8007": 39.43635586597957
+  },
+  "status": "supported"
+}

analysis/characterization/elastic_migration_v2/data/per_worker_unified_v2.json

@@ -0,0 +1,24 @@
+{
+  "hotspot_index_ttft_p90": 3.9096187869766164,
+  "per_worker_latency_p90_s": {
+    "http://127.0.0.1:8000": 27.12522437740119,
+    "http://127.0.0.1:8001": 15.299228341400166,
+    "http://127.0.0.1:8002": 49.346961313998335,
+    "http://127.0.0.1:8003": 22.404519376007386,
+    "http://127.0.0.1:8004": 22.470557069155618,
+    "http://127.0.0.1:8005": 17.487964828591807,
+    "http://127.0.0.1:8006": 21.76291022058577,
+    "http://127.0.0.1:8007": 18.311422476416926
+  },
+  "per_worker_ttft_p90_s": {
+    "http://127.0.0.1:8000": 9.26557928660186,
+    "http://127.0.0.1:8001": 5.734943528624719,
+    "http://127.0.0.1:8002": 38.812515752378395,
+    "http://127.0.0.1:8003": 10.589305737824198,
+    "http://127.0.0.1:8004": 10.83847834250191,
+    "http://127.0.0.1:8005": 5.034968857781501,
+    "http://127.0.0.1:8006": 3.5207203380181493,
+    "http://127.0.0.1:8007": 12.236044214287555
+  },
+  "status": "supported"
+}

analysis/characterization/elastic_migration_v2/data/per_worker_unified_v2_strict.json

@@ -0,0 +1,24 @@
+{
+  "hotspot_index_ttft_p90": 2.7334230011629197,
+  "per_worker_latency_p90_s": {
+    "http://127.0.0.1:8000": 11.098119341616997,
+    "http://127.0.0.1:8001": 23.1559918191866,
+    "http://127.0.0.1:8002": 22.57899510498975,
+    "http://127.0.0.1:8003": 9.956129518186204,
+    "http://127.0.0.1:8004": 28.072633931197924,
+    "http://127.0.0.1:8005": 47.2373243979877,
+    "http://127.0.0.1:8006": 23.23235769500608,
+    "http://127.0.0.1:8007": 27.031178803613876
+  },
+  "per_worker_ttft_p90_s": {
+    "http://127.0.0.1:8000": 3.1871710045961663,
+    "http://127.0.0.1:8001": 8.824780725361773,
+    "http://127.0.0.1:8002": 16.364250262192222,
+    "http://127.0.0.1:8003": 4.1765614019881445,
+    "http://127.0.0.1:8004": 14.026077619416176,
+    "http://127.0.0.1:8005": 24.662665293016516,
+    "http://127.0.0.1:8006": 9.220479947811697,
+    "http://127.0.0.1:8007": 8.441550621995741
+  },
+  "status": "supported"
+}

analysis/characterization/elastic_migration_v2/render_figures.py

@@ -0,0 +1,300 @@
+"""Render PNG figures for the elastic_migration_v2 section.
+
+Inputs in ./data/ :
+- b3_policy_comparison.json
+- breakdown_unified.json, breakdown_unified_kv_both.json,
+  breakdown_unified_v2.json, breakdown_unified_v2_strict.json
+- per_worker_<policy>.json for each of the four
+
+Outputs in ./figures/ :
+- fig_kv_both_overhead.png      — three-way latency bars (plain vs kv_both vs v2)
+- fig_v2_trigger_funnel.png     — request count per fall-through reason
+- fig_v2_predicted_vs_actual.png — cost-model migrate prediction vs realized TTFT
+- fig_three_way_hotspot.png      — per-worker TTFT p90 grouped bars
+"""
+
+from __future__ import annotations
+
+import json
+from collections import Counter
+from pathlib import Path
+
+import matplotlib
+matplotlib.use("Agg")
+import matplotlib.pyplot as plt
+
+ROOT = Path(__file__).parent
+DATA = ROOT / "data"
+OUT = ROOT / "figures"
+OUT.mkdir(parents=True, exist_ok=True)
+
+
+def _load(name: str):
+    return json.loads((DATA / name).read_text())
+
+
+POLICY_COLORS = {
+    "unified":             "#2ca02c",
+    "unified_kv_both":     "#9467bd",
+    "unified_nixl_both":   "#1f77b4",
+    "unified_v2":          "#d62728",
+    "unified_v2_strict":   "#ff7f0e",
+}
+
+
+def fig_kv_both_overhead():
+    comp = _load("b3_policy_comparison.json")
+    by = {r["policy"]: r for r in comp["rows"]}
+    pols = ["unified", "unified_kv_both", "unified_nixl_both", "unified_v2"]
+    metrics = [
+        ("TTFT p90 (s)",   lambda r: r["ttft_p90_s"]),
+        ("TPOT p90 (ms)",  lambda r: r["tpot_p90_s"] * 1000),
+        ("E2E p90 (s)",    lambda r: r["e2e_p90_s"]),
+        ("hotspot index",  lambda r: r["hotspot_index_ttft_p90"]),
+    ]
+    fig, axes = plt.subplots(1, 4, figsize=(15, 4.2))
+    for ax, (label, fn) in zip(axes, metrics):
+        vals = [fn(by[p]) for p in pols]
+        labels_short = [p.replace("unified_", "") for p in pols]
+        labels_short[0] = "plain"
+        bars = ax.bar(labels_short, vals,
+                       color=[POLICY_COLORS[p] for p in pols],
+                       edgecolor="black", linewidth=0.5)
+        ax.set_title(label)
+        ax.tick_params(axis="x", rotation=15, labelsize=9)
+        for b, v in zip(bars, vals):
+            ax.text(b.get_x() + b.get_width() / 2, v,
+                     f"{v:.2f}" if v < 100 else f"{v:.0f}",
+                     ha="center", va="bottom", fontsize=9)
+        ax.grid(alpha=0.3, axis="y")
+        baseline = vals[0]
+        for i, v in enumerate(vals):
+            if i == 0:
+                continue
+            pct = (v - baseline) / baseline * 100
+            ax.text(i, v * 0.5, f"{pct:+.0f}%", ha="center",
+                     fontsize=10, fontweight="bold",
+                     color="darkred" if pct > 0 else "darkgreen")
+    fig.suptitle(
+        "Mooncake substrate adds 19-45% across metrics; NIXL is 5-19pp better but\n"
+        "still 16-38% above plain. v2's 5 PD-sep events don't recover the substrate tax."
+    )
+    fig.tight_layout()
+    fig.savefig(OUT / "fig_kv_both_overhead.png", dpi=120)
+    plt.close(fig)
+
+
+def _bucket_reasons(data):
+    """Collapse v2_reason strings into the funnel buckets."""
+    buckets = Counter()
+    for r in data:
+        if r.get("v2_pd_sep") is True:
+            buckets["PD-sep TRIGGERED"] += 1
+            continue
+        reason = (r.get("v2_reason") or "no_v2_reason").split(" (")[0]
+        if reason.startswith("local_cost"):
+            reason = "cost_benefit not enough margin"
+        buckets[reason] += 1
+    return buckets
+
+
+def fig_v2_trigger_funnel():
+    strict = _load("breakdown_unified_v2_strict.json")
+    relaxed = _load("breakdown_unified_v2.json")
+    bs = _bucket_reasons(strict)
+    br = _bucket_reasons(relaxed)
+    order = [
+        "new_local_below_threshold",
+        "chosen_no_active_decode",
+        "chosen_few_decodes",
+        "src_cache_below_threshold",
+        "src_not_meaningfully_more_cache",
+        "cost_benefit not enough margin",
+        "PD-sep TRIGGERED",
+    ]
+    labels = [k for k in order if k in bs or k in br]
+    strict_vals = [bs.get(k, 0) for k in labels]
+    relaxed_vals = [br.get(k, 0) for k in labels]
+
+    x = range(len(labels))
+    width = 0.4
+    fig, ax = plt.subplots(figsize=(11, 5))
+    ax.bar([i - width / 2 for i in x], strict_vals, width,
+            label=f"v2.0 strict (PD-sep={bs['PD-sep TRIGGERED']}/{sum(bs.values())} "
+                  f"= {bs['PD-sep TRIGGERED']*100/sum(bs.values()):.2f}%)",
+            color="#ff7f0e", edgecolor="black", linewidth=0.5)
+    ax.bar([i + width / 2 for i in x], relaxed_vals, width,
+            label=f"v2.1 relaxed (PD-sep={br['PD-sep TRIGGERED']}/{sum(br.values())} "
+                  f"= {br['PD-sep TRIGGERED']*100/sum(br.values()):.2f}%)",
+            color="#d62728", edgecolor="black", linewidth=0.5)
+    ax.set_xticks(list(x))
+    ax.set_xticklabels(labels, rotation=20, ha="right", fontsize=9)
+    ax.set_ylabel("request count")
+    ax.set_yscale("log")
+    ax.set_title(
+        "Why v2 rarely PD-seps: 88-76% of requests have new_local < threshold\n"
+        "(intra-session cache already hot). Relaxing thresholds barely helps."
+    )
+    ax.legend()
+    ax.grid(alpha=0.3, axis="y", which="both")
+    for i, (s, r) in enumerate(zip(strict_vals, relaxed_vals)):
+        if s > 0:
+            ax.text(i - width / 2, s * 1.05, str(s), ha="center", fontsize=8)
+        if r > 0:
+            ax.text(i + width / 2, r * 1.05, str(r), ha="center", fontsize=8)
+    fig.tight_layout()
+    fig.savefig(OUT / "fig_v2_trigger_funnel.png", dpi=120)
+    plt.close(fig)
+
+
+def fig_v2_predicted_vs_actual():
+    """For each PD-sep'd request, plot model-predicted migrate cost
+    vs realized TTFT. Should sit near y=x if model is calibrated; sits
+    far above if mechanism is more expensive than modeled."""
+    relaxed = _load("breakdown_unified_v2.json")
+    triggered = [r for r in relaxed if r.get("v2_pd_sep") is True]
+    if not triggered:
+        return
+    predicted = []
+    actual = []
+    sizes = []
+    rids = []
+    for r in triggered:
+        cm = r.get("v2_cost_migrate_s")
+        t0 = r.get("t_proxy_recv")
+        t_first = r.get("t_first_token")
+        if cm is None or t0 is None or t_first is None:
+            continue
+        ttft = t_first - t0
+        predicted.append(cm)
+        actual.append(ttft)
+        sizes.append(r.get("input_length", 0))
+        rids.append(r.get("request_id", "?"))
+
+    fig, ax = plt.subplots(figsize=(7, 5))
+    ax.scatter(predicted, actual,
+                s=[max(100, sz / 100) for sz in sizes],
+                color="#d62728", edgecolors="black", alpha=0.75)
+    for p, a, sz, rid in zip(predicted, actual, sizes, rids):
+        ax.annotate(f"input={sz}",
+                     (p, a), xytext=(8, 6), textcoords="offset points",
+                     fontsize=9)
+    # y=x reference + 10x line + 20x line
+    lo = 0.5
+    hi = max(50, max(actual) * 1.2)
+    ax.plot([lo, hi], [lo, hi], "k--", alpha=0.5, label="y = x (calibrated)")
+    ax.plot([lo, hi], [lo * 10, hi * 10], color="gray", linestyle=":",
+             alpha=0.4, label="10x")
+    ax.plot([lo, hi], [lo * 20, hi * 20], color="lightgray", linestyle=":",
+             alpha=0.4, label="20x")
+    ax.set_xscale("log")
+    ax.set_yscale("log")
+    ax.set_xlim(lo, hi)
+    ax.set_ylim(lo, hi)
+    ax.set_xlabel("Cost model: predicted migrate cost (s)")
+    ax.set_ylabel("Realized TTFT (s)")
+    ax.set_title(
+        "All 5 PD-sep triggered requests in v2.1 sit far above y=x.\n"
+        "Real transfer cost ~10-20x what the calibrated model predicted."
+    )
+    ax.grid(alpha=0.3, which="both")
+    ax.legend(loc="lower right")
+    fig.tight_layout()
+    fig.savefig(OUT / "fig_v2_predicted_vs_actual.png", dpi=120)
+    plt.close(fig)
+
+
+def fig_three_way_hotspot():
+    pols = ["unified", "unified_kv_both", "unified_nixl_both", "unified_v2"]
+    per_worker = {p: _load(f"per_worker_{p}.json") for p in pols}
+    workers = sorted(per_worker["unified"]["per_worker_ttft_p90_s"].keys())
+
+    x = range(len(workers))
+    n = len(pols)
+    width = 0.85 / n
+    fig, ax = plt.subplots(figsize=(12, 5))
+    for i, p in enumerate(pols):
+        d = per_worker[p]["per_worker_ttft_p90_s"]
+        vals = [d[w] for w in workers]
+        offset = (i - (n - 1) / 2) * width
+        label = p.replace("unified_", "") if p != "unified" else "plain"
+        ax.bar([j + offset for j in x], vals, width,
+                label=f"{label} (hotspot={per_worker[p]['hotspot_index_ttft_p90']:.2f})",
+                color=POLICY_COLORS[p], edgecolor="black", linewidth=0.4)
+    short = [w.replace("http://127.0.0.1:", ":") for w in workers]
+    ax.set_xticks(list(x))
+    ax.set_xticklabels(short, rotation=0, fontsize=9)
+    ax.set_ylabel("worker TTFT p90 (s)")
+    ax.set_title(
+        "Per-worker TTFT p90 distribution across substrates. Mooncake (kv_both)\n"
+        "amplifies the hot worker (hotspot 4.36); NIXL keeps it close to plain (3.67)."
+    )
+    ax.legend(loc="upper left", fontsize=9)
+    ax.grid(alpha=0.3, axis="y")
+    fig.tight_layout()
+    fig.savefig(OUT / "fig_three_way_hotspot.png", dpi=120)
+    plt.close(fig)
+
+
+def fig_connector_substrate_attribution():
+    """Decomposes overhead into v1-framework cost (shared by all connectors,
+    proxied by NIXL since it's the leanest) and Mooncake-specific cost."""
+    comp = _load("b3_policy_comparison.json")
+    by = {r["policy"]: r for r in comp["rows"]}
+    metrics = [
+        ("TTFT p90 (s)",  "ttft_p90_s",  False),
+        ("TPOT p90 (ms)", "tpot_p90_s",  True),
+        ("E2E p90 (s)",   "e2e_p90_s",   False),
+        ("hotspot index", "hotspot_index_ttft_p90", False),
+    ]
+    fig, axes = plt.subplots(1, 4, figsize=(15, 4))
+    for ax, (label, key, scale_ms) in zip(axes, metrics):
+        plain = by["unified"][key] * (1000 if scale_ms else 1)
+        nixl = by["unified_nixl_both"][key] * (1000 if scale_ms else 1)
+        moon = by["unified_kv_both"][key] * (1000 if scale_ms else 1)
+        v2 = by["unified_v2"][key] * (1000 if scale_ms else 1)
+
+        framework_cost = nixl - plain   # what NIXL adds = v1 framework cost
+        mooncake_extra = moon - nixl    # extra on top from Mooncake
+        v2_branch_extra = v2 - moon     # extra from PD-sep branch (Mooncake + 5 events)
+
+        bottom = 0
+        ax.bar(["overhead"], [plain], color="#cccccc",
+                edgecolor="black", linewidth=0.4,
+                label=f"plain unified ({plain:.2f})")
+        bottom += plain
+        ax.bar(["overhead"], [framework_cost], bottom=[bottom],
+                color="#1f77b4", edgecolor="black", linewidth=0.4,
+                label=f"v1 framework (+{framework_cost:.2f})")
+        bottom += framework_cost
+        ax.bar(["overhead"], [mooncake_extra], bottom=[bottom],
+                color="#9467bd", edgecolor="black", linewidth=0.4,
+                label=f"Mooncake extra (+{mooncake_extra:.2f})")
+        bottom += mooncake_extra
+        ax.bar(["overhead"], [v2_branch_extra], bottom=[bottom],
+                color="#d62728", edgecolor="black", linewidth=0.4,
+                label=f"v2 PD-sep branch ({v2_branch_extra:+.2f})")
+        ax.set_title(label)
+        ax.legend(fontsize=8, loc="upper right")
+        ax.grid(alpha=0.3, axis="y")
+        ax.tick_params(axis="x", labelbottom=False)
+    fig.suptitle(
+        "Attribution: plain unified vs NIXL substrate vs Mooncake substrate vs v2.\n"
+        "Blue: cost shared by any v1 connector. Purple: cost specific to Mooncake."
+    )
+    fig.tight_layout()
+    fig.savefig(OUT / "fig_connector_substrate_attribution.png", dpi=120)
+    plt.close(fig)
+
+
+def main():
+    fig_kv_both_overhead()
+    fig_v2_trigger_funnel()
+    fig_v2_predicted_vs_actual()
+    fig_three_way_hotspot()
+    fig_connector_substrate_attribution()
+    print(f"wrote 5 figures to {OUT}")
+
+
+if __name__ == "__main__":
+    main()

analysis/characterization/joined_analysis.py

@@ -0,0 +1,596 @@
+"""A5: joined-record analysis from instrumented runs.
+
+Inputs (all optional; functions degrade gracefully when missing):
+
+- replayer metrics.jsonl with A1 fields
+  (t_dispatch_unix, t_first_token_unix, t_finish_unix, proxy_request_id,
+  endpoint_url, trace_hash_ids)
+- proxy breakdown.json with A2 fields
+  (session_id, candidate_scores, chosen_score_*, t_first_token_unix,
+  t_done_unix, t_decision_unix)
+- proxy worker_state.jsonl with A2 schema (one row per route decision)
+- vLLM scheduler engine_state JSONLs from A3
+  (one per engine, env AGENTIC_STEP_LOG_PATH)
+
+Outputs under <out_dir>/:
+
+- joined.jsonl           — per-request join across all sources
+- reuse_decomposition.json
+- interference_index.json
+- hotspot_index.json
+- failure_label.jsonl
+- window_summary.json    — when run_meta.json (from SRR loadgen) is present
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import math
+import statistics
+from collections import defaultdict
+from pathlib import Path
+from typing import Any, Iterable
+
+JsonDict = dict[str, Any]
+
+
+# ---------- I/O ---------------------------------------------------------
+
+def load_jsonl(path: Path) -> list[JsonDict]:
+    if not path.exists():
+        return []
+    out: list[JsonDict] = []
+    for line in path.read_text(encoding="utf-8").splitlines():
+        line = line.strip()
+        if not line:
+            continue
+        try:
+            out.append(json.loads(line))
+        except json.JSONDecodeError:
+            continue
+    return out
+
+
+def load_json(path: Path) -> Any:
+    if not path.exists():
+        return None
+    text = path.read_text(encoding="utf-8").strip()
+    if not text:
+        return None
+    return json.loads(text)
+
+
+def write_json(path: Path, data: Any) -> None:
+    path.parent.mkdir(parents=True, exist_ok=True)
+    path.write_text(json.dumps(data, indent=2, sort_keys=True))
+
+
+def write_jsonl(path: Path, rows: Iterable[JsonDict]) -> None:
+    path.parent.mkdir(parents=True, exist_ok=True)
+    with path.open("w", encoding="utf-8") as fh:
+        for row in rows:
+            fh.write(json.dumps(row, sort_keys=True) + "\n")
+
+
+# ---------- Joining -----------------------------------------------------
+
+def build_joined_records(
+    metrics: list[JsonDict],
+    breakdown: list[JsonDict],
+    worker_state: list[JsonDict],
+) -> list[JsonDict]:
+    """Join metrics + breakdown + worker_state by request_id.
+
+    Returns one row per metrics record (the load-generator's view of truth).
+    Missing sources leave the corresponding columns as None.
+    """
+    bk_by_id = {str(r.get("request_id")): r for r in breakdown if r.get("request_id")}
+    ws_by_id = {str(r.get("request_id")): r for r in worker_state if r.get("request_id")}
+
+    joined: list[JsonDict] = []
+    for m in metrics:
+        rid = str(m.get("request_id") or m.get("proxy_request_id") or "")
+        bk = bk_by_id.get(rid)
+        ws = ws_by_id.get(rid)
+        row: JsonDict = {
+            "request_id": rid,
+            "session_id": m.get("session_id"),
+            "turn_id": m.get("turn_id"),
+            "trace_timestamp_s": m.get("trace_timestamp_s"),
+            "input_length": m.get("input_length"),
+            "output_length": m.get("output_length"),
+            "cached_tokens": m.get("cached_tokens"),
+            "actual_output_tokens": m.get("actual_output_tokens"),
+            "latency_s": m.get("latency_s"),
+            "ttft_s": m.get("ttft_s"),
+            "tpot_s": m.get("tpot_s"),
+            "t_dispatch_unix": m.get("t_dispatch_unix"),
+            "t_first_token_unix": m.get("t_first_token_unix"),
+            "t_finish_unix": m.get("t_finish_unix"),
+            "endpoint_url": m.get("endpoint_url"),
+            "trace_hash_ids": m.get("trace_hash_ids") or [],
+            "error": m.get("error"),
+        }
+        if bk:
+            row["policy"] = bk.get("policy")
+            row["route_class"] = bk.get("route_class")
+            row["routed_to"] = bk.get("routed_to")
+            row["chosen_idx"] = bk.get("chosen_idx")
+            row["chosen_score_linear"] = bk.get("chosen_score_linear")
+            row["chosen_score_lmetric"] = bk.get("chosen_score_lmetric")
+            row["estimated_new_tokens"] = bk.get("estimated_new_tokens")
+            row["cache_hit_proxy"] = bk.get("cache_hit")
+            row["proxy_t_decision_unix"] = bk.get("t_decision_unix")
+            row["proxy_t_first_token_unix"] = bk.get("t_first_token_unix")
+            row["proxy_t_done_unix"] = bk.get("t_done_unix")
+        if ws:
+            row["worker_state_at_decision"] = ws.get("workers")
+        joined.append(row)
+    return joined
+
+
+# ---------- Reuse decomposition (real) ----------------------------------
+
+def reuse_decomposition(records: list[JsonDict], block_size: int = 16) -> JsonDict:
+    """Real intra/cross/shared decomposition keyed on (session, hash blocks)."""
+    if not records:
+        return {"status": "unavailable", "reason": "no joined records"}
+
+    # block first-seen index: hash_id -> (session_id, first_seen_seq)
+    first_seen: dict[int, tuple[str, int]] = {}
+    block_sessions: dict[int, set[str]] = defaultdict(set)
+    seq = 0
+    for r in sorted(records, key=lambda x: x.get("t_dispatch_unix") or 0.0):
+        sid = str(r.get("session_id"))
+        for h in r.get("trace_hash_ids") or []:
+            h_int = int(h)
+            if h_int not in first_seen:
+                first_seen[h_int] = (sid, seq)
+            block_sessions[h_int].add(sid)
+        seq += 1
+
+    total_cached = 0
+    intra = cross = shared = unclassified = 0
+    for r in records:
+        cached = r.get("cached_tokens") or 0
+        if not cached:
+            continue
+        total_cached += cached
+        sid = str(r.get("session_id"))
+        hashes = [int(h) for h in (r.get("trace_hash_ids") or [])]
+        if not hashes:
+            unclassified += cached
+            continue
+        # Approximate: classify the cached tokens by the first non-current
+        # owner of any hash block we've seen before.
+        block_tokens = max(cached // max(len(hashes), 1), 1)
+        for h in hashes:
+            first_sid, _ = first_seen.get(h, (None, None))
+            if first_sid is None:
+                unclassified += min(block_tokens, cached)
+            elif first_sid == sid:
+                intra += min(block_tokens, cached)
+            elif len(block_sessions[h]) >= 8:
+                shared += min(block_tokens, cached)
+            else:
+                cross += min(block_tokens, cached)
+            cached -= block_tokens
+            if cached <= 0:
+                break
+
+    return {
+        "status": "supported",
+        "total_cached_tokens": total_cached,
+        "intra_session_tokens": intra,
+        "cross_session_tokens": cross,
+        "shared_prefix_tokens": shared,
+        "unclassified_tokens": unclassified,
+        "fractions": _fractions(intra, cross, shared, unclassified),
+        "shared_prefix_min_sessions": 8,
+    }
+
+
+def _fractions(*parts: int) -> JsonDict:
+    total = sum(parts)
+    if total == 0:
+        return {"intra": 0.0, "cross": 0.0, "shared": 0.0, "unclassified": 0.0}
+    labels = ["intra", "cross", "shared", "unclassified"]
+    return {label: parts[i] / total for i, label in enumerate(labels)}
+
+
+# ---------- Interference index (B2) -------------------------------------
+
+def interference_index(
+    joined: list[JsonDict],
+    engine_state_by_worker: dict[str, list[JsonDict]],
+    worker_map: dict[str, str] | None = None,
+) -> JsonDict:
+    """Label each completed request's decode period as overlap / no-overlap.
+
+    A request 'overlaps same-worker prefill' if any scheduler step on the
+    chosen worker between (t_first_token_unix, t_finish_unix) had
+    prefill_tokens > 0 from a request other than this one.
+    """
+    if not joined or not engine_state_by_worker:
+        return {"status": "unavailable",
+                "reason": "missing joined records or engine state"}
+
+    tpot_overlap: list[float] = []
+    tpot_clean: list[float] = []
+    for r in joined:
+        rid = r.get("request_id")
+        # routed_to is the vLLM worker URL; endpoint_url is the proxy URL.
+        # For worker-id matching we want routed_to.
+        worker = _resolve_worker(
+            r.get("routed_to") or r.get("endpoint_url"),
+            worker_map,
+        )
+        steps = engine_state_by_worker.get(worker)
+        if not steps:
+            continue
+        t0 = r.get("t_first_token_unix")
+        t1 = r.get("t_finish_unix")
+        tpot = r.get("tpot_s")
+        if t0 is None or t1 is None or tpot is None or r.get("error"):
+            continue
+        overlap = False
+        for s in steps:
+            t = s.get("t_unix")
+            if t is None or t < t0 or t > t1:
+                continue
+            if not s.get("prefill_tokens"):
+                continue
+            # If the only prefill belongs to *this* request, that's still
+            # this request's own prefill warming up, not interference.
+            other_prefill = False
+            for pr in s.get("per_req", []) or []:
+                if pr.get("phase") != "prefill":
+                    continue
+                # vLLM rewrites rid as `cmpl-<our_id>-<idx>-<hash>`; strip
+                # the prefix and the trailing suffix so equality to our
+                # proxy request_id works.
+                if _vllm_rid_matches(pr.get("rid"), rid):
+                    continue
+                other_prefill = True
+                break
+            if other_prefill:
+                overlap = True
+                break
+        (tpot_overlap if overlap else tpot_clean).append(float(tpot))
+
+    p90_overlap = _percentile(tpot_overlap, 0.90) if tpot_overlap else None
+    p90_clean = _percentile(tpot_clean, 0.90) if tpot_clean else None
+    idx = None
+    if p90_overlap is not None and p90_clean and p90_clean > 0:
+        idx = p90_overlap / p90_clean
+
+    return {
+        "status": "supported" if idx is not None else "partial",
+        "n_overlap_requests": len(tpot_overlap),
+        "n_clean_requests": len(tpot_clean),
+        "tpot_p90_overlap_s": p90_overlap,
+        "tpot_p90_clean_s": p90_clean,
+        "interference_index": idx,
+    }
+
+
+def _normalize_worker(url_or_id: str | None) -> str | None:
+    """Best-effort: map URLs like http://h:8000 to engine_0 by base-port 8000.
+
+    Use `_resolve_worker(url, worker_map)` instead when you have an
+    explicit URL→worker_id map (e.g. from bench.sh).
+    """
+    if not url_or_id:
+        return None
+    if url_or_id.startswith("engine_"):
+        return url_or_id
+    try:
+        port = int(url_or_id.rsplit(":", 1)[1].split("/")[0])
+        return f"engine_{port - 8000}"
+    except (ValueError, IndexError):
+        return url_or_id
+
+
+def _resolve_worker(
+    url_or_id: str | None,
+    worker_map: dict[str, str] | None,
+) -> str | None:
+    if not url_or_id:
+        return None
+    if worker_map and url_or_id in worker_map:
+        return worker_map[url_or_id]
+    return _normalize_worker(url_or_id)
+
+
+def _vllm_rid_matches(vllm_rid: Any, proxy_rid: Any) -> bool:
+    """vLLM internally rewrites rid as `cmpl-<proxy_id>-<i>-<hash>`."""
+    if vllm_rid is None or proxy_rid is None:
+        return False
+    if vllm_rid == proxy_rid:
+        return True
+    s = str(vllm_rid)
+    p = str(proxy_rid)
+    return s.startswith(f"cmpl-{p}-") or s.startswith(f"chatcmpl-{p}-")
+
+
+# ---------- Hotspot index (B3) ------------------------------------------
+
+def hotspot_index(joined: list[JsonDict]) -> JsonDict:
+    """max/median per-worker queue-delay p90 across completed requests.
+
+    Worker key is the raw `routed_to` URL (or proxy `endpoint_url`
+    fallback), so per-worker rows match the user's mental model.
+    """
+    if not joined:
+        return {"status": "unavailable"}
+
+    by_worker_queue: dict[str, list[float]] = defaultdict(list)
+    by_worker_latency: dict[str, list[float]] = defaultdict(list)
+    for r in joined:
+        if r.get("error"):
+            continue
+        # routed_to is the vLLM worker URL; endpoint_url is the proxy URL.
+        worker = r.get("routed_to") or r.get("endpoint_url")
+        if not worker:
+            continue
+        # queue delay proxy: (t_first_token - t_dispatch) - prefill estimate
+        # is fragile; use raw TTFT as the queue-stressing signal.
+        ttft = r.get("ttft_s")
+        lat = r.get("latency_s")
+        if ttft is not None:
+            by_worker_queue[worker].append(float(ttft))
+        if lat is not None:
+            by_worker_latency[worker].append(float(lat))
+
+    worker_p90_q: dict[str, float] = {
+        w: _percentile(v, 0.90) for w, v in by_worker_queue.items() if v
+    }
+    worker_p90_lat: dict[str, float] = {
+        w: _percentile(v, 0.90) for w, v in by_worker_latency.items() if v
+    }
+    p90s_q = sorted(worker_p90_q.values())
+    idx = None
+    if len(p90s_q) >= 2:
+        # True median: average of two middle values for even-length lists.
+        # Previously used sorted[n//2] which returns the ~60th percentile
+        # for n=8 and systematically under-states hotspot_index.
+        median = statistics.median(p90s_q)
+        if median > 0:
+            idx = max(p90s_q) / median
+
+    return {
+        "status": "supported" if idx is not None else "partial",
+        "per_worker_ttft_p90_s": worker_p90_q,
+        "per_worker_latency_p90_s": worker_p90_lat,
+        "hotspot_index_ttft_p90": idx,
+    }
+
+
+# ---------- Failure label (B5) ------------------------------------------
+
+DEFAULT_SLO = {
+    "ttft_p90_s": 2.0,
+    "tpot_p90_s": 0.15,
+}
+
+
+def label_slow_requests(
+    joined: list[JsonDict],
+    engine_state_by_worker: dict[str, list[JsonDict]],
+    slo: JsonDict | None = None,
+    slow_ttft_factor: float = 2.0,
+    worker_map: dict[str, str] | None = None,
+) -> list[JsonDict]:
+    slo = slo or DEFAULT_SLO
+    ttft_threshold = float(slo["ttft_p90_s"]) * slow_ttft_factor
+
+    # Per-worker queue p90 to flag hot workers
+    by_worker_ttft: dict[str, list[float]] = defaultdict(list)
+    for r in joined:
+        if r.get("ttft_s") is not None:
+            by_worker_ttft[r.get("routed_to") or ""].append(float(r["ttft_s"]))
+    worker_p90 = {w: _percentile(v, 0.90) for w, v in by_worker_ttft.items() if v}
+    global_p90 = _percentile(
+        [v for vs in by_worker_ttft.values() for v in vs], 0.90,
+    ) if by_worker_ttft else None
+    hot_workers = {w for w, p in worker_p90.items()
+                   if global_p90 and p > global_p90 * 1.2}
+
+    labels: list[JsonDict] = []
+    for r in joined:
+        ttft = r.get("ttft_s")
+        if ttft is None or r.get("error"):
+            continue
+        if ttft <= ttft_threshold:
+            continue
+        label = _assign_label(r, hot_workers, engine_state_by_worker,
+                               worker_map)
+        labels.append({
+            "request_id": r.get("request_id"),
+            "session_id": r.get("session_id"),
+            "routed_to": r.get("routed_to"),
+            "ttft_s": ttft,
+            "latency_s": r.get("latency_s"),
+            "input_length": r.get("input_length"),
+            "cached_tokens": r.get("cached_tokens"),
+            "estimated_new_tokens": r.get("estimated_new_tokens"),
+            "label": label,
+        })
+    return labels
+
+
+def _assign_label(
+    r: JsonDict,
+    hot_workers: set[str],
+    engine_state_by_worker: dict[str, list[JsonDict]],
+    worker_map: dict[str, str] | None = None,
+) -> str:
+    worker = _resolve_worker(
+        r.get("routed_to") or r.get("endpoint_url"),
+        worker_map,
+    )
+    rid = r.get("request_id")
+    steps = engine_state_by_worker.get(worker, [])
+    t0 = r.get("t_first_token_unix")
+    t1 = r.get("t_finish_unix")
+    if steps and t0 and t1:
+        for s in steps:
+            t = s.get("t_unix")
+            if t is None or t < t0 or t > t1:
+                continue
+            for pr in s.get("per_req", []) or []:
+                if pr.get("phase") != "prefill":
+                    continue
+                if _vllm_rid_matches(pr.get("rid"), rid):
+                    continue
+                return "same_worker_prefill_overlap"
+    if (r.get("routed_to") or "") in hot_workers:
+        return "hot_worker_queue"
+    est = r.get("estimated_new_tokens") or 0
+    inp = r.get("input_length") or 0
+    if est and inp and est >= 0.5 * inp:
+        return "cache_miss_large_append"
+    snap = r.get("worker_state_at_decision") or []
+    if snap:
+        chosen_idx = r.get("chosen_idx")
+        if isinstance(chosen_idx, int) and 0 <= chosen_idx < len(snap):
+            cached = snap[chosen_idx].get("cached_blocks", 0)
+            if cached and cached > 50_000:
+                return "high_kv_occupancy"
+    return "unknown"
+
+
+# ---------- Window summary (B4) -----------------------------------------
+
+def window_summary(joined: list[JsonDict], run_meta: JsonDict | None) -> JsonDict:
+    if not run_meta:
+        return {"status": "unavailable", "reason": "no run_meta"}
+    warmup_end = float(run_meta["warmup_end_unix"])
+    steady_end = float(run_meta["steady_end_unix"])
+
+    buckets: dict[str, list[JsonDict]] = {"warmup": [], "steady": [], "drain": []}
+    for r in joined:
+        t = r.get("t_dispatch_unix")
+        if t is None:
+            continue
+        if t < warmup_end:
+            buckets["warmup"].append(r)
+        elif t < steady_end:
+            buckets["steady"].append(r)
+        else:
+            buckets["drain"].append(r)
+
+    out: JsonDict = {"run_meta": run_meta, "windows": {}}
+    for name, rows in buckets.items():
+        ttft = [r["ttft_s"] for r in rows if r.get("ttft_s") is not None]
+        tpot = [r["tpot_s"] for r in rows if r.get("tpot_s") is not None]
+        e2e = [r["latency_s"] for r in rows if r.get("latency_s") is not None]
+        errs = sum(1 for r in rows if r.get("error"))
+        out["windows"][name] = {
+            "attempted": len(rows),
+            "completed": len(rows) - errs,
+            "errored": errs,
+            "ttft_p50_s": _percentile(ttft, 0.50) if ttft else None,
+            "ttft_p90_s": _percentile(ttft, 0.90) if ttft else None,
+            "tpot_p50_s": _percentile(tpot, 0.50) if tpot else None,
+            "tpot_p90_s": _percentile(tpot, 0.90) if tpot else None,
+            "e2e_p50_s": _percentile(e2e, 0.50) if e2e else None,
+            "e2e_p90_s": _percentile(e2e, 0.90) if e2e else None,
+        }
+    return out
+
+
+# ---------- helpers -----------------------------------------------------
+
+def _percentile(values: list[float], pct: float) -> float | None:
+    if not values:
+        return None
+    sorted_vals = sorted(values)
+    if len(sorted_vals) == 1:
+        return sorted_vals[0]
+    rank = pct * (len(sorted_vals) - 1)
+    lo = int(rank)
+    hi = min(lo + 1, len(sorted_vals) - 1)
+    frac = rank - lo
+    return sorted_vals[lo] * (1 - frac) + sorted_vals[hi] * frac
+
+
+def load_engine_state(dir_path: Path) -> dict[str, list[JsonDict]]:
+    """Load all engine_*.jsonl files from a directory; key by worker id."""
+    if not dir_path.exists() or not dir_path.is_dir():
+        return {}
+    by_worker: dict[str, list[JsonDict]] = {}
+    for p in sorted(dir_path.glob("engine_*.jsonl")):
+        worker_id = p.stem  # 'engine_0', etc.
+        rows = load_jsonl(p)
+        # Sort steps by time for binary-search-friendly access.
+        rows.sort(key=lambda r: r.get("t_unix") or 0.0)
+        by_worker[worker_id] = rows
+    return by_worker
+
+
+# ---------- CLI ---------------------------------------------------------
+
+def main(argv: list[str] | None = None) -> None:
+    p = argparse.ArgumentParser(description="A5 joined analysis")
+    p.add_argument("--metrics", type=Path, required=True)
+    p.add_argument("--breakdown", type=Path, default=None)
+    p.add_argument("--worker-state", type=Path, default=None)
+    p.add_argument("--engine-state-dir", type=Path, default=None,
+                   help="Directory containing engine_*.jsonl from A3 patch")
+    p.add_argument("--run-meta", type=Path, default=None,
+                   help="run_meta or window_summary.json from SRR loadgen")
+    p.add_argument("--out-dir", type=Path, required=True)
+    p.add_argument("--slow-ttft-factor", type=float, default=2.0)
+    p.add_argument("--worker-map", type=str, default=None,
+                   help="Comma-separated URL=worker_id pairs, e.g. "
+                        "http://h:9100=engine_0,http://h:9101=engine_1")
+    args = p.parse_args(argv)
+    worker_map: dict[str, str] | None = None
+    if args.worker_map:
+        worker_map = {}
+        for entry in args.worker_map.split(","):
+            url, _, wid = entry.strip().partition("=")
+            if url and wid:
+                worker_map[url] = wid
+
+    metrics = load_jsonl(args.metrics)
+    breakdown_raw = load_json(args.breakdown) if args.breakdown else []
+    if isinstance(breakdown_raw, dict):
+        breakdown_raw = breakdown_raw.get("records", [breakdown_raw])
+    breakdown = list(breakdown_raw or [])
+    worker_state_raw = load_json(args.worker_state) if args.worker_state else []
+    if isinstance(worker_state_raw, dict):
+        worker_state_raw = worker_state_raw.get("records", [worker_state_raw])
+    worker_state = list(worker_state_raw or [])
+    engine_state = (
+        load_engine_state(args.engine_state_dir) if args.engine_state_dir else {}
+    )
+    run_meta = load_json(args.run_meta) if args.run_meta else None
+
+    joined = build_joined_records(metrics, breakdown, worker_state)
+
+    args.out_dir.mkdir(parents=True, exist_ok=True)
+    write_jsonl(args.out_dir / "joined.jsonl", joined)
+    write_json(args.out_dir / "reuse_decomposition.json",
+                reuse_decomposition(joined))
+    write_json(args.out_dir / "interference_index.json",
+                interference_index(joined, engine_state, worker_map))
+    write_json(args.out_dir / "hotspot_index.json",
+                hotspot_index(joined))
+    labels = label_slow_requests(joined, engine_state,
+                                  slow_ttft_factor=args.slow_ttft_factor,
+                                  worker_map=worker_map)
+    write_jsonl(args.out_dir / "failure_label.jsonl", labels)
+    counts: dict[str, int] = defaultdict(int)
+    for L in labels:
+        counts[L["label"]] += 1
+    write_json(args.out_dir / "failure_breakdown.json",
+                {"counts": dict(counts), "n_slow": len(labels)})
+    write_json(args.out_dir / "window_summary.json",
+                window_summary(joined, run_meta))
+
+
+if __name__ == "__main__":
+    main()

analysis/characterization/plot_current_results.py

@@ -0,0 +1,332 @@
+#!/usr/bin/env python3
+"""Generate matplotlib figures for the current characterization package."""
+
+from __future__ import annotations
+
+import json
+from pathlib import Path
+from typing import Any
+
+import matplotlib
+
+matplotlib.use("Agg")
+import matplotlib.pyplot as plt
+
+
+ROOT = Path("analysis/characterization/current_results")
+FIG_DIR = ROOT / "figures"
+
+
+def main() -> None:
+    FIG_DIR.mkdir(parents=True, exist_ok=True)
+    full_trace = load_json(ROOT / "full_trace_summary.json")
+    runs = load_json(ROOT / "run_summaries.json")
+    claims = load_json(ROOT / "claim_matrix.json")
+
+    paths = [
+        plot_full_trace_workload(full_trace),
+        plot_session_skew(full_trace),
+        plot_pdsep_vs_combined(runs),
+        plot_elastic_vs_baseline(runs),
+        plot_gpu_balance(runs),
+        plot_claim_status(claims),
+    ]
+    write_figures_index(paths)
+    for path in paths:
+        print(path)
+
+
+def load_json(path: Path) -> Any:
+    return json.loads(path.read_text(encoding="utf-8"))
+
+
+def plot_full_trace_workload(summary: dict[str, Any]) -> str:
+    labels = ["p50", "p90", "p99"]
+    series = {
+        "input tokens": [summary["input"][k] for k in labels],
+        "output tokens": [summary["output"][k] for k in labels],
+        "input/output": [summary["input_output_ratio"][k] for k in labels],
+    }
+    fig, ax = plt.subplots(figsize=(9, 5.5))
+    width = 0.24
+    x = range(len(labels))
+    colors = ["#2f6fab", "#dd8452", "#4c995c"]
+    for idx, (name, values) in enumerate(series.items()):
+        offset = (idx - 1) * width
+        ax.bar([v + offset for v in x], values, width=width, label=name, color=colors[idx])
+        for xpos, value in zip([v + offset for v in x], values):
+            ax.text(xpos, value * 1.08, short_num(value), ha="center", va="bottom", fontsize=9)
+    ax.set_yscale("log")
+    ax.set_xticks(list(x), labels)
+    ax.set_ylabel("value, log scale")
+    ax.set_title("Full Trace Workload Shape")
+    ax.text(
+        0.01,
+        -0.22,
+        f"Requests={summary['records']:,}; sessions={summary['sessions']:,}; span={summary['trace_span_s']:.1f}s",
+        transform=ax.transAxes,
+        fontsize=10,
+        color="#555",
+    )
+    ax.grid(True, axis="y", alpha=0.25)
+    ax.legend()
+    return save(fig, "fig_full_trace_workload.png")
+
+
+def plot_session_skew(summary: dict[str, Any]) -> str:
+    vals = summary["top_session_input_fraction"]
+    labels = ["top 1%", "top 5%", "top 10%"]
+    fractions = [vals["top1pct"] * 100, vals["top5pct"] * 100, vals["top10pct"] * 100]
+    fig, ax = plt.subplots(figsize=(8, 5))
+    bars = ax.bar(labels, fractions, color=["#c44e52", "#dd8452", "#2f6fab"])
+    for bar, value in zip(bars, fractions):
+        ax.text(bar.get_x() + bar.get_width() / 2, value + 1.5, f"{value:.1f}%", ha="center")
+    ax.set_ylim(0, 100)
+    ax.set_ylabel("% of input-token mass")
+    ax.set_title("Session Token-Mass Skew")
+    ax.text(
+        0.01,
+        -0.20,
+        "Session input-token p50/p90/p99/max = "
+        f"{short_num(summary['session_input_tokens']['p50'])} / "
+        f"{short_num(summary['session_input_tokens']['p90'])} / "
+        f"{short_num(summary['session_input_tokens']['p99'])} / "
+        f"{short_num(summary['session_input_tokens']['max'])}",
+        transform=ax.transAxes,
+        fontsize=10,
+        color="#555",
+    )
+    ax.grid(True, axis="y", alpha=0.25)
+    return save(fig, "fig_session_skew.png")
+
+
+def plot_pdsep_vs_combined(runs: list[dict[str, Any]]) -> str:
+    by_run = {run["run"]: run for run in runs}
+    combined = by_run["outputs/gpu_ab_combined"]
+    pdsep = by_run["outputs/gpu_ab_pdsep"]
+    labels = ["TTFT p50", "TTFT p90", "E2E p50", "E2E p90"]
+    combined_vals = [
+        stat(combined, "ttft_stats_s", "p50"),
+        stat(combined, "ttft_stats_s", "p90"),
+        stat(combined, "latency_stats_s", "p50"),
+        stat(combined, "latency_stats_s", "p90"),
+    ]
+    pdsep_vals = [
+        stat(pdsep, "ttft_stats_s", "p50"),
+        stat(pdsep, "ttft_stats_s", "p90"),
+        stat(pdsep, "latency_stats_s", "p50"),
+        stat(pdsep, "latency_stats_s", "p90"),
+    ]
+    fig, ax = plt.subplots(figsize=(9, 5))
+    grouped_bars(ax, labels, [("combined", combined_vals), ("PD-sep", pdsep_vals)], ["#2f6fab", "#c44e52"])
+    ax.set_ylabel("seconds")
+    ax.set_title("Static PD-Sep vs Combined Baseline")
+    ax.text(
+        0.01,
+        -0.22,
+        f"Errors: combined={combined['error_count']}, PD-sep={pdsep['error_count']}; "
+        f"wall-clock delta={pct_delta(combined['wall_clock_s'], pdsep['wall_clock_s']):+.1f}%",
+        transform=ax.transAxes,
+        fontsize=10,
+        color="#555",
+    )
+    ax.grid(True, axis="y", alpha=0.25)
+    ax.legend()
+    return save(fig, "fig_pdsep_vs_combined.png")
+
+
+def plot_elastic_vs_baseline(runs: list[dict[str, Any]]) -> str:
+    by_run = {run["run"]: run for run in runs}
+    baseline = by_run["outputs/contention_16s_ts10"]
+    elastic = by_run["outputs/contention_16s_elastic"]
+    labels = ["TTFT p50", "TTFT p90", "E2E p50", "E2E p90", "TPOT p90"]
+    baseline_vals = [
+        stat(baseline, "ttft_stats_s", "p50"),
+        stat(baseline, "ttft_stats_s", "p90"),
+        stat(baseline, "latency_stats_s", "p50"),
+        stat(baseline, "latency_stats_s", "p90"),
+        stat(baseline, "tpot_stats_s", "p90"),
+    ]
+    elastic_vals = [
+        stat(elastic, "ttft_stats_s", "p50"),
+        stat(elastic, "ttft_stats_s", "p90"),
+        stat(elastic, "latency_stats_s", "p50"),
+        stat(elastic, "latency_stats_s", "p90"),
+        stat(elastic, "tpot_stats_s", "p90"),
+    ]
+    fig, ax = plt.subplots(figsize=(10, 5))
+    grouped_bars(ax, labels, [("baseline", baseline_vals), ("elastic", elastic_vals)], ["#2f6fab", "#dd8452"])
+    ax.set_ylabel("seconds")
+    ax.set_title("Elastic Transfer-Based Migration vs High-Contention Baseline")
+    ax.text(
+        0.01,
+        -0.22,
+        f"GPU imbalance ratio: baseline={nested(baseline, ['gpu_summary', 'max_min_ratio']):.2f}x, "
+        f"elastic={nested(elastic, ['gpu_summary', 'max_min_ratio']):.2f}x",
+        transform=ax.transAxes,
+        fontsize=10,
+        color="#555",
+    )
+    ax.grid(True, axis="y", alpha=0.25)
+    ax.legend()
+    return save(fig, "fig_elastic_vs_baseline.png")
+
+
+def plot_gpu_balance(runs: list[dict[str, Any]]) -> str:
+    selected = [
+        ("combined", "outputs/gpu_ab_combined"),
+        ("PD-sep", "outputs/gpu_ab_pdsep"),
+        ("16s base", "outputs/contention_16s_ts10"),
+        ("16s elastic", "outputs/contention_16s_elastic"),
+    ]
+    by_run = {run["run"]: run for run in runs}
+    labels = [label for label, _ in selected]
+    mean_util = [nested(by_run[path], ["gpu_summary", "mean_util_pct"]) for _, path in selected]
+    imbalance = [nested(by_run[path], ["gpu_summary", "max_min_ratio"]) for _, path in selected]
+    fig, axes = plt.subplots(1, 2, figsize=(11, 4.8))
+    axes[0].bar(labels, mean_util, color="#4c995c")
+    axes[0].set_ylabel("mean GPU util (%)")
+    axes[0].set_title("Mean Utilization")
+    axes[0].tick_params(axis="x", rotation=20)
+    axes[0].grid(True, axis="y", alpha=0.25)
+    axes[1].bar(labels, imbalance, color="#76619c")
+    axes[1].set_ylabel("max/min mean util")
+    axes[1].set_title("Imbalance Ratio")
+    axes[1].tick_params(axis="x", rotation=20)
+    axes[1].grid(True, axis="y", alpha=0.25)
+    fig.suptitle("GPU Utilization Balance in Existing Runs")
+    fig.text(
+        0.02,
+        0.01,
+        "GPU util imbalance is suggestive only; hot-spot causality still needs per-worker queue and session mapping.",
+        fontsize=10,
+        color="#555",
+    )
+    return save(fig, "fig_gpu_balance.png")
+
+
+def plot_claim_status(claims: list[dict[str, Any]]) -> str:
+    order = [
+        "supported_by_existing_artifact",
+        "supported_for_trace_shape",
+        "partially_supported",
+        "not_yet_supported",
+    ]
+    counts = {status: 0 for status in order}
+    for claim in claims:
+        counts[claim["status"]] = counts.get(claim["status"], 0) + 1
+    labels = [status.replace("_", "\n") for status in order if counts.get(status)]
+    values = [counts[status] for status in order if counts.get(status)]
+    fig, ax = plt.subplots(figsize=(9, 5))
+    bars = ax.bar(labels, values, color=["#4c995c", "#2f6fab", "#dd8452", "#c44e52"][: len(values)])
+    for bar, value in zip(bars, values):
+        ax.text(bar.get_x() + bar.get_width() / 2, value + 0.05, str(value), ha="center")
+    ax.set_ylabel("claim count")
+    ax.set_title("Current Claim Support Status")
+    ax.grid(True, axis="y", alpha=0.25)
+    return save(fig, "fig_claim_status.png")
+
+
+def grouped_bars(ax: Any, labels: list[str], series: list[tuple[str, list[float]]], colors: list[str]) -> None:
+    x = list(range(len(labels)))
+    width = 0.35
+    for idx, ((name, values), color) in enumerate(zip(series, colors)):
+        offset = (idx - (len(series) - 1) / 2) * width
+        bars = ax.bar([pos + offset for pos in x], values, width=width, label=name, color=color)
+        for bar, value in zip(bars, values):
+            ax.text(bar.get_x() + bar.get_width() / 2, value * 1.02, short_num(value), ha="center", va="bottom", fontsize=8)
+    ax.set_xticks(x, labels)
+
+
+def stat(run: dict[str, Any], stat_name: str, key: str) -> float:
+    return float(run[stat_name][key])
+
+
+def nested(run: dict[str, Any], keys: list[str]) -> float:
+    current: Any = run
+    for key in keys:
+        current = current[key]
+    return float(current)
+
+
+def pct_delta(base: float, variant: float) -> float:
+    return (variant - base) / base * 100.0
+
+
+def short_num(value: float) -> str:
+    if abs(value) >= 1_000_000:
+        return f"{value / 1_000_000:.1f}M"
+    if abs(value) >= 10_000:
+        return f"{value / 1000:.1f}k"
+    if abs(value) >= 1000:
+        return f"{value / 1000:.2f}k"
+    if abs(value) >= 100:
+        return f"{value:.0f}"
+    if abs(value) >= 10:
+        return f"{value:.1f}"
+    return f"{value:.2f}"
+
+
+def save(fig: Any, name: str) -> str:
+    path = FIG_DIR / name
+    fig.tight_layout(rect=(0, 0.04, 1, 0.95))
+    fig.savefig(path, dpi=180)
+    plt.close(fig)
+    return str(path)
+
+
+def write_figures_index(paths: list[str]) -> None:
+    claims = {
+        "fig_full_trace_workload.png": (
+            "Full Trace Workload",
+            "Full GLM-5.1 trace is long-input, short-output, and high input/output ratio.",
+        ),
+        "fig_session_skew.png": (
+            "Session Skew",
+            "Session input-token mass is highly skewed; top sessions dominate work.",
+        ),
+        "fig_pdsep_vs_combined.png": (
+            "PD-Sep vs Combined",
+            "Existing static PD-sep A/B regresses TTFT/E2E vs combined.",
+        ),
+        "fig_elastic_vs_baseline.png": (
+            "Elastic vs Baseline",
+            "Existing elastic transfer-based run does not improve TTFT/TPOT over high-contention baseline.",
+        ),
+        "fig_gpu_balance.png": (
+            "GPU Balance",
+            "Existing runs show GPU-util imbalance, but more data is needed for hot-spot causality.",
+        ),
+        "fig_claim_status.png": (
+            "Claim Status",
+            "Current audit separates supported, partial, and unsupported claims.",
+        ),
+    }
+    lines = [
+        "# Figures Index",
+        "",
+        "Generated by:",
+        "",
+        "```bash",
+        ".venv/bin/python analysis/characterization/plot_current_results.py",
+        "```",
+        "",
+        "| Figure | Intended Claim |",
+        "|---|---|",
+    ]
+    for path in paths:
+        name = Path(path).name
+        title, claim = claims[name]
+        rel_path = f"figures/{name}"
+        lines.append(f"| [{name}]({rel_path}) | {claim} |")
+    lines.extend(["", "## Figure Previews", ""])
+    for path in paths:
+        name = Path(path).name
+        title, claim = claims[name]
+        rel_path = f"figures/{name}"
+        lines.extend([f"### {title}", "", claim, "", f"![{title}]({rel_path})", ""])
+    (ROOT / "all_figures_index.md").write_text("\n".join(lines).rstrip() + "\n", encoding="utf-8")
+
+
+if __name__ == "__main__":
+    main()

analysis/characterization/protocols.md

@@ -0,0 +1,264 @@
+# Characterization Protocols For Remaining Batches
+
+Status: implementation protocol and audit checklist
+Date: 2026-05-25
+
+This file completes the `analysis/characterization` scaffold for the TODO
+list. It separates what is already implemented from what requires fresh GPU
+runs or new engine/proxy instrumentation.
+
+## Implemented Now
+
+### Batch 0/1 Analyzer
+
+Use:
+
+```bash
+python3 analysis/characterization/analyze.py \
+  --trace traces/w600_r0.0015_st30.jsonl \
+  --kv-bytes-per-token 98304 \
+  --task-name w600_local_full_trace \
+  --overwrite
+```
+
+The analyzer writes:
+
+- `manifest.json`
+- `summary.json`
+- `summary.md`
+- `audit.md`
+- `session_concurrency.json`
+- `session_arrival_stats.json`
+- `turn_interval_stats.json`
+- `trace_profile.json`
+- `workload_summary.json`
+- `kv_footprint_summary.json`
+- `reuse_decomposition.json`
+- `session_skew.json`
+- `append_delta_stats.json`
+
+Limitations:
+
+- Actual online sequentiality requires dispatch and finish/error timestamps.
+  Existing `metrics.jsonl` artifacts generally do not contain these fields.
+- Actual reuse decomposition requires `cached_tokens`/`cache_hit`, `hash_ids`,
+  and `session_id` in the same joinable request record.
+
+### Existing-Run Audit
+
+Use:
+
+```bash
+python3 analysis/characterization/summarize_runs.py
+```
+
+The script writes an audit package under:
+
+```text
+analysis/characterization/current_results/
+```
+
+It summarizes already completed runs and explicitly marks which claims are
+supported, partially supported, or not yet supported.
+
+## Batch 2 Protocol: PD-Colo Prefill/Decode Interference
+
+Purpose:
+
+Prove whether same-worker prefill overlap increases decode TPOT/queue delay.
+
+Required new instrumentation:
+
+- per-request dispatch timestamp
+- per-request finish/error timestamp
+- per decode step timestamp
+- decode step worker id
+- prefill chunk start/end timestamp
+- prefill worker id
+- request/session id associated with each prefill chunk
+
+Required arms:
+
+1. decode-only steady load
+2. decode + same-worker heavy prefill injection
+3. decode + different-worker heavy prefill injection
+4. trace replay with overlap labels
+
+Required sweep:
+
+```text
+uncached_prefill_tokens in {2k, 8k, 16k, 32k, 64k}
+chunked_prefill_size in available engine values
+```
+
+Required outputs:
+
+- `interference_microbench_summary.json`
+- `decode_step_timeseries.csv`
+- `prefill_overlap_events.jsonl`
+- `interference_index.json`
+- TPOT timeline figure with prefill overlays
+- same-worker vs different-worker TPOT boxplot
+
+Pass condition:
+
+```text
+TPOT_p90(overlap_same_worker) / TPOT_p90(no_overlap) > 1
+```
+
+and the effect must be materially weaker in the different-worker control.
+
+## Batch 3 Protocol: Session Hot-Spot Residual Imbalance
+
+Purpose:
+
+Prove whether cache-aware/LMetric still leaves hot workers under
+session-heavy skew.
+
+Required new instrumentation:
+
+- route decision per request
+- chosen worker
+- candidate worker scores
+- cache hit / estimated uncached tokens per candidate
+- per-worker request queue length/delay
+- per-worker decode queue length/delay
+- per-worker KV occupancy
+- per-worker APC/cache-hit snapshot
+
+Required arms:
+
+1. corrected LMetric/cache-aware
+2. load-only routing
+3. hard sticky routing
+4. current Unified hybrid
+5. session-mass capped/equalized replay
+
+Required outputs:
+
+- `worker_balance_summary.json`
+- `session_to_worker_map.json`
+- `session_mass_summary.json`
+- `routing_policy_comparison.json`
+- `hotspot_index.json`
+- per-worker queue delay bar
+- APC vs queue delay scatter
+- top-session contribution bar
+- policy tradeoff plot: APC vs hot-spot index
+
+Pass condition:
+
+LMetric/cache-aware must show measurable residual worker skew, and that skew
+must correlate with session token mass or locality.
+
+GPU utilization alone is not enough for this claim.
+
+## Batch 4 Protocol: Sustainable Request Rate
+
+Purpose:
+
+Measure:
+
+```text
+SRR(SLO) = max arrival rate satisfying SLO in steady state
+```
+
+Required load generator behavior:
+
+- open-loop session arrivals, preferably Poisson
+- session-internal sequentiality
+- warmup window
+- steady-state measurement window
+- explicit attempted/completed/error counters
+
+Provisional SLO:
+
+```text
+TTFT_p90 <= T_ttft
+E2E_p90  <= T_e2e
+TPOT_p90 <= T_tpot
+error_rate <= epsilon
+queue length stable
+KV occupancy stable
+```
+
+Required arms:
+
+1. PD-colo corrected LMetric/cache-aware
+2. static PD-disagg
+3. current Unified hybrid
+4. optional hard sticky
+5. optional load-only
+
+Required outputs:
+
+- `srr_curve.json`
+- `lambda_runs/<lambda>/summary.json`
+- `slo_violation_reason.json`
+- `goodput_vs_arrival_rate.json`
+- SRR bar chart
+- latency vs arrival rate curves
+- goodput vs arrival rate
+- queue/KV stability plot near failure point
+
+Pass condition:
+
+Each policy has a measured max sustainable lambda under the same SLO and
+same session-causal arrival process.
+
+## Batch 5 Protocol: Failure Attribution Near SRR Boundary
+
+Purpose:
+
+Explain why each policy fails near SRR.
+
+Required rates:
+
+```text
+lambda = 0.9 * SRR
+lambda = 1.0 * SRR
+lambda = 1.1 * SRR
+```
+
+Labels for each slow/SLO-violating request:
+
+- same-worker prefill overlap
+- hot worker queue
+- high KV occupancy
+- cache miss / large uncached append
+- transfer wait
+- P queue wait
+- D admission wait
+- unknown
+
+Required outputs:
+
+- `slow_request_attribution.jsonl`
+- `failure_breakdown.json`
+- `case_studies.md`
+- `worker_failure_windows.json`
+- violation cause stacked bar
+- slow request waterfall
+- worker timeline near failure
+
+Pass condition:
+
+The analysis must explain whether PD-colo is limited by interference,
+hot-spot, KV pressure, or a mixture, and whether Unified/PUSH underperforms
+because of trigger quality, transfer cost, target admission, or load regime.
+
+## Batch 6 Protocol: Audit Package
+
+Implemented by `summarize_runs.py` for existing runs and extended by fresh
+Batch 2-5 outputs later.
+
+Required files:
+
+- `characterization_claim_matrix.md`
+- `all_figures_index.md`
+- `reviewer_risk_register.md`
+- `reproduction_commands.sh`
+- `main_claim_allowed_runs.md`
+
+Current package intentionally marks Batch 2/4/5 claims as not yet supported
+until fresh instrumented experiments exist.

analysis/characterization/render_window1_figures.py

@@ -0,0 +1,416 @@
+"""Render PNG figures for Window 1 results (B1', B2, B3).
+
+Inputs (all expected under <results-dir>):
+- b3_policy_comparison.json        (per-policy table)
+- b2_sweep_summary.json            (per-cell B2 sweep)
+- apc_upper_w600.json              (theoretical bounds)
+- lmetric_reuse.json               (intra/cross/shared decomp)
+- kv_footprint_summary.json        (full trace KV stats)
+
+Outputs (under <out-dir>):
+- fig_b3_apc_vs_hotspot.png
+- fig_b3_latency_bars.png
+- fig_b3_apc_vs_upper.png
+- fig_b3_failure_breakdown.png
+- fig_b3_per_worker_ttft_p90.png
+- fig_b2_tpot_vs_prefill.png
+- fig_b2_ttft_vs_prefill.png
+- fig_reuse_decomposition.png
+- fig_kv_footprint_cdf.png
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+from pathlib import Path
+
+import matplotlib
+matplotlib.use("Agg")
+import matplotlib.pyplot as plt
+
+POLICY_ORDER = ["lmetric", "load_only", "sticky", "unified", "capped"]
+POLICY_COLOR = {
+    "lmetric":   "#1f77b4",
+    "load_only": "#ff7f0e",
+    "sticky":    "#d62728",
+    "unified":   "#2ca02c",
+    "capped":    "#9467bd",
+}
+
+
+def _load(results_dir: Path, name: str) -> dict:
+    return json.loads((results_dir / name).read_text())
+
+
+def fig_b3_apc_vs_hotspot(comp: dict, upper: dict, out: Path) -> None:
+    upper_intra = upper["apc_upper_intra_session"]
+    fig, ax = plt.subplots(figsize=(6, 4.5))
+    for r in comp["rows"]:
+        pol = r["policy"]
+        if pol not in POLICY_ORDER:
+            continue
+        ax.scatter(r["apc_ratio"] * 100, r["hotspot_index_ttft_p90"],
+                    s=180, color=POLICY_COLOR.get(pol, "gray"), label=pol,
+                    edgecolors="black", linewidths=0.5)
+        ax.annotate(pol, (r["apc_ratio"] * 100, r["hotspot_index_ttft_p90"]),
+                     xytext=(7, 7), textcoords="offset points",
+                     fontsize=9)
+    ax.axvline(upper_intra * 100, linestyle="--", color="gray", alpha=0.6,
+                label=f"intra-session APC upper {upper_intra * 100:.1f}%")
+    ax.set_xlabel("APC achieved (%)")
+    ax.set_ylabel("hotspot_index = max(worker TTFT p90) / median")
+    ax.set_title("B3: APC vs hot-spot tradeoff across policies")
+    ax.grid(alpha=0.3)
+    fig.tight_layout()
+    fig.savefig(out, dpi=120)
+    plt.close(fig)
+
+
+def fig_b3_latency_bars(comp: dict, out: Path) -> None:
+    by = {r["policy"]: r for r in comp["rows"]}
+    pols = [p for p in POLICY_ORDER if p in by]
+    metrics = [("TTFT p90 (s)", "ttft_p90_s"),
+                ("TPOT p90 (ms)", "tpot_p90_s"),
+                ("E2E p90 (s)", "e2e_p90_s")]
+    fig, axes = plt.subplots(1, 3, figsize=(12, 4))
+    for ax, (label, key) in zip(axes, metrics):
+        vals = [by[p][key] * (1000 if "TPOT" in label else 1) for p in pols]
+        ax.bar(pols, vals, color=[POLICY_COLOR.get(p, "gray") for p in pols],
+                edgecolor="black", linewidth=0.5)
+        ax.set_title(label)
+        ax.tick_params(axis="x", rotation=20)
+        for i, v in enumerate(vals):
+            ax.text(i, v, f"{v:.1f}", ha="center", va="bottom", fontsize=9)
+        ax.grid(alpha=0.3, axis="y")
+    fig.suptitle("B3 headline latencies per policy")
+    fig.tight_layout()
+    fig.savefig(out, dpi=120)
+    plt.close(fig)
+
+
+def fig_b3_latency_full_grid(results_dir: Path, out: Path) -> None:
+    """4 rows (mean / p50 / p90 / p99) × 3 cols (TTFT / TPOT / E2E) per policy.
+
+    Reads per-policy metrics.summary.json caches under raw_stats/, which
+    expose mean alongside the percentiles (b3_policy_comparison.json does
+    not record mean).
+    """
+    raw_dir = results_dir / "raw_stats"
+    pols = [p for p in POLICY_ORDER if (raw_dir / f"{p}.json").exists()]
+    if not pols:
+        return
+    stats = {p: json.loads((raw_dir / f"{p}.json").read_text()) for p in pols}
+
+    rows = [("mean", "mean"), ("p50", "p50"), ("p90", "p90"), ("p99", "p99")]
+    cols = [
+        ("TTFT (s)", "ttft", 1.0),
+        ("TPOT (ms)", "tpot", 1000.0),
+        ("E2E (s)", "e2e", 1.0),
+    ]
+    fig, axes = plt.subplots(len(rows), len(cols), figsize=(11, 11), sharex=True)
+    for i, (row_label, agg_key) in enumerate(rows):
+        for j, (col_label, metric_key, scale) in enumerate(cols):
+            ax = axes[i][j]
+            vals = [stats[p][metric_key][agg_key] * scale for p in pols]
+            ax.bar(pols, vals,
+                   color=[POLICY_COLOR.get(p, "gray") for p in pols],
+                   edgecolor="black", linewidth=0.5)
+            for k, v in enumerate(vals):
+                ax.text(k, v, f"{v:.1f}", ha="center", va="bottom", fontsize=8)
+            if j == 0:
+                ax.set_ylabel(row_label, fontsize=11)
+            if i == 0:
+                ax.set_title(col_label, fontsize=11)
+            ax.grid(alpha=0.3, axis="y")
+            ax.tick_params(axis="x", rotation=20, labelsize=9)
+            ax.margins(y=0.18)
+    fig.suptitle("B3 latencies per policy — mean / p50 / p90 / p99")
+    fig.tight_layout()
+    fig.savefig(out, dpi=120)
+    plt.close(fig)
+
+
+def fig_b3_apc_vs_upper(comp: dict, upper: dict, out: Path) -> None:
+    by = {r["policy"]: r for r in comp["rows"]}
+    pols = [p for p in POLICY_ORDER if p in by]
+    achieved = [by[p]["apc_ratio"] * 100 for p in pols]
+    fig, ax = plt.subplots(figsize=(6.5, 4))
+    bars = ax.bar(pols, achieved,
+                    color=[POLICY_COLOR.get(p, "gray") for p in pols],
+                    edgecolor="black", linewidth=0.5)
+    ax.axhline(upper["apc_upper_intra_session"] * 100, linestyle="--",
+                color="black", alpha=0.7,
+                label=f"intra-session ceiling {upper['apc_upper_intra_session'] * 100:.1f}%")
+    ax.axhline(upper["apc_upper_any_session"] * 100, linestyle=":",
+                color="darkgray", alpha=0.7,
+                label=f"any-session ceiling {upper['apc_upper_any_session'] * 100:.1f}%")
+    for b, v in zip(bars, achieved):
+        ax.text(b.get_x() + b.get_width() / 2, v + 1, f"{v:.1f}%",
+                 ha="center", fontsize=9)
+    ax.set_ylim(0, 100)
+    ax.set_ylabel("APC ratio (%)")
+    ax.set_title("B3: APC achieved vs theoretical ceiling")
+    ax.legend(loc="upper right", fontsize=9)
+    ax.grid(alpha=0.3, axis="y")
+    fig.tight_layout()
+    fig.savefig(out, dpi=120)
+    plt.close(fig)
+
+
+def fig_b3_failure_breakdown(comp: dict, out: Path) -> None:
+    by = {r["policy"]: r for r in comp["rows"]}
+    pols = [p for p in POLICY_ORDER if p in by]
+    causes = ["same_worker_prefill_overlap", "hot_worker_queue",
+               "cache_miss_large_append", "high_kv_occupancy", "unknown"]
+    cause_color = {
+        "same_worker_prefill_overlap": "#d62728",
+        "hot_worker_queue":            "#ff7f0e",
+        "cache_miss_large_append":     "#1f77b4",
+        "high_kv_occupancy":           "#8c564b",
+        "unknown":                     "#7f7f7f",
+    }
+    fig, ax = plt.subplots(figsize=(7, 4.5))
+    bottom = [0.0] * len(pols)
+    for c in causes:
+        vals = [(by[p].get("failure_counts") or {}).get(c, 0) for p in pols]
+        ax.bar(pols, vals, bottom=bottom, label=c.replace("_", " "),
+                color=cause_color[c], edgecolor="black", linewidth=0.3)
+        bottom = [a + b for a, b in zip(bottom, vals)]
+    for i, total in enumerate(bottom):
+        ax.text(i, total + 3, f"n={int(total)}", ha="center", fontsize=9)
+    ax.set_ylabel("slow request count (TTFT > 2× p90 threshold)")
+    ax.set_title("B3: slow-request cause breakdown per policy")
+    ax.legend(fontsize=8, loc="upper right")
+    ax.grid(alpha=0.3, axis="y")
+    fig.tight_layout()
+    fig.savefig(out, dpi=120)
+    plt.close(fig)
+
+
+def fig_b3_per_worker_ttft(results_dir: Path, comp: dict, out: Path) -> None:
+    """Per-worker TTFT p90 grouped bars; title shows median + max worker p90.
+
+    We deliberately do NOT report a max/median 'hotspot index' here: it is a
+    ratio and treats unified (most workers fast, one hot) as worse than
+    sticky (all workers slow), which inverts the actual user-facing p90.
+    """
+    import statistics
+    by = {r["policy"]: r for r in comp["rows"]}
+    pols = [p for p in POLICY_ORDER if p in by]
+    fig, axes = plt.subplots(1, len(pols), figsize=(3 * len(pols), 4),
+                              sharey=True)
+    if len(pols) == 1:
+        axes = [axes]
+    for ax, pol in zip(axes, pols):
+        path = results_dir / f"per_worker_{pol}.json"
+        if not path.exists():
+            ax.text(0.5, 0.5, f"{pol}: no data", ha="center", va="center",
+                     transform=ax.transAxes)
+            continue
+        per = json.loads(path.read_text()).get("per_worker_ttft_p90_s") or {}
+        items = sorted(per.items(), key=lambda kv: int(kv[0].rsplit(":", 1)[1]))
+        labels = [f"e{int(k.rsplit(':', 1)[1]) - 8000}" for k, _ in items]
+        vals = [v for _, v in items]
+        ax.bar(labels, vals, color=POLICY_COLOR.get(pol, "gray"),
+                edgecolor="black", linewidth=0.5)
+        for i, v in enumerate(vals):
+            ax.text(i, v, f"{v:.1f}", ha="center", va="bottom", fontsize=8)
+        median_v = statistics.median(vals)
+        max_v = max(vals)
+        ax.set_title(
+            f"{pol}\nmedian {median_v:.1f}s · max {max_v:.1f}s",
+            fontsize=10,
+        )
+        ax.tick_params(axis="x", labelsize=8)
+        ax.grid(alpha=0.3, axis="y")
+    axes[0].set_ylabel("worker TTFT p90 (s)")
+    fig.suptitle("B3 per-worker TTFT p90 distribution")
+    fig.tight_layout()
+    fig.savefig(out, dpi=120)
+    plt.close(fig)
+
+
+def fig_b2_curves(b2: dict, out_tpot: Path, out_ttft: Path) -> None:
+    sizes = sorted({r["prefill_size"] for r in b2["rows"]})
+    by_var = {"same": {}, "different": {}}
+    for r in b2["rows"]:
+        by_var[r["variant"]][r["prefill_size"]] = r
+
+    for name, key, ylabel, ymax_log, out in [
+        ("TPOT", "tpot_p90", "TPOT p90 (ms)", True, out_tpot),
+        ("TTFT", "ttft_p90", "TTFT p90 (s)", True, out_ttft),
+    ]:
+        fig, axes = plt.subplots(1, 2, figsize=(11, 4))
+        ax_abs, ax_idx = axes
+        for variant in ("different", "same"):
+            xs, ys_o, ys_c, idxs = [], [], [], []
+            for sz in sizes:
+                r = by_var[variant].get(sz)
+                if not r: continue
+                ov = r.get(f"{key}_overlap_s")
+                cl = r.get(f"{key}_clean_s")
+                if ov is None or cl is None: continue
+                xs.append(sz)
+                scale = 1000 if name == "TPOT" else 1.0
+                ys_o.append(ov * scale)
+                ys_c.append(cl * scale)
+                idxs.append(ov / cl)
+            color = "#d62728" if variant == "same" else "#1f77b4"
+            ax_abs.plot(xs, ys_o, "o-", color=color,
+                         label=f"{variant} (overlap)")
+            ax_abs.plot(xs, ys_c, "s--", color=color, alpha=0.5,
+                         label=f"{variant} (clean)")
+            ax_idx.plot(xs, idxs, "o-", color=color, label=variant,
+                         linewidth=2)
+        ax_abs.set_xscale("log", base=2)
+        ax_abs.set_yscale("log")
+        ax_abs.set_xlabel("prefill injection size (tokens)")
+        ax_abs.set_ylabel(ylabel + " (log)")
+        ax_abs.set_title(f"B2 {name} absolute (overlap vs clean)")
+        ax_abs.legend(fontsize=8)
+        ax_abs.grid(alpha=0.3, which="both")
+
+        ax_idx.set_xscale("log", base=2)
+        if ymax_log:
+            ax_idx.set_yscale("log")
+        ax_idx.axhline(1.0, color="black", linestyle=":", alpha=0.5)
+        ax_idx.set_xlabel("prefill injection size (tokens)")
+        ax_idx.set_ylabel(f"{name} idx = overlap / clean")
+        ax_idx.set_title(f"B2 {name} interference index (same vs different worker)")
+        ax_idx.legend()
+        ax_idx.grid(alpha=0.3, which="both")
+        fig.tight_layout()
+        fig.savefig(out, dpi=120)
+        plt.close(fig)
+
+
+def fig_reuse_decomposition(reuse: dict, out: Path) -> None:
+    fr = reuse.get("fractions") or {}
+    labels = ["intra-session", "cross-session", "shared-prefix", "unclassified"]
+    vals = [fr.get("intra", 0), fr.get("cross", 0),
+             fr.get("shared", 0), fr.get("unclassified", 0)]
+    colors = ["#2ca02c", "#ff7f0e", "#9467bd", "#7f7f7f"]
+    fig, ax = plt.subplots(figsize=(6, 3))
+    bottom = 0.0
+    for label, v, c in zip(labels, vals, colors):
+        ax.barh(["lmetric run"], [v], left=[bottom], color=c, edgecolor="black",
+                 linewidth=0.5, label=f"{label} ({v * 100:.1f}%)")
+        bottom += v
+    ax.set_xlabel("fraction of cached_tokens")
+    ax.set_xlim(0, 1)
+    ax.set_title("Real reuse decomposition (w600 lmetric run)")
+    ax.legend(fontsize=9, loc="lower right")
+    ax.grid(alpha=0.3, axis="x")
+    fig.tight_layout()
+    fig.savefig(out, dpi=120)
+    plt.close(fig)
+
+
+def fig_kv_footprint_cdf(kv: dict, out: Path) -> None:
+    """How many concurrent decodes fit per percentile, under three deployments.
+
+    KV pool assumption: 96 GiB H20 HBM split ~50% model params (Qwen3-Coder-
+    30B-A3B in bf16 + headroom), ~10% runtime activations, leaving ~40% for
+    the KV cache pool — i.e. ~38.4 GiB per instance.
+
+    For each request-size percentile, we report system-wide concurrent
+    decode capacity = N_D × floor(KV_pool / req_size_MiB) under three 8-GPU
+    deployments: all-combined, 4P+4D, 6P+2D. The point is that going from
+    combined 8C to 4P+4D halves the system's decode population at the
+    same per-request KV pressure.
+    """
+    s = kv.get("kv_mib_per_request") or {}
+    pct_keys = ["p50", "p90", "p95", "p99"]
+    req_mib = [float(s.get(k, 0.0)) for k in pct_keys]
+    req_gib = [v / 1024 for v in req_mib]
+
+    hbm_gib = 96.0
+    kv_pool_frac = 0.40
+    kv_pool_mib = hbm_gib * kv_pool_frac * 1024  # ≈ 39322 MiB per instance
+
+    deploys = [
+        ("Combined 8C",     8, "#2ca02c"),
+        ("PD-disagg 4P+4D", 4, "#ff7f0e"),
+        ("PD-disagg 6P+2D", 2, "#d62728"),
+    ]
+
+    import numpy as _np
+    x = _np.arange(len(pct_keys))
+    bar_w = 0.26
+
+    fig, ax = plt.subplots(figsize=(9, 5.2))
+    for i, (label, n_d, color) in enumerate(deploys):
+        per_inst = [int(kv_pool_mib // r) if r > 0 else 0 for r in req_mib]
+        sys_cap = [n_d * pi for pi in per_inst]
+        bars = ax.bar(x + (i - 1) * bar_w, sys_cap, bar_w,
+                      label=f"{label} (N_D={n_d})",
+                      color=color, edgecolor="black", linewidth=0.5)
+        for j, (b, n) in enumerate(zip(bars, sys_cap)):
+            ax.text(b.get_x() + b.get_width() / 2, n, str(n),
+                    ha="center", va="bottom", fontsize=9, color="#333")
+
+    # Annotate per-request KV size and per-instance fit just above the x-axis
+    per_inst_combined = [int(kv_pool_mib // r) if r > 0 else 0 for r in req_mib]
+    annot = [
+        f"{pct}\n{rg:.1f} GiB / req\nfits {pi}/inst"
+        for pct, rg, pi in zip(pct_keys, req_gib, per_inst_combined)
+    ]
+    ax.set_xticks(x)
+    ax.set_xticklabels(annot, fontsize=10)
+
+    ax.set_ylabel("System-wide concurrent decodes")
+    ax.set_title(
+        f"Per-instance KV pool ≈ {kv_pool_mib / 1024:.1f} GiB "
+        f"(0.4 × H20 96 GiB; remaining 0.5 model + 0.1 activation)\n"
+        f"PD-disagg halves the decode population at p90+ "
+        f"(Qwen3-Coder-30B-A3B, 98304 B/token)"
+    )
+    ax.legend(loc="upper right")
+    ax.grid(alpha=0.3, axis="y")
+    ax.margins(y=0.15)
+    fig.tight_layout()
+    fig.savefig(out, dpi=120)
+    plt.close(fig)
+
+
+def main() -> None:
+    p = argparse.ArgumentParser()
+    p.add_argument("--results-dir", type=Path, required=True)
+    p.add_argument("--out-dir", type=Path, required=True)
+    p.add_argument("--exclude-policies", default="",
+                   help="Comma-separated policies to drop from per-policy figures")
+    args = p.parse_args()
+    args.out_dir.mkdir(parents=True, exist_ok=True)
+
+    excluded = {s.strip() for s in args.exclude_policies.split(",") if s.strip()}
+    if excluded:
+        global POLICY_ORDER
+        POLICY_ORDER = [p for p in POLICY_ORDER if p not in excluded]
+        print(f"excluding policies: {sorted(excluded)} -> kept {POLICY_ORDER}")
+
+    comp = _load(args.results_dir, "b3_policy_comparison.json")
+    upper = _load(args.results_dir, "apc_upper_w600.json")
+    b2 = _load(args.results_dir, "b2_sweep_summary.json")
+    reuse = _load(args.results_dir, "lmetric_reuse.json")
+    kv = _load(args.results_dir, "kv_footprint_summary.json")
+
+    fig_b3_apc_vs_hotspot(comp, upper, args.out_dir / "fig_b3_apc_vs_hotspot.png")
+    fig_b3_latency_bars(comp, args.out_dir / "fig_b3_latency_bars.png")
+    fig_b3_latency_full_grid(
+        args.results_dir, args.out_dir / "fig_b3_latency_full_grid.png"
+    )
+    fig_b3_apc_vs_upper(comp, upper, args.out_dir / "fig_b3_apc_vs_upper.png")
+    fig_b3_failure_breakdown(comp, args.out_dir / "fig_b3_failure_breakdown.png")
+    fig_b3_per_worker_ttft(args.results_dir, comp,
+                            args.out_dir / "fig_b3_per_worker_ttft_p90.png")
+    fig_b2_curves(b2,
+                   args.out_dir / "fig_b2_tpot_vs_prefill.png",
+                   args.out_dir / "fig_b2_ttft_vs_prefill.png")
+    fig_reuse_decomposition(reuse, args.out_dir / "fig_reuse_decomposition.png")
+    fig_kv_footprint_cdf(kv, args.out_dir / "fig_kv_footprint_cdf.png")
+    print(f"wrote 8 figures to {args.out_dir}")
+
+
+if __name__ == "__main__":
+    main()

analysis/characterization/summarize_runs.py

@@ -0,0 +1,666 @@
+#!/usr/bin/env python3
+"""Summarize existing benchmark artifacts for characterization review.
+
+This is a CPU-only companion to ``analyze.py``. It does not run benchmarks.
+It reads completed output directories and produces an audit-oriented package
+that helps decide which TODO claims are currently supported by existing data
+and which still need fresh GPU runs or additional instrumentation.
+"""
+
+from __future__ import annotations
+
+import argparse
+import csv
+import datetime as dt
+import json
+import math
+import statistics
+import subprocess
+from pathlib import Path
+from typing import Any
+
+
+JsonDict = dict[str, Any]
+
+
+DEFAULT_RUNS = [
+    "outputs/gpu_ab_combined",
+    "outputs/gpu_ab_pdsep",
+    "outputs/contention_16s_ts10",
+    "outputs/contention_16s_elastic",
+    "outputs/combined_1000req",
+    "outputs/exp3_pd_sep_tp1_mooncake",
+]
+
+
+def main() -> None:
+    args = parse_args()
+    out_dir = args.output_dir
+    out_dir.mkdir(parents=True, exist_ok=True)
+
+    run_dirs = [Path(p) for p in (args.runs or DEFAULT_RUNS)]
+    summaries = [summarize_run(path) for path in run_dirs]
+    comparisons = build_comparisons(summaries)
+    claim_matrix = build_claim_matrix(summaries, comparisons)
+    risk_register = build_risk_register(summaries)
+
+    write_json(out_dir / "run_summaries.json", summaries)
+    write_json(out_dir / "comparisons.json", comparisons)
+    write_json(out_dir / "claim_matrix.json", claim_matrix)
+    write_json(out_dir / "reviewer_risk_register.json", risk_register)
+    (out_dir / "current_results.md").write_text(
+        render_current_results(summaries, comparisons, claim_matrix, risk_register),
+        encoding="utf-8",
+    )
+    (out_dir / "characterization_claim_matrix.md").write_text(
+        render_claim_matrix(claim_matrix),
+        encoding="utf-8",
+    )
+    (out_dir / "reviewer_risk_register.md").write_text(
+        render_risk_register(risk_register),
+        encoding="utf-8",
+    )
+    (out_dir / "all_figures_index.md").write_text(
+        render_figures_index(summaries),
+        encoding="utf-8",
+    )
+    (out_dir / "reproduction_commands.sh").write_text(
+        render_reproduction_commands(args, run_dirs),
+        encoding="utf-8",
+    )
+    print(f"Wrote run summary package to {out_dir}")
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser(
+        description="Summarize existing characterization-relevant output dirs.",
+        formatter_class=argparse.ArgumentDefaultsHelpFormatter,
+    )
+    parser.add_argument(
+        "--runs",
+        nargs="*",
+        default=None,
+        help="Output directories to summarize. Defaults to a small curated set.",
+    )
+    parser.add_argument(
+        "--output-dir",
+        type=Path,
+        default=Path("analysis/characterization/current_results"),
+        help="Directory for generated review artifacts.",
+    )
+    return parser.parse_args()
+
+
+def summarize_run(path: Path) -> JsonDict:
+    metrics_summary = load_json(path / "metrics.summary.json")
+    metrics_rows = load_jsonl(path / "metrics.jsonl")
+    gpu_summary = summarize_gpu(path / "gpu_util.csv")
+    breakdown_summary = summarize_breakdown(path / "breakdown.json")
+    apc_summary = summarize_apc(path / "apc.txt")
+    return {
+        "run": str(path),
+        "exists": path.exists(),
+        "metrics_summary_available": bool(metrics_summary),
+        "metrics_jsonl_rows": len(metrics_rows),
+        "request_count": first_present(metrics_summary, ["request_count"]),
+        "success_count": first_present(metrics_summary, ["success_count"]),
+        "error_count": first_present(metrics_summary, ["error_count"]),
+        "wall_clock_s": first_present(metrics_summary, ["wall_clock_s"]),
+        "latency_stats_s": metrics_summary.get("latency_stats_s"),
+        "ttft_stats_s": metrics_summary.get("ttft_stats_s"),
+        "tpot_stats_s": metrics_summary.get("tpot_stats_s"),
+        "prefix_cache_hit_ratio": metrics_summary.get("prefix_cache_hit_ratio"),
+        "external_cache_hit_ratio": metrics_summary.get("external_cache_hit_ratio"),
+        "session_summary": summarize_sessions(metrics_rows),
+        "gpu_summary": gpu_summary,
+        "breakdown_summary": breakdown_summary,
+        "apc_summary": apc_summary,
+        "artifact_availability": {
+            "metrics_summary_json": (path / "metrics.summary.json").exists(),
+            "metrics_jsonl": (path / "metrics.jsonl").exists(),
+            "gpu_util_csv": (path / "gpu_util.csv").exists(),
+            "breakdown_json": (path / "breakdown.json").exists(),
+            "apc_txt": (path / "apc.txt").exists(),
+        },
+    }
+
+
+def summarize_sessions(rows: list[JsonDict]) -> JsonDict:
+    if not rows:
+        return {
+            "status": "unavailable",
+            "reason": "metrics.jsonl missing",
+        }
+    sessions: dict[str, JsonDict] = {}
+    input_values = []
+    output_values = []
+    cached_values = []
+    for row in rows:
+        sid = str(row.get("session_id", ""))
+        item = sessions.setdefault(
+            sid,
+            {
+                "turns": 0,
+                "input_tokens": 0.0,
+                "output_tokens": 0.0,
+                "cached_tokens": 0.0,
+            },
+        )
+        inp = to_float(row.get("input_length")) or 0.0
+        out = to_float(row.get("actual_output_tokens")) or to_float(row.get("output_length")) or 0.0
+        cached = to_float(row.get("cached_tokens")) or 0.0
+        item["turns"] += 1
+        item["input_tokens"] += inp
+        item["output_tokens"] += out
+        item["cached_tokens"] += cached
+        input_values.append(inp)
+        output_values.append(out)
+        cached_values.append(cached)
+    per_session_input = [v["input_tokens"] for v in sessions.values()]
+    return {
+        "status": "available",
+        "request_input_tokens": stats(input_values),
+        "request_output_tokens": stats(output_values),
+        "request_cached_tokens": stats(cached_values),
+        "session_count": len(sessions),
+        "turns_per_session": stats([v["turns"] for v in sessions.values()]),
+        "session_input_tokens": stats(per_session_input),
+        "top_session_input_fraction": top_contribution(per_session_input),
+    }
+
+
+def summarize_gpu(path: Path) -> JsonDict:
+    if not path.exists():
+        return {
+            "status": "unavailable",
+            "reason": "gpu_util.csv missing",
+        }
+    values: dict[str, list[float]] = {}
+    with path.open() as handle:
+        reader = csv.DictReader(handle)
+        for row in reader:
+            gpu = str(row.get("gpu", ""))
+            util = to_float(row.get("util_pct"))
+            if gpu and util is not None:
+                values.setdefault(gpu, []).append(util)
+    means = {gpu: statistics.fmean(vals) for gpu, vals in values.items() if vals}
+    if not means:
+        return {
+            "status": "unavailable",
+            "reason": "gpu_util.csv had no util_pct rows",
+        }
+    mean_values = list(means.values())
+    return {
+        "status": "available",
+        "gpu_count": len(means),
+        "per_gpu_mean_util_pct": means,
+        "mean_util_pct": statistics.fmean(mean_values),
+        "stddev_across_gpu_mean_util_pct": statistics.pstdev(mean_values),
+        "max_mean_util_pct": max(mean_values),
+        "min_mean_util_pct": min(mean_values),
+        "max_min_ratio": max(mean_values) / max(min(mean_values), 1e-9),
+    }
+
+
+def summarize_breakdown(path: Path) -> JsonDict:
+    if not path.exists():
+        return {
+            "status": "unavailable",
+            "reason": "breakdown.json missing",
+        }
+    try:
+        data = json.loads(path.read_text(encoding="utf-8"))
+    except Exception as exc:
+        return {
+            "status": "unavailable",
+            "reason": f"failed to parse breakdown: {exc}",
+        }
+    rows: list[JsonDict]
+    if isinstance(data, list):
+        rows = [r for r in data if isinstance(r, dict)]
+    elif isinstance(data, dict):
+        rows = data.get("records") if isinstance(data.get("records"), list) else []
+        if not rows:
+            rows = [data]
+    else:
+        rows = []
+    mode_counts: dict[str, int] = {}
+    route_counts: dict[str, int] = {}
+    for row in rows:
+        mode = row.get("mode") or row.get("execution_mode") or row.get("route_class")
+        route = row.get("route") or row.get("decision") or row.get("policy")
+        if mode is not None:
+            mode_counts[str(mode)] = mode_counts.get(str(mode), 0) + 1
+        if route is not None:
+            route_counts[str(route)] = route_counts.get(str(route), 0) + 1
+    return {
+        "status": "available",
+        "row_count": len(rows),
+        "mode_counts": mode_counts,
+        "route_counts": route_counts,
+        "field_sample": sorted(rows[0].keys()) if rows else [],
+    }
+
+
+def summarize_apc(path: Path) -> JsonDict:
+    if not path.exists():
+        return {
+            "status": "unavailable",
+            "reason": "apc.txt missing",
+        }
+    text = path.read_text(encoding="utf-8", errors="replace")
+    return {
+        "status": "available",
+        "line_count": len(text.splitlines()),
+        "preview": "\n".join(text.splitlines()[:20]),
+    }
+
+
+def build_comparisons(summaries: list[JsonDict]) -> list[JsonDict]:
+    by_run = {s["run"]: s for s in summaries}
+    pairs = [
+        ("combined_vs_pdsep_200", "outputs/gpu_ab_combined", "outputs/gpu_ab_pdsep"),
+        ("contention_baseline_vs_elastic_500", "outputs/contention_16s_ts10", "outputs/contention_16s_elastic"),
+        ("combined_1000_vs_pdsep_mooncake", "outputs/combined_1000req", "outputs/exp3_pd_sep_tp1_mooncake"),
+    ]
+    out = []
+    for name, base, variant in pairs:
+        if base not in by_run or variant not in by_run:
+            continue
+        out.append(compare_pair(name, by_run[base], by_run[variant]))
+    return out
+
+
+def compare_pair(name: str, base: JsonDict, variant: JsonDict) -> JsonDict:
+    return {
+        "name": name,
+        "baseline": base["run"],
+        "variant": variant["run"],
+        "request_count": [base.get("request_count"), variant.get("request_count")],
+        "success_count": [base.get("success_count"), variant.get("success_count")],
+        "error_count": [base.get("error_count"), variant.get("error_count")],
+        "ttft_p50_delta_pct": pct_delta(stat_value(base, "ttft_stats_s", "p50"), stat_value(variant, "ttft_stats_s", "p50")),
+        "ttft_p90_delta_pct": pct_delta(stat_value(base, "ttft_stats_s", "p90"), stat_value(variant, "ttft_stats_s", "p90")),
+        "e2e_p50_delta_pct": pct_delta(stat_value(base, "latency_stats_s", "p50"), stat_value(variant, "latency_stats_s", "p50")),
+        "e2e_p90_delta_pct": pct_delta(stat_value(base, "latency_stats_s", "p90"), stat_value(variant, "latency_stats_s", "p90")),
+        "tpot_p90_delta_pct": pct_delta(stat_value(base, "tpot_stats_s", "p90"), stat_value(variant, "tpot_stats_s", "p90")),
+        "wall_clock_delta_pct": pct_delta(base.get("wall_clock_s"), variant.get("wall_clock_s")),
+        "gpu_mean_util": [
+            nested(base, ["gpu_summary", "mean_util_pct"]),
+            nested(variant, ["gpu_summary", "mean_util_pct"]),
+        ],
+        "gpu_imbalance_ratio": [
+            nested(base, ["gpu_summary", "max_min_ratio"]),
+            nested(variant, ["gpu_summary", "max_min_ratio"]),
+        ],
+    }
+
+
+def build_claim_matrix(summaries: list[JsonDict], comparisons: list[JsonDict]) -> list[JsonDict]:
+    has_gpu = any((s.get("gpu_summary") or {}).get("status") == "available" for s in summaries)
+    has_metrics = any(s.get("metrics_summary_available") for s in summaries)
+    return [
+        {
+            "claim": "Batch 0 substrate audit is only partially complete for existing runs.",
+            "status": "partially_supported",
+            "supporting_data": "metrics.jsonl lacks actual dispatch/finish timestamps in current artifacts.",
+            "needed_next": "Add request dispatch and finish/error timestamps to future replayer/proxy metrics.",
+            "reviewer_risk": "Cannot use these runs to prove online per-session sequentiality.",
+        },
+        {
+            "claim": "Batch 1 workload shape can be characterized from formatted traces and metrics.",
+            "status": "supported_for_trace_shape",
+            "supporting_data": "analysis/characterization/analyze.py outputs workload_summary/session_skew/KV footprint when given trace and kv_bytes_per_token.",
+            "needed_next": "Run on dash0 compact formatted trace for canonical full-trace numbers.",
+            "reviewer_risk": "Actual cache reuse decomposition needs cached_tokens joined with hash_ids.",
+        },
+        {
+            "claim": "Static PD separation is worse than combined in existing 200-request GPU A/B.",
+            "status": "supported_by_existing_artifact" if has_metrics else "unavailable",
+            "supporting_data": "outputs/gpu_ab_combined vs outputs/gpu_ab_pdsep metrics.summary.json.",
+            "needed_next": "Refresh with PD matrix, multiple seeds, cudagraph-enabled methodology.",
+            "reviewer_risk": "Legacy run has no per-stage TTFT breakdown and no step-level KV occupancy.",
+        },
+        {
+            "claim": "Elastic transfer-based migration does not improve high-contention 500-request run.",
+            "status": "supported_by_existing_artifact" if has_metrics else "unavailable",
+            "supporting_data": "outputs/contention_16s_ts10 vs outputs/contention_16s_elastic metrics.summary.json and gpu_util.csv.",
+            "needed_next": "Attribute whether failure is trigger quality, transfer overhead, or wrong load regime.",
+            "reviewer_risk": "Existing metrics lack actual sequentiality proof and per-request transfer waterfall.",
+        },
+        {
+            "claim": "PD-colo prefill/decode interference is not yet directly proven by step-level data in this package.",
+            "status": "not_yet_supported",
+            "supporting_data": "No decode-step and prefill-overlap timestamp artifact found in summarized runs.",
+            "needed_next": "Run Batch 2 controlled same-worker/different-worker injection with step timestamps.",
+            "reviewer_risk": "Cannot claim interference as causal without Batch 2.",
+        },
+        {
+            "claim": "Session hot-spot residual imbalance is suggested but not fully attributed.",
+            "status": "partially_supported" if has_gpu else "unavailable",
+            "supporting_data": "gpu_util.csv shows per-GPU mean-util imbalance in existing runs.",
+            "needed_next": "Collect per-worker queue delay, session-to-worker map, and per-session token mass per worker.",
+            "reviewer_risk": "GPU util imbalance alone is not enough to prove session hot-spot.",
+        },
+        {
+            "claim": "SRR is not measured by existing fixed-request runs.",
+            "status": "not_yet_supported",
+            "supporting_data": "No arrival-rate sweep artifacts found.",
+            "needed_next": "Implement Batch 4 Poisson session-arrival SRR sweep.",
+            "reviewer_risk": "Latency-at-one-load cannot support sustainable throughput claim.",
+        },
+    ]
+
+
+def build_risk_register(summaries: list[JsonDict]) -> list[JsonDict]:
+    return [
+        {
+            "risk": "Session sequentiality not proven",
+            "severity": "high",
+            "evidence": "Current metrics include trace timestamp and latency but not actual dispatch/finish wall-clock timestamps.",
+            "mitigation": "Add dispatch/finish timestamps and run Batch 0 before SRR claims.",
+        },
+        {
+            "risk": "Legacy PD-sep data may not match final methodology",
+            "severity": "medium",
+            "evidence": "PD matrix scaffold exists separately; some old runs used earlier flags/methodology.",
+            "mitigation": "Use fresh PD matrix for paper-grade claims.",
+        },
+        {
+            "risk": "GPU util is not a sufficient hot-spot proof",
+            "severity": "medium",
+            "evidence": "Existing artifacts have gpu_util.csv but lack per-worker queue and session ownership.",
+            "mitigation": "Add route-decision and per-worker queue logs for Batch 3.",
+        },
+        {
+            "risk": "Cache reuse decomposition is incomplete without joined hash/cache-hit data",
+            "severity": "medium",
+            "evidence": "Trace has hash_ids; metrics have cached_tokens; request IDs may not join across all artifacts.",
+            "mitigation": "Emit hash_ids/session_id/cached_tokens in the same per-request record.",
+        },
+    ]
+
+
+def render_current_results(
+    summaries: list[JsonDict],
+    comparisons: list[JsonDict],
+    claim_matrix: list[JsonDict],
+    risk_register: list[JsonDict],
+) -> str:
+    lines = [
+        "# Current Characterization Results",
+        "",
+        f"Generated: {dt.datetime.now(dt.timezone.utc).isoformat()}",
+        f"Git commit: `{git_commit()}`",
+        "",
+        "## Existing Run Summaries",
+        "",
+        "| Run | OK/Req | TTFT p50/p90 | E2E p50/p90 | TPOT p90 | GPU mean util | GPU imbalance |",
+        "|---|---:|---:|---:|---:|---:|---:|",
+    ]
+    for s in summaries:
+        lines.append(
+            "| {run} | {ok}/{req} | {ttft50}/{ttft90} | {e2e50}/{e2e90} | {tpot90} | {gpu_mean} | {gpu_imb} |".format(
+                run=s["run"],
+                ok=fmt(s.get("success_count")),
+                req=fmt(s.get("request_count")),
+                ttft50=fmt(stat_value(s, "ttft_stats_s", "p50")),
+                ttft90=fmt(stat_value(s, "ttft_stats_s", "p90")),
+                e2e50=fmt(stat_value(s, "latency_stats_s", "p50")),
+                e2e90=fmt(stat_value(s, "latency_stats_s", "p90")),
+                tpot90=fmt(stat_value(s, "tpot_stats_s", "p90")),
+                gpu_mean=fmt(nested(s, ["gpu_summary", "mean_util_pct"])),
+                gpu_imb=fmt(nested(s, ["gpu_summary", "max_min_ratio"])),
+            )
+        )
+    lines.extend([
+        "",
+        "## Pairwise Comparisons",
+        "",
+        "| Comparison | TTFT p50 Δ | TTFT p90 Δ | E2E p50 Δ | E2E p90 Δ | TPOT p90 Δ | Wall-clock Δ |",
+        "|---|---:|---:|---:|---:|---:|---:|",
+    ])
+    for c in comparisons:
+        lines.append(
+            "| {name} | {ttft50} | {ttft90} | {e2e50} | {e2e90} | {tpot90} | {wall} |".format(
+                name=c["name"],
+                ttft50=fmt_pct(c.get("ttft_p50_delta_pct")),
+                ttft90=fmt_pct(c.get("ttft_p90_delta_pct")),
+                e2e50=fmt_pct(c.get("e2e_p50_delta_pct")),
+                e2e90=fmt_pct(c.get("e2e_p90_delta_pct")),
+                tpot90=fmt_pct(c.get("tpot_p90_delta_pct")),
+                wall=fmt_pct(c.get("wall_clock_delta_pct")),
+            )
+        )
+    lines.extend([
+        "",
+        "## What We Can Say Now",
+        "",
+    ])
+    for item in claim_matrix:
+        lines.append(f"- **{item['status']}**: {item['claim']}")
+        lines.append(f"  Supporting data: {item['supporting_data']}")
+        lines.append(f"  Next: {item['needed_next']}")
+    lines.extend([
+        "",
+        "## Main Reviewer Risks",
+        "",
+    ])
+    for item in risk_register:
+        lines.append(f"- **{item['severity']}**: {item['risk']} - {item['mitigation']}")
+    lines.append("")
+    return "\n".join(lines)
+
+
+def render_claim_matrix(items: list[JsonDict]) -> str:
+    lines = [
+        "# Characterization Claim Matrix",
+        "",
+        "| Claim | Status | Supporting Data | Needed Next | Reviewer Risk |",
+        "|---|---|---|---|---|",
+    ]
+    for item in items:
+        lines.append(
+            f"| {item['claim']} | `{item['status']}` | {item['supporting_data']} | {item['needed_next']} | {item['reviewer_risk']} |"
+        )
+    lines.append("")
+    return "\n".join(lines)
+
+
+def render_risk_register(items: list[JsonDict]) -> str:
+    lines = [
+        "# Reviewer Risk Register",
+        "",
+        "| Risk | Severity | Evidence | Mitigation |",
+        "|---|---|---|---|",
+    ]
+    for item in items:
+        lines.append(
+            f"| {item['risk']} | `{item['severity']}` | {item['evidence']} | {item['mitigation']} |"
+        )
+    lines.append("")
+    return "\n".join(lines)
+
+
+def render_figures_index(summaries: list[JsonDict]) -> str:
+    return "\n".join([
+        "# Figures Index",
+        "",
+        "No generated figures are committed by this script. Batch-specific figures should be generated from:",
+        "",
+        "- `analysis/characterization/analyze.py` for Batch 0/1 trace figures.",
+        "- future Batch 2 step-timeline artifacts for interference plots.",
+        "- future Batch 3 per-worker/session artifacts for hot-spot plots.",
+        "- future Batch 4 arrival-rate sweep artifacts for SRR curves.",
+        "",
+        "This file exists so the audit package has a stable placeholder until fresh figures are generated.",
+        "",
+    ])
+
+
+def render_reproduction_commands(args: argparse.Namespace, run_dirs: list[Path]) -> str:
+    runs = " ".join(str(p) for p in run_dirs)
+    return "\n".join([
+        "#!/usr/bin/env bash",
+        "set -euo pipefail",
+        "",
+        "# Rebuild this current-results audit package.",
+        f"python3 analysis/characterization/summarize_runs.py --output-dir {args.output_dir} --runs {runs}",
+        "",
+        "# Example Batch 0/1 local trace analysis.",
+        "python3 analysis/characterization/analyze.py \\",
+        "  --trace traces/w600_r0.0015_st30.jsonl \\",
+        "  --kv-bytes-per-token 98304 \\",
+        "  --task-name w600_local_full_trace \\",
+        "  --overwrite",
+        "",
+    ])
+
+
+def load_json(path: Path) -> JsonDict:
+    if not path.exists():
+        return {}
+    try:
+        data = json.loads(path.read_text(encoding="utf-8"))
+    except Exception:
+        return {}
+    return data if isinstance(data, dict) else {}
+
+
+def load_jsonl(path: Path) -> list[JsonDict]:
+    if not path.exists():
+        return []
+    rows = []
+    with path.open(encoding="utf-8") as handle:
+        for line in handle:
+            if not line.strip():
+                continue
+            try:
+                row = json.loads(line)
+            except Exception:
+                continue
+            if isinstance(row, dict):
+                rows.append(row)
+    return rows
+
+
+def write_json(path: Path, data: Any) -> None:
+    path.parent.mkdir(parents=True, exist_ok=True)
+    path.write_text(json.dumps(data, indent=2, sort_keys=True) + "\n", encoding="utf-8")
+
+
+def first_present(row: JsonDict, keys: list[str]) -> Any:
+    for key in keys:
+        if key in row:
+            return row[key]
+    return None
+
+
+def stat_value(run: JsonDict, stat_key: str, value_key: str) -> float | None:
+    stats_obj = run.get(stat_key)
+    if not isinstance(stats_obj, dict):
+        return None
+    return to_float(stats_obj.get(value_key))
+
+
+def nested(row: JsonDict, keys: list[str]) -> Any:
+    cur: Any = row
+    for key in keys:
+        if not isinstance(cur, dict):
+            return None
+        cur = cur.get(key)
+    return cur
+
+
+def pct_delta(base: Any, variant: Any) -> float | None:
+    b = to_float(base)
+    v = to_float(variant)
+    if b is None or v is None or b == 0:
+        return None
+    return (v - b) / b * 100.0
+
+
+def to_float(value: Any) -> float | None:
+    if value is None:
+        return None
+    try:
+        out = float(value)
+    except (TypeError, ValueError):
+        return None
+    return out if math.isfinite(out) else None
+
+
+def stats(values: list[float]) -> JsonDict | None:
+    clean = sorted(float(v) for v in values if math.isfinite(float(v)))
+    if not clean:
+        return None
+    return {
+        "count": len(clean),
+        "mean": statistics.fmean(clean),
+        "p50": percentile(clean, 0.50),
+        "p90": percentile(clean, 0.90),
+        "p95": percentile(clean, 0.95),
+        "p99": percentile(clean, 0.99),
+        "max": clean[-1],
+    }
+
+
+def percentile(values: list[float], q: float) -> float:
+    if len(values) == 1:
+        return values[0]
+    rank = q * (len(values) - 1)
+    lo = int(rank)
+    hi = min(lo + 1, len(values) - 1)
+    frac = rank - lo
+    return values[lo] * (1 - frac) + values[hi] * frac
+
+
+def top_contribution(values: list[float]) -> JsonDict:
+    clean = sorted([v for v in values if math.isfinite(v)], reverse=True)
+    total = sum(clean)
+    if not clean or total <= 0:
+        return {"top_1pct": None, "top_5pct": None, "top_10pct": None}
+
+    def frac(pct: float) -> float:
+        k = max(1, math.ceil(len(clean) * pct))
+        return sum(clean[:k]) / total
+
+    return {
+        "top_1pct": frac(0.01),
+        "top_5pct": frac(0.05),
+        "top_10pct": frac(0.10),
+    }
+
+
+def fmt(value: Any) -> str:
+    num = to_float(value)
+    if num is None:
+        return "n/a"
+    if abs(num - round(num)) < 1e-9 and abs(num) < 1_000_000:
+        return str(int(round(num)))
+    return f"{num:.3g}"
+
+
+def fmt_pct(value: Any) -> str:
+    num = to_float(value)
+    if num is None:
+        return "n/a"
+    return f"{num:+.1f}%"
+
+
+def git_commit() -> str:
+    try:
+        result = subprocess.run(
+            ["git", "rev-parse", "HEAD"],
+            check=True,
+            stdout=subprocess.PIPE,
+            stderr=subprocess.DEVNULL,
+            text=True,
+        )
+    except Exception:
+        return ""
+    return result.stdout.strip()
+
+
+if __name__ == "__main__":
+    main()

analysis/characterization/window_1_results.md

@@ -0,0 +1,259 @@
+# Window 1 Results: B1' + B2 + B3
+
+Status: Window 1 complete (CPU + 2 dash0 GPU windows on 2026-05-25)
+Sweep: `outputs/b3_sweep_20260525_095043` (B3) + `outputs/b2_microbench/` (B2)
+Trace: `traces/w600_r0.0015_st30.jsonl` (1214 requests / 274 sessions / 53.3 M input tokens)
+Model: Qwen3-Coder-30B-A3B-Instruct (TP1 × 8 instances on H20)
+
+Per-policy artifacts under `window_1_results/`. Figures under `window_1_results/figures/`.
+
+## Headline
+
+| Claim | Status | Evidence |
+|---|---|---|
+| Agentic workload reuse is overwhelmingly intra-session | **supported** | 93.2% of cached_tokens are intra-session (real); theoretical any-session APC ceiling 80.3% vs intra-session ceiling 79.6% → < 1pp gap |
+| LMetric leaves 23 pp of APC on the table | **supported** | lmetric achieved 56.9% vs intra-session ceiling 79.6% (theoretical) |
+| Hard session affinity recovers the locality lost by LMetric | **supported** | sticky APC 77.2% = 97% of theoretical ceiling |
+| Hard affinity inflates same-worker prefill-decode interference | **supported** | sticky interference_index 13.65 vs lmetric 6.53 |
+| Hybrid affinity (Unified) breaks the locality-vs-latency tradeoff | **supported** | unified hits 79.4% APC and TTFT p90 7.35 s (lmetric 15.67 s) simultaneously |
+| Same-worker prefill-decode interference is causal, not correlation | **supported** | different-worker control idx≈1.0; same-worker idx scales monotonically with prefill size |
+| Heavy-tail sessions are *a* contributor to hot-spot, not the sole cause | **supported** | cap=8 truncated trace cuts 37% of work; hotspot drops only ~10% (2.253→2.020) |
+| The agentic dispatch coupling amplifies policy gaps under saturation | **supported, framed as feature** | Slow policy → longer session lifetime → more concurrent in-flight → harder system. B3 measures the combined policy + feedback effect, which is what an agentic operator experiences. See `agentic_dispatch_coupling.md`. |
+
+## B1' Workload characterization
+
+### Per-request KV footprint (Qwen3-Coder-30B-A3B)
+
+`kv_bytes_per_token = 2 × num_layers × num_kv_heads × head_dim × dtype_bytes = 2 × 48 × 4 × 128 × 2 = 98304 B`
+
+Full GLM-5.1 trace (2.11 M requests, 1.31 M sessions):
+
+| | p50 | p90 | p95 | p99 | max |
+|---|---:|---:|---:|---:|---:|
+| KV per request | 1.83 GiB | 8.04 GiB | 9.59 GiB | **11.49 GiB** | 18.5 GiB |
+
+H20 has ~95 GiB usable per GPU. **A single p99 request occupies 12% of a single H20's HBM** purely for KV. Multi-request batching is bounded by this.
+
+Figure: `figures/fig_kv_footprint_cdf.png`.
+
+### Real reuse decomposition (from lmetric run on w600 trace)
+
+| class | tokens | fraction |
+|---|---:|---:|
+| intra-session | 28.3 M | **93.2%** |
+| cross-session | 1.72 M | 5.7% |
+| shared / system-prefix | 0.34 M | 1.1% |
+| unclassified | 0 | 0.0% |
+
+→ session-affinity routing covers >99% of the reuse signal. There is no meaningful "system prompt" in this trace.
+
+Figure: `figures/fig_reuse_decomposition.png`.
+
+### Theoretical APC ceilings on w600
+
+Computed by building a block-level trie of `hash_ids` per session (intra-session) or globally (any-session), then walking each request's `hash_ids` to count its longest prefix-match against previously-seen prefixes.
+
+| variant | upper bound | hit requests |
+|---|---:|---:|
+| any-session (perfect global cache) | **80.3%** | 961 / 1214 |
+| intra-session only | **79.6%** | 914 / 1214 |
+| shared-prefix only (pos 0, ≥8 sessions) | 0.10% | 107 / 1214 |
+
+Gap "any − intra" is 0.7 pp → no meaningful cross-session sharing in this trace.
+
+## B3 5-policy routing sweep
+
+8 vLLM instances on TP1, w600 trace, `--enable-prompt-tokens-details` so `cached_tokens` is reported per request.
+
+| policy | TTFT p50/p90/p99 | TPOT p50/p90/p99 ms | E2E p50/p90/p99 | **APC** | interference | **hotspot** | n_slow |
+|---|---|---|---|---:|---:|---:|---:|
+| lmetric | 0.94 / 15.67 / 53.57 | 8.9 / 21.2 / 176.9 | 2.75 / 24.82 / 79.83 | 56.9% | 6.53 | 2.253 | 295 |
+| load_only | 1.26 / 20.20 / 52.84 | 9.2 / 26.9 / 320.7 | 3.59 / 33.46 / 93.93 | 54.1% | 9.16 | **1.294** | 379 |
+| sticky | 0.54 / 18.02 / 74.09 | 8.9 / 36.4 / 357.2 | 2.08 / 34.63 / 134.36 | 77.2% | **13.65** | 2.728 | 234 |
+| **unified** | **0.50 / 7.35 / 42.34** | 8.1 / 17.1 / 118.3 | **1.75 / 18.03 / 68.43** | **79.4%** | n/a* | **3.667** | **189** |
+| capped | 1.20 / 12.83 / 46.62 | 7.2 / 16.0 / 101.7 | 2.59 / 21.25 / 73.79 | 31.6% | 6.33 | 2.020 | 185 |
+
+\*unified `engine_state` was overwritten by my analyzer's slice step before the `b3_analyze.sh` fix landed; vLLM and the patch worked correctly. The B2 microbench provides a cleaner interference proof.
+
+> **Methodology note (read before interpreting latency comparisons)**: B3 uses
+> session-sequential trace dispatch — turn N+1 fires the instant turn N
+> completes when the trace timestamp has already passed. This is the right
+> model of agentic workloads (tool-call driven, no user think-time), but it
+> means under saturation each policy's effective in-flight session count is
+> a function of its own per-turn latency (slower policy → longer mean
+> session lifetime → more concurrent in-flight). The reported gaps are
+> therefore "policy + agentic-feedback-amplification", which is what a
+> production agentic operator would experience when switching policies.
+> See `agentic_dispatch_coupling.md` for the full argument. B4 will report
+> the orthogonal "fixed-λ open-loop" measurement.
+
+**Mechanism indices**
+- `interference_index` = TPOT_p90(decode overlapping same-worker prefill) / TPOT_p90(clean)
+- `hotspot_index` = max(worker TTFT p90) / median(worker TTFT p90)
+
+Figures: `fig_b3_latency_bars.png`, `fig_b3_apc_vs_upper.png`,
+`fig_b3_apc_vs_hotspot.png`, `fig_b3_per_worker_ttft_p90.png`,
+`fig_b3_failure_breakdown.png`.
+
+### Per-policy reading
+
+- **lmetric** is the cache-aware baseline. APC 56.9% achieves only 71% of the intra-session ceiling — the missing 23 pp is the locality opportunity unified picks up.
+- **load_only** strips cache awareness. Hot-spot drops to 1.294 (best), but APC only drops 3 pp because the picker's `min(num_requests)` tie-break to instance 0 creates accidental stickiness at low concurrency.
+- **sticky** locks each session to one worker. APC climbs to 77.2% (97% of ceiling) but interference doubles to 13.65 and TPOT p99 hits 345 ms.
+- **unified** is the hybrid — affinity gate `(cache_ratio>0.5 AND num_req ≤ 2×avg)` keeps locality where it pays and drops it where it would hurt. The result is APC 79.4% **and** TTFT p90 cut in half from lmetric. The one bad worker (engine_4 at 37.7s p90) drives `hotspot_index=3.667`, but the other seven workers are all under 18 s.
+- **capped** runs lmetric on a turn-capped trace (max 8 turns/session). Removes 37% of requests but APC also crashes to 31.6% and hotspot only improves by ~10% (2.253 → 2.020). This is the session-mass ablation: heavy sessions are *a* contributor to hot-spot but not the sole cause.
+
+### Slow-request cause breakdown (from `joined_analysis.label_slow_requests`)
+
+| policy | n_slow | same-worker overlap | hot worker queue | cache miss large append | unknown |
+|---|---:|---:|---:|---:|---:|
+| lmetric | 295 | 69 (23%) | 68 (23%) | 94 (32%) | 64 (22%) |
+| load_only | 379 | 108 (29%) | 33 (9%) | 151 (40%) | 87 (23%) |
+| sticky | 234 | **134 (57%)** | 51 (22%) | **20 (9%)** | 29 (12%) |
+| unified | 189 | 0 (no engine_state) | 116 (61%) | 18 (10%) | 55 (29%) |
+| capped | 185 | 45 (24%) | 66 (36%) | 60 (32%) | 14 (8%) |
+
+PD-colo failures are mixed-mechanism: lmetric has no single dominant cause.
+Sticky concentrates failures into same-worker overlap (locality is on, cache misses are gone, but interference takes over).
+
+## B2 PD-colo interference microbench
+
+Setup: 2 vLLM instances on GPU 0 (decode endpoint) and GPU 1 (prefill endpoint). A continuous 4 req/s short-prompt decode load runs against GPU 0 for 60 s per cell. 4 large-prompt one-token "prefill injections" fire every 12 s, targeted at either the same instance (`same`) or the paired one (`different`). Decode requests are labeled overlap iff their `[t_first_token, t_finish]` intersects any injection window. We compare TPOT p90 (overlap vs clean) per cell.
+
+| variant | prefill | n_overlap | n_clean | **TPOT idx** | **TTFT idx** |
+|---|---:|---:|---:|---:|---:|
+| different | 2k–65k | 12–126 | 114–228 | **0.92–1.02** | **0.96–1.00** |
+| same | 2k | 12 | 228 | 1.16 | 2.15 |
+| same | 8k | 19 | 221 | 1.90 | **12.1×** |
+| same | 16k | 37 | 203 | 3.37 | **30.8×** |
+| same | 32k | 67 | 173 | **7.89** | **94.6×** |
+| same | 65k | 130 | 110 | 2.26* | **218×** |
+
+\*65k TPOT idx is non-monotone — see §"TPOT idx peaks at 32k, not 65k" below.
+
+Figures: `fig_b2_tpot_vs_prefill.png`, `fig_b2_ttft_vs_prefill.png`.
+
+**Why this matters**
+- The `different-worker` control sits at idx ≈ 1.0 across 32× variation in prefill size. This is the cleanest possible disproof of "any prefill anywhere hurts decode": prefill on a *different* worker is invisible to the decode worker.
+- The `same-worker` TTFT curve is monotone in prefill size all the way to 218× at 65k. TPOT p90 is monotone only up to 32k (7.89×), then drops at 65k — this is not "interference relaxing", it is the cost regime shifting from TPOT to TTFT (see below).
+- This is the mechanism behind the B3 sticky interference jump (13.65) and unified's single hot worker (engine_4 at 37.7 s TTFT p90).
+
+### TPOT idx peaks at 32k, not 65k — regime shift, not relief
+
+The naïve reading of the table is "interference gets worse up to 32k then drops at 65k". That is wrong; the cost is shifting from per-token rate (TPOT) to first-token wait (TTFT), and `p90 / clean` happens to compress the visible cost. Three superimposed effects.
+
+Same-variant detail across the regime boundary:
+
+```
+                          32k         65k       change
+n_overlap                  67         130       +94%  (most decodes now overlap)
+n_clean                   173         110       -37%
+TPOT p50 overlap (ms)    12.2        20.1       +1.6x
+TPOT p90 overlap (ms)    54.8        21.7       -2.5x  <- "improves"
+TPOT p99 overlap (ms)    59.0       169.5       +2.9x  <- tail explodes
+TTFT p90 overlap  (s)    4.17       14.06       +3.4x
+TPOT p90 clean   (ms)     6.9         9.6       +40%
+```
+
+**Mechanism 1 — Cost shifts from TPOT to TTFT.** TPOT is measured only *after* a request starts emitting tokens. A 32 k prefill (~5 s on H20) is short enough that vLLM's chunked-prefill scheduler keeps interleaving decode steps; overlapping decodes trickle tokens out at painfully slow per-token rates → p90 TPOT 54.8 ms. A 65 k prefill (~10 s) is long enough that many overlapping decodes get *zero* tokens for nearly the whole prefill window; when they finally break through, the injection is winding down so subsequent decode iterations are unobstructed. The cost goes onto the TTFT clock (14 s) instead of inflating TPOT.
+
+**Mechanism 2 — Bimodal TPOT distribution hides under p90.** At 65 k overlap, two populations of decodes coexist:
+- decodes blocked the entire prefill (high TTFT, then normal per-token rate)
+- decodes that did trickle slowly through prefill chunks (low TTFT, high TPOT)
+- The p99 jump 59 → 169.5 ms shows the second population is *worse* at 65 k. p90 happens to fall on the first (fast-after-block) population.
+
+**Mechanism 3 — "Clean" stops being clean.** With 4 × ~10 s injections spread across 60 s (40 s of injection time, 20 s of gaps), there are very few moments where the worker is truly idle. The 110 "clean" decodes at 65 k are squeezed into 2-3 s pockets where the system is recovering from the previous injection or about to be hit by the next. TPOT p90 clean rises 6.9 → 9.6 ms (the denominator of the idx ratio drifts up by 40%).
+
+**Reading rule for B2**: TTFT idx is the headline interference metric — it is monotone and reflects user-visible "no tokens for N seconds" latency. **TPOT p99** is the right tail-sensitivity indicator (also monotone). **TPOT p90 is non-monotone across regime shifts and should not be used alone**. This has direct implications for SLO design: TTFT and TPOT cannot share the same violation threshold under PD-colo interference, because they measure costs from *different* points in the request lifecycle and the cost migration between them is workload-dependent.
+
+This is also a finding the paper should call out: **once same-worker prefill grows beyond a TTFT-block threshold, overlapping decodes "give up" their per-token rate complaint and pay the cost in queueing instead**. The system looks faster on per-token metrics; users experience longer waits.
+
+## What Window 1 does *not* answer
+
+These need Window 2 (B4 SRR sweep + B5 failure attribution near SRR boundary):
+
+1. **Sustainable arrival rate (SRR) per policy under SLO**. B3 was driven by trace timestamps with strict session sequentiality; when 8 instances cannot keep up, requests pile up and the *effective* dispatch window stretches (lmetric: trace claims 600 s, actual replay 49 min). We measured *saturated* behavior but not the saturation point. B4 needs the A4 open-loop Poisson loadgen with per-class SLO thresholds.
+2. **Failure breakdown at the SRR boundary**. B5 will rerun each policy at 0.9× / 1.0× / 1.1× of its SRR_max and label each SLO-violating request — gives the paper its causal failure-attribution table.
+
+Optional / paper-polish runs (not blocking the story):
+
+3. unified isolated rerun to capture `interference_index` (B2 already provides cleaner causal proof; skip unless reviewer asks).
+4. B2 with the proxy in path — measure whether the production cache_aware routing actually pushes prefill and decode onto different workers in practice.
+5. KV-occupancy timeline per worker — needs polling `vllm:gpu_cache_usage` during B3 reruns; useful for "KV pressure drives cache miss" subsection.
+
+## Limitations (read this before quoting B3 numbers)
+
+1. **Agentic dispatch coupling is by design**. B3 is the
+   "production-replay under captured agentic load" experiment, not the
+   "controlled-load envelope" experiment. Latency p90 reflects both
+   per-request policy effect AND the agentic feedback amplification
+   (slow policy → longer mean session lifetime → more concurrent
+   in-flight). Both contributions are real and visible to a production
+   operator; **the paper must report both, not subtract one**. See
+   `agentic_dispatch_coupling.md`. The orthogonal "fixed-λ Poisson"
+   measurement is B4.
+
+2. **B3 `interference_index` is a binary indicator**. A decode is
+   labeled "overlap" iff *any* other request's prefill exists on the
+   chosen worker during `[t_first_token, t_finish]`, regardless of
+   prefill size. B2's per-prefill-size cells (2k = 1.16×, 65k = 2.26×)
+   cannot be directly read off B3's aggregate numbers (lmetric 6.53,
+   sticky 13.65). The B3 numbers are size-weighted averages of the
+   per-cell signal, valid for *within-B3 cross-policy* comparison but
+   not for direct cross-batch numerical comparison with B2.
+
+3. **Hot-sweep cache contamination (low)**: `lmetric` ran from cold;
+   `load_only` and `sticky` ran on the same 8 vLLMs without restart.
+   First-turn cached_tokens verification puts empirical contamination
+   at < 1% APC, well below the cross-policy gaps. `unified` and
+   `capped` were rerun cold-start specifically to remove any residual
+   concern.
+
+4. **Unified's `interference_index` is missing**. The original
+   `b3_analyze.sh` unconditionally truncate-wrote sliced engine_state
+   files; isolated runs that wrote engine_state into their own
+   per-policy directory were overwritten. Fixed in commit `df32499`;
+   capped was the first run to benefit and survived. **Implication**:
+   unified's slow-request mechanism breakdown (rows 0 / 116 / 18 / 55
+   for same-worker overlap / hot worker queue / cache miss / unknown)
+   has the "same-worker overlap" label *unrecoverable* and forced into
+   the catch-all buckets — do not read unified's failure attribution
+   as causal.
+
+5. **w600 is not the full GLM-5.1 trace** (1214 req vs 2.11 M). All
+   B3/B2 percentiles are on the sample. The full-trace KV-footprint
+   stats are on the full trace.
+
+6. **Reuse decomposition (intra/cross/shared/unclassified) is
+   per-cached-token only in expectation** — `joined_analysis.py`
+   distributes a request's `cached_tokens` count uniformly across its
+   `hash_ids` and classifies block-by-block. For the w600 trace with
+   <1% cross-session sharing the qualitative split is robust; for
+   workloads with mixed-class hashes within a single request the
+   classifier should be revisited.
+
+## Reproduction commands
+
+```bash
+# B3 5-policy sweep
+bash scripts/b3_sweep.sh                                   # lmetric, load_only, sticky (hot-cache)
+bash scripts/b3_isolated_policy.sh unified <trace> <dir>   # isolated cold-start
+bash scripts/b3_isolated_policy.sh lmetric <capped> <dir>  # capped variant
+
+bash scripts/b3_analyze.sh outputs/b3_sweep_<TS>
+python3 scripts/render_b3_report.py --sweep-dir outputs/b3_sweep_<TS>
+
+# B2 interference microbench
+# (launch 2 vLLM on ports 8100/8101 with --enable-prompt-tokens-details first)
+python3 scripts/b2_interference.py \
+    --decode-endpoint http://127.0.0.1:8100 \
+    --prefill-endpoint http://127.0.0.1:8101 \
+    --model <model> \
+    --out-dir outputs/b2_microbench/sweep
+python3 analysis/characterization/b2_sweep_analysis.py --sweep-dir outputs/b2_microbench/sweep
+
+# Figures
+python3 analysis/characterization/render_window1_figures.py \
+    --results-dir analysis/characterization/window_1_results \
+    --out-dir analysis/characterization/window_1_results/figures
+```

analysis/characterization/window_1_results/apc_upper_w600.json

@@ -0,0 +1,18 @@
+{
+  "trace": "/home/admin/cpfs/wjh/agentic-kv/traces/w600_r0.0015_st30.jsonl",
+  "n_requests": 1214,
+  "n_sessions": 274,
+  "block_size": 512,
+  "shared_prefix_min_sessions": 8,
+  "total_input_tokens": 53335690,
+  "apc_upper_any_session": 0.8030439654947747,
+  "apc_upper_intra_session": 0.7956783534627564,
+  "apc_upper_shared_prefix_only": 0.0010271546126055554,
+  "cached_tokens_any_session": 42830904,
+  "cached_tokens_intra_session": 42438054,
+  "cached_tokens_shared_prefix_only": 54784,
+  "n_requests_any_hit": 961,
+  "n_requests_intra_hit": 914,
+  "n_requests_shared_hit": 107,
+  "n_shared_pos0_blocks": 1
+}

analysis/characterization/window_1_results/b2_sweep_summary.json

@@ -0,0 +1,194 @@
+{
+  "rows": [
+    {
+      "decode_endpoint": "http://127.0.0.1:8100",
+      "interference_index": 0.9868436853823819,
+      "n_decode_clean": 207,
+      "n_decode_overlap": 33,
+      "n_decode_total": 240,
+      "n_prefill_injections": 4,
+      "prefill_endpoint": "http://127.0.0.1:8101",
+      "prefill_size": 16384,
+      "tpot_p50_clean_s": 0.0061757058808297825,
+      "tpot_p50_overlap_s": 0.006127697048765241,
+      "tpot_p90_clean_s": 0.006862485770023231,
+      "tpot_p90_overlap_s": 0.006772200748173878,
+      "tpot_p99_clean_s": 0.007128368820806946,
+      "tpot_p99_overlap_s": 0.0070623818792478,
+      "ttft_p90_clean_s": 0.043039703369140626,
+      "ttft_p90_overlap_s": 0.04307723045349121,
+      "variant": "different"
+    },
+    {
+      "decode_endpoint": "http://127.0.0.1:8100",
+      "interference_index": 1.0176125863449343,
+      "n_decode_clean": 228,
+      "n_decode_overlap": 12,
+      "n_decode_total": 240,
+      "n_prefill_injections": 4,
+      "prefill_endpoint": "http://127.0.0.1:8101",
+      "prefill_size": 2048,
+      "tpot_p50_clean_s": 0.0062349300191860005,
+      "tpot_p50_overlap_s": 0.006218204594621754,
+      "tpot_p90_clean_s": 0.006892242576136734,
+      "tpot_p90_overlap_s": 0.007013632793619174,
+      "tpot_p99_clean_s": 0.007111345902837888,
+      "tpot_p99_overlap_s": 0.007131954732567373,
+      "ttft_p90_clean_s": 0.04290406703948975,
+      "ttft_p90_overlap_s": 0.040976309776306154,
+      "variant": "different"
+    },
+    {
+      "decode_endpoint": "http://127.0.0.1:8100",
+      "interference_index": 0.9221676118155049,
+      "n_decode_clean": 176,
+      "n_decode_overlap": 64,
+      "n_decode_total": 240,
+      "n_prefill_injections": 4,
+      "prefill_endpoint": "http://127.0.0.1:8101",
+      "prefill_size": 32768,
+      "tpot_p50_clean_s": 0.00620933012528853,
+      "tpot_p50_overlap_s": 0.005991364970351711,
+      "tpot_p90_clean_s": 0.0069098352181791054,
+      "tpot_p90_overlap_s": 0.006372026241186894,
+      "tpot_p99_clean_s": 0.007242970394365715,
+      "tpot_p99_overlap_s": 0.006935877366499467,
+      "ttft_p90_clean_s": 0.04308474063873291,
+      "ttft_p90_overlap_s": 0.04266033172607422,
+      "variant": "different"
+    },
+    {
+      "decode_endpoint": "http://127.0.0.1:8100",
+      "interference_index": 1.0162810692345416,
+      "n_decode_clean": 114,
+      "n_decode_overlap": 126,
+      "n_decode_total": 240,
+      "n_prefill_injections": 4,
+      "prefill_endpoint": "http://127.0.0.1:8101",
+      "prefill_size": 65536,
+      "tpot_p50_clean_s": 0.006080349286397299,
+      "tpot_p50_overlap_s": 0.006312949488861392,
+      "tpot_p90_clean_s": 0.0068880830148253785,
+      "tpot_p90_overlap_s": 0.007000228371283021,
+      "tpot_p99_clean_s": 0.007222196574162956,
+      "tpot_p99_overlap_s": 0.00723441562267265,
+      "ttft_p90_clean_s": 0.04367616176605225,
+      "ttft_p90_overlap_s": 0.04332089424133301,
+      "variant": "different"
+    },
+    {
+      "decode_endpoint": "http://127.0.0.1:8100",
+      "interference_index": 0.92169565663476,
+      "n_decode_clean": 220,
+      "n_decode_overlap": 20,
+      "n_decode_total": 240,
+      "n_prefill_injections": 4,
+      "prefill_endpoint": "http://127.0.0.1:8101",
+      "prefill_size": 8192,
+      "tpot_p50_clean_s": 0.006260122915711066,
+      "tpot_p50_overlap_s": 0.006120474651606396,
+      "tpot_p90_clean_s": 0.006968991684191154,
+      "tpot_p90_overlap_s": 0.006423289366442748,
+      "tpot_p99_clean_s": 0.007601349209294174,
+      "tpot_p99_overlap_s": 0.006715166592838788,
+      "ttft_p90_clean_s": 0.04314079284667969,
+      "ttft_p90_overlap_s": 0.042817187309265134,
+      "variant": "different"
+    },
+    {
+      "decode_endpoint": "http://127.0.0.1:8100",
+      "interference_index": 3.3716068170318985,
+      "n_decode_clean": 203,
+      "n_decode_overlap": 37,
+      "n_decode_total": 240,
+      "n_prefill_injections": 4,
+      "prefill_endpoint": "http://127.0.0.1:8100",
+      "prefill_size": 16384,
+      "tpot_p50_clean_s": 0.006435276281954062,
+      "tpot_p50_overlap_s": 0.009116151116111061,
+      "tpot_p90_clean_s": 0.0071605749804564195,
+      "tpot_p90_overlap_s": 0.024142643417974917,
+      "tpot_p99_clean_s": 0.008356584539317119,
+      "tpot_p99_overlap_s": 0.024809808827409838,
+      "ttft_p90_clean_s": 0.04402604103088379,
+      "ttft_p90_overlap_s": 1.3574100017547606,
+      "variant": "same"
+    },
+    {
+      "decode_endpoint": "http://127.0.0.1:8100",
+      "interference_index": 1.1589170446597312,
+      "n_decode_clean": 228,
+      "n_decode_overlap": 12,
+      "n_decode_total": 240,
+      "n_prefill_injections": 4,
+      "prefill_endpoint": "http://127.0.0.1:8100",
+      "prefill_size": 2048,
+      "tpot_p50_clean_s": 0.006142637946388938,
+      "tpot_p50_overlap_s": 0.007610858088791972,
+      "tpot_p90_clean_s": 0.006933137142296993,
+      "tpot_p90_overlap_s": 0.008034930807171445,
+      "tpot_p99_clean_s": 0.007201877651792584,
+      "tpot_p99_overlap_s": 0.0084272463153107,
+      "ttft_p90_clean_s": 0.043091440200805665,
+      "ttft_p90_overlap_s": 0.09247522354125978,
+      "variant": "same"
+    },
+    {
+      "decode_endpoint": "http://127.0.0.1:8100",
+      "interference_index": 7.891276559921504,
+      "n_decode_clean": 173,
+      "n_decode_overlap": 67,
+      "n_decode_total": 240,
+      "n_prefill_injections": 4,
+      "prefill_endpoint": "http://127.0.0.1:8100",
+      "prefill_size": 32768,
+      "tpot_p50_clean_s": 0.006226602226796776,
+      "tpot_p50_overlap_s": 0.012180752224392362,
+      "tpot_p90_clean_s": 0.00694006813897027,
+      "tpot_p90_overlap_s": 0.054765997029314145,
+      "tpot_p99_clean_s": 0.010443444107518053,
+      "tpot_p99_overlap_s": 0.058983875428787386,
+      "ttft_p90_clean_s": 0.04411859512329101,
+      "ttft_p90_overlap_s": 4.174754428863525,
+      "variant": "same"
+    },
+    {
+      "decode_endpoint": "http://127.0.0.1:8100",
+      "interference_index": 2.259323176730457,
+      "n_decode_clean": 110,
+      "n_decode_overlap": 130,
+      "n_decode_total": 240,
+      "n_prefill_injections": 4,
+      "prefill_endpoint": "http://127.0.0.1:8100",
+      "prefill_size": 65536,
+      "tpot_p50_clean_s": 0.0064652375500611585,
+      "tpot_p50_overlap_s": 0.020095128001588764,
+      "tpot_p90_clean_s": 0.009607415488272014,
+      "tpot_p90_overlap_s": 0.021706256481132124,
+      "tpot_p99_clean_s": 0.016912007837584522,
+      "tpot_p99_overlap_s": 0.16948255478733715,
+      "ttft_p90_clean_s": 0.06447408199310305,
+      "ttft_p90_overlap_s": 14.060086917877197,
+      "variant": "same"
+    },
+    {
+      "decode_endpoint": "http://127.0.0.1:8100",
+      "interference_index": 1.8961314610807898,
+      "n_decode_clean": 221,
+      "n_decode_overlap": 19,
+      "n_decode_total": 240,
+      "n_prefill_injections": 4,
+      "prefill_endpoint": "http://127.0.0.1:8100",
+      "prefill_size": 8192,
+      "tpot_p50_clean_s": 0.00617263052198622,
+      "tpot_p50_overlap_s": 0.008303543533941712,
+      "tpot_p90_clean_s": 0.007060385713673601,
+      "tpot_p90_overlap_s": 0.013387419479061859,
+      "tpot_p99_clean_s": 0.0076809098022152696,
+      "tpot_p99_overlap_s": 0.013849472662415166,
+      "ttft_p90_clean_s": 0.04307150840759277,
+      "ttft_p90_overlap_s": 0.52073073387146,
+      "variant": "same"
+    }
+  ]
+}

analysis/characterization/window_1_results/b3_policy_comparison.json

@@ -0,0 +1,133 @@
+{
+  "rows": [
+    {
+      "policy": "capped",
+      "n_ok": 770,
+      "n_total": 770,
+      "ttft_p50_s": 1.1989156164927408,
+      "ttft_p90_s": 12.827629912580612,
+      "ttft_p99_s": 46.61752380923125,
+      "tpot_p50_s": 0.007231239004497606,
+      "tpot_p90_s": 0.015998617687440243,
+      "tpot_p99_s": 0.11515370831539476,
+      "e2e_p50_s": 2.598489043477457,
+      "e2e_p90_s": 21.245602010778384,
+      "e2e_p99_s": 74.60736650204846,
+      "apc_ratio": 0.3158312503528108,
+      "interference_index": 6.331064378362814,
+      "hotspot_index_ttft_p90": 2.0204268015410918,
+      "reuse_intra_frac": 0.9192657105586233,
+      "reuse_cross_frac": 0.0602232594931501,
+      "n_slow": 185,
+      "failure_counts": {
+        "cache_miss_large_append": 60,
+        "hot_worker_queue": 66,
+        "same_worker_prefill_overlap": 45,
+        "unknown": 14
+      }
+    },
+    {
+      "policy": "lmetric",
+      "n_ok": 1214,
+      "n_total": 1214,
+      "ttft_p50_s": 0.9387824369769078,
+      "ttft_p90_s": 15.671339168207492,
+      "ttft_p99_s": 53.56683189840049,
+      "tpot_p50_s": 0.008854518407308914,
+      "tpot_p90_s": 0.02122720699121469,
+      "tpot_p99_s": 0.18280341184277568,
+      "e2e_p50_s": 2.754255389008904,
+      "e2e_p90_s": 24.8209177934099,
+      "e2e_p99_s": 80.59924928059091,
+      "apc_ratio": 0.5694312382571595,
+      "interference_index": 6.530231061794441,
+      "hotspot_index_ttft_p90": 2.252837147833725,
+      "reuse_intra_frac": 0.9321238805590836,
+      "reuse_cross_frac": 0.05679481258506571,
+      "n_slow": 295,
+      "failure_counts": {
+        "cache_miss_large_append": 94,
+        "hot_worker_queue": 68,
+        "same_worker_prefill_overlap": 69,
+        "unknown": 64
+      }
+    },
+    {
+      "policy": "load_only",
+      "n_ok": 1214,
+      "n_total": 1214,
+      "ttft_p50_s": 1.2609447415161412,
+      "ttft_p90_s": 20.197147866390882,
+      "ttft_p99_s": 52.84285237012196,
+      "tpot_p50_s": 0.009231464695980247,
+      "tpot_p90_s": 0.026851662550158716,
+      "tpot_p99_s": 0.3211630676943426,
+      "e2e_p50_s": 3.58568156149704,
+      "e2e_p90_s": 33.459180271782685,
+      "e2e_p99_s": 93.95083751494239,
+      "apc_ratio": 0.5412093853102866,
+      "interference_index": 9.16424627504275,
+      "hotspot_index_ttft_p90": 1.2940319990630569,
+      "reuse_intra_frac": 0.9353191550754928,
+      "reuse_cross_frac": 0.053372184678592026,
+      "n_slow": 379,
+      "failure_counts": {
+        "cache_miss_large_append": 151,
+        "hot_worker_queue": 33,
+        "same_worker_prefill_overlap": 108,
+        "unknown": 87
+      }
+    },
+    {
+      "policy": "sticky",
+      "n_ok": 1214,
+      "n_total": 1214,
+      "ttft_p50_s": 0.5415176274836995,
+      "ttft_p90_s": 18.021296651283045,
+      "ttft_p99_s": 74.09429564891524,
+      "tpot_p50_s": 0.008952101894096181,
+      "tpot_p90_s": 0.03641285916619554,
+      "tpot_p99_s": 0.35152006935195085,
+      "e2e_p50_s": 2.081947358994512,
+      "e2e_p90_s": 34.62592205510591,
+      "e2e_p99_s": 139.68334607904353,
+      "apc_ratio": 0.7720092868396378,
+      "interference_index": 13.651718321568111,
+      "hotspot_index_ttft_p90": 2.727756623171119,
+      "reuse_intra_frac": 0.9327723488279339,
+      "reuse_cross_frac": 0.05495149683864246,
+      "n_slow": 234,
+      "failure_counts": {
+        "cache_miss_large_append": 20,
+        "hot_worker_queue": 51,
+        "same_worker_prefill_overlap": 134,
+        "unknown": 29
+      }
+    },
+    {
+      "policy": "unified",
+      "n_ok": 1213,
+      "n_total": 1214,
+      "ttft_p50_s": 0.4997710260213353,
+      "ttft_p90_s": 7.345769894809922,
+      "ttft_p99_s": 42.34170345296613,
+      "tpot_p50_s": 0.008079791456705824,
+      "tpot_p90_s": 0.017110194704198407,
+      "tpot_p99_s": 0.12655874612209597,
+      "e2e_p50_s": 1.7495028690318577,
+      "e2e_p90_s": 18.033410895219994,
+      "e2e_p99_s": 68.80023987947489,
+      "apc_ratio": 0.794261466256467,
+      "interference_index": null,
+      "hotspot_index_ttft_p90": 3.667136528736114,
+      "reuse_intra_frac": 0.9311187350942534,
+      "reuse_cross_frac": 0.056702150437367635,
+      "n_slow": 189,
+      "failure_counts": {
+        "cache_miss_large_append": 18,
+        "hot_worker_queue": 116,
+        "unknown": 55
+      }
+    }
+  ]
+}

analysis/characterization/window_1_results/b3_report.md

@@ -0,0 +1,114 @@
+# B3 Routing Sweep Report
+
+Sweep dir: `b3_sweep_20260525_095043`
+Trace: w600_r0.0015_st30.jsonl (~1.2k reqs, 8 × TP1)
+Policies present: lmetric, load_only, sticky, unified, capped
+Policies pending: —
+
+## Headline latencies + APC
+
+| policy | ok/total | TTFT p50/p90/p99 (s) | TPOT p50/p90/p99 (ms) | E2E p50/p90/p99 (s) | APC |
+|---|---:|---|---|---|---:|
+| **lmetric** | 1214/1214 | 0.94/15.59/52.95 | 8.9/21.2/175.9 | 2.75/24.75/79.62 | 56.9% |
+| **load_only** | 1214/1214 | 1.25/20.15/52.65 | 9.2/26.7/320.7 | 3.58/33.43/93.92 | 54.1% |
+| **sticky** | 1214/1214 | 0.54/18.02/71.37 | 8.9/36.1/345.2 | 2.08/34.61/133.58 | 77.2% |
+| **unified** | 1213/1214 | 0.50/7.24/42.02 | 8.1/17.1/118.1 | 1.75/17.89/68.18 | 79.4% |
+| **capped** | 770/770 | 1.20/12.76/46.05 | 7.2/16.0/101.5 | 2.59/21.24/73.39 | 31.6% |
+
+## Mechanism indices
+
+| policy | interference_index | hotspot_index (TTFT p90) | intra-session reuse | cross-session reuse | n_slow |
+|---|---:|---:|---:|---:|---:|
+| **lmetric** | 6.53 | 2.24 | 93.2% | 5.7% | 295 |
+| **load_only** | 9.16 | 1.14 | 93.5% | 5.3% | 379 |
+| **sticky** | 13.65 | 2.35 | 93.3% | 5.5% | 234 |
+| **unified** | — | 3.35 | 93.1% | 5.7% | 189 |
+| **capped** | 6.33 | 1.94 | 91.9% | 6.0% | 185 |
+
+- **interference_index** = TPOT_p90(decode overlapping same-worker prefill) / TPOT_p90(clean)
+- **hotspot_index** = max(worker TTFT_p90) / median(worker TTFT_p90)
+
+## Slow-request cause breakdown
+
+| policy | n_slow | same-worker overlap | hot worker queue | cache miss large append | high KV | unknown |
+|---|---:|---:|---:|---:|---:|---:|
+| **lmetric** | 295 | 69 | 68 | 94 | 0 | 64 |
+| **load_only** | 379 | 108 | 33 | 151 | 0 | 87 |
+| **sticky** | 234 | 134 | 51 | 20 | 0 | 29 |
+| **unified** | 189 | 0 | 116 | 18 | 0 | 55 |
+| **capped** | 185 | 45 | 66 | 60 | 0 | 14 |
+
+## Policy notes
+
+- **lmetric** — cache-aware P_tokens × BS (main baseline)
+- **load_only** — control: min(num_requests), no cache, no affinity
+- **sticky** — control: hard session affinity (never break)
+- **unified** — hybrid affinity + LMetric fallback
+- **capped** — lmetric on per-session turn-capped trace
+
+## Per-policy per-worker TTFT p90 (s)
+
+### lmetric
+
+| worker | TTFT p90 (s) |
+|---|---:|
+| http://127.0.0.1:8000 | 28.18 |
+| http://127.0.0.1:8001 | 13.15 |
+| http://127.0.0.1:8002 | 13.82 |
+| http://127.0.0.1:8003 | 14.00 |
+| http://127.0.0.1:8004 | 31.34 |
+| http://127.0.0.1:8005 | 7.87 |
+| http://127.0.0.1:8006 | 14.15 |
+| http://127.0.0.1:8007 | 11.78 |
+
+### load_only
+
+| worker | TTFT p90 (s) |
+|---|---:|
+| http://127.0.0.1:8000 | 22.06 |
+| http://127.0.0.1:8001 | 16.43 |
+| http://127.0.0.1:8002 | 16.81 |
+| http://127.0.0.1:8003 | 23.58 |
+| http://127.0.0.1:8004 | 25.14 |
+| http://127.0.0.1:8005 | 16.08 |
+| http://127.0.0.1:8006 | 23.96 |
+| http://127.0.0.1:8007 | 13.95 |
+
+### sticky
+
+| worker | TTFT p90 (s) |
+|---|---:|
+| http://127.0.0.1:8000 | 12.28 |
+| http://127.0.0.1:8001 | 23.57 |
+| http://127.0.0.1:8002 | 5.20 |
+| http://127.0.0.1:8003 | 55.38 |
+| http://127.0.0.1:8004 | 17.03 |
+| http://127.0.0.1:8005 | 25.49 |
+| http://127.0.0.1:8006 | 36.31 |
+| http://127.0.0.1:8007 | 2.50 |
+
+### unified
+
+| worker | TTFT p90 (s) |
+|---|---:|
+| http://127.0.0.1:8000 | 11.26 |
+| http://127.0.0.1:8001 | 3.61 |
+| http://127.0.0.1:8002 | 16.18 |
+| http://127.0.0.1:8003 | 9.31 |
+| http://127.0.0.1:8004 | 37.73 |
+| http://127.0.0.1:8005 | 18.33 |
+| http://127.0.0.1:8006 | 3.63 |
+| http://127.0.0.1:8007 | 7.77 |
+
+### capped
+
+| worker | TTFT p90 (s) |
+|---|---:|
+| http://127.0.0.1:8000 | 19.77 |
+| http://127.0.0.1:8001 | 15.79 |
+| http://127.0.0.1:8002 | 20.40 |
+| http://127.0.0.1:8003 | 10.54 |
+| http://127.0.0.1:8004 | 9.52 |
+| http://127.0.0.1:8005 | 9.46 |
+| http://127.0.0.1:8006 | 7.38 |
+| http://127.0.0.1:8007 | 9.66 |

analysis/characterization/window_1_results/kv_footprint_summary.json

@@ -0,0 +1,26 @@
+{
+  "formula": "kv_bytes_per_request = input_tokens * kv_bytes_per_token",
+  "kv_bytes_per_request": {
+    "count": 2114220,
+    "max": 19893878784.0,
+    "mean": 3306689367.3278427,
+    "min": 0.0,
+    "p50": 1969029120.0,
+    "p90": 8636507750.40001,
+    "p95": 10296164352.0,
+    "p99": 12339806208.0
+  },
+  "kv_bytes_per_token": 98304.0,
+  "kv_mib_per_request": {
+    "count": 2114220,
+    "max": 18972.28125,
+    "mean": 3153.5047219541957,
+    "min": 0.0,
+    "p50": 1877.8125,
+    "p90": 8236.415625000009,
+    "p95": 9819.1875,
+    "p99": 11768.15625
+  },
+  "status": "available",
+  "total_kv_gib": 6510940.188720703
+}

analysis/characterization/window_1_results/lmetric_hotspot.json

@@ -0,0 +1,24 @@
+{
+  "hotspot_index_ttft_p90": 2.237981740718548,
+  "per_worker_latency_p90_s": {
+    "http://127.0.0.1:8000": 34.71445541951107,
+    "http://127.0.0.1:8001": 21.922988962882666,
+    "http://127.0.0.1:8002": 23.936190764518685,
+    "http://127.0.0.1:8003": 26.22220957049285,
+    "http://127.0.0.1:8004": 40.318757307820505,
+    "http://127.0.0.1:8005": 12.26559703698149,
+    "http://127.0.0.1:8006": 27.904838753980588,
+    "http://127.0.0.1:8007": 18.430557113309625
+  },
+  "per_worker_ttft_p90_s": {
+    "http://127.0.0.1:8000": 28.18261351052206,
+    "http://127.0.0.1:8001": 13.147308969072796,
+    "http://127.0.0.1:8002": 13.818959677941162,
+    "http://127.0.0.1:8003": 14.003642184572524,
+    "http://127.0.0.1:8004": 31.339895512629305,
+    "http://127.0.0.1:8005": 7.870992770011071,
+    "http://127.0.0.1:8006": 14.149156623415186,
+    "http://127.0.0.1:8007": 11.777357225219024
+  },
+  "status": "supported"
+}

analysis/characterization/window_1_results/lmetric_reuse.json

@@ -0,0 +1,15 @@
+{
+  "cross_session_tokens": 1723017,
+  "fractions": {
+    "cross": 0.05679481258506571,
+    "intra": 0.9321238805590836,
+    "shared": 0.011081306855850749,
+    "unclassified": 0.0
+  },
+  "intra_session_tokens": 28278380,
+  "shared_prefix_min_sessions": 8,
+  "shared_prefix_tokens": 336180,
+  "status": "supported",
+  "total_cached_tokens": 30371008,
+  "unclassified_tokens": 0
+}

analysis/characterization/window_1_results/per_worker_capped.json

@@ -0,0 +1,24 @@
+{
+  "hotspot_index_ttft_p90": 2.0204268015410918,
+  "per_worker_latency_p90_s": {
+    "http://127.0.0.1:8000": 23.81083881931848,
+    "http://127.0.0.1:8001": 18.139674991380897,
+    "http://127.0.0.1:8002": 29.116712999995805,
+    "http://127.0.0.1:8003": 19.245074290811324,
+    "http://127.0.0.1:8004": 17.230851700413044,
+    "http://127.0.0.1:8005": 15.86663371440958,
+    "http://127.0.0.1:8006": 16.707309890014592,
+    "http://127.0.0.1:8007": 23.93718611740042
+  },
+  "per_worker_ttft_p90_s": {
+    "http://127.0.0.1:8000": 19.772570010094213,
+    "http://127.0.0.1:8001": 15.786850639013576,
+    "http://127.0.0.1:8002": 20.403525242628533,
+    "http://127.0.0.1:8003": 10.535247699997853,
+    "http://127.0.0.1:8004": 9.52290979558602,
+    "http://127.0.0.1:8005": 9.455131393985376,
+    "http://127.0.0.1:8006": 7.379608143202497,
+    "http://127.0.0.1:8007": 9.661995008389932
+  },
+  "status": "supported"
+}

analysis/characterization/window_1_results/per_worker_lmetric.json

@@ -0,0 +1,24 @@
+{
+  "hotspot_index_ttft_p90": 2.252837147833725,
+  "per_worker_latency_p90_s": {
+    "http://127.0.0.1:8000": 34.71445541951107,
+    "http://127.0.0.1:8001": 21.922988962882666,
+    "http://127.0.0.1:8002": 23.936190764518685,
+    "http://127.0.0.1:8003": 26.22220957049285,
+    "http://127.0.0.1:8004": 40.318757307820505,
+    "http://127.0.0.1:8005": 12.26559703698149,
+    "http://127.0.0.1:8006": 27.904838753980588,
+    "http://127.0.0.1:8007": 18.430557113309625
+  },
+  "per_worker_ttft_p90_s": {
+    "http://127.0.0.1:8000": 28.18261351052206,
+    "http://127.0.0.1:8001": 13.147308969072796,
+    "http://127.0.0.1:8002": 13.818959677941162,
+    "http://127.0.0.1:8003": 14.003642184572524,
+    "http://127.0.0.1:8004": 31.339895512629305,
+    "http://127.0.0.1:8005": 7.870992770011071,
+    "http://127.0.0.1:8006": 14.149156623415186,
+    "http://127.0.0.1:8007": 11.777357225219024
+  },
+  "status": "supported"
+}

analysis/characterization/window_1_results/per_worker_load_only.json

@@ -0,0 +1,24 @@
+{
+  "hotspot_index_ttft_p90": 1.2940319990630569,
+  "per_worker_latency_p90_s": {
+    "http://127.0.0.1:8000": 33.51168999259829,
+    "http://127.0.0.1:8001": 29.20308109278556,
+    "http://127.0.0.1:8002": 27.126518827211115,
+    "http://127.0.0.1:8003": 38.597240307606995,
+    "http://127.0.0.1:8004": 36.607777832809376,
+    "http://127.0.0.1:8005": 28.097025175404276,
+    "http://127.0.0.1:8006": 49.29610514297965,
+    "http://127.0.0.1:8007": 20.958507975534303
+  },
+  "per_worker_ttft_p90_s": {
+    "http://127.0.0.1:8000": 22.055091864388675,
+    "http://127.0.0.1:8001": 16.425856862741057,
+    "http://127.0.0.1:8002": 16.806352904380766,
+    "http://127.0.0.1:8003": 23.581166115606912,
+    "http://127.0.0.1:8004": 25.14397653030465,
+    "http://127.0.0.1:8005": 16.080231266201018,
+    "http://127.0.0.1:8006": 23.960470345703648,
+    "http://127.0.0.1:8007": 13.95184187250561
+  },
+  "status": "supported"
+}

analysis/characterization/window_1_results/per_worker_sticky.json

@@ -0,0 +1,24 @@
+{
+  "hotspot_index_ttft_p90": 2.727756623171119,
+  "per_worker_latency_p90_s": {
+    "http://127.0.0.1:8000": 30.185792533413043,
+    "http://127.0.0.1:8001": 47.49661003401852,
+    "http://127.0.0.1:8002": 22.069474861002554,
+    "http://127.0.0.1:8003": 83.73774532350944,
+    "http://127.0.0.1:8004": 22.03310715127737,
+    "http://127.0.0.1:8005": 33.024566102202556,
+    "http://127.0.0.1:8006": 61.65600914339302,
+    "http://127.0.0.1:8007": 6.077459598158019
+  },
+  "per_worker_ttft_p90_s": {
+    "http://127.0.0.1:8000": 12.284569517592924,
+    "http://127.0.0.1:8001": 23.570226482005094,
+    "http://127.0.0.1:8002": 5.202772857400123,
+    "http://127.0.0.1:8003": 55.37555769548635,
+    "http://127.0.0.1:8004": 17.031311958114394,
+    "http://127.0.0.1:8005": 25.48531596700202,
+    "http://127.0.0.1:8006": 36.31029207323453,
+    "http://127.0.0.1:8007": 2.4984901855932535
+  },
+  "status": "supported"
+}

analysis/characterization/window_1_results/per_worker_unified.json

@@ -0,0 +1,24 @@
+{
+  "hotspot_index_ttft_p90": 3.667136528736114,
+  "per_worker_latency_p90_s": {
+    "http://127.0.0.1:8000": 41.42001512600109,
+    "http://127.0.0.1:8001": 12.4878579101933,
+    "http://127.0.0.1:8002": 22.462878945574648,
+    "http://127.0.0.1:8003": 15.501050900109117,
+    "http://127.0.0.1:8004": 39.956250199786155,
+    "http://127.0.0.1:8005": 36.69850301651168,
+    "http://127.0.0.1:8006": 10.116177947795954,
+    "http://127.0.0.1:8007": 20.35038618039107
+  },
+  "per_worker_ttft_p90_s": {
+    "http://127.0.0.1:8000": 11.264844838529825,
+    "http://127.0.0.1:8001": 3.6063860427122614,
+    "http://127.0.0.1:8002": 16.175747957825664,
+    "http://127.0.0.1:8003": 9.314684258581842,
+    "http://127.0.0.1:8004": 37.73397144810297,
+    "http://127.0.0.1:8005": 18.328030522551852,
+    "http://127.0.0.1:8006": 3.6328767628350773,
+    "http://127.0.0.1:8007": 7.772977900883419
+  },
+  "status": "supported"
+}

analysis/characterization/window_1_results/raw_stats/lmetric.json

@@ -0,0 +1,23 @@
+{
+  "ttft": {
+    "count": 1214.0,
+    "mean": 5.111546324698484,
+    "p50": 0.9387824369769078,
+    "p90": 15.671339168207492,
+    "p99": 53.56683189840049
+  },
+  "tpot": {
+    "count": 1214.0,
+    "mean": 0.01757124870168204,
+    "p50": 0.008854518407308914,
+    "p90": 0.02122720699121469,
+    "p99": 0.18280341184277568
+  },
+  "e2e": {
+    "count": 1214.0,
+    "mean": 9.518126648903337,
+    "p50": 2.754255389008904,
+    "p90": 24.8209177934099,
+    "p99": 80.59924928059091
+  }
+}

analysis/characterization/window_1_results/raw_stats/load_only.json

@@ -0,0 +1,23 @@
+{
+  "ttft": {
+    "count": 1214.0,
+    "mean": 6.268620166597892,
+    "p50": 1.2609447415161412,
+    "p90": 20.197147866390882,
+    "p99": 52.84285237012196
+  },
+  "tpot": {
+    "count": 1214.0,
+    "mean": 0.02406975794215626,
+    "p50": 0.009231464695980247,
+    "p90": 0.026851662550158716,
+    "p99": 0.3211630676943426
+  },
+  "e2e": {
+    "count": 1214.0,
+    "mean": 11.702793988628443,
+    "p50": 3.58568156149704,
+    "p90": 33.459180271782685,
+    "p99": 93.95083751494239
+  }
+}

analysis/characterization/window_1_results/raw_stats/sticky.json

@@ -0,0 +1,23 @@
+{
+  "ttft": {
+    "count": 1214.0,
+    "mean": 5.55315460854824,
+    "p50": 0.5415176274836995,
+    "p90": 18.021296651283045,
+    "p99": 74.09429564891524
+  },
+  "tpot": {
+    "count": 1214.0,
+    "mean": 0.027834537397398284,
+    "p50": 0.008952101894096181,
+    "p90": 0.03641285916619554,
+    "p99": 0.35152006935195085
+  },
+  "e2e": {
+    "count": 1214.0,
+    "mean": 12.109200157184377,
+    "p50": 2.081947358994512,
+    "p90": 34.62592205510591,
+    "p99": 139.68334607904353
+  }
+}

analysis/characterization/window_1_results/raw_stats/unified.json

@@ -0,0 +1,23 @@
+{
+  "ttft": {
+    "count": 1213.0,
+    "mean": 3.2790960856202394,
+    "p50": 0.4997710260213353,
+    "p90": 7.345769894809922,
+    "p99": 42.34170345296613
+  },
+  "tpot": {
+    "count": 1213.0,
+    "mean": 0.012493800538265787,
+    "p50": 0.008079791456705824,
+    "p90": 0.017110194704198407,
+    "p99": 0.12655874612209597
+  },
+  "e2e": {
+    "count": 1213.0,
+    "mean": 6.961301470549104,
+    "p50": 1.7495028690318577,
+    "p90": 18.033410895219994,
+    "p99": 68.80023987947489
+  }
+}

analysis/characterization/window_1_results/summary.json

@@ -0,0 +1,136 @@
+{
+  "analyzed_records": 2114220,
+  "batch0": {
+    "attempted_requests": 2114220,
+    "completed_requests": null,
+    "error_requests": null,
+    "max_inflight_per_session": null,
+    "session_concurrency_status": "unavailable",
+    "session_sequential": null
+  },
+  "batch1": {
+    "append_status": "unavailable",
+    "input_stats": {
+      "count": 2114220,
+      "max": 202371.0,
+      "mean": 33637.38370084476,
+      "min": 0.0,
+      "p50": 20030.0,
+      "p90": 87855.1000000001,
+      "p95": 104738.0,
+      "p99": 125527.0
+    },
+    "kv_footprint_status": "available",
+    "output_stats": {
+      "count": 2114220,
+      "max": 132665.0,
+      "mean": 444.97059624826176,
+      "min": 0.0,
+      "p50": 80.0,
+      "p90": 811.0,
+      "p95": 2213.0,
+      "p99": 6614.810000000056
+    },
+    "reuse_status": "unavailable"
+  },
+  "classification": {
+    "label": "invalid_for_online_claim",
+    "reason": "actual dispatch/finish timestamps are unavailable, so online sequentiality cannot be proven",
+    "source": "auto",
+    "stress_indicators": []
+  },
+  "manifest": {
+    "canonical_trace_data_sources": {
+      "dash0_formatted_trace_dir": "~/ali-trace/trace-glm5.1-formatted/",
+      "dash0_raw_trace_dir": "~/ali-trace/trace-glm5.1/",
+      "usage_note": "Full trace analysis can be run CPU-only on dash0, or the needed JSONL files can be copied/rsynced from dash0 to this machine before running this analyzer."
+    },
+    "end_time": "2026-05-25T09:03:36.499002+00:00",
+    "figure_status": {
+      "reason": "matplotlib unavailable: ModuleNotFoundError(\"No module named 'matplotlib'\")",
+      "status": "skipped"
+    },
+    "git_commit": "",
+    "gpu_count": 0,
+    "gpu_type": "",
+    "host": "ds-6348bee4-1-765874c9c4-7zrvf",
+    "input_requirements": {
+      "actual_sequentiality_proof": [
+        "per-request dispatch timestamp",
+        "per-request finish or error/timeout timestamp",
+        "request_id join to trace/metrics when timing source is separate"
+      ],
+      "metrics_jsonl": [
+        "request_id",
+        "session_id",
+        "trace_timestamp_s",
+        "input_length",
+        "output_length",
+        "latency_s",
+        "ttft_s",
+        "tpot_s",
+        "error",
+        "optional cached_tokens"
+      ],
+      "reuse_decomposition": [
+        "cached_tokens or cache_hit",
+        "hash_ids",
+        "session_id"
+      ],
+      "trace_jsonl": [
+        "chat_id",
+        "parent_chat_id",
+        "timestamp",
+        "input_length",
+        "output_length",
+        "turn",
+        "hash_ids",
+        "optional session_id"
+      ]
+    },
+    "input_status": {
+      "analyzed_records": 2114220,
+      "breakdown_records": 0,
+      "merge_warnings": [],
+      "metrics_records": 0,
+      "trace_records": 2114220,
+      "trace_warnings": [],
+      "unmatched_breakdown": 0,
+      "unmatched_metrics": 0
+    },
+    "launch_command": "analysis/characterization/analyze.py --trace /home/admin/cpfs/wjh/ali-trace/trace-glm5.1-formatted/051315-051317.jsonl --kv-bytes-per-token 98304 --task-name full_trace_with_kv --output-root outputs/characterization --overwrite",
+    "output_dir": "outputs/characterization/2026-05-25/full_trace_with_kv",
+    "policy": "",
+    "request_limit": null,
+    "session_sampling_method": "",
+    "session_sequential": null,
+    "start_time": "2026-05-25T08:59:11.618919+00:00",
+    "time_scale": null,
+    "trace_file_info": {
+      "exists": true,
+      "mtime_s": 1778772033.2788928,
+      "path": "/home/admin/cpfs/wjh/ali-trace/trace-glm5.1-formatted/051315-051317.jsonl",
+      "sha256": "",
+      "sha256_status": "skipped_use_--hash-inputs",
+      "size_bytes": 1561266372
+    },
+    "trace_path": "/home/admin/cpfs/wjh/ali-trace/trace-glm5.1-formatted/051315-051317.jsonl",
+    "trace_sha256": ""
+  },
+  "outputs": [
+    "append_delta_stats.json",
+    "invalid_runs.md",
+    "kv_footprint_summary.json",
+    "manifest.json",
+    "raw/merged_requests.jsonl",
+    "raw/unmatched_breakdown.jsonl",
+    "raw/unmatched_metrics.jsonl",
+    "reuse_decomposition.json",
+    "session_arrival_stats.json",
+    "session_concurrency.json",
+    "session_skew.json",
+    "trace_profile.json",
+    "turn_interval_stats.json",
+    "workload_summary.json"
+  ]
+}

analysis/characterization_todo_for_interns.md

@@ -0,0 +1,684 @@
+# Agentic Workload Characterization TODO
+
+Status: execution checklist for interns
+Date: 2026-05-25
+Last progress audit: 2026-05-25
+
+## Progress Snapshot (2026-05-25, post-Window-1)
+
+| Batch | State | Evidence |
+|---|---|---|
+| B0 Substrate audit | **DONE for new runs**, legacy still partial | A1+A2 instrumentation lands per-request unix timestamps and X-Request-Id passthrough; B3 sweep 2026-05-25 achieves 100% join coverage on all 5 policy runs |
+| B1 Workload characterization | **DONE** | `window_1_results/kv_footprint_summary.json` (98304 B/token, p99 = 11.49 GiB); real reuse decomposition (`lmetric_reuse.json`: 93.2% intra-session, 5.7% cross, 1.1% shared); theoretical APC ceilings (`apc_upper_w600.json`: 79.6% intra / 80.3% any) |
+| B2 PD interference | **DONE** | `outputs/b2_microbench/sweep/` 5 × 2 cells. Different-worker control idx 0.92-1.02 across 32× prefill size variation; same-worker TTFT idx scales 2.15× → 218×. Causal proof complete. |
+| B3 5-policy routing sweep | **DONE** | `outputs/b3_sweep_20260525_095043/` lmetric/load_only/sticky (warm-cache) + unified/capped (isolated cold-start). Aggregated in `b3_policy_comparison.json`. Unified hits APC 79.4% (97% of ceiling) AND TTFT p90 7.24 s. |
+| B4 SRR sweep | NOT DONE | Window 2 task. A4 loadgen + per-class SLO + λ binary search per policy. |
+| B5 Failure attribution | NOT DONE | Window 2 task. Depends on B4 SRR boundaries. |
+| B6 Audit package | **DONE for Window 1** | `current_results/{characterization_claim_matrix.md, all_figures_index.md, reviewer_risk_register.md, main_claim_allowed_runs.md, reproduction_commands.sh}` refreshed; Window 1 results aggregated in `window_1_results.md` + 8 PNG figures |
+
+Reusable assets already in repo:
+
+- `analysis/characterization/analyze.py` — B0+B1 CPU-only analyzer
+- `analysis/characterization/summarize_runs.py` — existing-run audit producing the B6 scaffold
+- `analysis/characterization/plot_current_results.py` — figure regeneration script
+- `analysis/characterization/protocols.md` — B2–B6 protocol with required instrumentation, sweep, pass condition
+- `analysis/characterization/current_results/` — current audit package (claim matrix + risk register + allowed-runs gate + 6 PNG figures)
+
+Hard gates still blocking main claims:
+
+1. Replayer/proxy must emit per-request dispatch + finish/error wall-clock timestamps (blocks B0 actual sequentiality, B4 SRR validity).
+2. Per-request record must carry `session_id` + `hash_ids` + `cached_tokens` jointly (blocks B1 reuse decomposition).
+3. Engine/proxy must log decode-step and prefill-chunk timestamps with worker id (blocks B2 interference index).
+4. Proxy must log route decision, chosen worker, candidate scores, per-worker queue/KV/APC snapshot (blocks B3 hot-spot proof).
+
+
+## 0. Purpose
+
+We are not starting from the assumption that Unified routing or PUSH
+migration is already the answer.
+
+The first goal is to build a rigorous characterization package that proves:
+
+1. which dimensions make agentic serving different;
+2. where static PD-disaggregation works poorly;
+3. where PD-colocation/cache-aware routing still has residual failure modes;
+4. how these failure modes reduce sustainable request rate under SLO.
+
+Only after these facts are established should we refine the positive system
+design.
+
+Primary system goal:
+
+```text
+maximize sustainable request rate under SLO
+```
+
+Prefill-decode interference and session hot-spot imbalance are mechanisms
+that may reduce SRR. They are not the final metric by themselves.
+
+## 1. Global Delivery Rules
+
+Every task must produce data, figures, and an audit trail. A task is not
+complete if it only produces a written conclusion.
+
+Use this output layout:
+
+```text
+outputs/characterization/<date>/<task_name>/
+├── manifest.json
+├── raw/
+├── summary.json
+├── summary.md
+├── figures/
+└── audit.md
+```
+
+Required fields in `manifest.json`:
+
+```json
+{
+  "git_commit": "",
+  "host": "",
+  "gpu_type": "",
+  "gpu_count": 0,
+  "trace_path": "",
+  "trace_sha256": "",
+  "policy": "",
+  "launch_command": "",
+  "request_limit": null,
+  "time_scale": null,
+  "session_sampling_method": "",
+  "session_sequential": true,
+  "start_time": "",
+  "end_time": ""
+}
+```
+
+Every comparison must report:
+
+- attempted requests
+- completed requests
+- errors / timeouts
+- goodput
+- TTFT p50/p90/p99
+- E2E p50/p90/p99
+- TPOT p50/p90/p99
+- per-worker queue metrics
+- per-worker GPU utilization
+- per-worker KV occupancy if available
+- per-worker APC / cache-hit metrics
+
+Every figure must be reproducible from raw data by a script committed or
+saved alongside the artifact.
+
+## 2. Batch 0: Benchmark Substrate Audit
+
+Status: analyzer DONE (`analyze.py`); legacy-run sequentiality claim BLOCKED by missing dispatch/finish timestamps in `metrics.jsonl`. New replayer must add those fields before any `online_realistic` classification is allowed.
+
+### Goal
+
+Prove the load generator and trace replay are valid before trusting any
+performance result.
+
+The most important invariant:
+
+```text
+For online agentic serving, each session must have at most one in-flight turn.
+Turn N+1 must not be sent before turn N completes.
+```
+
+### TODO
+
+1. Implement or run an analyzer that reconstructs per-session request
+   intervals:
+   - dispatch timestamp
+   - first-token timestamp
+   - finish timestamp
+   - error / timeout timestamp
+2. Compute max concurrent in-flight turns per session.
+3. Compute session start-time distribution.
+4. Compute turn inter-arrival distribution.
+5. Classify each existing run as one of:
+   - `online_realistic`
+   - `burst_stress`
+   - `synthetic_microbench`
+   - `invalid_for_online_claim`
+6. For any run where session sequentiality is violated, write down exactly
+   which claim it can still support.
+
+### Data Artifacts
+
+- `session_concurrency.json`
+- `session_arrival_stats.json`
+- `turn_interval_stats.json`
+- `trace_profile.json`
+- `invalid_runs.md`
+
+### Figures
+
+- session start-time CDF
+- per-session max in-flight histogram
+- turns per session CDF
+- turn inter-arrival CDF
+
+### Audit Checks
+
+The `audit.md` must answer:
+
+1. Does the main trace satisfy `max_inflight_per_session == 1`?
+2. If not, is the run explicitly labeled as stress or invalid?
+3. Are attempted/completed/error counts included?
+4. Are latency percentiles computed only over successes, and if so, is
+   goodput also reported?
+
+### Pass Criteria
+
+- Main online-serving experiments must have `max_inflight_per_session == 1`.
+- Any violation must be clearly labeled and excluded from SRR claims.
+
+## 3. Batch 1: Workload Characterization
+
+Status: trace-shape items (1, 2, 3, 6, 8) DONE on full 7200 s GLM-5.1 trace; recorded in `current_results/full_trace_summary.json`. Items 4 (KV footprint), 5 (reuse decomposition), 7 (uncached append delta) are PENDING because they need `--kv-bytes-per-token` for the production model and joinable `cached_tokens`+`hash_ids` per request.
+
+### Goal
+
+Establish agentic workload facts independent of any proposed system.
+
+Required facts:
+
+1. long input, short output;
+2. large per-request KV footprint;
+3. reuse is mostly intra-session;
+4. session token mass is heavy-tailed;
+5. total prompt length and effective uncached prefill work are different.
+
+### TODO
+
+1. Compute input token CDF.
+2. Compute output token CDF.
+3. Compute input/output ratio.
+4. Estimate KV footprint per request:
+
+   ```text
+   kv_bytes_per_request = input_tokens * kv_bytes_per_token
+   ```
+
+5. Decompose reusable KV into:
+   - intra-session reuse
+   - cross-session reuse
+   - shared/system-prefix reuse
+6. Compute session-level skew:
+   - turns per session
+   - cumulative input tokens per session
+   - cumulative output tokens per session
+   - cumulative uncached tokens per session
+   - top-k session contribution
+7. Compute append / effective-prefill distribution:
+
+   ```text
+   uncached_tokens = input_tokens - cached_tokens
+   ```
+
+8. Compare total input length vs uncached tokens.
+
+### Data Artifacts
+
+- `workload_summary.json`
+- `kv_footprint_summary.json`
+- `reuse_decomposition.json`
+- `session_skew.json`
+- `append_delta_stats.json`
+
+### Figures
+
+- input/output token CDF
+- input/output ratio CDF
+- KV footprint CDF
+- reuse decomposition stacked bar
+- turns per session CDF
+- per-session token mass Lorenz curve
+- top-k sessions token contribution bar
+- total input vs uncached tokens scatter
+
+### Audit Checks
+
+The `audit.md` must answer:
+
+1. What are input p50/p90/p99?
+2. What are output p50/p90/p99?
+3. What is the estimated KV footprint p50/p90/p99?
+4. What fraction of reuse is intra-session?
+5. What fraction of total token mass comes from top 1% / 5% sessions?
+6. Are long prompts often small appends after cache reuse?
+
+### Pass Criteria
+
+The batch passes only if these facts can be stated numerically with raw data
+links and plotted figures.
+
+## 4. Batch 2: PD-Colo Prefill-Decode Interference Proof
+
+Status: protocol DONE (`analysis/characterization/protocols.md` §"Batch 2 Protocol"); execution NOT STARTED — needs new engine instrumentation for decode-step and prefill-chunk timestamps.
+
+### Goal
+
+Prove that PD-colocation can suffer from prefill-decode interference under
+high load, and quantify how much this affects TPOT, decode queueing, and SLO.
+
+Hypothesis:
+
+```text
+When heavy uncached prefill overlaps with active decode on the same worker,
+decode TPOT and/or decode queue delay increases.
+```
+
+### TODO
+
+1. Run controlled microbenchmarks:
+   - decode-only steady load;
+   - decode load plus same-worker heavy prefill injection;
+   - decode load plus different-worker heavy prefill injection.
+2. Sweep uncached prefill sizes:
+   - 2k
+   - 8k
+   - 16k
+   - 32k
+   - 64k
+3. If supported, sweep chunked prefill size.
+4. Log timestamps for:
+   - decode steps;
+   - prefill start/end;
+   - prefill chunks;
+   - queue admission;
+   - request completion.
+5. In trace replay, label decode steps by whether they overlap with
+   same-worker prefill.
+6. Compute:
+
+   ```text
+   interference_index =
+     TPOT_p90(decode steps overlapping same-worker prefill)
+     / TPOT_p90(decode steps without same-worker prefill)
+   ```
+
+7. Compare same-worker vs different-worker controls.
+
+### Data Artifacts
+
+- `interference_microbench_summary.json`
+- `decode_step_timeseries.csv`
+- `prefill_overlap_events.jsonl`
+- `interference_index.json`
+- `trace_overlap_summary.json`
+
+### Figures
+
+- TPOT time series with prefill overlap annotation
+- interference index vs uncached prefill size
+- same-worker vs different-worker TPOT boxplot
+- chunk size vs TTFT/TPOT tradeoff
+- trace replay overlap vs non-overlap TPOT comparison
+
+### Audit Checks
+
+The `audit.md` must answer:
+
+1. Is the interference observed on the same worker?
+2. Is the different-worker control significantly weaker?
+3. Does interference grow with uncached prefill size?
+4. Does the phenomenon appear in real trace replay, not only microbench?
+5. Could the result be explained by global load instead of local colocation?
+
+### Pass Criteria
+
+- Same-worker overlap must measurably increase TPOT or decode queue delay.
+- The effect must be weaker or absent in the different-worker control.
+- The effect must be visible in at least one trace replay setting.
+
+## 5. Batch 3: Session Hot-Spot Residual Imbalance Proof
+
+Status: protocol DONE; partial signal from legacy `gpu_util.csv` (GPU-util imbalance visible) but causal proof NOT STARTED — needs per-worker queue/KV/APC and session→worker map from instrumented proxy.
+
+### Goal
+
+Prove that cache-aware/LMetric is a strong baseline but still leaves residual
+hot-worker imbalance due to session skew and locality.
+
+Hypothesis:
+
+```text
+Cache-aware routing preserves locality by attracting future turns to cached
+workers. This is usually good, but heavy-tailed sessions can create hot
+workers whose queue delay/SLO violations are much worse than the median
+worker even when other workers still have headroom.
+```
+
+### TODO
+
+1. Run the same session-causal trace with:
+   - corrected LMetric/cache-aware;
+   - load-only routing;
+   - hard sticky routing;
+   - current Unified hybrid, if available.
+2. For each worker, record:
+   - assigned session count;
+   - cumulative input tokens;
+   - cumulative uncached tokens;
+   - cumulative output tokens;
+   - request queue delay;
+   - decode queue delay;
+   - GPU utilization;
+   - KV occupancy;
+   - APC / cache-hit rate;
+   - SLO violations.
+3. For each session, record:
+   - worker set used;
+   - primary worker;
+   - cumulative token mass;
+   - number of turns;
+   - latency contribution;
+   - whether it appears in slow-request set.
+4. Create a session-mass capped or equalized replay:
+   - cap max session turns or token mass;
+   - rerun LMetric/cache-aware;
+   - compare hot-spot index.
+5. Compute:
+
+   ```text
+   hotspot_index =
+     max_worker_queue_delay_p90 / median_worker_queue_delay_p90
+   ```
+
+6. Compute locality/load tradeoff:
+
+   ```text
+   locality_gain = APC(policy) - APC(load_only)
+   imbalance_cost =
+     max_worker_latency_p90(policy) - median_worker_latency_p90(policy)
+   ```
+
+### Data Artifacts
+
+- `worker_balance_summary.json`
+- `session_to_worker_map.json`
+- `session_mass_summary.json`
+- `routing_policy_comparison.json`
+- `hotspot_index.json`
+- `capped_session_replay_summary.json`
+
+### Figures
+
+- per-worker queue delay bar
+- per-worker token mass bar
+- GPU utilization timeline by worker
+- KV occupancy timeline by worker
+- APC vs queue delay scatter
+- top sessions contribution bar
+- policy tradeoff plot: APC vs hotspot_index
+- original vs session-capped hot-spot comparison
+
+### Audit Checks
+
+The `audit.md` must answer:
+
+1. Does LMetric/cache-aware still show worker-level skew?
+2. Are SLO violations concentrated on hot workers or hot sessions?
+3. Does load-only routing improve balance but reduce APC/locality?
+4. Does hard sticky improve locality but worsen hot-spot/HOL?
+5. Does session-mass capping reduce hot spots?
+
+### Pass Criteria
+
+- LMetric/cache-aware must be shown as strong but imperfect.
+- There must be measurable residual hot-worker imbalance.
+- The imbalance must correlate with session token mass or locality.
+
+## 6. Batch 4: Sustainable Request Rate Sweep
+
+Status: protocol DONE; execution NOT STARTED — requires open-loop session-causal loadgen and policy-comparable arrival process.
+
+### Goal
+
+Connect interference and hot-spot mechanisms to the final metric:
+
+```text
+SRR(SLO) = max arrival rate satisfying SLO in steady state
+```
+
+### TODO
+
+1. Define provisional SLO thresholds. Use configurable values, for example:
+
+   ```text
+   TTFT_p90 <= T_ttft
+   E2E_p90  <= T_e2e
+   TPOT_p90 <= T_tpot
+   error_rate <= epsilon
+   queue length stable
+   KV occupancy stable
+   ```
+
+2. Implement arrival-rate sweep:
+   - Poisson session arrivals;
+   - session-internal sequentiality;
+   - warmup window;
+   - steady-state measurement window.
+3. For each arrival rate `lambda`, run:
+   - PD-colo cache-aware/LMetric;
+   - static PD-disagg;
+   - current Unified hybrid;
+   - optional hard sticky;
+   - optional load-only.
+4. Find maximum sustainable lambda for each policy.
+5. Report instability reasons:
+   - SLO violation;
+   - queue growth;
+   - KV occupancy growth;
+   - error/timeout growth.
+
+### Data Artifacts
+
+- `srr_curve.json`
+- `lambda_runs/<lambda>/summary.json`
+- `slo_violation_reason.json`
+- `goodput_vs_arrival_rate.json`
+- `stability_summary.json`
+
+### Figures
+
+- SRR bar chart
+- TTFT p90 vs arrival rate
+- E2E p90 vs arrival rate
+- TPOT p90 vs arrival rate
+- goodput vs arrival rate
+- error rate vs arrival rate
+- queue length over time near failure point
+- KV occupancy over time near failure point
+
+### Audit Checks
+
+The `audit.md` must answer:
+
+1. Are session arrivals open-loop and Poisson?
+2. Is session-internal sequentiality enforced?
+3. How long are warmup and steady-state windows?
+4. Is SRR failure persistent rather than transient?
+5. Are completed/requested counts reported at every lambda?
+6. Are policies compared on the same trace and same arrival process?
+
+### Pass Criteria
+
+- Each policy must have a measured SRR under the same SLO.
+- Failure must be attributed to persistent SLO violation, queue growth, KV
+  growth, or error growth.
+- Data must be session-causal.
+
+## 7. Batch 5: Failure Attribution Near SRR Boundary
+
+Status: protocol DONE; execution NOT STARTED — depends on B2 instrumentation and B4 SRR boundary.
+
+### Goal
+
+At and around the PD-colo/LMetric failure point, determine whether SLO
+violations are caused by prefill-decode interference, session hot spots, KV
+pressure, cache misses, or other mechanisms.
+
+### TODO
+
+1. Select three arrival rates:
+
+   ```text
+   lambda = 0.9 * SRR
+   lambda = 1.0 * SRR
+   lambda = 1.1 * SRR
+   ```
+
+2. For every slow or SLO-violating request, assign labels:
+   - same-worker prefill overlap;
+   - hot worker queue;
+   - high KV occupancy;
+   - cache miss / large uncached append;
+   - transfer wait;
+   - P queue wait;
+   - D admission wait;
+   - unknown.
+3. Produce per-request waterfall for representative slow requests.
+4. Produce per-worker timeline around failure windows.
+5. Summarize cause distribution.
+
+### Data Artifacts
+
+- `slow_request_attribution.jsonl`
+- `failure_breakdown.json`
+- `case_studies.md`
+- `worker_failure_windows.json`
+
+### Figures
+
+- SLO violation cause stacked bar
+- slow request waterfall
+- worker timeline near failure
+- prefill/decode/KV/queue stacked breakdown
+- failure cause vs arrival rate
+
+### Audit Checks
+
+The `audit.md` must answer:
+
+1. What fraction of slow requests overlap same-worker prefill?
+2. What fraction are on hot workers?
+3. What fraction happen under high KV occupancy?
+4. What fraction are large uncached append requests?
+5. For PD-disagg/Unified migration, how much time is transfer/P queue/D wait?
+6. What remains unexplained?
+
+### Pass Criteria
+
+The batch must answer:
+
+1. Why PD-colo/LMetric hits its SRR limit.
+2. Why static PD-disagg hits its SRR limit.
+3. If Unified/PUSH underperforms, whether the cause is trigger quality, cost
+   model, transfer overhead, wrong load regime, or something else.
+
+## 8. Batch 6: Audit Package
+
+Status: scaffold DONE — all five final artifacts exist under `analysis/characterization/current_results/` and are regenerated by `summarize_runs.py` + `plot_current_results.py`. Future B2–B5 outputs must be merged into the same package by re-running `summarize_runs.py` after new runs.
+
+### Goal
+
+Make the whole characterization package reviewable by a strict systems
+reviewer.
+
+### TODO
+
+1. Write a claim matrix:
+
+   ```text
+   claim -> data artifact -> figure -> script -> caveat -> reviewer risk
+   ```
+
+2. Write a figure index:
+   - figure filename;
+   - source data;
+   - generation command;
+   - intended claim.
+3. Write a reviewer risk register:
+   - loadgen validity risks;
+   - trace representativeness risks;
+   - metric bias risks;
+   - implementation-specific risks;
+   - generalization risks.
+4. Write a reproduction script or command list.
+5. Mark experiments that cannot support main claims.
+
+### Final Artifacts
+
+- `characterization_claim_matrix.md`
+- `all_figures_index.md`
+- `reviewer_risk_register.md`
+- `reproduction_commands.sh`
+- `main_claim_allowed_runs.md`
+
+### Audit Checks
+
+The final package must satisfy:
+
+1. Every claim links to raw data.
+2. Every figure can be regenerated.
+3. Every experiment has a manifest.
+4. Every caveat is explicit.
+5. Invalid or stress-only runs are not used for online-serving claims.
+
+## 9. Priority Order
+
+### Priority 1
+
+Do these first:
+
+1. Batch 0: Benchmark Substrate Audit
+2. Batch 1: Workload Characterization
+3. Batch 3: Session Hot-Spot Residual Imbalance Proof
+
+Reason:
+
+These define whether the trace and routing problem are real. Without them,
+SRR sweeps and system experiments are not trustworthy.
+
+### Priority 2
+
+Do these after the substrate and workload facts are stable:
+
+1. Batch 2: PD-Colo Prefill-Decode Interference Proof
+2. Batch 5: Failure Attribution Near SRR Boundary
+
+Reason:
+
+These explain the mechanisms behind SLO/SRR failure and determine what the
+positive system should actually fix.
+
+### Priority 3
+
+Do these after instrumentation and attribution are ready:
+
+1. Batch 4: Sustainable Request Rate Sweep
+2. Batch 6: Audit Package
+
+Reason:
+
+SRR sweeps are expensive. They should run only after trace validity,
+logging, and attribution labels are ready.
+
+## 10. Non-Negotiable Reviewer Rules
+
+1. Do not use session-nonsequential loadgen for online-serving claims.
+2. Do not compare latency percentiles without attempted/completed/error counts.
+3. Do not use APC alone as a success metric.
+4. Do not use average GPU utilization as proof of load balance.
+5. Do not compare policies on different traces unless explicitly labeled.
+6. Do not hide failed requests or timeouts.
+7. Do not claim Unified/PUSH is the answer before failure attribution proves
+   the relevant bottleneck and cost budget.
+8. Treat corrected LMetric/cache-aware PD-colo as the main baseline.
+9. Treat static PD-disagg as an important baseline, not a strawman.
+10. Every result must be reproducible from raw artifacts and commands.

analysis/claude_characterization_work_plan.md

@@ -0,0 +1,360 @@
+# Claude Characterization Work Plan
+
+Status: planning, awaiting dash0 idle
+Date: 2026-05-25
+Owner: Claude (not interns)
+Source of requirements: `analysis/characterization_todo_for_interns.md`
+
+## Scope
+
+This plan covers the four hard gates and the B2–B5 GPU experiments that the
+intern TODO marks as `NOT DONE` / `protocol DONE`. The B0 analyzer, the
+B1 trace-shape statistics, and the B6 audit scaffold are already done; this
+plan does **not** re-do them, only refreshes their inputs.
+
+The work is split into:
+
+- **Phase A (CPU-only)** — instrumentation + analyzer extensions. Can run
+  on the local dev box; does **not** need dash0. Must finish before any
+  GPU run.
+- **Phase B (dash0 GPU)** — controlled microbench + routing sweep + SRR
+  sweep + failure attribution.
+- **Phase C (CPU-only)** — final audit package refresh.
+
+## Phase A: Instrumentation + Analyzer (CPU-only, before dash0)
+
+### A1. Replayer instrumentation — close Gate 1 + Gate 2
+
+File: `replayer/metrics.py`, `replayer/replay.py`
+
+Add these fields to `RequestMetrics`:
+
+```text
+t_dispatch_unix     float   # absolute wall-clock when POST starts
+t_first_token_unix  float   # absolute wall-clock at first stream chunk
+t_finish_unix       float   # absolute wall-clock at stream done or error
+proxy_request_id    str     # value sent in X-Request-Id (matches breakdown)
+endpoint_url        str     # which proxy/instance the request hit
+trace_hash_ids      list[int]  # carried from trace for reuse joins
+```
+
+Change `_dispatch_request` to:
+
+- send a deterministic `X-Request-Id: <session_id>:<turn_id>` header (so
+  proxy breakdown can be joined to metrics by exact key);
+- record `time.time()` (unix) at dispatch, first token, finish; keep
+  `perf_counter` for the latency arithmetic.
+
+Acceptance: a 30-request smoke run produces `metrics.jsonl` where every
+row has those fields; `breakdown.json` rows from the proxy have the same
+`request_id` keys.
+
+Effort: 1 small PR. Pure CPU.
+
+### A2. Proxy instrumentation — close Gate 1 + Gate 3 + Gate 4
+
+File: `scripts/cache_aware_proxy.py`
+
+Changes:
+
+1. Honor incoming `X-Request-Id`: if header present, use it instead of
+   generating a new uuid. Falls back to uuid otherwise.
+2. Record on every breakdown row:
+   - `session_id` (already on header, not currently stored)
+   - `input_length`
+   - `estimated_new_tokens` (already produced by router)
+   - `candidate_scores` (list of `{url, p_tokens_score, cache_score, bs,
+     occupancy}`)
+   - `chosen_score`
+3. At route decision time, snapshot per-worker state:
+   - `pending_prefill_tokens` per worker
+   - `running_decode_requests` per worker
+   - `kv_blocks_used` / `kv_blocks_total` per worker
+   - `apc_hits` / `apc_queries` cumulative per worker
+   Write to a separate `worker_state.jsonl` (one line per route decision)
+   with `(t_decision_unix, request_id, per_worker_state)`.
+4. New endpoint `GET /worker_state` returns the latest snapshot per worker
+   (for sanity / live debugging).
+
+Acceptance: smoke run produces `breakdown.json` with new fields and a
+non-empty `worker_state.jsonl` that joins to breakdown by `request_id`.
+
+Effort: 1 medium PR. Pure CPU + light proxy work.
+
+### A3. Engine-side step timestamps — close Gate 3 for B2
+
+vLLM 0.18.1 already exposes:
+
+- `vllm:request_prefill_time_seconds` (histogram, per-request)
+- `vllm:request_decode_time_seconds`
+- `vllm:time_per_output_token_seconds`
+- step-level scheduler stats via `engine.async_step` logging
+
+For B2 we need decode-step and prefill-chunk timestamps with worker id.
+Plan:
+
+1. Inspect whether the vLLM proxy can be polled at high rate (e.g.
+   100 Hz) for per-engine scheduler counters
+   (`num_running`, `num_waiting`, `gpu_cache_usage`,
+   `prefix_cache_queries`, `prefix_cache_hits`). If yes, sample
+   into `engine_state.jsonl` during runs.
+2. If finer step-level data is needed, patch one vLLM file
+   (`vllm/engine/async_llm_engine.py` step loop or
+   `vllm/v1/core/sched/scheduler.py`) to emit a JSONL line per
+   scheduler step with `(t_unix, worker_id, num_prefill_tokens_scheduled,
+   num_decode_steps, running_request_ids)`. Patch goes under `patches/`
+   so it can be applied/reverted cleanly.
+3. Worker id mapping: when running TP1xDP8 or similar, each engine
+   listens on a distinct port; `worker_id == endpoint_url`.
+
+Acceptance: a single 10-minute run produces `engine_state.jsonl` from
+which a decode step at time T on worker W can be classified as
+"overlapping a same-worker prefill chunk" or not.
+
+Effort: 1 medium investigation (decide poll vs patch) + 1 medium PR.
+
+### A4. Open-loop session-causal loadgen for B4
+
+File: `replayer/replay.py` (new mode) or new `replayer/srr_loadgen.py`
+
+Current replayer dispatches by trace timestamps. SRR sweep needs:
+
+- pool of session templates (each = ordered list of turns from the
+  trace);
+- Poisson arrivals of new sessions at rate `lambda`;
+- within a session: strict sequentiality (turn N+1 waits for turn N
+  finish);
+- per-run warmup window (e.g. 60s) + steady-state window (e.g. 300s);
+- attempted / completed / error counters per window.
+
+Add a new mode `--mode srr --arrival-rate <lambda>
+--warmup-s 60 --steady-s 300 --session-pool-size N`. The trace
+file becomes the pool; sessions are drawn with replacement.
+
+Acceptance: at `lambda = 0.5 sess/s`, the run shows exponential inter-
+arrival times and per-session sequentiality in `metrics.jsonl`. A
+`window_summary.json` lists warmup vs steady-state attempted/completed.
+
+Effort: 1 medium PR.
+
+### A5. Analyzer extensions
+
+File: `analysis/characterization/analyze.py` (extend, do not rewrite)
+
+Add:
+
+1. **Joined-record builder.** Given `--metrics metrics.jsonl
+   --breakdown breakdown.json --worker-state worker_state.jsonl
+   --engine-state engine_state.jsonl`, produce
+   `joined.jsonl` keyed on `request_id` with all fields merged.
+2. **Reuse decomposition (real).** Using joined records that carry
+   `session_id` + `hash_ids` + `cached_tokens`, compute
+   `intra_session` / `cross_session` / `shared_prefix` /
+   `unclassified` cached-token mass. Replaces the current
+   `status: unavailable` placeholder when fields are present.
+3. **Interference index.** Per decode step, label "overlap same-
+   worker prefill" using `engine_state.jsonl`. Compute
+   `TPOT_p90(overlap) / TPOT_p90(no_overlap)`.
+4. **Hotspot index.** Per worker queue delay p90, output
+   `max_worker_q_p90 / median_worker_q_p90`.
+5. **Failure label.** For each slow / SLO-violating request, assign
+   one of: `same_worker_prefill_overlap`, `hot_worker_queue`,
+   `high_kv_occupancy`, `cache_miss_large_append`, `transfer_wait`,
+   `p_queue_wait`, `d_admission_wait`, `unknown`.
+6. **Window summary.** For SRR runs, compute attempted/completed/
+   error/goodput plus latency percentiles on the steady-state
+   window only.
+
+Acceptance: re-run analyzer on smoke output and confirm `reuse_decomposition`
+no longer says `unavailable`; `interference_index.json` produced when
+engine state present; `failure_breakdown.json` populated when
+labels assigned.
+
+Effort: 1 large PR. CPU-only.
+
+## Phase B: GPU experiments (needs dash0)
+
+### B1' Workload characterization closure
+
+Inputs: instrumented replayer + small smoke trace (≤500 req).
+
+Steps:
+
+1. Pick `kv_bytes_per_token` for the production model. For
+   Qwen3-Coder TP1 the value depends on layer/head config; compute
+   from `vllm.config` once at run start and record in manifest.
+2. Re-run analyzer on full GLM-5.1 trace with `--kv-bytes-per-token`.
+   Output: KV footprint p50/p90/p99 in `kv_footprint_summary.json`.
+3. Run a 1k-request session-causal smoke replay with instrumented
+   proxy. Use the joined records to populate real reuse decomposition
+   for the small sample. (Full-trace replay is too expensive; sample
+   is acceptable for the decomposition claim.)
+
+Wall-clock: ~30 min GPU. Produces 2 figures: KV footprint CDF, reuse
+decomposition stacked bar.
+
+### B2 PD-colo interference microbench
+
+Setup: 1 combined instance on TP1. Two synthetic load generators:
+
+1. **Decode-only steady load** — short-prompt sessions at fixed
+   per-second arrival, designed to saturate decode without prefill
+   contention.
+2. **Prefill injector** — single-shot long-prompt requests at
+   controlled cadence; same worker (target the decode worker) vs
+   different worker (route to a paired idle instance).
+
+Sweep `uncached_prefill_tokens ∈ {2k, 8k, 16k, 32k, 64k}` × `{same,
+different} worker`.
+
+Outputs: `interference_microbench_summary.json`,
+`decode_step_timeseries.csv` (from `engine_state.jsonl`),
+`prefill_overlap_events.jsonl`, `interference_index.json`,
+TPOT-with-overlay figure, interference-index-vs-prefill-size figure.
+
+Wall-clock: ~2–3 h GPU including warm-up between sweeps.
+
+### B3 Routing sweep on session-causal trace
+
+Setup: 8 combined instances (TP1 × DP8) with the cache-aware proxy.
+
+Run the same session-causal trace (e.g. r=0.0015 st=30 850-req config
+from auto-mem `feedback-bench-config.md`) under five policies:
+
+1. corrected LMetric / cache-aware (`--policy lmetric`)
+2. load-only (new policy `--policy load_only` — picks min running)
+3. hard sticky (new policy `--policy sticky` — once a session lands
+   on a worker, never moves)
+4. current Unified hybrid (`--policy unified`)
+5. session-mass capped replay (filter the trace so no session exceeds
+   `cap_turns` or `cap_input_tokens`; rerun policy 1)
+
+Per run, collect: replayer metrics, proxy breakdown, worker_state,
+engine_state. Compute per-worker queue delay, GPU util, KV occupancy,
+APC, session-to-worker map.
+
+Outputs: `worker_balance_summary.json`, `session_to_worker_map.json`,
+`session_mass_summary.json`, `routing_policy_comparison.json`,
+`hotspot_index.json`, `capped_session_replay_summary.json`,
+8 figures from the TODO list (§5.figures).
+
+Wall-clock: 5 runs × ~13 min ≈ 1.5 h GPU.
+
+Implementation note: `load_only` and `sticky` are small additions to
+`scripts/cache_aware_proxy.py` — they reuse existing affinity / score
+machinery.
+
+### B4 Sustainable Request Rate sweep
+
+Setup: same 8 instances. Use Phase-A `--mode srr` loadgen.
+
+SLO (locked per-class):
+
+```text
+TTFT_p90 <= 2.0 s
+TPOT_p90 <= 0.15 s
+error_rate <= 0.5%
+queue length stable (no monotone growth over steady window)
+KV occupancy stable
+E2E_p90 <= T_class[c]  for each output-length decile c
+```
+
+`T_class[c]` is derived from a low-load reference run as
+`E2E_p90_low_load(c) * 2` (factor configurable). The reference run
+is done once and cached as `analysis/characterization/srr/slo_classes.json`.
+
+Per policy sweep `lambda` from low (clearly safe) to high (clearly
+broken) using a bisection-ish search:
+
+```
+λ_low  = 0.1 sess/s
+λ_high = doubling until first SLO violation
+binary-search λ_low .. λ_high for max sustainable λ
+```
+
+Policies covered: LMetric, static PD-disagg, Unified, hard sticky,
+load-only.
+
+Outputs: `srr_curve.json`, `lambda_runs/<lambda>/summary.json`,
+`slo_violation_reason.json`, `goodput_vs_arrival_rate.json`,
+`stability_summary.json`, all 8 figures from §6.figures.
+
+Wall-clock: this is the most expensive batch. With binary search,
+~6 lambda points × 5 policies × ~8 min (warmup + steady) ≈ 4 h GPU.
+
+### B5 Failure attribution near SRR boundary
+
+For each policy: pick `λ ∈ {0.9, 1.0, 1.1} × SRR`, run with full
+instrumentation, then run the analyzer's failure-label step.
+
+Outputs: `slow_request_attribution.jsonl`, `failure_breakdown.json`,
+`case_studies.md`, `worker_failure_windows.json`, 5 figures from §7.
+
+Wall-clock: 3 lambdas × 5 policies × 8 min ≈ 2 h GPU.
+
+## Phase C: Audit package refresh (CPU)
+
+Re-run `summarize_runs.py` and `plot_current_results.py` after each
+GPU batch. Final pass after B5: refresh `claim_matrix`, `risk_register`,
+`allowed_runs`, regenerate all figures, update
+`reproduction_commands.sh`.
+
+Effort: ~1 h CPU.
+
+## Sequencing & rough timeline
+
+```text
+Phase A (CPU, before dash0):
+  A1 + A2     (parallel)               ~half day CPU
+  A3 patch    (scheduler.py)           ~half day CPU
+  A4 SRR loadgen                       ~half day CPU
+  A5 analyzer extensions               ~1 day   CPU
+
+Window 1 on dash0 (B2 + B3 only, ~5 h GPU):
+  smoke validation of A1–A4            ~30 min GPU
+  B1' KV footprint + reuse decomp      ~30 min GPU
+  B2 interference microbench           ~3 h    GPU
+  B3 routing sweep (5 policies)        ~1.5 h  GPU
+  Phase C partial refresh              ~30 min CPU
+  ── HARD STOP, hand results back ──
+
+Window 2 on dash0 (B4 + B5, ~6 h GPU, only after review):
+  B4 SRR sweep (5 policies × bisect)   ~4 h    GPU
+  B5 failure attribution               ~2 h    GPU
+  Phase C final refresh                ~1 h    CPU
+```
+
+## Decisions (locked 2026-05-25)
+
+1. **Target model**: Qwen3-Coder-30B-A3B. Compute
+   `kv_bytes_per_token` from this model's config at manifest time.
+2. **GPU topology**: TP1 × 8 vLLM instances (DP8). All proxies and
+   sweeps assume 8 worker endpoints.
+3. **Trace for B3/B4**: `traces/w600_r0.0015_st30.jsonl` (~850
+   requests). No resampling.
+4. **E2E SLO**: per-class. Split requests by `requested_output_tokens`
+   decile, set separate E2E thresholds per class. No normalized-E2E
+   headline.
+5. **vLLM scheduler patch**: accepted. Step-level JSONL log goes
+   through a patch under `patches/`. Polling falls back to per-engine
+   `/metrics` for sanity only.
+6. **GPU phasing**: hard stop after B2 and B3. Hand results back for
+   review before committing to B4 SRR sweep or B5 attribution.
+
+## What stays with the interns
+
+- Re-running `summarize_runs.py` after each GPU batch (mechanical).
+- Reviewing the auto-generated `current_results.md` for typos.
+- Maintaining `main_claim_allowed_runs.md` if new traces are added.
+- Anything reading the audit package — not extending it.
+
+## Out of scope for this plan
+
+- New routing policy design (Unified-v2 / PUSH variants).
+- Production-grade KV transfer engineering.
+- Any change to the production paper figures in
+  `analysis/pd_sep_paper_section/`.
+- vLLM upstream contributions.
+
+These are downstream of characterization; once B2/B3/B5 attribution is
+in, we decide separately.

analysis/crossover/d1_i1024_goodput.json

@@ -0,0 +1,142 @@
+{
+  "baseline": "8C-proxy",
+  "arms": {
+    "8C-proxy": {
+      "name": "8C-proxy",
+      "n_offered": 1167,
+      "n_success": 1167,
+      "completion_rate": 1.0,
+      "offered_window_s": 299.736197,
+      "offered_qps": 3.8934236561358655,
+      "wall_clock_s": 300.1385939740576,
+      "amplification": 1.0013425037685975,
+      "ttft": {
+        "count": 1167,
+        "mean": 0.08144469193556772,
+        "p50": 0.07862715201918036,
+        "p90": 0.08015060934703797,
+        "p99": 0.0875979653932154
+      },
+      "tpot": {
+        "count": 1167,
+        "mean": 0.005001699398049616,
+        "p50": 0.004988961030788246,
+        "p90": 0.005045765990923557,
+        "p99": 0.005062779263327164
+      },
+      "e2e": {
+        "count": 1167,
+        "mean": 0.3968209869372152,
+        "p50": 0.393534954986535,
+        "p90": 0.39730903680901974,
+        "p99": 0.40925762055674536
+      }
+    },
+    "4P+4D": {
+      "name": "4P+4D",
+      "n_offered": 1167,
+      "n_success": 1167,
+      "completion_rate": 1.0,
+      "offered_window_s": 299.736197,
+      "offered_qps": 3.8934236561358655,
+      "wall_clock_s": 300.1604231200181,
+      "amplification": 1.0014153316291596,
+      "ttft": {
+        "count": 1167,
+        "mean": 0.09946277569807849,
+        "p50": 0.09600010397844017,
+        "p90": 0.10452785079833121,
+        "p99": 0.11205230774357905
+      },
+      "tpot": {
+        "count": 1167,
+        "mean": 0.005007447102661814,
+        "p50": 0.004987124730611131,
+        "p90": 0.005003212126977151,
+        "p99": 0.005478902989961502
+      },
+      "e2e": {
+        "count": 1167,
+        "mean": 0.415208436744531,
+        "p50": 0.41056320699863136,
+        "p90": 0.4200975856045261,
+        "p99": 0.44871115096379066
+      }
+    },
+    "6P+2D": {
+      "name": "6P+2D",
+      "n_offered": 1167,
+      "n_success": 1167,
+      "completion_rate": 1.0,
+      "offered_window_s": 299.736197,
+      "offered_qps": 3.8934236561358655,
+      "wall_clock_s": 300.2032543020323,
+      "amplification": 1.0015582278907484,
+      "ttft": {
+        "count": 1167,
+        "mean": 0.10561635944505095,
+        "p50": 0.10468761203810573,
+        "p90": 0.11257308297790587,
+        "p99": 0.12065987563692024
+      },
+      "tpot": {
+        "count": 1167,
+        "mean": 0.005328901365752947,
+        "p50": 0.005144592110318915,
+        "p90": 0.005990574603515958,
+        "p99": 0.006688486758013448
+      },
+      "e2e": {
+        "count": 1167,
+        "mean": 0.4416300980896939,
+        "p50": 0.42991435900330544,
+        "p90": 0.4854830394731835,
+        "p99": 0.5404117306252005
+      }
+    }
+  },
+  "slo_grid": [
+    {
+      "ttft_slo_s": 2.0,
+      "tpot_slo_s": 0.05,
+      "arms": {
+        "8C-proxy": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 1167
+        },
+        "4P+4D": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 1167
+        },
+        "6P+2D": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 1167
+        }
+      }
+    },
+    {
+      "ttft_slo_s": 5.0,
+      "tpot_slo_s": 0.05,
+      "arms": {
+        "8C-proxy": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 1167
+        },
+        "4P+4D": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 1167
+        },
+        "6P+2D": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 1167
+        }
+      }
+    }
+  ]
+}

analysis/crossover/d1_i16384_goodput.json

@@ -0,0 +1,142 @@
+{
+  "baseline": "8C-proxy",
+  "arms": {
+    "8C-proxy": {
+      "name": "8C-proxy",
+      "n_offered": 1167,
+      "n_success": 1167,
+      "completion_rate": 1.0,
+      "offered_window_s": 299.736197,
+      "offered_qps": 3.8934236561358655,
+      "wall_clock_s": 303.05708941399644,
+      "amplification": 1.0110793839624128,
+      "ttft": {
+        "count": 1167,
+        "mean": 1.674437926279444,
+        "p50": 1.5353219069947954,
+        "p90": 2.0787689138029237,
+        "p99": 3.039117059087727
+      },
+      "tpot": {
+        "count": 1167,
+        "mean": 0.035498316425319934,
+        "p50": 0.02951206674587743,
+        "p90": 0.085249871320677,
+        "p99": 0.15422643764865662
+      },
+      "e2e": {
+        "count": 1167,
+        "mean": 3.9111985703531236,
+        "p50": 3.392241114997887,
+        "p90": 7.760864628604043,
+        "p99": 11.30427318874542
+      }
+    },
+    "4P+4D": {
+      "name": "4P+4D",
+      "n_offered": 1167,
+      "n_success": 1136,
+      "completion_rate": 0.9734361610968295,
+      "offered_window_s": 299.736197,
+      "offered_qps": 3.8934236561358655,
+      "wall_clock_s": 866.9596620649972,
+      "amplification": 2.8924089607535697,
+      "ttft": {
+        "count": 1136,
+        "mean": 65.09021699308856,
+        "p50": 66.22900710900285,
+        "p90": 112.5535424454938,
+        "p99": 124.55262411334482
+      },
+      "tpot": {
+        "count": 1136,
+        "mean": 0.005710658520121912,
+        "p50": 0.005725543936557461,
+        "p90": 0.005750613698356098,
+        "p99": 0.0058447879207267845
+      },
+      "e2e": {
+        "count": 1136,
+        "mean": 65.4504681098121,
+        "p50": 66.59053339700768,
+        "p90": 112.9150809329949,
+        "p99": 124.91415351489852
+      }
+    },
+    "6P+2D": {
+      "name": "6P+2D",
+      "n_offered": 1167,
+      "n_success": 1167,
+      "completion_rate": 1.0,
+      "offered_window_s": 299.736197,
+      "offered_qps": 3.8934236561358655,
+      "wall_clock_s": 307.42712411200046,
+      "amplification": 1.0256589867656205,
+      "ttft": {
+        "count": 1167,
+        "mean": 3.6233640342625417,
+        "p50": 3.255483777000336,
+        "p90": 6.0935156565916255,
+        "p99": 7.349482456580735
+      },
+      "tpot": {
+        "count": 1167,
+        "mean": 0.006360297526341433,
+        "p50": 0.006324973206372104,
+        "p90": 0.007198417158741947,
+        "p99": 0.007942749238420567
+      },
+      "e2e": {
+        "count": 1167,
+        "mean": 4.024414356593621,
+        "p50": 3.6796783979953034,
+        "p90": 6.510242249601289,
+        "p99": 7.7530036393977895
+      }
+    }
+  },
+  "slo_grid": [
+    {
+      "ttft_slo_s": 2.0,
+      "tpot_slo_s": 0.05,
+      "arms": {
+        "8C-proxy": {
+          "attainment": 0.6563838903170522,
+          "pd_advantage": 1.0,
+          "n_slo": 766
+        },
+        "4P+4D": {
+          "attainment": 0.0,
+          "pd_advantage": 0.0,
+          "n_slo": 0
+        },
+        "6P+2D": {
+          "attainment": 0.20565552699228792,
+          "pd_advantage": 0.3133159268929504,
+          "n_slo": 240
+        }
+      }
+    },
+    {
+      "ttft_slo_s": 5.0,
+      "tpot_slo_s": 0.05,
+      "arms": {
+        "8C-proxy": {
+          "attainment": 0.7095115681233933,
+          "pd_advantage": 1.0,
+          "n_slo": 828
+        },
+        "4P+4D": {
+          "attainment": 0.012853470437017995,
+          "pd_advantage": 0.018115942028985508,
+          "n_slo": 15
+        },
+        "6P+2D": {
+          "attainment": 0.7746358183376179,
+          "pd_advantage": 1.0917874396135265,
+          "n_slo": 904
+        }
+      }
+    }
+  ]
+}

analysis/crossover/d1_i2048_goodput.json

@@ -0,0 +1,142 @@
+{
+  "baseline": "8C-proxy",
+  "arms": {
+    "8C-proxy": {
+      "name": "8C-proxy",
+      "n_offered": 1167,
+      "n_success": 1167,
+      "completion_rate": 1.0,
+      "offered_window_s": 299.736197,
+      "offered_qps": 3.8934236561358655,
+      "wall_clock_s": 300.1927249849541,
+      "amplification": 1.0015230992770423,
+      "ttft": {
+        "count": 1167,
+        "mean": 0.1367582682356152,
+        "p50": 0.1337889520218596,
+        "p90": 0.13568343156948687,
+        "p99": 0.1417779766954481
+      },
+      "tpot": {
+        "count": 1167,
+        "mean": 0.005039774583944486,
+        "p50": 0.005025791809968059,
+        "p90": 0.005088228446667984,
+        "p99": 0.0051380019550329985
+      },
+      "e2e": {
+        "count": 1167,
+        "mean": 0.45452996390628697,
+        "p50": 0.45110413199290633,
+        "p90": 0.4555703255813569,
+        "p99": 0.4688122991356063
+      }
+    },
+    "4P+4D": {
+      "name": "4P+4D",
+      "n_offered": 1167,
+      "n_success": 1167,
+      "completion_rate": 1.0,
+      "offered_window_s": 299.736197,
+      "offered_qps": 3.8934236561358655,
+      "wall_clock_s": 300.2166367470054,
+      "amplification": 1.0016028753010615,
+      "ttft": {
+        "count": 1167,
+        "mean": 0.1660798433322169,
+        "p50": 0.16653224604669958,
+        "p90": 0.17306958099361508,
+        "p99": 0.1832630861806683
+      },
+      "tpot": {
+        "count": 1167,
+        "mean": 0.005046875948133808,
+        "p50": 0.0050270736812510425,
+        "p90": 0.0050664691831029595,
+        "p99": 0.005532995437730161
+      },
+      "e2e": {
+        "count": 1167,
+        "mean": 0.48433690918382055,
+        "p50": 0.4845748710213229,
+        "p90": 0.4916891472181305,
+        "p99": 0.5235905526624989
+      }
+    },
+    "6P+2D": {
+      "name": "6P+2D",
+      "n_offered": 1167,
+      "n_success": 1167,
+      "completion_rate": 1.0,
+      "offered_window_s": 299.736197,
+      "offered_qps": 3.8934236561358655,
+      "wall_clock_s": 300.2855537990108,
+      "amplification": 1.001832800991369,
+      "ttft": {
+        "count": 1167,
+        "mean": 0.17076528707394278,
+        "p50": 0.16959826194215566,
+        "p90": 0.17923957279417663,
+        "p99": 0.18792458378709853
+      },
+      "tpot": {
+        "count": 1167,
+        "mean": 0.005377765550956012,
+        "p50": 0.005177637602808693,
+        "p90": 0.006078562941697855,
+        "p99": 0.007001848420689976
+      },
+      "e2e": {
+        "count": 1167,
+        "mean": 0.5098518045703324,
+        "p50": 0.4961305959150195,
+        "p90": 0.5569328068522736,
+        "p99": 0.6314893902023319
+      }
+    }
+  },
+  "slo_grid": [
+    {
+      "ttft_slo_s": 2.0,
+      "tpot_slo_s": 0.05,
+      "arms": {
+        "8C-proxy": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 1167
+        },
+        "4P+4D": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 1167
+        },
+        "6P+2D": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 1167
+        }
+      }
+    },
+    {
+      "ttft_slo_s": 5.0,
+      "tpot_slo_s": 0.05,
+      "arms": {
+        "8C-proxy": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 1167
+        },
+        "4P+4D": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 1167
+        },
+        "6P+2D": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 1167
+        }
+      }
+    }
+  ]
+}

analysis/crossover/d1_i32768_goodput.json

@@ -0,0 +1,142 @@
+{
+  "baseline": "8C-proxy",
+  "arms": {
+    "8C-proxy": {
+      "name": "8C-proxy",
+      "n_offered": 1167,
+      "n_success": 951,
+      "completion_rate": 0.8149100257069408,
+      "offered_window_s": 299.736197,
+      "offered_qps": 3.8934236561358655,
+      "wall_clock_s": 567.1572082909988,
+      "amplification": 1.8921879104611408,
+      "ttft": {
+        "count": 951,
+        "mean": 156.08624654169503,
+        "p50": 159.80228465000982,
+        "p90": 263.44082942500245,
+        "p99": 267.3189407904938
+      },
+      "tpot": {
+        "count": 951,
+        "mean": 0.5120599246480225,
+        "p50": 0.5265210192539223,
+        "p90": 0.5403215261268621,
+        "p99": 0.5436686666190138
+      },
+      "e2e": {
+        "count": 951,
+        "mean": 188.3470215559188,
+        "p50": 192.9057134770119,
+        "p90": 294.68771972299146,
+        "p99": 300.2205298934932
+      }
+    },
+    "4P+4D": {
+      "name": "4P+4D",
+      "n_offered": 1167,
+      "n_success": 596,
+      "completion_rate": 0.5107112253641817,
+      "offered_window_s": 299.736197,
+      "offered_qps": 3.8934236561358655,
+      "wall_clock_s": 789.8321366070013,
+      "amplification": 2.63509093833936,
+      "ttft": {
+        "count": 596,
+        "mean": 240.99040966441748,
+        "p50": 259.1066791820049,
+        "p90": 379.7789027979961,
+        "p99": 381.22047349565617
+      },
+      "tpot": {
+        "count": 596,
+        "mean": 0.006412497327210905,
+        "p50": 0.006444117785865692,
+        "p90": 0.0064974606745333095,
+        "p99": 0.006660929446938789
+      },
+      "e2e": {
+        "count": 596,
+        "mean": 241.39516468475196,
+        "p50": 259.5132920899996,
+        "p90": 380.1822811405,
+        "p99": 381.620277955458
+      }
+    },
+    "6P+2D": {
+      "name": "6P+2D",
+      "n_offered": 1167,
+      "n_success": 712,
+      "completion_rate": 0.6101113967437874,
+      "offered_window_s": 299.736197,
+      "offered_qps": 3.8934236561358655,
+      "wall_clock_s": 880.7267912379903,
+      "amplification": 2.938339780290167,
+      "ttft": {
+        "count": 712,
+        "mean": 175.69983718378106,
+        "p50": 172.43730882649834,
+        "p90": 250.6456608123961,
+        "p99": 594.014642834873
+      },
+      "tpot": {
+        "count": 712,
+        "mean": 0.007946936024875445,
+        "p50": 0.007568808706382119,
+        "p90": 0.010293392149296345,
+        "p99": 0.011275891952114068
+      },
+      "e2e": {
+        "count": 712,
+        "mean": 176.20114515147316,
+        "p50": 172.91078094949626,
+        "p90": 251.09661155799986,
+        "p99": 594.4257701966026
+      }
+    }
+  },
+  "slo_grid": [
+    {
+      "ttft_slo_s": 2.0,
+      "tpot_slo_s": 0.05,
+      "arms": {
+        "8C-proxy": {
+          "attainment": 0.0,
+          "pd_advantage": NaN,
+          "n_slo": 0
+        },
+        "4P+4D": {
+          "attainment": 0.0,
+          "pd_advantage": NaN,
+          "n_slo": 0
+        },
+        "6P+2D": {
+          "attainment": 0.0,
+          "pd_advantage": NaN,
+          "n_slo": 0
+        }
+      }
+    },
+    {
+      "ttft_slo_s": 5.0,
+      "tpot_slo_s": 0.05,
+      "arms": {
+        "8C-proxy": {
+          "attainment": 0.0,
+          "pd_advantage": NaN,
+          "n_slo": 0
+        },
+        "4P+4D": {
+          "attainment": 0.0,
+          "pd_advantage": NaN,
+          "n_slo": 0
+        },
+        "6P+2D": {
+          "attainment": 0.0,
+          "pd_advantage": NaN,
+          "n_slo": 0
+        }
+      }
+    }
+  ]
+}

analysis/crossover/d1_i4096_goodput.json

@@ -0,0 +1,142 @@
+{
+  "baseline": "8C-proxy",
+  "arms": {
+    "8C-proxy": {
+      "name": "8C-proxy",
+      "n_offered": 1167,
+      "n_success": 1167,
+      "completion_rate": 1.0,
+      "offered_window_s": 299.736197,
+      "offered_qps": 3.8934236561358655,
+      "wall_clock_s": 300.32634208898526,
+      "amplification": 1.0019688816195438,
+      "ttft": {
+        "count": 1167,
+        "mean": 0.2628433253752162,
+        "p50": 0.2604120689211413,
+        "p90": 0.2637787788407877,
+        "p99": 0.2976114549371393
+      },
+      "tpot": {
+        "count": 1167,
+        "mean": 0.005143964614959416,
+        "p50": 0.005124552334318795,
+        "p90": 0.005180289564166395,
+        "p99": 0.0052467077378080115
+      },
+      "e2e": {
+        "count": 1167,
+        "mean": 0.5871811069348818,
+        "p50": 0.5839827890740708,
+        "p90": 0.5883447280153632,
+        "p99": 0.6408826243295329
+      }
+    },
+    "4P+4D": {
+      "name": "4P+4D",
+      "n_offered": 1167,
+      "n_success": 1167,
+      "completion_rate": 1.0,
+      "offered_window_s": 299.736197,
+      "offered_qps": 3.8934236561358655,
+      "wall_clock_s": 300.3759389320621,
+      "amplification": 1.0021343499332585,
+      "ttft": {
+        "count": 1167,
+        "mean": 0.30861241175426407,
+        "p50": 0.3091614870354533,
+        "p90": 0.3227507559815422,
+        "p99": 0.3547600568598136
+      },
+      "tpot": {
+        "count": 1167,
+        "mean": 0.005148936446188915,
+        "p50": 0.0051316127147791645,
+        "p90": 0.005184003428393413,
+        "p99": 0.00547781713853871
+      },
+      "e2e": {
+        "count": 1167,
+        "mean": 0.6332931331915089,
+        "p50": 0.6365591660141945,
+        "p90": 0.6472898541716858,
+        "p99": 0.7053035433730102
+      }
+    },
+    "6P+2D": {
+      "name": "6P+2D",
+      "n_offered": 1167,
+      "n_success": 1167,
+      "completion_rate": 1.0,
+      "offered_window_s": 299.736197,
+      "offered_qps": 3.8934236561358655,
+      "wall_clock_s": 300.4041277950164,
+      "amplification": 1.0022283955081221,
+      "ttft": {
+        "count": 1167,
+        "mean": 0.31527804771431056,
+        "p50": 0.31678270106203854,
+        "p90": 0.3284846659982577,
+        "p99": 0.3402980738645419
+      },
+      "tpot": {
+        "count": 1167,
+        "mean": 0.0055213440500518985,
+        "p50": 0.005325577683776381,
+        "p90": 0.006278165857056304,
+        "p99": 0.0070092806818761975
+      },
+      "e2e": {
+        "count": 1167,
+        "mean": 0.6634175265073216,
+        "p50": 0.6494167000055313,
+        "p90": 0.7135140666039661,
+        "p99": 0.7741130576422437
+      }
+    }
+  },
+  "slo_grid": [
+    {
+      "ttft_slo_s": 2.0,
+      "tpot_slo_s": 0.05,
+      "arms": {
+        "8C-proxy": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 1167
+        },
+        "4P+4D": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 1167
+        },
+        "6P+2D": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 1167
+        }
+      }
+    },
+    {
+      "ttft_slo_s": 5.0,
+      "tpot_slo_s": 0.05,
+      "arms": {
+        "8C-proxy": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 1167
+        },
+        "4P+4D": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 1167
+        },
+        "6P+2D": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 1167
+        }
+      }
+    }
+  ]
+}

analysis/crossover/d1_i8192_goodput.json

@@ -0,0 +1,142 @@
+{
+  "baseline": "8C-proxy",
+  "arms": {
+    "8C-proxy": {
+      "name": "8C-proxy",
+      "n_offered": 1167,
+      "n_success": 1167,
+      "completion_rate": 1.0,
+      "offered_window_s": 299.736197,
+      "offered_qps": 3.8934236561358655,
+      "wall_clock_s": 300.66455180699995,
+      "amplification": 1.0030972395602922,
+      "ttft": {
+        "count": 1167,
+        "mean": 0.584800394000121,
+        "p50": 0.5817365200055065,
+        "p90": 0.5870239008043427,
+        "p99": 0.6088061455392736
+      },
+      "tpot": {
+        "count": 1167,
+        "mean": 0.005585180932058873,
+        "p50": 0.005308644777941813,
+        "p90": 0.005355837698406836,
+        "p99": 0.014590345752427337
+      },
+      "e2e": {
+        "count": 1167,
+        "mean": 0.9369382756572071,
+        "p50": 0.9158835929993074,
+        "p90": 0.9229304600012256,
+        "p99": 1.5052119748995754
+      }
+    },
+    "4P+4D": {
+      "name": "4P+4D",
+      "n_offered": 1167,
+      "n_success": 1167,
+      "completion_rate": 1.0,
+      "offered_window_s": 299.736197,
+      "offered_qps": 3.8934236561358655,
+      "wall_clock_s": 300.7607197400066,
+      "amplification": 1.003418081467173,
+      "ttft": {
+        "count": 1167,
+        "mean": 0.7293765576349805,
+        "p50": 0.6887177099997643,
+        "p90": 0.8911576298065484,
+        "p99": 1.1879012519156076
+      },
+      "tpot": {
+        "count": 1167,
+        "mean": 0.005308951736224922,
+        "p50": 0.0053098901269203495,
+        "p90": 0.005347346425488857,
+        "p99": 0.005420954272433716
+      },
+      "e2e": {
+        "count": 1167,
+        "mean": 1.064155324243521,
+        "p50": 1.0238807429996086,
+        "p90": 1.22566919879755,
+        "p99": 1.521622942700049
+      }
+    },
+    "6P+2D": {
+      "name": "6P+2D",
+      "n_offered": 1167,
+      "n_success": 1167,
+      "completion_rate": 1.0,
+      "offered_window_s": 299.736197,
+      "offered_qps": 3.8934236561358655,
+      "wall_clock_s": 300.8507765819959,
+      "amplification": 1.003718535142407,
+      "ttft": {
+        "count": 1167,
+        "mean": 0.7028102769056583,
+        "p50": 0.6950660040020011,
+        "p90": 0.7394948218076024,
+        "p99": 0.8658222487580485
+      },
+      "tpot": {
+        "count": 1167,
+        "mean": 0.005782890183689697,
+        "p50": 0.00558671395230617,
+        "p90": 0.006629064533303286,
+        "p99": 0.007366264003757681
+      },
+      "e2e": {
+        "count": 1167,
+        "mean": 1.0674261210719522,
+        "p50": 1.0612060459970962,
+        "p90": 1.1425091800017981,
+        "p99": 1.3045775910036173
+      }
+    }
+  },
+  "slo_grid": [
+    {
+      "ttft_slo_s": 2.0,
+      "tpot_slo_s": 0.05,
+      "arms": {
+        "8C-proxy": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 1167
+        },
+        "4P+4D": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 1167
+        },
+        "6P+2D": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 1167
+        }
+      }
+    },
+    {
+      "ttft_slo_s": 5.0,
+      "tpot_slo_s": 0.05,
+      "arms": {
+        "8C-proxy": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 1167
+        },
+        "4P+4D": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 1167
+        },
+        "6P+2D": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 1167
+        }
+      }
+    }
+  ]
+}

analysis/crossover/d2_o1024_goodput.json

@@ -0,0 +1,121 @@
+{
+  "baseline": "8C-proxy",
+  "arms": {
+    "8C-proxy": {
+      "name": "8C-proxy",
+      "n_offered": 3552,
+      "n_success": 1950,
+      "completion_rate": 0.5489864864864865,
+      "offered_window_s": 299.74141899999995,
+      "offered_qps": 11.850214134070008,
+      "wall_clock_s": 348.4953830049999,
+      "amplification": 1.1626534102882857,
+      "ttft": {
+        "count": 1950,
+        "mean": 0.8245253885559363,
+        "p50": 0.7988759850013594,
+        "p90": 1.5384256363991882,
+        "p99": 2.125257717882705
+      },
+      "tpot": {
+        "count": 1950,
+        "mean": 0.07444567704430907,
+        "p50": 0.08914986800879843,
+        "p90": 0.09127071481447005,
+        "p99": 0.09233926755616054
+      },
+      "e2e": {
+        "count": 1950,
+        "mean": 77.04034660455785,
+        "p50": 92.29079191399796,
+        "p90": 94.47993849659906,
+        "p99": 95.72745552870882
+      }
+    },
+    "4P+4D": {
+      "name": "4P+4D",
+      "n_offered": 3552,
+      "n_success": 2754,
+      "completion_rate": 0.7753378378378378,
+      "offered_window_s": 299.74141899999995,
+      "offered_qps": 11.850214134070008,
+      "wall_clock_s": 744.3694221920014,
+      "amplification": 2.4833719166185753,
+      "ttft": {
+        "count": 2754,
+        "mean": 4.5006646864741695,
+        "p50": 2.329877773499902,
+        "p90": 10.863291689799375,
+        "p99": 21.572781211729307
+      },
+      "tpot": {
+        "count": 2754,
+        "mean": 0.046089308743682535,
+        "p50": 0.04704797274047173,
+        "p90": 0.04792202048768524,
+        "p99": 0.059544689888886115
+      },
+      "e2e": {
+        "count": 2754,
+        "mean": 51.68415100813728,
+        "p50": 50.91020956099965,
+        "p90": 59.22212425199977,
+        "p99": 68.42946432940865
+      }
+    },
+    "6P+2D": {
+      "name": "6P+2D",
+      "n_offered": 3552,
+      "n_success": 1928,
+      "completion_rate": 0.5427927927927928,
+      "offered_window_s": 299.74141899999995,
+      "offered_qps": 11.850214134070008,
+      "wall_clock_s": 821.7403777359868,
+      "amplification": 2.7414975897474716,
+      "ttft": {
+        "count": 1928,
+        "mean": 39.07585104131927,
+        "p50": 42.6814165695032,
+        "p90": 63.28579387369681,
+        "p99": 73.84470144698193
+      },
+      "tpot": {
+        "count": 1928,
+        "mean": 0.04064862157373329,
+        "p50": 0.0398688508558163,
+        "p90": 0.04277554483881431,
+        "p99": 0.0695667276081441
+      },
+      "e2e": {
+        "count": 1928,
+        "mean": 80.66499786808757,
+        "p50": 84.05398454950046,
+        "p90": 105.42814423799427,
+        "p99": 113.52591980495636
+      }
+    }
+  },
+  "slo_grid": [
+    {
+      "ttft_slo_s": 2.0,
+      "tpot_slo_s": 0.05,
+      "arms": {
+        "8C-proxy": {
+          "attainment": 0.10191441441441441,
+          "pd_advantage": 1.0,
+          "n_slo": 362
+        },
+        "4P+4D": {
+          "attainment": 0.3502252252252252,
+          "pd_advantage": 3.43646408839779,
+          "n_slo": 1244
+        },
+        "6P+2D": {
+          "attainment": 0.06447072072072071,
+          "pd_advantage": 0.6325966850828729,
+          "n_slo": 229
+        }
+      }
+    }
+  ]
+}

analysis/crossover/d2_o128_goodput.json

@@ -0,0 +1,121 @@
+{
+  "baseline": "8C-proxy",
+  "arms": {
+    "8C-proxy": {
+      "name": "8C-proxy",
+      "n_offered": 3552,
+      "n_success": 3552,
+      "completion_rate": 1.0,
+      "offered_window_s": 299.74141899999995,
+      "offered_qps": 11.850214134070008,
+      "wall_clock_s": 300.61600343801547,
+      "amplification": 1.0029177964157683,
+      "ttft": {
+        "count": 3552,
+        "mean": 0.139721406387908,
+        "p50": 0.13759525847854093,
+        "p90": 0.14188919204752892,
+        "p99": 0.17052793529117438
+      },
+      "tpot": {
+        "count": 3552,
+        "mean": 0.006496044329480822,
+        "p50": 0.006430442302898453,
+        "p90": 0.007889476366154117,
+        "p99": 0.009282101713493095
+      },
+      "e2e": {
+        "count": 3552,
+        "mean": 0.9651016481538636,
+        "p50": 0.9548319909954444,
+        "p90": 1.1440699087572284,
+        "p99": 1.3324073514551853
+      }
+    },
+    "4P+4D": {
+      "name": "4P+4D",
+      "n_offered": 3552,
+      "n_success": 3552,
+      "completion_rate": 1.0,
+      "offered_window_s": 299.74141899999995,
+      "offered_qps": 11.850214134070008,
+      "wall_clock_s": 300.7410873459885,
+      "amplification": 1.0033351024670651,
+      "ttft": {
+        "count": 3552,
+        "mean": 0.18467717330343286,
+        "p50": 0.1806962049449794,
+        "p90": 0.1929074571467936,
+        "p99": 0.2592483600333798
+      },
+      "tpot": {
+        "count": 3552,
+        "mean": 0.006980159784640828,
+        "p50": 0.006945047342387123,
+        "p90": 0.008164690015837552,
+        "p99": 0.009163911323187739
+      },
+      "e2e": {
+        "count": 3552,
+        "mean": 1.0715517728354285,
+        "p50": 1.063076548918616,
+        "p90": 1.2275727374362762,
+        "p99": 1.4021264237130526
+      }
+    },
+    "6P+2D": {
+      "name": "6P+2D",
+      "n_offered": 3552,
+      "n_success": 3552,
+      "completion_rate": 1.0,
+      "offered_window_s": 299.74141899999995,
+      "offered_qps": 11.850214134070008,
+      "wall_clock_s": 301.1243950970238,
+      "amplification": 1.004613897210595,
+      "ttft": {
+        "count": 3552,
+        "mean": 0.20078434001432569,
+        "p50": 0.19958186394069344,
+        "p90": 0.213167638995219,
+        "p99": 0.23225504373782313
+      },
+      "tpot": {
+        "count": 3552,
+        "mean": 0.010393430778126047,
+        "p50": 0.010396533917478015,
+        "p90": 0.012065167099743436,
+        "p99": 0.01323438493101344
+      },
+      "e2e": {
+        "count": 3552,
+        "mean": 1.5212831554822883,
+        "p50": 1.5218885459471494,
+        "p90": 1.7440477049094625,
+        "p99": 1.9191367691196497
+      }
+    }
+  },
+  "slo_grid": [
+    {
+      "ttft_slo_s": 2.0,
+      "tpot_slo_s": 0.05,
+      "arms": {
+        "8C-proxy": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 3552
+        },
+        "4P+4D": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 3552
+        },
+        "6P+2D": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 3552
+        }
+      }
+    }
+  ]
+}

analysis/crossover/d2_o2048_goodput.json

@@ -0,0 +1,121 @@
+{
+  "baseline": "8C-proxy",
+  "arms": {
+    "8C-proxy": {
+      "name": "8C-proxy",
+      "n_offered": 3552,
+      "n_success": 1050,
+      "completion_rate": 0.2956081081081081,
+      "offered_window_s": 299.74141899999995,
+      "offered_qps": 11.850214134070008,
+      "wall_clock_s": 414.92026689198974,
+      "amplification": 1.3842607013613617,
+      "ttft": {
+        "count": 1050,
+        "mean": 0.8393797435154972,
+        "p50": 0.8442945880015031,
+        "p90": 1.4299255040969001,
+        "p99": 2.1686189359705894
+      },
+      "tpot": {
+        "count": 1050,
+        "mean": 0.08418485046407832,
+        "p50": 0.08937374470713316,
+        "p90": 0.09085719287396188,
+        "p99": 0.09190563933350096
+      },
+      "e2e": {
+        "count": 1050,
+        "mean": 173.22149556513375,
+        "p50": 183.9983977669981,
+        "p90": 187.03014375649946,
+        "p99": 189.36978534828302
+      }
+    },
+    "4P+4D": {
+      "name": "4P+4D",
+      "n_offered": 3552,
+      "n_success": 1320,
+      "completion_rate": 0.3716216216216216,
+      "offered_window_s": 299.74141899999995,
+      "offered_qps": 11.850214134070008,
+      "wall_clock_s": 802.1317602040072,
+      "amplification": 2.6760791447511205,
+      "ttft": {
+        "count": 1320,
+        "mean": 19.92388417242484,
+        "p50": 8.059029546493548,
+        "p90": 64.71132232169766,
+        "p99": 87.07121913193099
+      },
+      "tpot": {
+        "count": 1320,
+        "mean": 0.05007250707090057,
+        "p50": 0.0492711045202749,
+        "p90": 0.07142825628519621,
+        "p99": 0.08267229383702501
+      },
+      "e2e": {
+        "count": 1320,
+        "mean": 122.44309290737698,
+        "p50": 118.35192646600626,
+        "p90": 167.89536100080443,
+        "p99": 187.19021692074878
+      }
+    },
+    "6P+2D": {
+      "name": "6P+2D",
+      "n_offered": 3552,
+      "n_success": 1041,
+      "completion_rate": 0.29307432432432434,
+      "offered_window_s": 299.74141899999995,
+      "offered_qps": 11.850214134070008,
+      "wall_clock_s": 879.8981197069952,
+      "amplification": 2.935523968100636,
+      "ttft": {
+        "count": 1041,
+        "mean": 86.06142147925078,
+        "p50": 92.45557322700915,
+        "p90": 153.47880471601093,
+        "p99": 159.16995789420034
+      },
+      "tpot": {
+        "count": 1041,
+        "mean": 0.04275836784091199,
+        "p50": 0.038143618996089604,
+        "p90": 0.06297858072350163,
+        "p99": 0.07100282797010407
+      },
+      "e2e": {
+        "count": 1041,
+        "mean": 173.5925440399473,
+        "p50": 199.81516771799943,
+        "p90": 230.71809448600106,
+        "p99": 234.9438766648003
+      }
+    }
+  },
+  "slo_grid": [
+    {
+      "ttft_slo_s": 2.0,
+      "tpot_slo_s": 0.05,
+      "arms": {
+        "8C-proxy": {
+          "attainment": 0.009009009009009009,
+          "pd_advantage": 1.0,
+          "n_slo": 32
+        },
+        "4P+4D": {
+          "attainment": 0.11486486486486487,
+          "pd_advantage": 12.75,
+          "n_slo": 408
+        },
+        "6P+2D": {
+          "attainment": 0.04954954954954955,
+          "pd_advantage": 5.5,
+          "n_slo": 176
+        }
+      }
+    }
+  ]
+}

analysis/crossover/d2_o256_goodput.json

@@ -0,0 +1,121 @@
+{
+  "baseline": "8C-proxy",
+  "arms": {
+    "8C-proxy": {
+      "name": "8C-proxy",
+      "n_offered": 3552,
+      "n_success": 3552,
+      "completion_rate": 1.0,
+      "offered_window_s": 299.74141899999995,
+      "offered_qps": 11.850214134070008,
+      "wall_clock_s": 301.78380814404227,
+      "amplification": 1.0068138369093473,
+      "ttft": {
+        "count": 3552,
+        "mean": 0.14506312318632342,
+        "p50": 0.14291562204016373,
+        "p90": 0.1491016250802204,
+        "p99": 0.1832571337954128
+      },
+      "tpot": {
+        "count": 3552,
+        "mean": 0.0089611879488046,
+        "p50": 0.008925844482420122,
+        "p90": 0.010698060123752072,
+        "p99": 0.011754058378410762
+      },
+      "e2e": {
+        "count": 3552,
+        "mean": 2.430673889741114,
+        "p50": 2.419925262918696,
+        "p90": 2.874893436266575,
+        "p99": 3.1480205075151746
+      }
+    },
+    "4P+4D": {
+      "name": "4P+4D",
+      "n_offered": 3552,
+      "n_success": 3551,
+      "completion_rate": 0.9997184684684685,
+      "offered_window_s": 299.74141899999995,
+      "offered_qps": 11.850214134070008,
+      "wall_clock_s": 744.8868488909211,
+      "amplification": 2.4850981601942745,
+      "ttft": {
+        "count": 3551,
+        "mean": 0.20500909556101396,
+        "p50": 0.20017725799698383,
+        "p90": 0.21613375598099083,
+        "p99": 0.29023518750909716
+      },
+      "tpot": {
+        "count": 3551,
+        "mean": 0.010307210080394247,
+        "p50": 0.010299421709450874,
+        "p90": 0.011793581666607482,
+        "p99": 0.012639003840969035
+      },
+      "e2e": {
+        "count": 3551,
+        "mean": 2.8339171693501317,
+        "p50": 2.8360355379991233,
+        "p90": 3.2198793930001557,
+        "p99": 3.438588996999897
+      }
+    },
+    "6P+2D": {
+      "name": "6P+2D",
+      "n_offered": 3552,
+      "n_success": 3552,
+      "completion_rate": 1.0,
+      "offered_window_s": 299.74141899999995,
+      "offered_qps": 11.850214134070008,
+      "wall_clock_s": 303.4300506779691,
+      "amplification": 1.0123060459587976,
+      "ttft": {
+        "count": 3552,
+        "mean": 0.2333820717117756,
+        "p50": 0.23283391550648957,
+        "p90": 0.24394672318594532,
+        "p99": 0.2734717815916518
+      },
+      "tpot": {
+        "count": 3552,
+        "mean": 0.016453822599812207,
+        "p50": 0.016576926972415737,
+        "p90": 0.017214638463623238,
+        "p99": 0.01769411424845092
+      },
+      "e2e": {
+        "count": 3552,
+        "mean": 4.430368151474328,
+        "p50": 4.463736370031256,
+        "p90": 4.628522685484495,
+        "p99": 4.75812664218829
+      }
+    }
+  },
+  "slo_grid": [
+    {
+      "ttft_slo_s": 2.0,
+      "tpot_slo_s": 0.05,
+      "arms": {
+        "8C-proxy": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 3552
+        },
+        "4P+4D": {
+          "attainment": 0.9997184684684685,
+          "pd_advantage": 0.9997184684684685,
+          "n_slo": 3551
+        },
+        "6P+2D": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 3552
+        }
+      }
+    }
+  ]
+}

analysis/crossover/d2_o4096_goodput.json

@@ -0,0 +1,121 @@
+{
+  "baseline": "8C-proxy",
+  "arms": {
+    "8C-proxy": {
+      "name": "8C-proxy",
+      "n_offered": 3552,
+      "n_success": 545,
+      "completion_rate": 0.15343468468468469,
+      "offered_window_s": 299.74141899999995,
+      "offered_qps": 11.850214134070008,
+      "wall_clock_s": 505.4139580530027,
+      "amplification": 1.6861665622961597,
+      "ttft": {
+        "count": 545,
+        "mean": 4.137136383750316,
+        "p50": 0.5702517530007754,
+        "p90": 1.6976309113961179,
+        "p99": 56.1814190310361
+      },
+      "tpot": {
+        "count": 545,
+        "mean": 0.08390926873267023,
+        "p50": 0.08933810225128375,
+        "p90": 0.09449654152820693,
+        "p99": 0.10591519025133134
+      },
+      "e2e": {
+        "count": 545,
+        "mean": 347.7765141811284,
+        "p50": 366.3568219640001,
+        "p90": 388.1241692415934,
+        "p99": 435.2848098113155
+      }
+    },
+    "4P+4D": {
+      "name": "4P+4D",
+      "n_offered": 3552,
+      "n_success": 771,
+      "completion_rate": 0.2170608108108108,
+      "offered_window_s": 299.74141899999995,
+      "offered_qps": 11.850214134070008,
+      "wall_clock_s": 850.1328976760124,
+      "amplification": 2.8362209684341706,
+      "ttft": {
+        "count": 771,
+        "mean": 63.210636654403324,
+        "p50": 1.1097561890055658,
+        "p90": 179.64081536799495,
+        "p99": 186.67013871119852
+      },
+      "tpot": {
+        "count": 771,
+        "mean": 0.05484690890217986,
+        "p50": 0.04795774376874187,
+        "p90": 0.08067800507863548,
+        "p99": 0.09099416300415113
+      },
+      "e2e": {
+        "count": 771,
+        "mean": 287.81779562259663,
+        "p50": 312.2160719559906,
+        "p90": 372.0004520520015,
+        "p99": 379.3279856524052
+      }
+    },
+    "6P+2D": {
+      "name": "6P+2D",
+      "n_offered": 3552,
+      "n_success": 627,
+      "completion_rate": 0.17652027027027026,
+      "offered_window_s": 299.74141899999995,
+      "offered_qps": 11.850214134070008,
+      "wall_clock_s": 867.8833550199925,
+      "amplification": 2.8954402028102515,
+      "ttft": {
+        "count": 627,
+        "mean": 179.58769048342904,
+        "p50": 238.1998468660022,
+        "p90": 378.29023678940143,
+        "p99": 385.40577973942356
+      },
+      "tpot": {
+        "count": 627,
+        "mean": 0.04188420961205498,
+        "p50": 0.03654626756630041,
+        "p90": 0.06031132874202571,
+        "p99": 0.06738955674930582
+      },
+      "e2e": {
+        "count": 627,
+        "mean": 351.1066520824709,
+        "p50": 387.2650127700035,
+        "p90": 507.9008203571953,
+        "p99": 570.6463984230224
+      }
+    }
+  },
+  "slo_grid": [
+    {
+      "ttft_slo_s": 2.0,
+      "tpot_slo_s": 0.05,
+      "arms": {
+        "8C-proxy": {
+          "attainment": 0.0,
+          "pd_advantage": NaN,
+          "n_slo": 0
+        },
+        "4P+4D": {
+          "attainment": 0.05855855855855856,
+          "pd_advantage": NaN,
+          "n_slo": 208
+        },
+        "6P+2D": {
+          "attainment": 0.036036036036036036,
+          "pd_advantage": NaN,
+          "n_slo": 128
+        }
+      }
+    }
+  ]
+}

analysis/crossover/d2_o512_goodput.json

@@ -0,0 +1,121 @@
+{
+  "baseline": "8C-proxy",
+  "arms": {
+    "8C-proxy": {
+      "name": "8C-proxy",
+      "n_offered": 3552,
+      "n_success": 3552,
+      "completion_rate": 1.0,
+      "offered_window_s": 299.74141899999995,
+      "offered_qps": 11.850214134070008,
+      "wall_clock_s": 304.97662343096454,
+      "amplification": 1.0174657357946404,
+      "ttft": {
+        "count": 3552,
+        "mean": 0.16242270423816507,
+        "p50": 0.15836620499612764,
+        "p90": 0.17363918052287775,
+        "p99": 0.24847999344696287
+      },
+      "tpot": {
+        "count": 3552,
+        "mean": 0.013691483182659388,
+        "p50": 0.013865661168213087,
+        "p90": 0.015242900696529327,
+        "p99": 0.01626683903372128
+      },
+      "e2e": {
+        "count": 3552,
+        "mean": 7.160272567887971,
+        "p50": 7.247483663493767,
+        "p90": 7.962273431208451,
+        "p99": 8.480892732607899
+      }
+    },
+    "4P+4D": {
+      "name": "4P+4D",
+      "n_offered": 3552,
+      "n_success": 3551,
+      "completion_rate": 0.9997184684684685,
+      "offered_window_s": 299.74141899999995,
+      "offered_qps": 11.850214134070008,
+      "wall_clock_s": 744.9637431279989,
+      "amplification": 2.4853546954349977,
+      "ttft": {
+        "count": 3551,
+        "mean": 0.2446379999283825,
+        "p50": 0.24067969399038702,
+        "p90": 0.2630795220611617,
+        "p99": 0.3426029055262916
+      },
+      "tpot": {
+        "count": 3551,
+        "mean": 0.016916919575073883,
+        "p50": 0.016998299133030737,
+        "p90": 0.01775875886689886,
+        "p99": 0.018166751548973206
+      },
+      "e2e": {
+        "count": 3551,
+        "mean": 8.89104466149889,
+        "p50": 8.933106195996515,
+        "p90": 9.333998591057025,
+        "p99": 9.536390463996213
+      }
+    },
+    "6P+2D": {
+      "name": "6P+2D",
+      "n_offered": 3552,
+      "n_success": 3551,
+      "completion_rate": 0.9997184684684685,
+      "offered_window_s": 299.74141899999995,
+      "offered_qps": 11.850214134070008,
+      "wall_clock_s": 312.45468625507783,
+      "amplification": 1.0424141157985172,
+      "ttft": {
+        "count": 3551,
+        "mean": 0.36231219612768273,
+        "p50": 0.3462264990666881,
+        "p90": 0.4059687410481274,
+        "p99": 0.9837204645154998
+      },
+      "tpot": {
+        "count": 3551,
+        "mean": 0.03268022218101953,
+        "p50": 0.03333031399418835,
+        "p90": 0.03557429400772957,
+        "p99": 0.038558618127279086
+      },
+      "e2e": {
+        "count": 3551,
+        "mean": 17.068018403084057,
+        "p50": 17.392655145958997,
+        "p90": 18.56302695896011,
+        "p99": 20.159411454980727
+      }
+    }
+  },
+  "slo_grid": [
+    {
+      "ttft_slo_s": 2.0,
+      "tpot_slo_s": 0.05,
+      "arms": {
+        "8C-proxy": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 3552
+        },
+        "4P+4D": {
+          "attainment": 0.9997184684684685,
+          "pd_advantage": 0.9997184684684685,
+          "n_slo": 3551
+        },
+        "6P+2D": {
+          "attainment": 0.9997184684684685,
+          "pd_advantage": 0.9997184684684685,
+          "n_slo": 3551
+        }
+      }
+    }
+  ]
+}

analysis/crossover/d2_o64_goodput.json

@@ -0,0 +1,121 @@
+{
+  "baseline": "8C-proxy",
+  "arms": {
+    "8C-proxy": {
+      "name": "8C-proxy",
+      "n_offered": 3552,
+      "n_success": 3552,
+      "completion_rate": 1.0,
+      "offered_window_s": 299.74141899999995,
+      "offered_qps": 11.850214134070008,
+      "wall_clock_s": 300.21845804993063,
+      "amplification": 1.001591501940313,
+      "ttft": {
+        "count": 3552,
+        "mean": 0.1369056966029782,
+        "p50": 0.13516780693316832,
+        "p90": 0.13835511771030723,
+        "p99": 0.15317223175661632
+      },
+      "tpot": {
+        "count": 3552,
+        "mean": 0.005430781936309906,
+        "p50": 0.005039893150780468,
+        "p90": 0.007063257358303028,
+        "p99": 0.0077793481296283135
+      },
+      "e2e": {
+        "count": 3552,
+        "mean": 0.4793568905444592,
+        "p50": 0.45412574551301077,
+        "p90": 0.5810393166844734,
+        "p99": 0.6301419002050533
+      }
+    },
+    "4P+4D": {
+      "name": "4P+4D",
+      "n_offered": 3552,
+      "n_success": 3551,
+      "completion_rate": 0.9997184684684685,
+      "offered_window_s": 299.74141899999995,
+      "offered_qps": 11.850214134070008,
+      "wall_clock_s": 735.1059510430787,
+      "amplification": 2.452467041410379,
+      "ttft": {
+        "count": 3551,
+        "mean": 0.17374673821534295,
+        "p50": 0.17102365801110864,
+        "p90": 0.18273873499128968,
+        "p99": 0.24688138795318082
+      },
+      "tpot": {
+        "count": 3551,
+        "mean": 0.005485491774206834,
+        "p50": 0.0053864502226046865,
+        "p90": 0.006091995366168992,
+        "p99": 0.007109403222178419
+      },
+      "e2e": {
+        "count": 3551,
+        "mean": 0.5196563635758822,
+        "p50": 0.5100997349945828,
+        "p90": 0.5655082209268585,
+        "p99": 0.6639982180204242
+      }
+    },
+    "6P+2D": {
+      "name": "6P+2D",
+      "n_offered": 3552,
+      "n_success": 3552,
+      "completion_rate": 1.0,
+      "offered_window_s": 299.74141899999995,
+      "offered_qps": 11.850214134070008,
+      "wall_clock_s": 300.38101644301787,
+      "amplification": 1.0021338307036503,
+      "ttft": {
+        "count": 3552,
+        "mean": 0.18293367427152893,
+        "p50": 0.1822461549891159,
+        "p90": 0.1938482352765277,
+        "p99": 0.21272844232735222
+      },
+      "tpot": {
+        "count": 3552,
+        "mean": 0.007192309629699456,
+        "p50": 0.007143509595484902,
+        "p90": 0.008732455453672816,
+        "p99": 0.009842920153335268
+      },
+      "e2e": {
+        "count": 3552,
+        "mean": 0.636424056327808,
+        "p50": 0.6324848984950222,
+        "p90": 0.7393875011475757,
+        "p99": 0.8261980937235056
+      }
+    }
+  },
+  "slo_grid": [
+    {
+      "ttft_slo_s": 2.0,
+      "tpot_slo_s": 0.05,
+      "arms": {
+        "8C-proxy": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 3552
+        },
+        "4P+4D": {
+          "attainment": 0.9997184684684685,
+          "pd_advantage": 0.9997184684684685,
+          "n_slo": 3551
+        },
+        "6P+2D": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 3552
+        }
+      }
+    }
+  ]
+}

analysis/elastic_hypotheses.md

@@ -244,7 +244,11 @@ Offloaded:      —               13/500 (2.6%)   too few to matter
 ### What DOESN'T work for agentic workloads:
 
 1. **PD-Sep**: net negative — KV cache memory wall on decode instances
-2. **LMetric (OSDI'26)**: ≈ linear routing — session affinity limits routing freedom
+2. **LMetric (OSDI'26)**: ≈ linear routing — `P_tokens` already includes
+   `new_uncached_tokens`, so cache-hit scoring gives LMetric an implicit
+   soft affinity that converges to similar placements as explicit sticky
+   affinity (see `analysis/research_findings.md` §2.2 for the corrected
+   framing)
 3. **Elastic P2P RDMA offload**: net negative — Mooncake transfer overhead, no layerwise pipeline
 4. **OVERLOAD_FACTOR tuning**: no effect — imbalance from workload skew, not routing
 5. **Dedicated Prefill Service (PS)**: cannot win cost comparison without KV pull, PS is always slower than cached C
@@ -270,3 +274,21 @@ Instead of fixed chunk size, dynamically adjust based on decode pressure:
 - When decode queue is deep: smaller chunks → more decode slots → better TPOT
 - When decode queue is empty: larger chunks → faster prefill → better TTFT
 - This is a vLLM scheduler modification, not a routing change
+
+---
+
+## Current routing direction (cross-reference)
+
+The hypotheses above produced the following positive results that informed
+the current `--policy unified` implementation:
+
+- H1 / H7 / H9 (negative): PD-sep offload, OVERLOAD_FACTOR tuning, and
+  elastic RDMA at high concurrency all regressed or stayed within noise.
+- H3 / H4 / H6 (partial): cache-gated offload exists but only ~10-12% of
+  HEAVY requests have cache, and the offloaded subset pays RDMA penalty.
+
+The active algorithm (commit `255c8e6`) is **hybrid LMetric + high-cache
+affinity** in baseline mode (no Mooncake). The retired migration variants
+are catalogued in `docs/migration-policy-design.md` (Approach A and the
+revert chain `cc6e562` / `4c583f2`). H7's rejection (OVERLOAD_FACTOR within
+noise) is why the active default stays at `overload_factor=2.0`.

analysis/lpwl_5policy_600s.md

@@ -0,0 +1,131 @@
+# LPWL vs 4 baselines — parameter-free routing for agentic workloads
+
+Date: 2026-05-29. Hardware: dash1, 8×H20, Qwen3-Coder-30B-A3B, TP=1,
+max_model_len=200000, fresh vLLM per arm (cold APC), `--policy` via
+`scripts/b3_isolated_policy.sh`. Analyzer: `scripts/bench_report.py`.
+
+## Motivation
+
+unified+A+B carries too many knobs (`overload_factor`, `lmetric_decode_weight`,
+the 0.5 `cache_ratio` gate). Goal: a policy derived from the agentic *pattern*
+with no tuned constants, that does not overfit.
+
+## LPWL (Least-Prefill-Work-Left)
+
+`scripts/cache_aware_proxy.py:pick_instance_leastwork`, `--policy leastwork`:
+
+```
+score_i = pending_prefill_tokens_i + max(0, input_len − cache_hit_i)   → argmin
+```
+
+Tie-break: fewest `num_requests`, then round-robin. **Zero hyperparameters.**
+
+Why this shape (straight from the workload):
+- Decode is cheap (I/O ~217×) ⇒ the only load worth modeling is outstanding
+  *prefill* token-work. No decode weight; dropping LMetric's `×num_requests`
+  also makes an idle-but-decoding host score `input` (its true marginal cost),
+  not 0 — fixing the empty-batch degeneracy for free.
+- Cache-awareness *is* the affinity mechanism: a returning session's owner has
+  `new_uncached ≈ 0`, so it sticks unless its prefill backlog exceeds the cache
+  saving (`input`). The stick-vs-spill crossover is computed from real
+  token-work — no `overload_factor`, no `cache_ratio` gate.
+- Session skew degrades gracefully: a heavy session inflates its owner's
+  `pending_prefill`, auto-diverting *other* sessions while the heavy one stays
+  put (no cold re-prefill).
+
+## Results — 600s trace (`w600_r0.0015_st30_first600s.jsonl`, 807 reqs)
+
+This is the colder regime (theoretical APC ceiling ≈ 70% vs 80% for full w600).
+
+| policy | knobs | TTFT mean | TTFT p90 | E2E mean | E2E p90 | E2E p99 | TPOT p90 | APC | req-bal |
+|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|
+| **LPWL** | **0** | **3398** | **7983** | **8116** | **19014** | 87024 | 26 | 0.648 | **1.55×** |
+| unified+A+B | 3 | 3876 | 11562 | 8199 | 22569 | **74266** | **25** | 0.661 | 1.56× |
+| unified default | 2 | 5066 | 16389 | 10481 | 28427 | 96361 | 34 | 0.689 | 2.28× |
+| LMetric | 0 | 4809 | 14037 | 10051 | 26726 | 97442 | 32 | 0.507 | 2.11× |
+| sticky | 0 | 5758 | 20356 | 10815 | 34734 | 82732 | 28 | **0.696** | 3.86× |
+
+(latencies ms; req-bal = max:min per-worker request count; this batch predates
+the GPU-capture harness change so per-worker GPU util reads N/A.)
+
+### Findings
+
+1. **LPWL is overall best with zero knobs:** TTFT mean −12% / p90 −31%, E2E
+   mean ~tie / p90 −16% vs the tuned unified+A+B; best request balance; TPOT
+   tied-best. Only loss is E2E p99 (+17%) from heavy-class decode concentration.
+2. **The baselines bracket the problem and explain why LPWL works:** sticky has
+   the highest APC (0.696) but worst latency (hot-pin, 3.86× imbalance); LMetric
+   has the worst APC (0.507) because `×num_requests` swallows the cache signal.
+   LPWL drops exactly that factor, so locality re-emerges (APC 0.648, beside the
+   explicit-affinity policies) while balance stays tight — the sweet spot, no
+   gate, no tuning.
+3. **Anti-overfit, demonstrated:** unified+A+B was tuned (of=1.3, lmw=0.01) on
+   the *full* w600; on the colder 600s regime the parameter-free policy beats it
+   by 31% TTFT p90. The tuning did not transfer; LPWL did.
+
+### Per-class TTFT (ms, mean / p50 / p90 / p99) — LPWL dominates except the floor
+
+| class | LPWL p90 | A+B p90 | LPWL p99 | A+B p99 |
+|---|---:|---:|---:|---:|
+| WARM<5k | 319 | 324 | 1032 | 2092 |
+| MED5-20k | 1618 | 1952 | 3013 | 33189 |
+| HEAVY20-50k | 4851 | 6198 | 14599 | 29044 |
+| HEAVY+>50k | 28942 | 33777 | 52651 | 50778 |
+
+LPWL's only weak class is the workload-inherent HEAVY+>50k floor (≈tied across
+all policies). Elsewhere it avoids the mid-class tails the unified gate creates
+when it pins a mid request behind a 50k-token turn on a barely-warm owner.
+
+## Full-w600 cross-check (1214 reqs, `outputs/lpwl_vs_ab_live/`)
+
+On the warmer full trace, LPWL vs unified+A+B is a wash: LPWL wins TTFT p90
+(−14%) but loses TPOT (+38%) and per-worker balance. Combined claim across both
+regimes: **LPWL ∈ [tied, clearly-better] vs a tuned baseline, at zero knobs.**
+
+## Ablation: derived-κ decode term (`leastwork_kappa`) — NET-NEGATIVE
+
+Tested the proposed knob-free fix for LPWL's E2E-p99: `--policy leastwork_kappa`,
+`score = (pending_prefill + new_uncached) × (1 + κ·ongoing_decode_tokens)`, with
+κ = 2.5e-6 *derived* from hardware (KV ~100 KB/tok ÷ HBM 4 TB/s ÷ TPOT 10 ms on
+H20+Qwen3-30B-A3B), not trace-tuned. Same 600s trace, fresh vLLM, cold APC.
+
+| metric | leastwork | leastwork_kappa | Δ |
+|---|---:|---:|---:|
+| TTFT p90 | 7983 | 9390 | +18% (worse) |
+| TTFT p99 | 44891 | 42370 | −6% |
+| E2E p90 | 19014 | 21674 | +14% (worse) |
+| E2E p99 | 87024 | 90155 | +4% (did NOT fix) |
+| APC | 0.648 | 0.647 | tie |
+| req-balance | 1.55× | 1.97× | worse |
+
+**Verdict: decode-awareness is the wrong lever for agentic.** The κ term is
+correct physics aimed at a negligible effect (decode is cheap, output p50≈80),
+so it mostly bounces heavy requests off their cache-owner → cold re-prefill
+elsewhere → new hotspots (balance degrades 1.55×→1.97×). It does NOT fix E2E-p99
+because that tail is the **structural HEAVY+>50k floor** (per-class p99 ≈51–52k
+for *all* policies), not decode interference — i.e. not routing-fixable. This is
+a negative result that *justifies* LPWL's omission of any decode term. The policy
+is kept in-tree as a documented ablation; do not revive without a decode-heavy
+regime. (First run on the GPU-capturing harness: per-worker GPU util mean 42–83%,
+1.95× spread — it even shows the κ-induced imbalance.)
+
+## Caveats / open work
+
+- n=1 per arm. The 600s −31% TTFT p90 is corroborated by mean/p50/per-class, but
+  repeat to bound run-to-run noise (no 3× repeats yet, by request — quick single
+  set first).
+- E2E-p99 deep tail is the one consistent LPWL weak spot (heavy-session decode
+  concentration). Proposed knob-free fix: add `+ κ·ongoing_decode` with
+  `κ = measured(TPOT/token) / prefill_throughput` (a derived hardware ratio, not
+  a tuned scalar). Not yet implemented.
+
+## Repro
+
+```bash
+# 5-policy, 600s trace (≈18 min/arm, ~90 min total)
+OUTROOT=.../outputs/policy5_600s \
+    bash microbench/connector_tax/cache_sweep/run_5policy_600s.sh
+# unified report
+.venv/bin/python scripts/bench_report.py --root .../outputs/policy5_600s \
+    leastwork unified_ab unified_def lmetric sticky
+```

analysis/mb1/README.md

@@ -0,0 +1,193 @@
+# MB1 — Prefill–Decode Interference (chunked-prefill on, vLLM 0.18.1 default)
+
+Persistent record of the phase-interference microbench used to put a
+quantitative upper bound on **what PD-disaggregation can buy** under the
+chunked-prefill-on baseline. Re-runs append a dated section at the
+bottom; the **Summary** block is what gets cited.
+
+---
+
+## Summary (latest)
+
+| Headline | Value |
+|---|---|
+| Baseline single-stream TPOT (D=1, idle GPU) | **4.8 ms** |
+| Effective per-stream TPOT during **8k-token** prefill burst (D=8) | **114 ms (≈15× baseline)** |
+| Effective per-stream TPOT during **32k-token** prefill burst (D=8) | **388 ms (≈52×)** |
+| Effective per-stream TPOT during **131k-token** prefill burst (D=8) | **1419 ms (≈183×)** |
+
+**What MB1 actually measures**:
+
+> During a prefill burst, every ongoing decode stream is essentially
+> halted (per-stream effective TPOT is 15×–2000× baseline, scaling with
+> prefill size). The **total decode time lost per prefill event is
+> `D × T_prefill`** (D concurrent decodes each lose ~T_prefill of useful
+> work). For the trace mean (P ≈ 33k tokens, T_prefill ≈ 4.5 s) at D=8
+> that's **~36 seconds of decode-equivalent work lost per request**.
+> This is the **upper bound on what PD-disaggregation's phase isolation
+> could recover** on the decode side.
+
+**⚠ Correction (2026-05-27)**: an earlier version of this README framed
+the §3.2 PD-disagg argument as "phase-isolation benefit is capped at
+the decode duration of the new request (~50–200 ms), so MB2 transfer
+cost dominates". That framing was wrong. The correct accounting is
+benefit-per-prefill-event = D × T_prefill (aggregate decode time saved
+across all stalled streams), which is **much larger than per-request
+transfer cost**. The actual reason static PD-disagg fails in agentic
+is **D-side KV pool capacity** (`figs/f4b_pdsep_kv_wall.png`), not a
+cost-vs-benefit imbalance on phase isolation. See `RESULTS_SUMMARY.md`
+section 4 for the corrected framing.
+
+## Setup
+
+| Component | Value |
+|---|---|
+| Host | dash1, H20 96 GiB, driver 570.133.20 |
+| Venv | `/home/admin/cpfs/wjh/agentic-kv-fresh/.venv` |
+| vLLM | 0.18.1 official wheel (chunked-prefill default-on, V1 engine) |
+| Model | `/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct` |
+| Launch flags | `--tensor-parallel-size 1 --enable-prefix-caching --gpu-memory-utilization 0.9 --max-model-len 200000 --max-num-batched-tokens 8192` |
+| kv_connector | **none** (this measures pure single-GPU phase interference; PD-disagg cost lives in MB2) |
+
+## Method
+
+Adapted from `microbench/interference/driver.py`:
+
+1. Start D streaming decode requests on `/v1/chat/completions` with a
+   long max_tokens cap. Discard the first 32 tokens as warmup.
+2. After 1 s, inject one prefill-only request with `max_tokens=1` and
+   an input of `P` synthetic tokens (uuid-seeded for zero prefix-cache
+   reuse). Measure the prefill's TTFT.
+3. Bin the *during-prefill* tokens from each decode stream by whether
+   their wall-clock falls inside `[prefill_inject_ts, prefill_inject_ts +
+   prefill_ttft]`. Report inter-token p50 / p90.
+4. Bin a baseline run (D streams, no prefill injection) the same way.
+
+We additionally compute the **effective per-stream TPOT during the
+prefill burst** as the single most informative summary:
+
+```
+eff_TPOT_during = prefill_ttft_ms / (num_tokens_during_prefill / D)
+```
+
+This is the average rate at which each decode stream produces tokens
+while a prefill is in flight. Compared to baseline TPOT it gives the
+real per-stream throughput penalty (chunked-prefill p50 looks deceptively
+fine because most decode-token intervals during the burst are at normal
+speed; p90 sees the stall but is itself noisy; the effective TPOT is
+the cleanest "average over the whole burst window" number).
+
+## Results — 2026-05-27, dash1 GPU 0, chunk_tokens=8192
+
+3 D × 5 P × 3 reps. Aggregated by `analyze_mb1.py`.
+
+| D | P (tok) | base TPOT (ms) | prefill_ttft (ms) | per-stream tokens during | effective TPOT during (ms) | penalty | max PD-disagg benefit per stream (ms) |
+|--:|--:|--:|--:|--:|--:|--:|--:|
+| 1 |   2 048 | 4.79 |   163 | 4.0 |     41 |     8× |    144 |
+| 1 |   8 192 | 4.78 |   584 | 5.0 |    117 |    24× |    560 |
+| 1 |  32 768 | 4.78 | 4 515 | 5.0 |    903 |   189× |  4 491 |
+| 1 |  65 536 | 4.78 | 15 568 | 5.3 |  2 919 |   610× | 15 542 |
+| 1 | 131 072 | 4.78 | 56 765 | 5.7 | 10 017 | 2 094× | 56 738 |
+| 4 |   2 048 | 5.62 |   138 | 3.9 |     36 |     6× |    117 |
+| 4 |   8 192 | 6.08 |   574 | 4.5 |    128 |    21× |    547 |
+| 4 |  32 768 | 6.09 | 4 529 | 11.9 |    381 |    63× |  4 457 |
+| 4 |  65 536 | 5.85 | 15 587 | 19.8 |    789 |   135× | 15 471 |
+| 4 | 131 072 | 6.27 | 56 697 | 37.4 |  1 517 |   242× | 56 463 |
+| 8 |   2 048 | 7.71 |   143 | 4.5 |     32 |     4× |    109 |
+| 8 |   8 192 | 7.69 |   583 | 5.1 |    114 |    15× |    544 |
+| 8 |  32 768 | 7.42 | 4 520 | 11.7 |    387 |    52× |  4 434 |
+| 8 |  65 536 | 7.67 | 15 615 | 20.6 |    757 |    99× | 15 457 |
+| 8 | 131 072 | 7.74 | 56 991 | 40.2 |  1 419 |   183× | 56 680 |
+
+**Reading the table**:
+
+- *Baseline TPOT* grows mildly with D (4.8 ms → 7.7 ms as D goes 1 → 8).
+  Multi-stream decoding has small but nonzero contention even without
+  prefill.
+- *Effective TPOT during* grows mostly with P: a single 8k prefill stalls
+  decode for ~580 ms regardless of D, so each stream emits only a handful
+  of tokens during that 580 ms window — effective per-stream TPOT
+  collapses to 100–130 ms. Larger prefill = more chunks = larger stall.
+- *Penalty* is the eff/baseline ratio. Above 50× for P ≥ 32k. Above
+  500× for D=1 at P ≥ 65k.
+- *Max PD-disagg benefit per stream* = `prefill_ttft − per_stream_tokens
+  × baseline_TPOT` ≈ `prefill_ttft` (since interference essentially
+  halts decode). This is the entire prefill duration's worth of decode
+  time that could in principle be recovered.
+
+**Connecting to the §3.2 PD-disagg argument** (corrected):
+
+PD-disagg's promised phase-isolation benefit is **per prefill event**,
+not per request. When a new prefill arrives, it stalls every concurrent
+decode stream on the same GPU. The aggregate decode time lost across
+those D streams is `D × T_prefill`. PD-disagg moving prefill off-decode-GPU
+recovers all of it.
+
+Plugging numbers per prefill event:
+
+| Prefill size | T_prefill | PD-disagg cost (MB2 T_transfer) | PD-disagg benefit (D=8 × T_prefill) | Ratio |
+|---:|---:|---:|---:|---:|
+| 2k tok (trace lower) | 0.14 s | 8 ms | 1.1 s | 0.7 % |
+| 33k tok (trace mean) | 4.5 s | 320 ms | 36 s | 0.9 % |
+| 125k tok (~p99) | 57 s | 1.9 s | 456 s | 0.4 % |
+
+On the **phase-isolation axis alone**, PD-disagg wins by 100×–250×.
+The reason static PD-disagg nonetheless **fails in agentic** is a
+*different* failure mode: the D-side KV pool cannot fit p90+ requests
+(p99 = 11.5 GiB; D-instance pool ≈ 38 GiB; 4P+4D halves system-wide
+decode capacity → TTFT p50 62×, success rate 99.5% → 52% in colleague's
+4P+4D experiment). The structural problem is **capacity** (see
+`figs/f4b_pdsep_kv_wall.png`), not transfer-cost vs phase-isolation
+trade-off.
+
+## Reproduction
+
+```bash
+# vllm pair-free single-instance launch
+ssh dash1 'GPU=0 PORT=8000 CHUNK_TOKENS=8192 \
+  bash /home/admin/cpfs/wjh/agentic-kv-fresh/scripts/mb1_launch.sh start'
+
+# sweep
+ssh dash1 'source /home/admin/cpfs/wjh/agentic-kv-fresh/.venv/bin/activate && \
+  python /home/admin/cpfs/wjh/agentic-kv-fresh/scripts/mb1_driver.py \
+    --host 127.0.0.1 --port 8000 \
+    --model /home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct \
+    --decode-batch-sizes 1,4,8 --prefill-tokens 2048,8192,32768,65536,131072 \
+    --reps 3 --output-dir /home/admin/cpfs/wjh/agentic-kv-fresh/mb1_results'
+
+# pull + analyze
+scp dash1:/home/admin/cpfs/wjh/agentic-kv-fresh/mb1_results/chunk8192/summary.csv \
+    analysis/mb1/summary.csv
+.venv/bin/python microbench/fresh_setup/analyze_mb1.py \
+  --summary analysis/mb1/summary.csv --out analysis/mb1/breakdown.json
+.venv/bin/python microbench/fresh_setup/plot_mb1.py \
+  --mb1 analysis/mb1/breakdown.json \
+  --mb2-intra analysis/mb2/intra_kvboth_breakdown.json \
+  --mb2-inter analysis/mb2/inter_kvboth_breakdown.json
+
+# teardown
+ssh dash1 'bash /home/admin/cpfs/wjh/agentic-kv-fresh/scripts/mb1_launch.sh stop'
+```
+
+## Open questions / next runs
+
+- **Chunk size sensitivity**: this run uses `--max-num-batched-tokens
+  8192`. Sarathi-Serve goes smaller (e.g. 1024) and recovers more
+  decode interleaving inside each prefill burst. Worth running
+  chunk_tokens ∈ {1024, 2048, 4096, 16384} to map the chunk-size axis.
+- **Higher D**: 12, 16 streams to see whether the penalty saturates or
+  keeps shrinking per-stream.
+- **Cross-validate effective_TPOT_during with token-time-series plot**:
+  raw per-token timestamps could reveal whether the stall is a few big
+  spikes or many small ones (currently inferred from p50/p90 spread).
+
+## Run log
+
+### 2026-05-27 — dash1 GPU 0, chunk_tokens=8192
+
+3 × 5 × 3 sweep. CSV: `analysis/mb1/summary.csv`. Per-config JSONs on
+dash1 at `/home/admin/cpfs/wjh/agentic-kv-fresh/mb1_results/chunk8192/`.
+Figure: `figs/mb1_interference.png`. The figure
+`figs/pd_cost_vs_benefit.png` from the original commit `029821c` was
+based on the wrong "benefit ≤ decode duration" accounting; **deleted in
+the correction commit**.