MB5 PD ablation v2 results: concurrency axis + reuse 3-way writeup

- fig3_conc32k.json + fig3_concurrency_axis.png: agentic-corner concurrency
  sweep (in=32768, reuse=0.984, out=128), N 8->128, PD capped 600s / colo
  uncapped. colo completes 100% at every N (graceful, E2E 2.4s->81s); every
  static PD split collapses, earlier as N rises (viable only N<=16; <27% by N=32).
- analysis/mb5_pd_ablation/README.md: self-contained v2 writeup. Reuse axis
  3-way (A=d2048/o256, C=d2048/o128, B=d1024/o128) decomposes shape: output
  ~negligible, delta (real prefill/turn) dominant; crossover to colo at reuse
  ~90-95% robust. Run on dash2 (dash0 NICs faulted for Mooncake).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

.gitignore

@@ -3,7 +3,14 @@ __pycache__/
 .venv/
 *.egg-info/
 outputs/
-traces/
+traces/*
+# ship the anonymized sampled trace + its provenance (metadata only, no cleartext)
+!traces/w600_r0.0015_st30.jsonl
+!traces/README.md
 *.log
 .claude/
 # third_party/vllm tracked in git for patch management
+!traces/w600_r0.0015_st30_first600s.jsonl
+# + time_to_parent_chat annotation (for --dispatch-mode thinktime); same anon data
+!traces/w600_r0.0015_st30_ttp.jsonl
+!traces/w600_r0.0015_st30_first600s_ttp.jsonl

MEETING.md

@@ -0,0 +1,137 @@
+# EAR — Agentic Serving Scheduler 汇报
+
+**One-liner**：Agentic workload 的 KV reuse 93% 在 session 内，turn 间 tool-call 反馈耦合把单 request 延迟差放大成 throughput 差距 —— locality 因此是主导调度杠杆；现有 load-balance 丢 locality、static PD-disagg 撞 D 侧 KV 墙、pure sticky 造 hot pin；我们提 EAR (Elastic Affinity Router) = session-affinity routing + hot-instance 触发 session migration。
+
+---
+
+## 1. 关键洞察：Dispatch Coupling
+
+每个 turn 间有一段外部 gap `T_external`（chatbot 是人类读+想+打字；agentic 是 tool 执行）。**Little's Law `L = Λ · N · (W_turn + T_external)`** 在两种 workload 下都成立 —— 差异在于 `T_external` 的分布相对于 `W_turn` 的位置：
+- `T_external ≫ W_turn` → 开环 regime：scheduler 退一步不动 L
+- `T_external ≲ W_turn` → 闭环 regime：`W_turn(L)` 因 KV 竞争耦合到 L，反馈环把 scheduler 的 ε 退步放大几倍
+
+**Production trace 实测 `T_external` 分布**（next.start − prev.end，formatted session 链作 ground truth）：
+
+![](figs/f3a_inter_turn_gap.png)
+
+| | Agentic | Chatbot |
+|---|---:|---:|
+| p50 | **1.6s** | **7.2s** |
+| gap < 1s | **39%** | 4% |
+| gap < 5s | 67% | 29% |
+| p99 | 738s | 43s |
+
+两个分布形状完全不同：chatbot unimodal 集中在 5–15s（人类节奏）；agentic bimodal —— **39% 的 gap 在 sub-second 里（autonomous tool-call mode）**，外加 13% > 30s 的长尾。**Agentic 的 sub-second mass 是 chatbot 没有的**，正是 dispatch coupling 激活的来源。
+
+**实测 regime**：在 unified（TTFT p90 = 7.3s）下，**73% 的 agentic turn 把系统推进闭环**（W_turn > T_external），chatbot 仅 32%。在 lmetric（15.7s）下 agentic 80%、**chatbot 也到 88%** —— lmetric 把 chatbot 自己也拖进闭环，这就是它在两种 workload 都 underperform 的根因。
+
+**结果**：lmetric 跑 600s trace 用 49 min wall-clock = **8x amplification**。**per-turn W 的小差异被放大成 wall-clock 数量级差距** —— locality 不是 nice-to-have，是 dominant lever。
+
+---
+
+## 2. Workload 实证（三件事）
+
+| | 数据 | 图 |
+|---|---|---|
+| KV reuse 几乎只在 session 内 | intra 93.2% / cross 5.7% / shared 1.1% | ![](figs/f2a_reuse_topology.png) |
+| Session 极度偏斜 | production trace 上 top 1% / 5% / 10% / 25% / 50% = **46.5% / 66.5% / 74.6% / 87.5% / 96.0%** input mass | ![](figs/f2b_session_skew.png) |
+| 单请求 KV footprint 大，单 instance KV pool 很快被占满 | per-instance KV pool ≈ **38 GiB**（0.4 × 96 GiB H20，剩 50% params + 10% activation）；p99 req 11.5 GiB → 一个 instance 只装 **3 个 p99 decode**；4P+4D 让系统 decode 容量直接减半 | ![](figs/f2c_kv_footprint_cdf.png) |
+
+理论 APC 上界 = intra-session 79.6% / any-session 80.3%，差 <1pp。**任何不 affinity 的调度都丢绝大部分 reuse。**
+
+---
+
+## 3. 现有调度的三种失败模式
+
+### Load-balanced (LMetric / round-robin / kv-aware)：丢 locality
+
+![](figs/f4a_apc_loss.png)
+
+LMetric 56.9%、load_only 54.1% APC，远低于 79.6% 上界。23pp 缺口直接来自跨 instance 路由丢的 intra-session hit。
+
+注意 LMetric 比 load_only 只好 **+3.3pp**：LMetric 的 score = `(pending_prefill + input − cache_hit) × num_requests`，cache_hit 只作 cost-model 减项，但 score 是**乘性**的 —— 一个有 affinity 的 instance 因 num_requests 高被乘式吃掉 cache 收益，LMetric 仍然会选冷 instance。sticky 把 cache 作硬约束直接拉到 77.2%。**结论：cache-aware-load routing 不够 —— affinity 必须是独立路由路径，不能折叠进 load cost 里**。
+
+### 静态 PD-disagg：D 侧 KV 容量墙
+
+![](figs/f4b_pdsep_kv_wall.png)
+
+agentic 平均请求 33.6k token 需 3.3GB KV；4P+4D / 6P+2D 在 agentic regime 都穿过 90% 内存墙。**TTFT p50 暴涨 62-72x，成功率 99.5% → 52-68%**。
+
+### Pure sticky：全员被 hot session 拖累
+
+![](figs/f4c_per_worker_ttft.png)
+
+我们刻意 **用 (median, max) 两个绝对数**衡量 worker 不平衡，不用 `max/median` 单一比值 —— 比值会把 unified（一个 worker 牺牲、其他 7 个快）算成比 sticky（全员一起慢）更不平衡，与系统 e2e p90 实际排序反向。下面是绝对数：
+
+| | median worker TTFT p90 | max worker | system e2e p90 |
+|---|---:|---:|---:|
+| sticky | **20.3s** | 55.4s | **34.6s** |
+| unified | **10.3s** | 37.7s | **18.0s** |
+
+机制：production trace 上 top 1% 的 session 占 46.5% input 量、且 hot session 数量远多于 instance 数（8 个），sticky 的 hash 绑定让 **每个 worker 都自己承接一份 hot session**，median worker 也被拖慢。Unified 用 LMetric fallback 把 cold/new session 重路由到非 hot worker，保留 7/8 worker 的速度。系统 p90 由大多数请求决定，所以 unified 几乎 2x 快。
+
+---
+
+## 4. EAR 设计
+
+两个 pillar，所有 instance 对称 PD-colocated（无静态 P/D 分区）：
+
+**Pillar 1 — Affinity-default routing（已实现）**
+新 session 用 load-balance 分配 host；后续 turn 按 session→host binding 路由。
+→ 这就是当前 `unified` 算法（hybrid LMetric + high-cache affinity），APC 79.4%，达到上界 97%。
+
+**Pillar 2 — Hot-triggered session migration（end-to-end 实证待补，substrate 已验证）**
+当 host 的 `pending_prefill_tokens > T_hot`，把整个 session 的 KV 通过 mooncake `kv_connector` migrate 到更轻 instance；session binding 更新；后续 turn 路由到新 host。
+
+> 🆕 **2026-05-27 数据**（commit `ef9e010`）：之前认为是 migration blocker 的 `kv_both` substrate overhead 已经不存在。在 8×TP1 trace replay 上 A/B/C 对比：
+> - plain unified: TTFT p90 = 11.97s
+> - unified + `kv_both`（未 DR-fix）: 9.74s（**−18.6%** vs plain）
+> - unified + `kv_both` + DR-fix: 7.58s（**−36.6%** vs plain）
+>
+> 即原 elastic_migration_v2 论文里 "+45% kv_both penalty" 已 obsolete；当前 substrate 是 **net positive**（connector mode 的 `delay_free_blocks=True` 在 93% intra-session-reuse trace 上把跨 turn cache hit 窗口拉长）。Migration 之前 4 次 revert 的主因消失。
+
+关键 design：
+- Target 选择用 **observable pending prefill tokens**，**不用** cost-model prediction（实测 mooncake cost model 误差 10-21x，绕过）
+- Per-session cooldown 防 thrashing
+- 若无候选 instance 能装下 session context → 保留当前 binding，opportunistic 不 mandatory
+
+---
+
+## 5. 进展 & TODO
+
+### ✅ 已完成
+
+- Workload characterization 三件事的实证齐全（`f2a/b/c`）
+- 三类 baseline 失败的实证齐全（`f4a/b/c/d`）
+- Anchor + paper outline（`PAPER_OUTLINE.md`）
+- Pillar 1 affinity routing 已实现并测过（current `unified` 算法）
+- Dispatch coupling 的 Little's Law 形式化推导
+- `replayer/replay.py` patched 输出 `amplification`
+- 🆕 **kv_both substrate validation**（commit `ef9e010`）：trace replay A/B/C 证明 substrate 已经是 net positive（TTFT p90 −18.6% / DR-fix 后 −36.6% vs plain），原 +45% penalty obsolete
+
+### 🟢 不依赖 migration 可以现在做
+
+1. **5 baseline × 3 runs wall-clock sweep**（patched replayer 直接出 amplification 字段）— §2.3 的实证 closure，**最高优先级**，一晚能跑完
+2. Static PD-disagg 补进 end-to-end 表
+3. λ / skew / KV pool 三轴 sensitivity
+4. Draft §1-§4 正文（数据已齐）
+
+### 🚧 待 migration end-to-end validation
+
+- §4.3 migration mechanism 的 e2e trigger + target selection 实验（substrate 已通，只缺策略层）
+- Full ablation（migration-only + both）
+- §5.6 migration microbench
+
+### 风险
+
+- Migration 之前 4 次尝试（`6b255fa`, `e991960/5772149`, `cc6e562`, `4c583f2`）都被 transfer overhead 吞掉而 revert ——  **该 overhead 已在 2026-05-27 验证不再存在**（substrate net positive）
+- 仍未直接验证 e2e migration 策略层（trigger + target 选择）能在反馈环里产生正收益；中间还有"决策错误 + cooldown thrashing"两类风险，独立于 substrate
+- 即便 migration e2e 仍 marginal，affinity-only pillar 的实证已经独立成立，paper 至少有 strong-affinity 的 storyline 可写
+
+---
+
+## 6. 一句话总结要 sell 的事
+
+> **Agentic 让 locality 从 nice-to-have 变成 dominant lever（dispatch coupling 论证）；EAR 用 affinity-default + hot-triggered migration 单一方案同时拿到 locality 和 balance。Pillar 1 已实证（APC 79.4%）；Pillar 2 design 完整、validation pending in DR-fix 之上的重测。**
+
+下一步主战场：跑 wall-clock sweep 把 §2.3 dispatch coupling 论证钉死。

PAPER_OUTLINE.md

@@ -0,0 +1,362 @@
+# GPU-Hit-First: Serving Agentic LLM Workloads by Keeping the Working Set in HBM
+
+> **Thesis (one-liner)**: 对 agentic LLM 负载，用户感受到的端到端 metric 是 **request latency / TPS / GPU
+> utilization**，而它们由一件事主导 —— **KV cache 命中是否发生在 GPU HBM 上**。Agentic 的 KV reuse 93% 在
+> session 内、且活跃 working set 小到一个节点就能常驻 HBM；命中层级 `GPU ≫ CPU-local > remote-RDMA-store ≫
+> recompute` 的代价差随 context 拉大。由此得到一条统一原则 —— **GPU-hit-first**：把活跃 working set 留在 HBM，
+> 而不是建深的 CPU/storage hierarchy 去追长尾。三个推论分别修复现有系统的三处失配：(3.1) 让 PD-colocation 重新
+> 成为默认；(3.2) 在全局路由里做 biased KV-cache-awareness；(3.3) 用 KV migration 而非 replication 做跨实例
+> GPU 去重。
+
+> **Framing note (2026-05-30)**：本 outline 取代早期的 "EAR: Elastic Affinity Routing" 版本（保留在 git
+> 历史里）。原 EAR 的 dispatch-coupling 形式化在此 **降级为 §2 的 metric 论证**（解释"为什么是 request latency
+> 而不是 TTFT/TPOT"），不再是 headline；headline 升格为 GPU-hit-first 原则，affinity routing / migration
+> 成为该原则的两个推论（§3.2 / §3.3）。
+
+---
+
+## 📊 Validation Status (2026-05-30)
+
+| 章节 | 论点 | 证据 | 状态 |
+|---|---|---|---|
+| §1 | 背景：PD-colo / PD-disagg / KV storage hierarchy | — | 写作 |
+| §2 metric | request latency 而非 TTFT/TPOT；TPS；GPU util | dispatch coupling + amplification；`bench_report.py` | 🟡 论证全，wall-clock sweep 待补 |
+| §2.1 | KV hit 普遍且关键 | `f2a` intra 93.2%、APC 上界 79.6%、C1/C2 | ✅ |
+| **§2.2** | **GPU hit > CPU hit > RDMA-store hit ≫ miss** | **`v2/figs/exp_a_tier_latency.png`（四层实测）** | ✅ **NEW** |
+| **§2.2 Ev#1** | **GPU 足以常驻"有价值的" working set** | **`v2/figs/exp_b_capacity_knee.png`（knee）+ working_set + cluster-scale 校正** | ✅ **NEW** |
+| §3.1 | Make PD-colocation great again | `PD_DISAGG_RESULTS`、`crossover_pd_advantage`、MB1/MB2、C2/C3 | ✅ |
+| §3.2 | Biased KV-aware global routing | LPWL（TTFT p90 −31%）、LMetric 乘性稀释、sticky hot-pin、ES ablation | ✅ |
+| §3.3 | GPU dedup via migration not replication | substrate net-positive（−18.6%/−36.6%）；correctness smoke tests | 🟡 substrate 通，policy e2e 待验证 |
+| §4 | 集成系统端到端 eval | 散落 mb1/mb2/mb5/crossover/lpwl，需统一 | 🚧 |
+| §5 | Related work（含 storage hierarchy 正面回应）| — | 写作 |
+
+---
+
+## §1 Background and System Setup
+
+### §1.1 LLM 与 KV cache
+
+Transformer 自回归推理分两段：**prefill**（一次性算完 prompt 的全部 KV，compute-bound）与 **decode**（逐 token
+生成，memory-bandwidth-bound）。每个 token 的 KV 常驻 GPU HBM 才能被后续 attention 复用。Prefix caching（APC）让
+相同 prompt 前缀直接命中已算好的 KV，省掉重复 prefill —— 这是本文全部优化的物理基础。
+
+> Qwen3-Coder-30B-A3B（GQA, 48 层, 4 KV heads, head_dim 128, bf16）：**KV = 96 KiB/token**，1 GiB = 10,923
+> token，block(16 tok) = 1.573 MB。
+
+### §1.2 Agentic workflow
+
+Agentic 负载 = LLM 通过 tool-call 自驱、多 turn 完成任务。与 chatbot 的本质差异：每个 turn 由上一个 turn 的
+tool-call 结果触发（无人类 think-time），prefill-dominated（input/output ≈ 75×）。
+
+**但它是一个 mixture，不是"全多轮"**（C1, `figs/workload_chars/c1_session_mixture.png`）：
+
+- **90.3%** 的 session 是单轮（mean 1.62 turns）；但多轮 session（9.7%）= **44.2% 的请求**、**66.9% 的
+  prefill 质量**。
+- Continuation hazard（Lindy）：turn1→2 仅 **10.2%**，turn5→6 87%，turn12→13 **94.3%** —— heaviness 在
+  cold-start 几乎不可预测（corr(turn1_input, n_turns) = **0.04**）。
+
+> **Routing 含义（贯穿 §3.2）**：heaviness 在 session 起点不可预测 → 必须 **reactive**（观测累计负载），不能
+> proactive 预判；单轮海洋与深尾的最优策略相反（前者 load-balance、后者 affinity-pin），且 turn-1 无法区分 →
+> 唯一可行的策略是"人人 load-balanced 起步、随 turn 累积变 sticky" —— 正是 LPWL 的 emergent 行为。
+
+### §1.3 Serving agents in the wild
+
+- **PD colocation（8C）**：每个 instance 对称，prefill 与 decode 在同一 GPU 上由 chunked-prefill +
+  continuous batching 交错。弹性 KV 池，无静态分区。
+- **PD disaggregation**：把 instance 静态分成 prefill 池（P）与 decode 池（D），物理隔离两个阶段
+  （DistServe / Splitwise）。
+- **KV cache storage hierarchy**：GPU HBM → CPU DRAM → 远端 pooled store（RDMA/SSD，如 Mooncake Store /
+  LMCache）。把被淘汰/跨实例的 KV 下沉到更慢但更大的层，用传输换重算。
+
+> 三者各被本文一节回应：PD-colo 在 §3.1 被"复活"为默认；PD-disagg 在 §3.1 被证否（agentic regime）；storage
+> hierarchy 在 §2.2 被定量地"限位"（GPU 命中远胜下层，且活跃 working set 本就装得下）。
+
+---
+
+## §2 GPU memory hit is the key to serving agents
+
+### §2.0 正确的 metric：request latency / TPS / GPU utilization（不是 TTFT/TPOT）
+
+**为什么不是 per-request TTFT/TPOT**：agentic 的 turn 之间有反馈环，单 turn 的延迟会跨 turn **复利**成 session
+端到端时间与系统吞吐差距。只有 **request/session latency、tokens-per-second、GPU utilization** 能 capture 这件事。
+
+**Dispatch coupling（降级为本节论证）**。每个 turn 间有外部 gap `T_external`（chatbot 是人在读/想/打字；agentic
+是 tool 执行）。Little's Law：`L = Λ · N · (W_turn(L) + T_external)`。当 `W_turn ≳ T_external` 时
+`W_turn(L)` 经 KV 竞争耦合到并发 L，scheduler 的 ε 退步被反馈环放大成 wall-clock 数倍差。
+
+production trace 实测 `T_external` CDF（`figs/f3a_inter_turn_gap.png`）：
+
+| | Agentic | Chatbot |
+|---|---:|---:|
+| p50 | **1.6 s** | 7.2 s |
+| gap < 1 s | **39%** | 4% |
+| gap < 5 s | 67% | 29% |
+| p99 | 738 s | 43 s |
+
+agentic 有一段 chatbot 没有的 **sub-second tool-call mass（39% vs 4%）**，几乎天然 `W_turn ≫ T_external` → 闭环。
+**实测**：lmetric 跑 600s trace 用 49 min wall-clock = **8× amplification**。**结论：per-turn 延迟的小差被放大成
+端到端数量级差 → 必须用 request latency / TPS / GPU util 衡量。**
+
+- **TPS / GPU util** 的工具：`microbench/fresh_setup/bench_report.py`（TTFT/TPOT/E2E 全分位 + TPS +
+  per-worker GPU util）。PD stall 时 GPU ~0% util vs colo 34%（§3.1）即是 GPU-util 作为 metric 的直接体现。
+
+> **🚧 待补**：5 baseline × ≥3 runs 的 wall-clock amplification sweep（replayer 已输出 `amplification` 字段），
+> 钉死本节实证 closure。优先级高。
+
+### §2.1 KV$ hit is common and critical
+
+Trace 上 KV reuse 的分解（`figs/f2a_reuse_topology.png`）：**intra-session 93.2% / cross-session 5.7% /
+shared-prefix 1.1%**。理论 APC 上界：intra-only **79.6%** vs any-session 80.3%，差 <1pp —— **cache 本质上是
+session-local 的**。
+
+per-turn 视角（C2, `figs/workload_chars/c2_work_amortization.png`）：resident context 11k→56k+ token 增长而
+new-prefill 从 2.7k 坍缩到 ~200 token，per-turn reuse 爬到 **99.6%**，resident/new（"PD tax"）到 turn 12 ≈
+250×、turn 30 ≈ 450×。**绝大部分 prefill 工作是可被命中省掉的**；命中与否直接决定 TTFT。
+
+### §2.2 Hits on GPU is more important than the CPU
+
+**命中层级的代价是实测的，不是断言的**（Qwen3-Coder-30B-A3B / H20）。TTFT(s, p50) 服务一段长 L 的复用前缀，
+来自每个 KV 层：
+
+![四层命中延迟：GPU < CPU-local < remote-RDMA-store ≪ miss，差距随 context 拉大](v2/figs/exp_a_tier_latency.png)
+
+
+| prefix L | miss(recompute) | **remote RDMA store** | CPU-local(DRAM,PCIe) | GPU(HBM) | miss/RDMA | RDMA/CPU | CPU/GPU |
+|---:|---:|---:|---:|---:|---:|---:|---:|
+| 8k  | 0.588 | 0.151 | 0.076 | 0.053 | 3.9× | 2.0× | 1.5× |
+| 16k | 1.547 | 0.262 | 0.105 | 0.063 | 5.9× | 2.5× | 1.7× |
+| 32k | 4.604 | 0.680 | 0.158 | 0.080 | 6.8× | 4.3× | 2.0× |
+| **64k** | **15.23** | **0.97** | **0.27** | **0.11** | **15.8×** | **3.6×** | **2.4×** |
+
+- **GPU hit ~flat**（42→111 ms / 1k→64k）：命中即整段前缀在 HBM，只重算最后一个 token。
+- **CPU-local hit** transfer-bound（PCIe H2D 实测 ~54 GB/s）；CPU-hit ≈ GPU-hit + KV/PCIe + ~0.15s 开销。
+  （native KV offload，命中经 `vllm:external_prefix_cache_hits` 100% 验证。）
+- **remote RDMA-store hit** = Mooncake-Store 机制（实测：两 instance，B 用 `do_remote_prefill` 经 RDMA 从 A 拉取
+  缓存前缀而非重算；`mb2_kv_transfer.py` / `v2/.../run_rdma.sh`）。对 recompute 是大赢（**最高 16×**，与 blog 的
+  46× 同向），但付 **NIC 税**（有效 ~5–7 GB/s，cf. MB2 raw ~9.7 GB/s；multi-NIC pooling 可抬高），故比 CPU-local
+  慢 3.6×、比 GPU 慢 ~9×（64k），**代价差随 context 拉大**。
+- **结论 —— 层级严格且随 context 拉大：`GPU < CPU-local < remote-RDMA-store ≪ miss`**。global KV store 确实
+  有用（这也是该路线存在的理由），但每靠近 GPU 一层就再省 1.4–4× TTFT。**最值钱的复用是 GPU-resident 的那种。**
+
+#### Evidence #1：GPU is sufficient to hold most KV requests
+
+**realized APC 与 latency 在很小的 GPU 容量就饱和**（closed-loop 多轮负载，并发 4，扫 GPU KV 容量）：
+
+![容量 knee：APC 与 TTFT p90 在 3.6 GB（=活跃 working set）饱和，之后 dead-flat](v2/figs/exp_b_capacity_knee.png)
+
+| GPU KV (GB) | realized APC | TTFT p90 |
+|---:|---:|---:|
+| 1.2 | 7.4% | 13.00 s |
+| 2.4 | 36.3% | 4.62 s |
+| **3.6** | **80.3%** | **0.53 s** |
+| 9.7 | 72.9% | 0.65 s |
+| 14.5| 72.9% | 0.65 s |
+
+**Knee 出现在 3.6 GB = 恰好 = 活跃 working set（4 session × 0.91 GB）**：APC 饱和到上界、TTFT p90 从 13.0s 坍缩
+到 0.53s，之后 dead-flat。**超过 working set 的 HBM 买不到额外收益；为追长尾而建的 CPU/storage tier 同理 ≈ 0。**
+
+**Cluster-scale 校正（关键）**：working_set 分析（`analysis/working_set/`，`figs/working_set/`）显示"装下整个
+2h cluster 的全部 reuse 尾巴需 ~14 节点"，**这不构成 CPU offload 的动机** —— 那是用 1 个 replica 容量去装**整个
+cluster** 的 reuse；产出该 trace 的真实 cluster 远不止 14 节点（trace 是 cluster 聚合，见
+`project-trace-is-cluster-level`）。在 per-cluster 的 HBM 总量下，活跃 working set 本就 GPU-resident（live KV
+533–1157 GB ≪ 单节点 1528 GB）。**knee 位置随并发线性增长 = 随 cluster GPU 数增长**，而 cluster 本就提供了它。
+
+> ⚠ **Scope**：本小节"装得下"指的是**活跃 working set 产生的近期高价值复用**，不是"全部 reuse 尾巴"。冷
+> session 长 gap 后回来的深尾命中（既低价值/byte 又贵于 fetch）正是 storage-hierarchy 派追的东西；本文论点是
+> 在 agentic 下这条尾巴不值得为之建深层级。§5 正面回应该派。
+
+### §2.3 Takeaway
+
+正确的 metric（§2.0）+ 命中集中在 GPU 才便宜（§2.2）+ 活跃 working set 装得下 HBM（Ev#1）⇒ **GPU-hit-first**：
+设计目标是最大化活跃 working set 的 **GPU 常驻 + 命中**，而非建深 CPU/storage hierarchy。§3 给出三个推论。
+
+---
+
+## §3 Optimizing agent serving with the GPU-Hit-First Principle
+
+### §3.1 Make PD-colocation great again
+
+静态 PD-disaggregation 对 chatbot 有效，对 agentic 结构性失败 —— colocation 才应是默认。
+
+**端到端证据**（`microbench/fresh_setup/PD_DISAGG_RESULTS.md`，8×H20，trace replay）：**没有任何静态 P/D 比能赢
+8-way colocation（8C）**，且失败模式随比例移动：
+
+| Metric | **8C** | 6P+2D | 4P+4D | 2P+6D |
+|---|---:|---:|---:|---:|
+| completion | **100%** | 100% | 100% | **9%** 💀 |
+| wall-clock | **2994 s** | 3419 | 4171 | 5762 |
+| prefix-cache hit | **19.4%** | 0% | 0% | 0% |
+| TTFT p50 | **7.0 s** | 41.0 | 56.4 | 23.6 |
+| E2E p90 | **83.3 s** | 91.8 | 157.1 | 499 |
+
+- **D-heavy（4P+4D）**：decode 池饱和 **97.5%**、prefill 池 ~30% —— 半个 cluster 的 KV 被困在错的一侧；agentic
+  请求大（p99 KV **11.5 GiB**），4D 让系统 decode 容量直接减半（24→12 并发，`figs/f2c_kv_footprint_cdf.png`、
+  `figs/f4b_pdsep_kv_wall.png`）。
+- **P-heavy（2P+6D）**：prefill 池 jam 99.7%，872 请求堆积，**91% 永不完成**。
+- **更聪明的路由救不了**（§6.x）：给 P 侧加 session-affinity 反而更差（4P+4D completion 100%→36%），GPU ~0%
+  util，cluster 卡在 KV-transfer 协调而非 compute —— 复现 producer hot-pinning。
+
+**为什么 colo 赢（正确论证，C2/C3 支撑）**：
+
+- **时变 P:D 需求**：agentic 同时在 roofline 两侧有实质工作 —— compute-bound prefill（~30% 时间）+
+  memory-bound decode（**~70% 时间**，C3 token≠time 校正，`figs/workload_chars/c3_prefill_decode_balance.png`）。
+  colo 的弹性池吸收当下热的那一相；静态分区让 P-instance 带宽闲、D-instance 算力闲。
+- **resident KV 本地化**（C2）：下一 turn 的 prefix = [prevPrompt+prevAnswer] 横跨 P/D 两侧，disagg 必须
+  gather/transfer，colo 免费本地保留。
+- **transfer 不便宜且拓扑无关**（MB2，`figs/mb2_transfer_time_compare.png`）：Mooncake `batch_transfer_sync_write`
+  恒走 RDMA NIC（~9.7 GB/s），intra ≈ inter；PD-disagg 的 per-request transfer 税无法靠拓扑买回。
+- **phase-isolation 是 disagg 唯一的真赢面但被压倒**（MB1，`figs/mb1_interference.png`：32k prefill 让
+  per-stream TPOT 退化 52×，131k → 183×）—— 但被 D 侧容量天花板压倒（见上）。
+
+**边界（不 overclaim）**：crossover sweep（`analysis/crossover/`，`figs/crossover_pd_advantage.png`）给出 colo
+停止占优的 input 长度 —— colo 在 agentic 工作点赢，且我们知道边界在哪。
+
+### §3.2 Biased KV-cache-awareness in global routing
+
+GPU-hit-first 在路由层 = **把 cache-awareness 作为带偏置的独立路由路径**，而不是折叠进 load cost。
+
+**反例：load-balanced / 朴素 cache-aware-load 丢 locality**（`figs/f4a_apc_loss.png`）。LMetric（OSDI'26）打分
+`P = pending_prefill + (input − cache_hit)`，`score = P × num_requests` —— cache 只作 cost-model **减项**，而 score
+是**乘性**的，有 affinity 的 instance 因 num_requests 高被乘式吃掉 cache 收益：
+
+| 策略 | APC | vs load_only | cache 处理方式 |
+|---|---:|---:|---|
+| load_only | 53.9% | — | 无 |
+| LMetric | 57.2% | **+3.3pp** | cost-model 减项（被稀释）|
+| sticky | 77.7% | **+23.8pp** | 硬约束 |
+| unified | 78.7% | **+24.8pp** | 硬+软混合 |
+
+`load_only→LMetric` 的 +3.3pp 几乎可忽略；**+20.5pp 的回报来自把 cache 作独立路由路径**。
+
+**本文方法：LPWL（least-prefill-work，parameter-free）**（`project-lpwl-policy`、`analysis/lpwl_5policy_600s.md`）：
+按 `new_uncached ≈ input − cache_hit` 路由 —— `new_uncached≈input`（冷/新 session）自动按负载分散，`new_uncached≈0`
+（暖 session）自动 stick。**零旋钮**，在 600s trace 上击败 tuned unified+A+B（TTFT p90 **−31%**），full w600 上打平。
+这正是 C1 mixture 要的形状：无需 classifier 自动切分单轮海洋与深尾。
+
+**两条被否的杠杆（节省读者时间）**：
+
+- **real-time engine state 不是路由杠杆**（ES ablation，`project-es-ablation-sweep`）：只对一次性 placement 有用
+  （sticky −26%），对 per-req load-chasing 有害（load_only +27%）。
+- **derived-κ decode 项 net-negative**：decode-awareness 是错的杠杆。
+
+**Pure sticky 的失败用绝对 per-worker latency 衡量**（`figs/f4c_per_worker_ttft.png`，不能用 max/median 比值）：
+
+| | median worker TTFT p90 | max worker | system e2e p90 |
+|---|---:|---:|---:|
+| sticky | **20.3 s** | 55.4 s | **34.6 s** |
+| unified | **10.3 s** | 37.7 s | **18.0 s** |
+
+hot session 数 ≫ instance 数（8），sticky 的 hash 绑定让**每个 worker 都自接一份 hot session**，median 也被拖慢；
+biased 路由把 cold/new 重路由到非 hot worker，保 7/8 worker 速度 → e2e p90 ~2× 快。**这引出 §3.3：sticky 的残余
+hot-pin 需要 migration 解。**
+
+### §3.3 GPU KV-cache dedup with migration instead of replication
+
+**视角**：多个 instance 各自缓存同一段 prefix = GPU 容量被 **replication** 浪费；GPU-hit-first 要求**全局只留一份 +
+把 session 搬到那一份**（migration/dedup），既保 GPU 命中又均衡负载、修复 §3.2 的残余 hot-pin。
+
+- **Trigger**：host 的 `pending_prefill > T_hot` 且 session 在 `T_cool` 内未迁移过。
+- **Target**：用 **observable pending prefill tokens** 选最轻 instance，**不用** cost-model 预测（mooncake cost
+  model 误差 10–21×，by construction 绕过）。
+- **Mechanism**：当前 request 重定向到 target，session KV 经 Mooncake `kv_connector` 迁移；binding 更新；后续
+  turn 按 affinity 路由到新 host。迁移成本在该 session 剩余 turn 上摊销。
+- **Thrashing prevention**：per-session cooldown `T_cool`。
+
+> **状态**：**substrate 已验证 net-positive**（`kv_both` connector 在 trace replay 上 TTFT p90 **−18.6%**，
+> DR-fix 后 **−36.6%** vs plain；之前认为是 blocker 的 transfer overhead 已不存在，4 次历史 revert 的主因消失）。
+> migration correctness smoke tests 通过。**policy 层 e2e（trigger + target 在反馈环里的真实收益）仍未直接验证**
+> —— 这是全文最弱的一环，独立风险是"决策错误 + cooldown thrashing"。affinity-only pillar（§3.2 LPWL）已独立成立，
+> 即便 migration 仍 marginal，paper 也有 strong-routing 主线。
+
+---
+
+## §4 System Evaluation
+
+> **🚧 关键缺口**：目前证据散落在 mb1/mb2/mb5/crossover/lpwl/v2；§4 需要一个**集成系统**（colocation +
+> biased routing + dedup-migration，统一命名）跑端到端、用 §2.0 的新 metric（request latency / TPS / GPU util）
+> 评测，并把 §3.1/§3.2/§3.3 做成 ablation。
+
+### §4.1 Setup
+- **Trace**：真实 Qwen3-Coder agentic trace（cluster-level，见 `project-trace-is-cluster-level`）；日常迭代
+  `w600_r0.0015_st30`（~850 req / ~13 min / peak QPS ~1.6 / APC headroom ~76%）。
+- **Hardware**：8× H20 (96 GB)；vLLM 0.18.1 (V1, chunked-prefill) + Mooncake。
+- **Baselines**：① round-robin ② LMetric (OSDI'26) ③ kvcache-aware+load 线性混合 ④ sticky ⑤ static PD-disagg
+  (4P/4D) ⑥ **本文系统**（colo + LPWL + dedup-migration）。
+- **Metrics**：request latency (mean/p50/p90/p99)、**TPS**、**GPU util (median/max worker)**、APC、
+  wall-clock amplification（不单看 TTFT/TPOT，§2.0）。
+
+### §4.2 End-to-end
+- `figs/f6_e2e_latency_bars.png` / `f6_e2e_latency_full_grid.png`（现有 4–5 baseline；🚧 补 static PD-disagg
+  列 + 本文系统列）。
+
+### §4.3 Ablation（GPU-hit-first 三推论各关一个）
+- −colocation（→ static PD-disagg，§3.1）
+- −biased routing（→ load-balance / LMetric，§3.2）
+- −dedup-migration（→ pure sticky，§3.3）
+- 🚧 migration-only / full 待 policy e2e 验证。
+
+### §4.4 Microbench 支撑（已有，复用）
+- §2.2 四层命中：`v2/figs/exp_a_tier_latency.png`
+- §2.2 容量 knee：`v2/figs/exp_b_capacity_knee.png`
+- §3.1 PD-disagg 失败：`figs/f4b`、`PD_DISAGG_RESULTS`、`crossover_pd_advantage`
+- transfer cost：`figs/mb2_*`；phase interference：`figs/mb1_interference.png`
+
+### §4.5 Sensitivity
+- 到达率 λ / skew(Zipf α) / KV pool size（不依赖 migration，可做）；`T_hot`/`T_cool`（依赖 migration，deferred）。
+
+---
+
+## §5 Related Work
+
+- **Serving systems**：vLLM, Mooncake, SGLang, DistServe, Splitwise。本文基于 vLLM+Mooncake，与
+  DistServe/Splitwise 不同在于**不做静态 P/D 分区**（§3.1 给出 agentic regime 的证否 + crossover 边界）。
+- **Cache-aware routing**：LMCache, Production-Stack, LMetric (OSDI'26)。本文指出 cache 信号不能折叠进乘性 load
+  score（§3.2 LMetric 稀释分析），须作独立 biased 路由路径。
+- **KV storage hierarchy / offload（主要对手，正面回应）**：Mooncake Store, LMCache, AttentionStore 等把 KV 下沉
+  到 CPU/SSD/远端 pool。**本文不否认其对 recompute 的收益**（§2.2 实测 remote-RDMA 命中比 recompute 快达 16×，
+  与 Mooncake-Store blog 的 46× 同向）；但论证在 **agentic** 下 (i) 命中层级 GPU ≫ CPU ≫ RDMA-store（§2.2），
+  (ii) 活跃 working set 本就 GPU-resident（§2.2 Ev#1）⇒ 深层级的边际收益低，应优先 GPU 常驻而非建深 hierarchy。
+- **Stateful migration**：Pollux, Gandiva（RL training 的 migration-as-rebalancing）。本文把该思路迁到 LLM KV
+  cache 的 dedup（§3.3）。
+
+---
+
+## §6 Conclusion
+
+对 agentic LLM 负载，用户感受的 request latency / TPS / GPU util 由 **KV 命中是否在 GPU HBM 上** 主导。命中层级
+`GPU ≫ CPU > RDMA-store ≫ recompute` 且代价差随 context 拉大（§2.2），而活跃 working set 小到本就 GPU-resident
+（§2.2 Ev#1）。**GPU-hit-first** 原则由此统一三件事：复活 PD-colocation（§3.1）、在路由里做 biased
+KV-awareness（§3.2）、用 migration 而非 replication 做 GPU 去重（§3.3）。
+
+---
+
+## Work Plan
+
+### ✅ Done
+- §1/§2 背景与 metric 论证；dispatch-coupling 数学；inter-turn gap CDF（`f3a`）
+- §2.1 reuse topology / C1–C3（`f2a`、`workload_chars/*`）
+- **§2.2 四层命中实测（`v2/exp_a_tier_latency`：GPU/CPU/RDMA/miss）**
+- **§2.2 Ev#1 容量 knee（`v2/exp_b_capacity_knee`）+ working_set + cluster-scale 校正**
+- §3.1 PD-disagg 证否（`PD_DISAGG_RESULTS`、crossover、MB1/MB2）
+- §3.2 LPWL（−31%）、LMetric 稀释、sticky hot-pin、ES ablation
+- §3.3 migration substrate net-positive + correctness smoke tests
+
+### 🟢 不依赖 migration，现在可做
+- §2.0 wall-clock amplification sweep（5 baseline × ≥3 runs）— **优先级最高**
+- §4 集成系统命名 + 端到端 baseline 矩阵（含 static PD-disagg 列）
+- §4.5 λ / skew / KV pool sensitivity
+- 草拟 §1–§3 正文（证据/图已齐）
+
+### 🚧 Deferred（待 migration policy e2e）
+- §3.3 migration trigger + target 的反馈环收益验证
+- §4.3 full / migration-only ablation
+- §4.5 `T_hot` / `T_cool` sensitivity
+
+### 🎨 待画
+- §1.3 storage-hierarchy 示意（GPU HBM → CPU DRAM → RDMA store）
+- §2.0 dispatch-coupling schematic（chatbot vs agentic timeline + 反馈环）
+- 集成系统 architecture 图
+
+### ❓ Open
+- 集成系统最终命名（GPU-hit-first 是原则；系统名待定）
+- §4 instance 数 / trace 总长定稿

README.md

@@ -0,0 +1,77 @@
+# agentic-kv
+
+Serving agentic LLM workloads by keeping the KV working set in GPU HBM
+(GPU-hit-first). Research outline: [`PAPER_OUTLINE.md`](PAPER_OUTLINE.md).
+Evidence + experiments: [`v2/`](v2/).
+
+---
+
+## ⚠️ Benchmarking methodology — read this first
+
+> **Replay agentic traces with `--dispatch-mode thinktime`, not the default
+> `tracets`.** It is the faithful, more realistic load — and the dispatch mode
+> materially changes the performance you measure.
+
+The replayer offers two ways to time each turn:
+
+| mode | turn-k dispatched at | what it models |
+|---|---|---|
+| `tracets` (default) | `max(prev_turn_finished, trace_ts)` | absolute production schedule |
+| **`thinktime` (use this)** | `prev_turn_finished + time_to_parent_chat` | real closed-loop agent pacing |
+
+**Why it matters.** `tracets` collapses the inter-turn think-time to ~0 whenever
+the system falls behind (it fires the next turn immediately because the trace
+timestamp is already in the past). That manufactures **artificial request
+bursts** — spiking instantaneous concurrency → KV-pool pressure → preemption →
+inflated tail latency and wasted throughput. `thinktime` keeps each turn's real
+gap (tool-exec + agent think), so the offered load is what a real agent produces.
+
+**Measured (w600 first-300s window, 8×H20, round-robin, 100% completion):**
+
+| metric (N=8) | `tracets` (Mode 1) | **`thinktime` (Mode 2)** | Δ |
+|---|---:|---:|---:|
+| E2E p90 | 102.8 s | **73.5 s** | **−28%** |
+| E2E p99 | 245 s | **227 s** | −7% |
+| TTFT p90 | 56.1 s | **39.7 s** | **−29%** |
+| system TPS | 111.8 | **119.3** | **+7%** |
+| wall-clock | 967 s | **787 s** | −19% |
+| TPOT p90 | 0.174 s | 0.188 s | ~flat |
+
+So under realistic capacity, `tracets` makes the system look **~30% worse on
+tail latency** than it actually is. Tell-tale: scaling 6→8 instances barely helped
+`tracets` (975→967 s — its bursts re-saturate regardless of capacity) but helped
+`thinktime` a lot (1125→787 s). Under heavy saturation (N=6) the two converge
+(E2E p90 ≈ 118–120 s), since there is no slack for bursts to harm. Decode (TPOT)
+is dispatch-independent everywhere.
+
+**Recommendation:** benchmark with `--dispatch-mode thinktime`; use `tracets`
+only as an explicit bursty stress case. Full ablation:
+[`v2/exp_c_dispatch_ablation/`](v2/exp_c_dispatch_ablation/).
+
+### How to use it
+
+```bash
+# 1. annotate a trace with the real per-turn gap (one-time; scans the raw trace)
+python scripts/add_time_to_parent.py traces/w600_r0.0015_st30.jsonl traces/w600_ttp.jsonl
+
+# 2. replay closed-loop with faithful think-time
+python -m replayer --trace traces/w600_ttp.jsonl --endpoint <eps> \
+    --model <model> --dispatch-mode thinktime
+```
+
+`time_to_parent_chat = this_turn.request_ready_time_ms − parent_turn.request_end_time_ms`,
+computed from the raw trace and stored per request; turn-1 has none (fires at its
+trace arrival). Traces without the field fall back to `tracets`.
+
+---
+
+## Project map
+
+- [`PAPER_OUTLINE.md`](PAPER_OUTLINE.md) — GPU-hit-first paper outline (the thesis).
+- [`v2/`](v2/) — evidence experiments:
+  - `exp_a_tier_latency/` — KV-hit cost by tier (GPU < CPU-local < remote-RDMA < miss).
+  - `exp_b_capacity_knee/` — realized APC / latency knee vs GPU capacity.
+  - `exp_c_dispatch_ablation/` — the replay-mode study above.
+- `replayer/` — trace replayer (`--dispatch-mode`, closed-loop think-time).
+- `scripts/add_time_to_parent.py` — trace annotation for `thinktime`.
+- `microbench/`, `analysis/` — PD-disagg, routing, workload characterization.

RESULTS_SUMMARY.md

@@ -0,0 +1,134 @@
+# 目前已成立的结论（2026-05-27）
+
+EAR 项目目前能用实测数据支撑的论点汇总。每条都标了对应的图/数据路径。
+
+---
+
+## 1. Workload 性质（§2）
+
+Production trace = Qwen3-Coder agentic，1.3 M sessions / 2.1 M reqs / 7200 s。
+
+| 性质 | 数据 | 实证 |
+|---|---|---|
+| **KV 复用几乎全在 session 内** | intra 93.2% / cross 5.7% / shared 1.1%；理论 APC 上界 79.6% | `figs/f2a_reuse_topology.png` |
+| **Session 极度偏斜** | top 1%/5%/10%/25%/50% = 46.5%/66.5%/74.6%/87.5%/**96.0%** input mass | `figs/f2b_session_skew.png` |
+| **单请求 KV 已经很大** | p50 1.8 GiB / p90 8.0 / p95 9.6 / **p99 11.5 GiB**；KV pool 38 GiB/instance（0.4 × H20 96 GiB）→ p99 req 只能装 **3 个/instance** | `figs/f2c_kv_footprint_cdf.png` |
+
+**结论**：cache 是 session-local 的，scheduling 必须保留 session affinity；单 request KV 接近 pool 上限，**PD-disagg 4P+4D 让系统 decode 容量直接减半**。
+
+## 2. Dispatch Coupling（§2.3）
+
+| 数据 | Agentic (Qwen3-Coder) | Chatbot (qwen3-max) |
+|---|---:|---:|
+| Inter-turn `T_external` p50 | **1.6 s** | 7.2 s |
+| `gap < 1 s` 比例 | **39%** | 4% |
+| `gap < 5 s` 比例 | 67% | 29% |
+| p99 | 738 s | 43 s |
+
+参考图：`figs/f3a_inter_turn_gap.png`。
+
+**结论**：agentic 有一段 chatbot 没有的 **sub-second tool-call mode**（39% vs 4%）。当 `W_turn ≳ T_external`（任何 W_turn > 1 s 的 scheduler 在 agentic 上都满足这条件），Little's Law `L = Λ · N · (W_turn(L) + T_external)` 进入闭环 regime，scheduler 的 ε 退步通过 KV 竞争反馈环被放大成 wall-clock 数倍差距。**实测**：lmetric 跑 600 s trace 用 49 min wall-clock = **8x amplification**。
+
+## 3. 现有调度的三类失败（§3）
+
+| Baseline | 失败模式 | 数据 |
+|---|---|---|
+| **load-balance / LMetric** | 丢 locality | lmetric APC **56.9%**（vs 上界 79.6%）；LMetric 比 load_only 只好 +3.3pp，因为 cache 信号在乘性 score `(pending+input−hit) × num_req` 里被 num_req 吞掉 |
+| **静态 PD-disagg** | D 侧 KV 容量墙 + transfer 成本 | 见 §4 cost-vs-benefit |
+| **Pure sticky** | 全员被 hot session 拖累，不是单一热点 | sticky median worker 20.3 s vs unified 10.3 s；system e2e p90 sticky 34.6 s vs unified 18.0 s（**用 max/median ratio 衡量是误导**，§3.3 用 absolute per-worker latency）|
+
+参考图：`figs/f4a_apc_loss.png`、`figs/f4b_pdsep_kv_wall.png`、`figs/f4c_per_worker_ttft.png`、`figs/f6_e2e_latency_bars.png`、`figs/f6_e2e_latency_full_grid.png`。
+
+## 4. Static PD-disagg 为什么失败（§3.2）—— 容量问题，不是 cost-benefit 问题
+
+⚠ **2026-05-27 纠正**：本节前一版本论证"PD-disagg 因为 transfer cost > phase isolation benefit 而失败"。**这个论证算错了**。正确的 phase-isolation benefit 是**每个 prefill 事件 × D 个 concurrent stream** 的总和（≈ `D × T_prefill`），不是单个 request 的 decode 时长。用正确公式，PD-disagg 在 phase-isolation 这一维上**赢 colo 一两个数量级**。Static PD-disagg 在 agentic 上失败的**真正根因是 D 侧 KV pool 容量**，不是这一维。
+
+### 4.1 真正的失败模式：D 侧 KV 容量天花板
+
+| | 8C colo | 4P+4D PD-disagg |
+|---|---:|---:|
+| Per-D-instance KV pool（0.4 × 96 GiB） | 38 GiB | 38 GiB |
+| 系统总 decode 容量（D 实例数 × 单池） | 8 × 38 = **304 GiB** | 4 × 38 = **152 GiB** |
+| p99 单请求 KV = 11.5 GiB → 最多并发 decode | 24 | **12（减半）** |
+
+Colleague 4P+4D 实测：TTFT p50 0.91 s → **62.8 s（62×）**、success rate **99.5% → 52%**。失败模式：**D 池溢出 + 排队**，不是 transfer 延迟。
+
+参考图：`figs/f4b_pdsep_kv_wall.png`（pdf 版本是高质量 paper figure）。
+
+### 4.2 MB2 — KV transfer cost（per-request 一次性成本，**不**是 dominant cost）
+
+dash1 GPU 0+1（intra）和 dash1 ↔ dash2（inter, 200 Gbps RoCE）扫 9 个 size × 5 reps。
+
+| 路径 | 稳态带宽（≤ 3 GiB） | p99 agentic 请求 11.5 GiB transfer |
+|---|---|---|
+| Intra-node | **9.7 GB/s** | p50 **1.9 s** · max 10 s |
+| Inter-node | **10.0 GB/s**（差 <3%） | p50 **1.7 s** · max 9.2 s |
+
+**新发现**：intra/inter 几乎重合 → **Mooncake `batch_transfer_sync_write` 永远走 RDMA NIC**，不走 NVLink。200 Gbps NIC 是天花板。**PD-disagg transfer cost 与拓扑无关**。
+
+参考图：`figs/mb2_transfer_time_compare.png`、doc `analysis/mb2/README.md`。
+
+### 4.3 MB1 — Phase interference（PD-disagg 的潜在 benefit 上界）
+
+dash1 GPU 0 单 instance（无 kv_connector），chunked-prefill 默认开启，D × P 扫描。D=8 结果：
+
+| Prefill | T_prefill | per-stream TPOT during | penalty |
+|---|---:|---:|---:|
+| 2k tok | 143 ms | 32 ms | 4× |
+| 32k tok | 4.5 s | 388 ms | **52×** |
+| 131k tok | 57 s | 1419 ms | **183×** |
+
+**Decode 在 prefill 期间被几乎完全 halted**，单 stream 损失 ≈ `T_prefill` 的时间。**每个 prefill event 总 decode 损失 ≈ `D × T_prefill`**。
+
+参考图：`figs/mb1_interference.png`、doc `analysis/mb1/README.md`。
+
+### 4.4 联合 cost-benefit（per-prefill event）
+
+| Prefill (KV size) | T_prefill | Cost = T_transfer | Benefit = D × T_prefill (D=8) | Cost / Benefit |
+|---:|---:|---:|---:|---:|
+| 2k tok (192 MiB) | 0.14 s | 8 ms | 1.1 s | **0.7%** |
+| 33k tok (3 GiB, trace mean) | 4.5 s | 0.32 s | 36 s | **0.9%** |
+| 125k tok (12 GiB, ~p99) | 57 s | 1.9 s | 456 s | **0.4%** |
+
+**PD-disagg 在 phase-isolation 这一维赢 100×–250×**。但**这不是 §3.2 该用的论证**，因为 §3.2 真正的 dominant failure 是 §4.1 的 D 池容量天花板（颠覆了上面的全部数学）。
+
+**总结**：
+- D 侧 KV 容量天花板（§4.1）→ PD-disagg 在 agentic 上**结构性失败**。
+- MB1 + MB2 的合计 cost-benefit 在 phase isolation 维度上 PD-disagg 是赢的，**但这件事被容量天花板压倒**。
+- Paper §3.2 论证应该聚焦"D 池装不下"，MB1/MB2 数据用作 supporting context（per-request transfer charge 量化、phase isolation benefit 量化）而不是 main argument。
+
+> ✅ **2026-05-30 更新 — 干净栈三轴 ablation 证实本节、并加 regime 细化。**
+> 本节的容量论点（D 池容量天花板 / decode 减半）在修复 `e13391e` 污染后的 clean stack
+> 上**得到确认**：concurrency 轴 N=64 时 PD 倾覆，**producer APC 从 71% 崩到 1.4%、TPS −30%**，
+> 而 colo 线性 scale（Fig 3）。**细化**：PD 并非"在 agentic 上一律失败"——它在
+> *低负载 / decode-heavy / 低复用* 区间**赢** colo；真正失败的是 agentic corner（高复用 +
+> 短输出 + 大上下文 + 高并发）——静态 P:D split 无法同时给出复用所需的 producer 容量
+> *和* decode 容量，而 colo 的弹性池两者兼得。
+> **另注**：旧 MB5 文档（`PD_DISAGG_RESULTS.md` §6）"session-affinity 救不了 PD / PD 复用=0%"
+> 的结论是 `e13391e`（producer 每次 KV 传输后 evict prefix）的**污染产物，已撤回**；
+> 干净栈上 session 路由的 producer APC 与 colo 持平（71–82%）。
+> 图：[`figs/mb5_pd_ablation/`](figs/mb5_pd_ablation/)。
+
+## 5. EAR 设计的实证状态（§4）
+
+| Pillar | 已实证 | 待实证 |
+|---|---|---|
+| **Affinity-default routing** (Pillar 1) | ✅ Current `unified` 算法 = LMetric + high-cache affinity；APC **79.4%**（达到 79.6% 上界 97%），TTFT p90 **7.3 s**，median worker p90 **10.3 s** | — |
+| **Hot-triggered session migration** (Pillar 2) | substrate 已通：`kv_both` connector 在 trace replay 上 net positive（TTFT p90 −18.6%，DR-fix 后 −36.6%），原 elastic_migration_v2 paper 的 "+45% kv_both penalty" obsolete | e2e 策略层（trigger 阈值 + target selection 在反馈环里）未直接验证 |
+
+## 6. 已经能写的 paper 主张（按 confidence 排序）
+
+1. **Agentic vs chatbot 在调度上是不同 regime**（dispatch coupling + sub-second tool-call mass）—— 实证完整
+2. **三类现有调度 baseline 各自的失败模式**（load-balance / static PD-disagg / pure sticky）—— 实证完整
+3. **Static PD-disagg 在 agentic 下失败的 dominant 根因是 D 侧 KV 容量**（不是 phase-isolation cost-benefit）—— 实证完整（`f4b` + colleague 4P+4D 数据）
+4. **Mooncake transfer cost 拓扑无关**（intra ≈ inter，~9.7 GB/s NIC 上限）—— 实证完整（MB2）
+5. **Phase isolation interference 在 chunked-prefill on 下仍然显著**（per-stream TPOT during prefill 15×–2000× baseline）—— 实证完整（MB1）。**注意**：这条数据本身不直接论证 "PD-disagg 失败"，因为算正确账后 PD-disagg 反而在这一维上赢；它的用途是给 §3.2 提供 phase-isolation benefit 上界的量化。
+6. **Affinity-default 调度（current unified）达到 APC 上界**，per-worker latency 也压倒 sticky —— 实证完整
+7. **Hot-triggered migration 修复 sticky 的 hot pin** —— design 完整、e2e 待验证
+
+## 7. 待做
+
+- **MB3-5**（end-to-end PD-disagg deployment）：D-pool runtime occupancy、cache reuse × PD interaction、PD ratio sweep。这些是 §5 完整实验矩阵的事
+- **EAR Pillar 2 migration e2e validation**（在 connector_tax DR-fix 之上重测）
+- **§5.4 wall-clock amplification sweep**（5 baseline × 3 runs，钉死 dispatch coupling 论证的实证 closure）
+- **Scale-out 验证**（dash1+dash2 = 16 GPU，等 dash0 + 3-node 可用时扩到 80 GPU）

analysis/characterization/elastic_migration_v2/README.md

@@ -9,82 +9,127 @@ Model: Qwen3-Coder-30B-A3B-Instruct, 8 × TP1 on H20
 This section explores whether the **B2-confirmed same-worker
 prefill–decode interference** can be relieved by selectively
 migrating prefill to a different worker for the requests where the
-interference cost would dominate the transfer cost. We implement two
-flavors of the policy (strict gates, then relaxed gates) and a clean
-isolation control (`unified_kv_both`: same picker as `unified`, but
-the vLLMs are launched in `kv_role=kv_both` so the Mooncake
-substrate is on but never triggers).
+interference cost would dominate the transfer cost. We implement
+two flavors of the routing policy (strict gates, then relaxed
+gates) and **two isolation controls** that use the unified picker
+but launch vLLMs in `kv_role=kv_both` so the connector substrate
+is on but never PD-seps:
 
-Three findings:
+- `unified_kv_both`: with **MooncakeConnector**
+- `unified_nixl_both`: with **NixlConnector** (NVIDIA's official
+  v1 connector; isolates connector implementation from policy)
 
-1. **`kv_role=kv_both` alone imposes a heavy always-on tax**: TTFT
-   p90 +45%, TPOT p90 +25%, hotspot index +19% vs plain `unified`,
-   with no PD-sep ever firing.
-2. **PD-sep almost never triggers on a real agentic workload**:
+Four findings:
+
+1. **`kv_role=kv_both` imposes a substantial always-on tax even
+   when no PD-sep ever fires**: with Mooncake it's TTFT p90 +45%,
+   TPOT p90 +25%, hotspot +19%; with NIXL it's TTFT p90 +38%,
+   TPOT p90 +16%, hotspot +0.2%.
+2. **About half of the substrate cost is generic v1-connector
+   framework overhead** (proxied by NIXL since it's the leanest
+   implementation): KV buffer GPU memory cut from the model's
+   working budget, `SchedulerOutput.kv_connector_metadata`
+   round-trip, and altered `kv_cache_manager` block-lifecycle
+   semantics. **NIXL is meaningfully better than Mooncake** but
+   still imposes a 16-38% tax vs no connector.
+3. **PD-sep almost never triggers on a real agentic workload**:
    0.16% with strict gates, 0.41% with relaxed gates. Agentic
-   workloads have 93% intra-session reuse, so most requests land on
-   workers that already hold cache — the uncached tail is too small
-   to be worth migrating.
-3. **When PD-sep does fire, the cost model is wrong by ~10–20×**:
+   workloads have 93% intra-session reuse, so most requests land
+   on workers that already hold cache — the uncached tail is too
+   small to be worth migrating.
+4. **When PD-sep does fire, the cost model is wrong by ~10–20×**:
    the calibrated `0.3s + bytes / 2.7 GB/s` predicts 1–2 s migrate
-   cost; observed TTFT on triggered requests is 12–45 s. The same
-   D-side block-reservation pressure and absence of layerwise
-   pipelining that the E2 audit flagged still dominate.
+   cost; observed TTFT on triggered requests is 12–45 s.
 
 The net latency of `unified_v2` is **not better than plain
-`unified`**. Improving agentic PD-sep requires fixing the underlying
-Mooncake transfer mechanism (E2 patches 6.1 lazy block reservation
-and 6.3 layerwise pipelining), not the routing decision.
+`unified`** under either Mooncake or NIXL substrate. Improving
+agentic PD-sep requires (a) using the leaner connector (NIXL >
+Mooncake by 5-19 pp across metrics), and (b) fixing the underlying
+transfer mechanism (E2 patches 6.1 lazy block reservation and 6.3
+layerwise pipelining), not just the routing decision.
 
 ## Substrate
 
-We compare three policies on identical traces:
+We compare four policies on identical traces:
 
 | policy | picker | vLLM launch mode | what's it for |
 |---|---|---|---|
-| `unified` | hybrid affinity + LMetric | plain (no Mooncake) | the headline baseline |
-| `unified_kv_both` | same as `unified` | `kv_role=kv_both` + bootstrap | isolation control: how much does kv_both *alone* cost? |
-| `unified_v2` | unified + selective PD-sep | `kv_role=kv_both` + bootstrap | the actual experiment |
+| `unified` | hybrid affinity + LMetric | plain (no connector) | the headline baseline |
+| `unified_kv_both` | same as `unified` | `MooncakeConnector` + `kv_both` | substrate control: Mooncake cost without PD-sep |
+| `unified_nixl_both` | same as `unified` | `NixlConnector` + `kv_both` | substrate control: NIXL cost without PD-sep, attributes overhead to "framework vs Mooncake" |
+| `unified_v2` | unified + selective PD-sep | `MooncakeConnector` + `kv_both` + bootstrap | the actual experiment |
 
-All three use the same trace, the same 8-instance topology, the same
+All four use the same trace, the same 8-instance topology, the same
 shadow-drift–corrected proxy (`scripts/cache_aware_proxy.py` post-fix
 `95c8ef8`). Plain `unified` was rerun on the patched proxy
 (`b3_sweep_20260525_095043/unified`) under the same conditions.
 
-## Result 1 — kv_both is expensive by itself
+NIXL required two launch fixes beyond Mooncake:
+- `VLLM_NIXL_SIDE_CHANNEL_PORT` must be unique per instance
+  (default 5600 → 5600..5607); otherwise instances 2..8 silently
+  hang in `zmq.error.ZMQError: Address already in use`.
+- Health-check timeout had to be raised from 180 s to 360 s
+  because NIXL initialization (UCX agent + memory registration)
+  takes ~100-150 s per instance under 8-way concurrent launch.
+
+## Result 1 — kv_both is expensive by itself, and only partly Mooncake's fault
 
 ![](figures/fig_kv_both_overhead.png)
 
 Switching the vLLM launch from plain to `kv_role=kv_both` without
-ever triggering PD-sep already costs:
+ever triggering PD-sep imposes a substrate tax. We compare the two
+connectors available in vendored vLLM:
 
-| metric | plain `unified` | `unified_kv_both` | Δ |
-|---|---:|---:|---|
-| TTFT p50 | 0.50 s | 0.50 s | +0% |
-| TTFT p90 | 7.35 s | 10.67 s | **+45%** |
-| TTFT p99 | 42.34 s | 45.19 s | +7% |
-| TPOT p90 | 17.1 ms | 21.3 ms | **+25%** |
-| E2E p90 | 18.03 s | 22.89 s | **+27%** |
-| APC | 79.4% | 78.3% | −1.1 pp |
-| hotspot index | 3.667 | **4.363** | **+19%** |
+| metric | plain `unified` | `unified_nixl_both` | `unified_kv_both` (Mooncake) |
+|---|---:|---:|---:|
+| TTFT p50 | 0.50 s | 0.51 s (+1%) | 0.50 s (+0%) |
+| **TTFT p90** | 7.35 s | **10.13 s (+38%)** | **10.67 s (+45%)** |
+| TTFT p99 | 42.34 s | 44.58 s (+5%) | 45.19 s (+7%) |
+| TPOT p90 | 17.1 ms | **19.8 ms (+16%)** | **21.3 ms (+25%)** |
+| E2E p90 | 18.03 s | **21.18 s (+17%)** | **22.89 s (+27%)** |
+| APC | 79.4% | 79.1% (−0.3 pp) | 78.3% (−1.1 pp) |
+| **hotspot index** | 3.667 | **3.674 (+0.2%)** | **4.363 (+19%)** |
+| interference index | n/a | 5.58 | 8.57 |
 
-Two contributing factors:
+![](figures/fig_connector_substrate_attribution.png)
 
-1. **The Mooncake `MooncakeConnector` runs even when no transfer is
-   pending.** Every scheduler step it walks `set(cache.keys())`
-   against `_known_hash_keys` (E2 audit §6.5) and updates the
-   `KVConnectorMetadata`. This is O(|cache|) per step on every
-   engine, even when no producer/consumer relationship is active.
-2. **Block reservation semantics differ** under kv_both. The
-   scheduler treats blocks as candidates for export-to-others, so
-   the prefix cache LRU pressure is slightly different (we lose 1
-   pp APC).
+Reading the table from left to right gives a clean attribution:
 
-Practical implication: **you don't enable kv_both for free**. If a
-deployment wants the option to do PD-sep selectively, the 45% TTFT
-p90 tax applies even on requests that stay local. This needs to
-recoverable cost before any selective-PD-sep policy is worth
-shipping.
+- **NIXL−plain** = the **v1-connector framework's irreducible cost**
+  (TTFT p90 +38%, TPOT p90 +16%, E2E p90 +17%). This is the cost
+  *any* v1 KV connector imposes:
+  - the 1 GB `kv_buffer_size` carved from `gpu-memory-utilization`,
+    reducing the KV cache budget;
+  - per-step `SchedulerOutput.kv_connector_metadata` serialization
+    and round-trip through the connector worker;
+  - altered block-lifecycle semantics in `kv_cache_manager`
+    (`delay_free_blocks=True` is the default once any connector is
+    loaded, slowing LRU eviction).
+- **Mooncake−NIXL** = the **Mooncake-implementation-specific extra**
+  (TTFT p90 +7 pp, TPOT p90 +9 pp, E2E p90 +10 pp, hotspot +19 pp).
+  This is the cost Mooncake's design choices add on top of the
+  generic framework:
+  - per-scheduler-step `set(self._block_pool.cache.keys())` diff
+    against `_known_hash_keys` (`mooncake_connector.py:432-456`)
+    walks O(|cache|) on every step on every engine, costing ~4 M
+    set operations per second on a 200 k-block cache;
+  - the hash sync runs even when no `direct_read` consumer is
+    present, so the cost is paid unconditionally;
+  - block-lifecycle is further constrained because Mooncake
+    requires `delay_free` until the explicit `finished_sending`
+    arrives, vs NIXL which can release blocks earlier.
+
+The **most striking gap is hotspot**: Mooncake's per-step hash
+sync runs on the scheduler's GIL and disrupts the timeliness of
+routing decisions, amplifying load imbalance by 19%. NIXL has no
+equivalent global-state maintenance and preserves the plain-unified
+hotspot to within 0.2%.
+
+Practical implication: **you don't enable any v1 KV connector for
+free**, but if you have to enable one, NIXL is meaningfully cheaper
+than Mooncake. Even NIXL's 38% TTFT p90 tax is large enough that
+PD-sep needs to recover it on a non-trivial fraction of requests
+before being worth it.
 
 ## Result 2 — PD-sep rarely fires on a real agentic trace
 
@@ -153,24 +198,24 @@ The first-token clock for the 49 k request is **21× the model's
 prediction**. This is not a small mis-tuning — it's a structurally
 different model.
 
-## Result 4 — three-way comparison
+## Result 4 — four-way comparison
 
 ![](figures/fig_three_way_hotspot.png)
 
 The full table:
 
-| metric | unified (plain) | unified_kv_both | unified_v2 (relaxed) |
-|---|---:|---:|---:|
-| n_ok | 1214 | 1214 | 1214 |
-| TTFT p50 | 0.50 s | 0.50 s | 0.49 s |
-| TTFT p90 | 7.35 s | 10.67 s | 10.98 s |
-| TTFT p99 | 42.34 s | 45.19 s | 49.45 s |
-| TPOT p90 | 17.1 ms | 21.3 ms | 18.4 ms |
-| E2E p90 | 18.03 s | 22.89 s | 22.53 s |
-| APC | 79.4% | 78.3% | 77.6% |
-| interference index | n/a (no engine_state) | 8.57 | 8.46 |
-| hotspot index | 3.667 | 4.363 | 3.910 |
-| n_slow | 189 | 198 | 198 |
+| metric | unified (plain) | unified_nixl_both | unified_kv_both (Mooncake) | unified_v2 (relaxed) |
+|---|---:|---:|---:|---:|
+| n_ok | 1214 | 1214 | 1214 | 1214 |
+| TTFT p50 | 0.50 s | 0.51 s | 0.50 s | 0.49 s |
+| TTFT p90 | 7.35 s | 10.13 s | 10.67 s | 10.98 s |
+| TTFT p99 | 42.34 s | 44.58 s | 45.19 s | 49.45 s |
+| TPOT p90 | 17.1 ms | 19.8 ms | 21.3 ms | 18.4 ms |
+| E2E p90 | 18.03 s | 21.18 s | 22.89 s | 22.53 s |
+| APC | 79.4% | 79.1% | 78.3% | 77.6% |
+| interference index | n/a | 5.58 | 8.57 | 8.46 |
+| hotspot index | 3.667 | 3.674 | 4.363 | 3.910 |
+| n_slow | 189 | 192 | 198 | 198 |
 
 ### v2 vs the kv_both control (the right comparison)
 

analysis/characterization/elastic_migration_v2/data/b3_policy_comparison.json

@@ -155,6 +155,32 @@
         "unknown": 49
       }
     },
+    {
+      "policy": "unified_nixl_both",
+      "n_ok": 1214,
+      "n_total": 1214,
+      "ttft_p50_s": 0.5138550130068325,
+      "ttft_p90_s": 10.127110345300755,
+      "ttft_p99_s": 44.5789094621703,
+      "tpot_p50_s": 0.008423213202440761,
+      "tpot_p90_s": 0.019759515867947428,
+      "tpot_p99_s": 0.1079433335279151,
+      "e2e_p50_s": 1.866590676479973,
+      "e2e_p90_s": 21.179128799570027,
+      "e2e_p99_s": 96.01196486203865,
+      "apc_ratio": 0.791441828164218,
+      "interference_index": 5.580715970433481,
+      "hotspot_index_ttft_p90": 3.673957447190547,
+      "reuse_intra_frac": 0.930632797070364,
+      "reuse_cross_frac": 0.05718149217603143,
+      "n_slow": 192,
+      "failure_counts": {
+        "cache_miss_large_append": 21,
+        "hot_worker_queue": 75,
+        "same_worker_prefill_overlap": 72,
+        "unknown": 24
+      }
+    },
     {
       "policy": "unified_v2",
       "n_ok": 1214,

analysis/characterization/elastic_migration_v2/data/per_worker_unified_nixl_both.json

@@ -0,0 +1,24 @@
+{
+  "hotspot_index_ttft_p90": 3.673957447190547,
+  "per_worker_latency_p90_s": {
+    "http://127.0.0.1:8000": 21.5702620673168,
+    "http://127.0.0.1:8001": 21.44246501957532,
+    "http://127.0.0.1:8002": 7.497513776784784,
+    "http://127.0.0.1:8003": 18.975387462502113,
+    "http://127.0.0.1:8004": 27.733961877820548,
+    "http://127.0.0.1:8005": 14.178356938017535,
+    "http://127.0.0.1:8006": 25.44877168269595,
+    "http://127.0.0.1:8007": 54.500166546402035
+  },
+  "per_worker_ttft_p90_s": {
+    "http://127.0.0.1:8000": 7.380765471985799,
+    "http://127.0.0.1:8001": 14.109222683508415,
+    "http://127.0.0.1:8002": 3.001173847797329,
+    "http://127.0.0.1:8003": 14.087287129514152,
+    "http://127.0.0.1:8004": 14.151121024426537,
+    "http://127.0.0.1:8005": 6.165523712011057,
+    "http://127.0.0.1:8006": 6.314287615299688,
+    "http://127.0.0.1:8007": 39.43635586597957
+  },
+  "status": "supported"
+}

analysis/characterization/elastic_migration_v2/render_figures.py

@@ -34,37 +34,39 @@ def _load(name: str):
 
 
 POLICY_COLORS = {
-    "unified":            "#2ca02c",
-    "unified_kv_both":    "#9467bd",
-    "unified_v2":         "#d62728",
-    "unified_v2_strict":  "#ff7f0e",
+    "unified":             "#2ca02c",
+    "unified_kv_both":     "#9467bd",
+    "unified_nixl_both":   "#1f77b4",
+    "unified_v2":          "#d62728",
+    "unified_v2_strict":   "#ff7f0e",
 }
 
 
 def fig_kv_both_overhead():
     comp = _load("b3_policy_comparison.json")
     by = {r["policy"]: r for r in comp["rows"]}
-    pols = ["unified", "unified_kv_both", "unified_v2"]
+    pols = ["unified", "unified_kv_both", "unified_nixl_both", "unified_v2"]
     metrics = [
         ("TTFT p90 (s)",   lambda r: r["ttft_p90_s"]),
         ("TPOT p90 (ms)",  lambda r: r["tpot_p90_s"] * 1000),
         ("E2E p90 (s)",    lambda r: r["e2e_p90_s"]),
         ("hotspot index",  lambda r: r["hotspot_index_ttft_p90"]),
     ]
-    fig, axes = plt.subplots(1, 4, figsize=(14, 4))
+    fig, axes = plt.subplots(1, 4, figsize=(15, 4.2))
     for ax, (label, fn) in zip(axes, metrics):
         vals = [fn(by[p]) for p in pols]
-        bars = ax.bar(pols, vals,
+        labels_short = [p.replace("unified_", "") for p in pols]
+        labels_short[0] = "plain"
+        bars = ax.bar(labels_short, vals,
                        color=[POLICY_COLORS[p] for p in pols],
                        edgecolor="black", linewidth=0.5)
         ax.set_title(label)
-        ax.tick_params(axis="x", rotation=20, labelsize=9)
+        ax.tick_params(axis="x", rotation=15, labelsize=9)
         for b, v in zip(bars, vals):
             ax.text(b.get_x() + b.get_width() / 2, v,
                      f"{v:.2f}" if v < 100 else f"{v:.0f}",
                      ha="center", va="bottom", fontsize=9)
         ax.grid(alpha=0.3, axis="y")
-        # delta annotation
         baseline = vals[0]
         for i, v in enumerate(vals):
             if i == 0:
@@ -74,8 +76,8 @@ def fig_kv_both_overhead():
                      fontsize=10, fontweight="bold",
                      color="darkred" if pct > 0 else "darkgreen")
     fig.suptitle(
-        "kv_both adds ~45% to TTFT p90 even without PD-sep firing.\n"
-        "v2's PD-sep barely recovers the gap (and overshoots TTFT p99)."
+        "Mooncake substrate adds 19-45% across metrics; NIXL is 5-19pp better but\n"
+        "still 16-38% above plain. v2's 5 PD-sep events don't recover the substrate tax."
     )
     fig.tight_layout()
     fig.savefig(OUT / "fig_kv_both_overhead.png", dpi=120)
@@ -203,27 +205,29 @@ def fig_v2_predicted_vs_actual():
 
 
 def fig_three_way_hotspot():
-    pols = ["unified", "unified_kv_both", "unified_v2"]
+    pols = ["unified", "unified_kv_both", "unified_nixl_both", "unified_v2"]
     per_worker = {p: _load(f"per_worker_{p}.json") for p in pols}
     workers = sorted(per_worker["unified"]["per_worker_ttft_p90_s"].keys())
 
     x = range(len(workers))
-    width = 0.27
-    fig, ax = plt.subplots(figsize=(11, 5))
+    n = len(pols)
+    width = 0.85 / n
+    fig, ax = plt.subplots(figsize=(12, 5))
     for i, p in enumerate(pols):
         d = per_worker[p]["per_worker_ttft_p90_s"]
         vals = [d[w] for w in workers]
-        offset = (i - 1) * width
+        offset = (i - (n - 1) / 2) * width
+        label = p.replace("unified_", "") if p != "unified" else "plain"
         ax.bar([j + offset for j in x], vals, width,
-                label=f"{p} (hotspot={per_worker[p]['hotspot_index_ttft_p90']:.2f})",
+                label=f"{label} (hotspot={per_worker[p]['hotspot_index_ttft_p90']:.2f})",
                 color=POLICY_COLORS[p], edgecolor="black", linewidth=0.4)
     short = [w.replace("http://127.0.0.1:", ":") for w in workers]
     ax.set_xticks(list(x))
     ax.set_xticklabels(short, rotation=0, fontsize=9)
     ax.set_ylabel("worker TTFT p90 (s)")
     ax.set_title(
-        "Per-worker TTFT p90 distribution. kv_both alone makes the hot worker hotter\n"
-        "(unified→kv_both: 37.7s→43.5s peak); v2's 5 PD-sep triggers nudge it back."
+        "Per-worker TTFT p90 distribution across substrates. Mooncake (kv_both)\n"
+        "amplifies the hot worker (hotspot 4.36); NIXL keeps it close to plain (3.67)."
     )
     ax.legend(loc="upper left", fontsize=9)
     ax.grid(alpha=0.3, axis="y")
@@ -232,12 +236,64 @@ def fig_three_way_hotspot():
     plt.close(fig)
 
 
+def fig_connector_substrate_attribution():
+    """Decomposes overhead into v1-framework cost (shared by all connectors,
+    proxied by NIXL since it's the leanest) and Mooncake-specific cost."""
+    comp = _load("b3_policy_comparison.json")
+    by = {r["policy"]: r for r in comp["rows"]}
+    metrics = [
+        ("TTFT p90 (s)",  "ttft_p90_s",  False),
+        ("TPOT p90 (ms)", "tpot_p90_s",  True),
+        ("E2E p90 (s)",   "e2e_p90_s",   False),
+        ("hotspot index", "hotspot_index_ttft_p90", False),
+    ]
+    fig, axes = plt.subplots(1, 4, figsize=(15, 4))
+    for ax, (label, key, scale_ms) in zip(axes, metrics):
+        plain = by["unified"][key] * (1000 if scale_ms else 1)
+        nixl = by["unified_nixl_both"][key] * (1000 if scale_ms else 1)
+        moon = by["unified_kv_both"][key] * (1000 if scale_ms else 1)
+        v2 = by["unified_v2"][key] * (1000 if scale_ms else 1)
+
+        framework_cost = nixl - plain   # what NIXL adds = v1 framework cost
+        mooncake_extra = moon - nixl    # extra on top from Mooncake
+        v2_branch_extra = v2 - moon     # extra from PD-sep branch (Mooncake + 5 events)
+
+        bottom = 0
+        ax.bar(["overhead"], [plain], color="#cccccc",
+                edgecolor="black", linewidth=0.4,
+                label=f"plain unified ({plain:.2f})")
+        bottom += plain
+        ax.bar(["overhead"], [framework_cost], bottom=[bottom],
+                color="#1f77b4", edgecolor="black", linewidth=0.4,
+                label=f"v1 framework (+{framework_cost:.2f})")
+        bottom += framework_cost
+        ax.bar(["overhead"], [mooncake_extra], bottom=[bottom],
+                color="#9467bd", edgecolor="black", linewidth=0.4,
+                label=f"Mooncake extra (+{mooncake_extra:.2f})")
+        bottom += mooncake_extra
+        ax.bar(["overhead"], [v2_branch_extra], bottom=[bottom],
+                color="#d62728", edgecolor="black", linewidth=0.4,
+                label=f"v2 PD-sep branch ({v2_branch_extra:+.2f})")
+        ax.set_title(label)
+        ax.legend(fontsize=8, loc="upper right")
+        ax.grid(alpha=0.3, axis="y")
+        ax.tick_params(axis="x", labelbottom=False)
+    fig.suptitle(
+        "Attribution: plain unified vs NIXL substrate vs Mooncake substrate vs v2.\n"
+        "Blue: cost shared by any v1 connector. Purple: cost specific to Mooncake."
+    )
+    fig.tight_layout()
+    fig.savefig(OUT / "fig_connector_substrate_attribution.png", dpi=120)
+    plt.close(fig)
+
+
 def main():
     fig_kv_both_overhead()
     fig_v2_trigger_funnel()
     fig_v2_predicted_vs_actual()
     fig_three_way_hotspot()
-    print(f"wrote 4 figures to {OUT}")
+    fig_connector_substrate_attribution()
+    print(f"wrote 5 figures to {OUT}")
 
 
 if __name__ == "__main__":

analysis/characterization/render_window1_figures.py

@@ -48,6 +48,8 @@ def fig_b3_apc_vs_hotspot(comp: dict, upper: dict, out: Path) -> None:
     fig, ax = plt.subplots(figsize=(6, 4.5))
     for r in comp["rows"]:
         pol = r["policy"]
+        if pol not in POLICY_ORDER:
+            continue
         ax.scatter(r["apc_ratio"] * 100, r["hotspot_index_ttft_p90"],
                     s=180, color=POLICY_COLOR.get(pol, "gray"), label=pol,
                     edgecolors="black", linewidths=0.5)
@@ -87,6 +89,48 @@ def fig_b3_latency_bars(comp: dict, out: Path) -> None:
     plt.close(fig)
 
 
+def fig_b3_latency_full_grid(results_dir: Path, out: Path) -> None:
+    """4 rows (mean / p50 / p90 / p99) × 3 cols (TTFT / TPOT / E2E) per policy.
+
+    Reads per-policy metrics.summary.json caches under raw_stats/, which
+    expose mean alongside the percentiles (b3_policy_comparison.json does
+    not record mean).
+    """
+    raw_dir = results_dir / "raw_stats"
+    pols = [p for p in POLICY_ORDER if (raw_dir / f"{p}.json").exists()]
+    if not pols:
+        return
+    stats = {p: json.loads((raw_dir / f"{p}.json").read_text()) for p in pols}
+
+    rows = [("mean", "mean"), ("p50", "p50"), ("p90", "p90"), ("p99", "p99")]
+    cols = [
+        ("TTFT (s)", "ttft", 1.0),
+        ("TPOT (ms)", "tpot", 1000.0),
+        ("E2E (s)", "e2e", 1.0),
+    ]
+    fig, axes = plt.subplots(len(rows), len(cols), figsize=(11, 11), sharex=True)
+    for i, (row_label, agg_key) in enumerate(rows):
+        for j, (col_label, metric_key, scale) in enumerate(cols):
+            ax = axes[i][j]
+            vals = [stats[p][metric_key][agg_key] * scale for p in pols]
+            ax.bar(pols, vals,
+                   color=[POLICY_COLOR.get(p, "gray") for p in pols],
+                   edgecolor="black", linewidth=0.5)
+            for k, v in enumerate(vals):
+                ax.text(k, v, f"{v:.1f}", ha="center", va="bottom", fontsize=8)
+            if j == 0:
+                ax.set_ylabel(row_label, fontsize=11)
+            if i == 0:
+                ax.set_title(col_label, fontsize=11)
+            ax.grid(alpha=0.3, axis="y")
+            ax.tick_params(axis="x", rotation=20, labelsize=9)
+            ax.margins(y=0.18)
+    fig.suptitle("B3 latencies per policy — mean / p50 / p90 / p99")
+    fig.tight_layout()
+    fig.savefig(out, dpi=120)
+    plt.close(fig)
+
+
 def fig_b3_apc_vs_upper(comp: dict, upper: dict, out: Path) -> None:
     by = {r["policy"]: r for r in comp["rows"]}
     pols = [p for p in POLICY_ORDER if p in by]
@@ -145,7 +189,13 @@ def fig_b3_failure_breakdown(comp: dict, out: Path) -> None:
 
 
 def fig_b3_per_worker_ttft(results_dir: Path, comp: dict, out: Path) -> None:
-    """Per-worker TTFT p90 grouped bars; reads each policy's hotspot_index.json."""
+    """Per-worker TTFT p90 grouped bars; title shows median + max worker p90.
+
+    We deliberately do NOT report a max/median 'hotspot index' here: it is a
+    ratio and treats unified (most workers fast, one hot) as worse than
+    sticky (all workers slow), which inverts the actual user-facing p90.
+    """
+    import statistics
     by = {r["policy"]: r for r in comp["rows"]}
     pols = [p for p in POLICY_ORDER if p in by]
     fig, axes = plt.subplots(1, len(pols), figsize=(3 * len(pols), 4),
@@ -166,8 +216,12 @@ def fig_b3_per_worker_ttft(results_dir: Path, comp: dict, out: Path) -> None:
                 edgecolor="black", linewidth=0.5)
         for i, v in enumerate(vals):
             ax.text(i, v, f"{v:.1f}", ha="center", va="bottom", fontsize=8)
-        ax.set_title(f"{pol}\nhotspot={by[pol]['hotspot_index_ttft_p90']:.2f}",
-                      fontsize=10)
+        median_v = statistics.median(vals)
+        max_v = max(vals)
+        ax.set_title(
+            f"{pol}\nmedian {median_v:.1f}s · max {max_v:.1f}s",
+            fontsize=10,
+        )
         ax.tick_params(axis="x", labelsize=8)
         ax.grid(alpha=0.3, axis="y")
     axes[0].set_ylabel("worker TTFT p90 (s)")
@@ -254,19 +308,67 @@ def fig_reuse_decomposition(reuse: dict, out: Path) -> None:
 
 
 def fig_kv_footprint_cdf(kv: dict, out: Path) -> None:
+    """How many concurrent decodes fit per percentile, under three deployments.
+
+    KV pool assumption: 96 GiB H20 HBM split ~50% model params (Qwen3-Coder-
+    30B-A3B in bf16 + headroom), ~10% runtime activations, leaving ~40% for
+    the KV cache pool — i.e. ~38.4 GiB per instance.
+
+    For each request-size percentile, we report system-wide concurrent
+    decode capacity = N_D × floor(KV_pool / req_size_MiB) under three 8-GPU
+    deployments: all-combined, 4P+4D, 6P+2D. The point is that going from
+    combined 8C to 4P+4D halves the system's decode population at the
+    same per-request KV pressure.
+    """
     s = kv.get("kv_mib_per_request") or {}
-    vals = [s.get(k) for k in ("p50", "p90", "p95", "p99")]
-    labels = ["p50", "p90", "p95", "p99"]
-    fig, ax = plt.subplots(figsize=(6, 3.5))
-    ax.bar(labels, vals, color="#1f77b4", edgecolor="black", linewidth=0.5)
-    for i, v in enumerate(vals):
-        ax.text(i, v, f"{v:.0f} MiB", ha="center", va="bottom", fontsize=9)
-    ax.axhline(95 * 1024, color="red", linestyle="--", alpha=0.5,
-                label="H20 ~95 GiB usable")
-    ax.set_ylabel("KV bytes per request (MiB)")
-    ax.set_title("B1' Per-request KV footprint (Qwen3-Coder-30B-A3B, 98304 B/token)")
-    ax.legend()
+    pct_keys = ["p50", "p90", "p95", "p99"]
+    req_mib = [float(s.get(k, 0.0)) for k in pct_keys]
+    req_gib = [v / 1024 for v in req_mib]
+
+    hbm_gib = 96.0
+    kv_pool_frac = 0.40
+    kv_pool_mib = hbm_gib * kv_pool_frac * 1024  # ≈ 39322 MiB per instance
+
+    deploys = [
+        ("Combined 8C",     8, "#2ca02c"),
+        ("PD-disagg 4P+4D", 4, "#ff7f0e"),
+        ("PD-disagg 6P+2D", 2, "#d62728"),
+    ]
+
+    import numpy as _np
+    x = _np.arange(len(pct_keys))
+    bar_w = 0.26
+
+    fig, ax = plt.subplots(figsize=(9, 5.2))
+    for i, (label, n_d, color) in enumerate(deploys):
+        per_inst = [int(kv_pool_mib // r) if r > 0 else 0 for r in req_mib]
+        sys_cap = [n_d * pi for pi in per_inst]
+        bars = ax.bar(x + (i - 1) * bar_w, sys_cap, bar_w,
+                      label=f"{label} (N_D={n_d})",
+                      color=color, edgecolor="black", linewidth=0.5)
+        for j, (b, n) in enumerate(zip(bars, sys_cap)):
+            ax.text(b.get_x() + b.get_width() / 2, n, str(n),
+                    ha="center", va="bottom", fontsize=9, color="#333")
+
+    # Annotate per-request KV size and per-instance fit just above the x-axis
+    per_inst_combined = [int(kv_pool_mib // r) if r > 0 else 0 for r in req_mib]
+    annot = [
+        f"{pct}\n{rg:.1f} GiB / req\nfits {pi}/inst"
+        for pct, rg, pi in zip(pct_keys, req_gib, per_inst_combined)
+    ]
+    ax.set_xticks(x)
+    ax.set_xticklabels(annot, fontsize=10)
+
+    ax.set_ylabel("System-wide concurrent decodes")
+    ax.set_title(
+        f"Per-instance KV pool ≈ {kv_pool_mib / 1024:.1f} GiB "
+        f"(0.4 × H20 96 GiB; remaining 0.5 model + 0.1 activation)\n"
+        f"PD-disagg halves the decode population at p90+ "
+        f"(Qwen3-Coder-30B-A3B, 98304 B/token)"
+    )
+    ax.legend(loc="upper right")
     ax.grid(alpha=0.3, axis="y")
+    ax.margins(y=0.15)
     fig.tight_layout()
     fig.savefig(out, dpi=120)
     plt.close(fig)
@@ -276,9 +378,17 @@ def main() -> None:
     p = argparse.ArgumentParser()
     p.add_argument("--results-dir", type=Path, required=True)
     p.add_argument("--out-dir", type=Path, required=True)
+    p.add_argument("--exclude-policies", default="",
+                   help="Comma-separated policies to drop from per-policy figures")
     args = p.parse_args()
     args.out_dir.mkdir(parents=True, exist_ok=True)
 
+    excluded = {s.strip() for s in args.exclude_policies.split(",") if s.strip()}
+    if excluded:
+        global POLICY_ORDER
+        POLICY_ORDER = [p for p in POLICY_ORDER if p not in excluded]
+        print(f"excluding policies: {sorted(excluded)} -> kept {POLICY_ORDER}")
+
     comp = _load(args.results_dir, "b3_policy_comparison.json")
     upper = _load(args.results_dir, "apc_upper_w600.json")
     b2 = _load(args.results_dir, "b2_sweep_summary.json")
@@ -287,6 +397,9 @@ def main() -> None:
 
     fig_b3_apc_vs_hotspot(comp, upper, args.out_dir / "fig_b3_apc_vs_hotspot.png")
     fig_b3_latency_bars(comp, args.out_dir / "fig_b3_latency_bars.png")
+    fig_b3_latency_full_grid(
+        args.results_dir, args.out_dir / "fig_b3_latency_full_grid.png"
+    )
     fig_b3_apc_vs_upper(comp, upper, args.out_dir / "fig_b3_apc_vs_upper.png")
     fig_b3_failure_breakdown(comp, args.out_dir / "fig_b3_failure_breakdown.png")
     fig_b3_per_worker_ttft(args.results_dir, comp,

analysis/characterization/window_1_results/raw_stats/lmetric.json

@@ -0,0 +1,23 @@
+{
+  "ttft": {
+    "count": 1214.0,
+    "mean": 5.111546324698484,
+    "p50": 0.9387824369769078,
+    "p90": 15.671339168207492,
+    "p99": 53.56683189840049
+  },
+  "tpot": {
+    "count": 1214.0,
+    "mean": 0.01757124870168204,
+    "p50": 0.008854518407308914,
+    "p90": 0.02122720699121469,
+    "p99": 0.18280341184277568
+  },
+  "e2e": {
+    "count": 1214.0,
+    "mean": 9.518126648903337,
+    "p50": 2.754255389008904,
+    "p90": 24.8209177934099,
+    "p99": 80.59924928059091
+  }
+}

analysis/characterization/window_1_results/raw_stats/load_only.json

@@ -0,0 +1,23 @@
+{
+  "ttft": {
+    "count": 1214.0,
+    "mean": 6.268620166597892,
+    "p50": 1.2609447415161412,
+    "p90": 20.197147866390882,
+    "p99": 52.84285237012196
+  },
+  "tpot": {
+    "count": 1214.0,
+    "mean": 0.02406975794215626,
+    "p50": 0.009231464695980247,
+    "p90": 0.026851662550158716,
+    "p99": 0.3211630676943426
+  },
+  "e2e": {
+    "count": 1214.0,
+    "mean": 11.702793988628443,
+    "p50": 3.58568156149704,
+    "p90": 33.459180271782685,
+    "p99": 93.95083751494239
+  }
+}

analysis/characterization/window_1_results/raw_stats/sticky.json

@@ -0,0 +1,23 @@
+{
+  "ttft": {
+    "count": 1214.0,
+    "mean": 5.55315460854824,
+    "p50": 0.5415176274836995,
+    "p90": 18.021296651283045,
+    "p99": 74.09429564891524
+  },
+  "tpot": {
+    "count": 1214.0,
+    "mean": 0.027834537397398284,
+    "p50": 0.008952101894096181,
+    "p90": 0.03641285916619554,
+    "p99": 0.35152006935195085
+  },
+  "e2e": {
+    "count": 1214.0,
+    "mean": 12.109200157184377,
+    "p50": 2.081947358994512,
+    "p90": 34.62592205510591,
+    "p99": 139.68334607904353
+  }
+}

analysis/characterization/window_1_results/raw_stats/unified.json

@@ -0,0 +1,23 @@
+{
+  "ttft": {
+    "count": 1213.0,
+    "mean": 3.2790960856202394,
+    "p50": 0.4997710260213353,
+    "p90": 7.345769894809922,
+    "p99": 42.34170345296613
+  },
+  "tpot": {
+    "count": 1213.0,
+    "mean": 0.012493800538265787,
+    "p50": 0.008079791456705824,
+    "p90": 0.017110194704198407,
+    "p99": 0.12655874612209597
+  },
+  "e2e": {
+    "count": 1213.0,
+    "mean": 6.961301470549104,
+    "p50": 1.7495028690318577,
+    "p90": 18.033410895219994,
+    "p99": 68.80023987947489
+  }
+}

analysis/crossover/d1_i1024_goodput.json

@@ -0,0 +1,142 @@
+{
+  "baseline": "8C-proxy",
+  "arms": {
+    "8C-proxy": {
+      "name": "8C-proxy",
+      "n_offered": 1167,
+      "n_success": 1167,
+      "completion_rate": 1.0,
+      "offered_window_s": 299.736197,
+      "offered_qps": 3.8934236561358655,
+      "wall_clock_s": 300.1385939740576,
+      "amplification": 1.0013425037685975,
+      "ttft": {
+        "count": 1167,
+        "mean": 0.08144469193556772,
+        "p50": 0.07862715201918036,
+        "p90": 0.08015060934703797,
+        "p99": 0.0875979653932154
+      },
+      "tpot": {
+        "count": 1167,
+        "mean": 0.005001699398049616,
+        "p50": 0.004988961030788246,
+        "p90": 0.005045765990923557,
+        "p99": 0.005062779263327164
+      },
+      "e2e": {
+        "count": 1167,
+        "mean": 0.3968209869372152,
+        "p50": 0.393534954986535,
+        "p90": 0.39730903680901974,
+        "p99": 0.40925762055674536
+      }
+    },
+    "4P+4D": {
+      "name": "4P+4D",
+      "n_offered": 1167,
+      "n_success": 1167,
+      "completion_rate": 1.0,
+      "offered_window_s": 299.736197,
+      "offered_qps": 3.8934236561358655,
+      "wall_clock_s": 300.1604231200181,
+      "amplification": 1.0014153316291596,
+      "ttft": {
+        "count": 1167,
+        "mean": 0.09946277569807849,
+        "p50": 0.09600010397844017,
+        "p90": 0.10452785079833121,
+        "p99": 0.11205230774357905
+      },
+      "tpot": {
+        "count": 1167,
+        "mean": 0.005007447102661814,
+        "p50": 0.004987124730611131,
+        "p90": 0.005003212126977151,
+        "p99": 0.005478902989961502
+      },
+      "e2e": {
+        "count": 1167,
+        "mean": 0.415208436744531,
+        "p50": 0.41056320699863136,
+        "p90": 0.4200975856045261,
+        "p99": 0.44871115096379066
+      }
+    },
+    "6P+2D": {
+      "name": "6P+2D",
+      "n_offered": 1167,
+      "n_success": 1167,
+      "completion_rate": 1.0,
+      "offered_window_s": 299.736197,
+      "offered_qps": 3.8934236561358655,
+      "wall_clock_s": 300.2032543020323,
+      "amplification": 1.0015582278907484,
+      "ttft": {
+        "count": 1167,
+        "mean": 0.10561635944505095,
+        "p50": 0.10468761203810573,
+        "p90": 0.11257308297790587,
+        "p99": 0.12065987563692024
+      },
+      "tpot": {
+        "count": 1167,
+        "mean": 0.005328901365752947,
+        "p50": 0.005144592110318915,
+        "p90": 0.005990574603515958,
+        "p99": 0.006688486758013448
+      },
+      "e2e": {
+        "count": 1167,
+        "mean": 0.4416300980896939,
+        "p50": 0.42991435900330544,
+        "p90": 0.4854830394731835,
+        "p99": 0.5404117306252005
+      }
+    }
+  },
+  "slo_grid": [
+    {
+      "ttft_slo_s": 2.0,
+      "tpot_slo_s": 0.05,
+      "arms": {
+        "8C-proxy": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 1167
+        },
+        "4P+4D": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 1167
+        },
+        "6P+2D": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 1167
+        }
+      }
+    },
+    {
+      "ttft_slo_s": 5.0,
+      "tpot_slo_s": 0.05,
+      "arms": {
+        "8C-proxy": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 1167
+        },
+        "4P+4D": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 1167
+        },
+        "6P+2D": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 1167
+        }
+      }
+    }
+  ]
+}

analysis/crossover/d1_i16384_goodput.json

@@ -0,0 +1,142 @@
+{
+  "baseline": "8C-proxy",
+  "arms": {
+    "8C-proxy": {
+      "name": "8C-proxy",
+      "n_offered": 1167,
+      "n_success": 1167,
+      "completion_rate": 1.0,
+      "offered_window_s": 299.736197,
+      "offered_qps": 3.8934236561358655,
+      "wall_clock_s": 303.05708941399644,
+      "amplification": 1.0110793839624128,
+      "ttft": {
+        "count": 1167,
+        "mean": 1.674437926279444,
+        "p50": 1.5353219069947954,
+        "p90": 2.0787689138029237,
+        "p99": 3.039117059087727
+      },
+      "tpot": {
+        "count": 1167,
+        "mean": 0.035498316425319934,
+        "p50": 0.02951206674587743,
+        "p90": 0.085249871320677,
+        "p99": 0.15422643764865662
+      },
+      "e2e": {
+        "count": 1167,
+        "mean": 3.9111985703531236,
+        "p50": 3.392241114997887,
+        "p90": 7.760864628604043,
+        "p99": 11.30427318874542
+      }
+    },
+    "4P+4D": {
+      "name": "4P+4D",
+      "n_offered": 1167,
+      "n_success": 1136,
+      "completion_rate": 0.9734361610968295,
+      "offered_window_s": 299.736197,
+      "offered_qps": 3.8934236561358655,
+      "wall_clock_s": 866.9596620649972,
+      "amplification": 2.8924089607535697,
+      "ttft": {
+        "count": 1136,
+        "mean": 65.09021699308856,
+        "p50": 66.22900710900285,
+        "p90": 112.5535424454938,
+        "p99": 124.55262411334482
+      },
+      "tpot": {
+        "count": 1136,
+        "mean": 0.005710658520121912,
+        "p50": 0.005725543936557461,
+        "p90": 0.005750613698356098,
+        "p99": 0.0058447879207267845
+      },
+      "e2e": {
+        "count": 1136,
+        "mean": 65.4504681098121,
+        "p50": 66.59053339700768,
+        "p90": 112.9150809329949,
+        "p99": 124.91415351489852
+      }
+    },
+    "6P+2D": {
+      "name": "6P+2D",
+      "n_offered": 1167,
+      "n_success": 1167,
+      "completion_rate": 1.0,
+      "offered_window_s": 299.736197,
+      "offered_qps": 3.8934236561358655,
+      "wall_clock_s": 307.42712411200046,
+      "amplification": 1.0256589867656205,
+      "ttft": {
+        "count": 1167,
+        "mean": 3.6233640342625417,
+        "p50": 3.255483777000336,
+        "p90": 6.0935156565916255,
+        "p99": 7.349482456580735
+      },
+      "tpot": {
+        "count": 1167,
+        "mean": 0.006360297526341433,
+        "p50": 0.006324973206372104,
+        "p90": 0.007198417158741947,
+        "p99": 0.007942749238420567
+      },
+      "e2e": {
+        "count": 1167,
+        "mean": 4.024414356593621,
+        "p50": 3.6796783979953034,
+        "p90": 6.510242249601289,
+        "p99": 7.7530036393977895
+      }
+    }
+  },
+  "slo_grid": [
+    {
+      "ttft_slo_s": 2.0,
+      "tpot_slo_s": 0.05,
+      "arms": {
+        "8C-proxy": {
+          "attainment": 0.6563838903170522,
+          "pd_advantage": 1.0,
+          "n_slo": 766
+        },
+        "4P+4D": {
+          "attainment": 0.0,
+          "pd_advantage": 0.0,
+          "n_slo": 0
+        },
+        "6P+2D": {
+          "attainment": 0.20565552699228792,
+          "pd_advantage": 0.3133159268929504,
+          "n_slo": 240
+        }
+      }
+    },
+    {
+      "ttft_slo_s": 5.0,
+      "tpot_slo_s": 0.05,
+      "arms": {
+        "8C-proxy": {
+          "attainment": 0.7095115681233933,
+          "pd_advantage": 1.0,
+          "n_slo": 828
+        },
+        "4P+4D": {
+          "attainment": 0.012853470437017995,
+          "pd_advantage": 0.018115942028985508,
+          "n_slo": 15
+        },
+        "6P+2D": {
+          "attainment": 0.7746358183376179,
+          "pd_advantage": 1.0917874396135265,
+          "n_slo": 904
+        }
+      }
+    }
+  ]
+}

analysis/crossover/d1_i2048_goodput.json

@@ -0,0 +1,142 @@
+{
+  "baseline": "8C-proxy",
+  "arms": {
+    "8C-proxy": {
+      "name": "8C-proxy",
+      "n_offered": 1167,
+      "n_success": 1167,
+      "completion_rate": 1.0,
+      "offered_window_s": 299.736197,
+      "offered_qps": 3.8934236561358655,
+      "wall_clock_s": 300.1927249849541,
+      "amplification": 1.0015230992770423,
+      "ttft": {
+        "count": 1167,
+        "mean": 0.1367582682356152,
+        "p50": 0.1337889520218596,
+        "p90": 0.13568343156948687,
+        "p99": 0.1417779766954481
+      },
+      "tpot": {
+        "count": 1167,
+        "mean": 0.005039774583944486,
+        "p50": 0.005025791809968059,
+        "p90": 0.005088228446667984,
+        "p99": 0.0051380019550329985
+      },
+      "e2e": {
+        "count": 1167,
+        "mean": 0.45452996390628697,
+        "p50": 0.45110413199290633,
+        "p90": 0.4555703255813569,
+        "p99": 0.4688122991356063
+      }
+    },
+    "4P+4D": {
+      "name": "4P+4D",
+      "n_offered": 1167,
+      "n_success": 1167,
+      "completion_rate": 1.0,
+      "offered_window_s": 299.736197,
+      "offered_qps": 3.8934236561358655,
+      "wall_clock_s": 300.2166367470054,
+      "amplification": 1.0016028753010615,
+      "ttft": {
+        "count": 1167,
+        "mean": 0.1660798433322169,
+        "p50": 0.16653224604669958,
+        "p90": 0.17306958099361508,
+        "p99": 0.1832630861806683
+      },
+      "tpot": {
+        "count": 1167,
+        "mean": 0.005046875948133808,
+        "p50": 0.0050270736812510425,
+        "p90": 0.0050664691831029595,
+        "p99": 0.005532995437730161
+      },
+      "e2e": {
+        "count": 1167,
+        "mean": 0.48433690918382055,
+        "p50": 0.4845748710213229,
+        "p90": 0.4916891472181305,
+        "p99": 0.5235905526624989
+      }
+    },
+    "6P+2D": {
+      "name": "6P+2D",
+      "n_offered": 1167,
+      "n_success": 1167,
+      "completion_rate": 1.0,
+      "offered_window_s": 299.736197,
+      "offered_qps": 3.8934236561358655,
+      "wall_clock_s": 300.2855537990108,
+      "amplification": 1.001832800991369,
+      "ttft": {
+        "count": 1167,
+        "mean": 0.17076528707394278,
+        "p50": 0.16959826194215566,
+        "p90": 0.17923957279417663,
+        "p99": 0.18792458378709853
+      },
+      "tpot": {
+        "count": 1167,
+        "mean": 0.005377765550956012,
+        "p50": 0.005177637602808693,
+        "p90": 0.006078562941697855,
+        "p99": 0.007001848420689976
+      },
+      "e2e": {
+        "count": 1167,
+        "mean": 0.5098518045703324,
+        "p50": 0.4961305959150195,
+        "p90": 0.5569328068522736,
+        "p99": 0.6314893902023319
+      }
+    }
+  },
+  "slo_grid": [
+    {
+      "ttft_slo_s": 2.0,
+      "tpot_slo_s": 0.05,
+      "arms": {
+        "8C-proxy": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 1167
+        },
+        "4P+4D": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 1167
+        },
+        "6P+2D": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 1167
+        }
+      }
+    },
+    {
+      "ttft_slo_s": 5.0,
+      "tpot_slo_s": 0.05,
+      "arms": {
+        "8C-proxy": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 1167
+        },
+        "4P+4D": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 1167
+        },
+        "6P+2D": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 1167
+        }
+      }
+    }
+  ]
+}

analysis/crossover/d1_i32768_goodput.json

@@ -0,0 +1,142 @@
+{
+  "baseline": "8C-proxy",
+  "arms": {
+    "8C-proxy": {
+      "name": "8C-proxy",
+      "n_offered": 1167,
+      "n_success": 951,
+      "completion_rate": 0.8149100257069408,
+      "offered_window_s": 299.736197,
+      "offered_qps": 3.8934236561358655,
+      "wall_clock_s": 567.1572082909988,
+      "amplification": 1.8921879104611408,
+      "ttft": {
+        "count": 951,
+        "mean": 156.08624654169503,
+        "p50": 159.80228465000982,
+        "p90": 263.44082942500245,
+        "p99": 267.3189407904938
+      },
+      "tpot": {
+        "count": 951,
+        "mean": 0.5120599246480225,
+        "p50": 0.5265210192539223,
+        "p90": 0.5403215261268621,
+        "p99": 0.5436686666190138
+      },
+      "e2e": {
+        "count": 951,
+        "mean": 188.3470215559188,
+        "p50": 192.9057134770119,
+        "p90": 294.68771972299146,
+        "p99": 300.2205298934932
+      }
+    },
+    "4P+4D": {
+      "name": "4P+4D",
+      "n_offered": 1167,
+      "n_success": 596,
+      "completion_rate": 0.5107112253641817,
+      "offered_window_s": 299.736197,
+      "offered_qps": 3.8934236561358655,
+      "wall_clock_s": 789.8321366070013,
+      "amplification": 2.63509093833936,
+      "ttft": {
+        "count": 596,
+        "mean": 240.99040966441748,
+        "p50": 259.1066791820049,
+        "p90": 379.7789027979961,
+        "p99": 381.22047349565617
+      },
+      "tpot": {
+        "count": 596,
+        "mean": 0.006412497327210905,
+        "p50": 0.006444117785865692,
+        "p90": 0.0064974606745333095,
+        "p99": 0.006660929446938789
+      },
+      "e2e": {
+        "count": 596,
+        "mean": 241.39516468475196,
+        "p50": 259.5132920899996,
+        "p90": 380.1822811405,
+        "p99": 381.620277955458
+      }
+    },
+    "6P+2D": {
+      "name": "6P+2D",
+      "n_offered": 1167,
+      "n_success": 712,
+      "completion_rate": 0.6101113967437874,
+      "offered_window_s": 299.736197,
+      "offered_qps": 3.8934236561358655,
+      "wall_clock_s": 880.7267912379903,
+      "amplification": 2.938339780290167,
+      "ttft": {
+        "count": 712,
+        "mean": 175.69983718378106,
+        "p50": 172.43730882649834,
+        "p90": 250.6456608123961,
+        "p99": 594.014642834873
+      },
+      "tpot": {
+        "count": 712,
+        "mean": 0.007946936024875445,
+        "p50": 0.007568808706382119,
+        "p90": 0.010293392149296345,
+        "p99": 0.011275891952114068
+      },
+      "e2e": {
+        "count": 712,
+        "mean": 176.20114515147316,
+        "p50": 172.91078094949626,
+        "p90": 251.09661155799986,
+        "p99": 594.4257701966026
+      }
+    }
+  },
+  "slo_grid": [
+    {
+      "ttft_slo_s": 2.0,
+      "tpot_slo_s": 0.05,
+      "arms": {
+        "8C-proxy": {
+          "attainment": 0.0,
+          "pd_advantage": NaN,
+          "n_slo": 0
+        },
+        "4P+4D": {
+          "attainment": 0.0,
+          "pd_advantage": NaN,
+          "n_slo": 0
+        },
+        "6P+2D": {
+          "attainment": 0.0,
+          "pd_advantage": NaN,
+          "n_slo": 0
+        }
+      }
+    },
+    {
+      "ttft_slo_s": 5.0,
+      "tpot_slo_s": 0.05,
+      "arms": {
+        "8C-proxy": {
+          "attainment": 0.0,
+          "pd_advantage": NaN,
+          "n_slo": 0
+        },
+        "4P+4D": {
+          "attainment": 0.0,
+          "pd_advantage": NaN,
+          "n_slo": 0
+        },
+        "6P+2D": {
+          "attainment": 0.0,
+          "pd_advantage": NaN,
+          "n_slo": 0
+        }
+      }
+    }
+  ]
+}

analysis/crossover/d1_i4096_goodput.json

@@ -0,0 +1,142 @@
+{
+  "baseline": "8C-proxy",
+  "arms": {
+    "8C-proxy": {
+      "name": "8C-proxy",
+      "n_offered": 1167,
+      "n_success": 1167,
+      "completion_rate": 1.0,
+      "offered_window_s": 299.736197,
+      "offered_qps": 3.8934236561358655,
+      "wall_clock_s": 300.32634208898526,
+      "amplification": 1.0019688816195438,
+      "ttft": {
+        "count": 1167,
+        "mean": 0.2628433253752162,
+        "p50": 0.2604120689211413,
+        "p90": 0.2637787788407877,
+        "p99": 0.2976114549371393
+      },
+      "tpot": {
+        "count": 1167,
+        "mean": 0.005143964614959416,
+        "p50": 0.005124552334318795,
+        "p90": 0.005180289564166395,
+        "p99": 0.0052467077378080115
+      },
+      "e2e": {
+        "count": 1167,
+        "mean": 0.5871811069348818,
+        "p50": 0.5839827890740708,
+        "p90": 0.5883447280153632,
+        "p99": 0.6408826243295329
+      }
+    },
+    "4P+4D": {
+      "name": "4P+4D",
+      "n_offered": 1167,
+      "n_success": 1167,
+      "completion_rate": 1.0,
+      "offered_window_s": 299.736197,
+      "offered_qps": 3.8934236561358655,
+      "wall_clock_s": 300.3759389320621,
+      "amplification": 1.0021343499332585,
+      "ttft": {
+        "count": 1167,
+        "mean": 0.30861241175426407,
+        "p50": 0.3091614870354533,
+        "p90": 0.3227507559815422,
+        "p99": 0.3547600568598136
+      },
+      "tpot": {
+        "count": 1167,
+        "mean": 0.005148936446188915,
+        "p50": 0.0051316127147791645,
+        "p90": 0.005184003428393413,
+        "p99": 0.00547781713853871
+      },
+      "e2e": {
+        "count": 1167,
+        "mean": 0.6332931331915089,
+        "p50": 0.6365591660141945,
+        "p90": 0.6472898541716858,
+        "p99": 0.7053035433730102
+      }
+    },
+    "6P+2D": {
+      "name": "6P+2D",
+      "n_offered": 1167,
+      "n_success": 1167,
+      "completion_rate": 1.0,
+      "offered_window_s": 299.736197,
+      "offered_qps": 3.8934236561358655,
+      "wall_clock_s": 300.4041277950164,
+      "amplification": 1.0022283955081221,
+      "ttft": {
+        "count": 1167,
+        "mean": 0.31527804771431056,
+        "p50": 0.31678270106203854,
+        "p90": 0.3284846659982577,
+        "p99": 0.3402980738645419
+      },
+      "tpot": {
+        "count": 1167,
+        "mean": 0.0055213440500518985,
+        "p50": 0.005325577683776381,
+        "p90": 0.006278165857056304,
+        "p99": 0.0070092806818761975
+      },
+      "e2e": {
+        "count": 1167,
+        "mean": 0.6634175265073216,
+        "p50": 0.6494167000055313,
+        "p90": 0.7135140666039661,
+        "p99": 0.7741130576422437
+      }
+    }
+  },
+  "slo_grid": [
+    {
+      "ttft_slo_s": 2.0,
+      "tpot_slo_s": 0.05,
+      "arms": {
+        "8C-proxy": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 1167
+        },
+        "4P+4D": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 1167
+        },
+        "6P+2D": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 1167
+        }
+      }
+    },
+    {
+      "ttft_slo_s": 5.0,
+      "tpot_slo_s": 0.05,
+      "arms": {
+        "8C-proxy": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 1167
+        },
+        "4P+4D": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 1167
+        },
+        "6P+2D": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 1167
+        }
+      }
+    }
+  ]
+}

analysis/crossover/d1_i8192_goodput.json

@@ -0,0 +1,142 @@
+{
+  "baseline": "8C-proxy",
+  "arms": {
+    "8C-proxy": {
+      "name": "8C-proxy",
+      "n_offered": 1167,
+      "n_success": 1167,
+      "completion_rate": 1.0,
+      "offered_window_s": 299.736197,
+      "offered_qps": 3.8934236561358655,
+      "wall_clock_s": 300.66455180699995,
+      "amplification": 1.0030972395602922,
+      "ttft": {
+        "count": 1167,
+        "mean": 0.584800394000121,
+        "p50": 0.5817365200055065,
+        "p90": 0.5870239008043427,
+        "p99": 0.6088061455392736
+      },
+      "tpot": {
+        "count": 1167,
+        "mean": 0.005585180932058873,
+        "p50": 0.005308644777941813,
+        "p90": 0.005355837698406836,
+        "p99": 0.014590345752427337
+      },
+      "e2e": {
+        "count": 1167,
+        "mean": 0.9369382756572071,
+        "p50": 0.9158835929993074,
+        "p90": 0.9229304600012256,
+        "p99": 1.5052119748995754
+      }
+    },
+    "4P+4D": {
+      "name": "4P+4D",
+      "n_offered": 1167,
+      "n_success": 1167,
+      "completion_rate": 1.0,
+      "offered_window_s": 299.736197,
+      "offered_qps": 3.8934236561358655,
+      "wall_clock_s": 300.7607197400066,
+      "amplification": 1.003418081467173,
+      "ttft": {
+        "count": 1167,
+        "mean": 0.7293765576349805,
+        "p50": 0.6887177099997643,
+        "p90": 0.8911576298065484,
+        "p99": 1.1879012519156076
+      },
+      "tpot": {
+        "count": 1167,
+        "mean": 0.005308951736224922,
+        "p50": 0.0053098901269203495,
+        "p90": 0.005347346425488857,
+        "p99": 0.005420954272433716
+      },
+      "e2e": {
+        "count": 1167,
+        "mean": 1.064155324243521,
+        "p50": 1.0238807429996086,
+        "p90": 1.22566919879755,
+        "p99": 1.521622942700049
+      }
+    },
+    "6P+2D": {
+      "name": "6P+2D",
+      "n_offered": 1167,
+      "n_success": 1167,
+      "completion_rate": 1.0,
+      "offered_window_s": 299.736197,
+      "offered_qps": 3.8934236561358655,
+      "wall_clock_s": 300.8507765819959,
+      "amplification": 1.003718535142407,
+      "ttft": {
+        "count": 1167,
+        "mean": 0.7028102769056583,
+        "p50": 0.6950660040020011,
+        "p90": 0.7394948218076024,
+        "p99": 0.8658222487580485
+      },
+      "tpot": {
+        "count": 1167,
+        "mean": 0.005782890183689697,
+        "p50": 0.00558671395230617,
+        "p90": 0.006629064533303286,
+        "p99": 0.007366264003757681
+      },
+      "e2e": {
+        "count": 1167,
+        "mean": 1.0674261210719522,
+        "p50": 1.0612060459970962,
+        "p90": 1.1425091800017981,
+        "p99": 1.3045775910036173
+      }
+    }
+  },
+  "slo_grid": [
+    {
+      "ttft_slo_s": 2.0,
+      "tpot_slo_s": 0.05,
+      "arms": {
+        "8C-proxy": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 1167
+        },
+        "4P+4D": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 1167
+        },
+        "6P+2D": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 1167
+        }
+      }
+    },
+    {
+      "ttft_slo_s": 5.0,
+      "tpot_slo_s": 0.05,
+      "arms": {
+        "8C-proxy": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 1167
+        },
+        "4P+4D": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 1167
+        },
+        "6P+2D": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 1167
+        }
+      }
+    }
+  ]
+}

analysis/crossover/d2_o1024_goodput.json

@@ -0,0 +1,121 @@
+{
+  "baseline": "8C-proxy",
+  "arms": {
+    "8C-proxy": {
+      "name": "8C-proxy",
+      "n_offered": 3552,
+      "n_success": 1950,
+      "completion_rate": 0.5489864864864865,
+      "offered_window_s": 299.74141899999995,
+      "offered_qps": 11.850214134070008,
+      "wall_clock_s": 348.4953830049999,
+      "amplification": 1.1626534102882857,
+      "ttft": {
+        "count": 1950,
+        "mean": 0.8245253885559363,
+        "p50": 0.7988759850013594,
+        "p90": 1.5384256363991882,
+        "p99": 2.125257717882705
+      },
+      "tpot": {
+        "count": 1950,
+        "mean": 0.07444567704430907,
+        "p50": 0.08914986800879843,
+        "p90": 0.09127071481447005,
+        "p99": 0.09233926755616054
+      },
+      "e2e": {
+        "count": 1950,
+        "mean": 77.04034660455785,
+        "p50": 92.29079191399796,
+        "p90": 94.47993849659906,
+        "p99": 95.72745552870882
+      }
+    },
+    "4P+4D": {
+      "name": "4P+4D",
+      "n_offered": 3552,
+      "n_success": 2754,
+      "completion_rate": 0.7753378378378378,
+      "offered_window_s": 299.74141899999995,
+      "offered_qps": 11.850214134070008,
+      "wall_clock_s": 744.3694221920014,
+      "amplification": 2.4833719166185753,
+      "ttft": {
+        "count": 2754,
+        "mean": 4.5006646864741695,
+        "p50": 2.329877773499902,
+        "p90": 10.863291689799375,
+        "p99": 21.572781211729307
+      },
+      "tpot": {
+        "count": 2754,
+        "mean": 0.046089308743682535,
+        "p50": 0.04704797274047173,
+        "p90": 0.04792202048768524,
+        "p99": 0.059544689888886115
+      },
+      "e2e": {
+        "count": 2754,
+        "mean": 51.68415100813728,
+        "p50": 50.91020956099965,
+        "p90": 59.22212425199977,
+        "p99": 68.42946432940865
+      }
+    },
+    "6P+2D": {
+      "name": "6P+2D",
+      "n_offered": 3552,
+      "n_success": 1928,
+      "completion_rate": 0.5427927927927928,
+      "offered_window_s": 299.74141899999995,
+      "offered_qps": 11.850214134070008,
+      "wall_clock_s": 821.7403777359868,
+      "amplification": 2.7414975897474716,
+      "ttft": {
+        "count": 1928,
+        "mean": 39.07585104131927,
+        "p50": 42.6814165695032,
+        "p90": 63.28579387369681,
+        "p99": 73.84470144698193
+      },
+      "tpot": {
+        "count": 1928,
+        "mean": 0.04064862157373329,
+        "p50": 0.0398688508558163,
+        "p90": 0.04277554483881431,
+        "p99": 0.0695667276081441
+      },
+      "e2e": {
+        "count": 1928,
+        "mean": 80.66499786808757,
+        "p50": 84.05398454950046,
+        "p90": 105.42814423799427,
+        "p99": 113.52591980495636
+      }
+    }
+  },
+  "slo_grid": [
+    {
+      "ttft_slo_s": 2.0,
+      "tpot_slo_s": 0.05,
+      "arms": {
+        "8C-proxy": {
+          "attainment": 0.10191441441441441,
+          "pd_advantage": 1.0,
+          "n_slo": 362
+        },
+        "4P+4D": {
+          "attainment": 0.3502252252252252,
+          "pd_advantage": 3.43646408839779,
+          "n_slo": 1244
+        },
+        "6P+2D": {
+          "attainment": 0.06447072072072071,
+          "pd_advantage": 0.6325966850828729,
+          "n_slo": 229
+        }
+      }
+    }
+  ]
+}

analysis/crossover/d2_o128_goodput.json

@@ -0,0 +1,121 @@
+{
+  "baseline": "8C-proxy",
+  "arms": {
+    "8C-proxy": {
+      "name": "8C-proxy",
+      "n_offered": 3552,
+      "n_success": 3552,
+      "completion_rate": 1.0,
+      "offered_window_s": 299.74141899999995,
+      "offered_qps": 11.850214134070008,
+      "wall_clock_s": 300.61600343801547,
+      "amplification": 1.0029177964157683,
+      "ttft": {
+        "count": 3552,
+        "mean": 0.139721406387908,
+        "p50": 0.13759525847854093,
+        "p90": 0.14188919204752892,
+        "p99": 0.17052793529117438
+      },
+      "tpot": {
+        "count": 3552,
+        "mean": 0.006496044329480822,
+        "p50": 0.006430442302898453,
+        "p90": 0.007889476366154117,
+        "p99": 0.009282101713493095
+      },
+      "e2e": {
+        "count": 3552,
+        "mean": 0.9651016481538636,
+        "p50": 0.9548319909954444,
+        "p90": 1.1440699087572284,
+        "p99": 1.3324073514551853
+      }
+    },
+    "4P+4D": {
+      "name": "4P+4D",
+      "n_offered": 3552,
+      "n_success": 3552,
+      "completion_rate": 1.0,
+      "offered_window_s": 299.74141899999995,
+      "offered_qps": 11.850214134070008,
+      "wall_clock_s": 300.7410873459885,
+      "amplification": 1.0033351024670651,
+      "ttft": {
+        "count": 3552,
+        "mean": 0.18467717330343286,
+        "p50": 0.1806962049449794,
+        "p90": 0.1929074571467936,
+        "p99": 0.2592483600333798
+      },
+      "tpot": {
+        "count": 3552,
+        "mean": 0.006980159784640828,
+        "p50": 0.006945047342387123,
+        "p90": 0.008164690015837552,
+        "p99": 0.009163911323187739
+      },
+      "e2e": {
+        "count": 3552,
+        "mean": 1.0715517728354285,
+        "p50": 1.063076548918616,
+        "p90": 1.2275727374362762,
+        "p99": 1.4021264237130526
+      }
+    },
+    "6P+2D": {
+      "name": "6P+2D",
+      "n_offered": 3552,
+      "n_success": 3552,
+      "completion_rate": 1.0,
+      "offered_window_s": 299.74141899999995,
+      "offered_qps": 11.850214134070008,
+      "wall_clock_s": 301.1243950970238,
+      "amplification": 1.004613897210595,
+      "ttft": {
+        "count": 3552,
+        "mean": 0.20078434001432569,
+        "p50": 0.19958186394069344,
+        "p90": 0.213167638995219,
+        "p99": 0.23225504373782313
+      },
+      "tpot": {
+        "count": 3552,
+        "mean": 0.010393430778126047,
+        "p50": 0.010396533917478015,
+        "p90": 0.012065167099743436,
+        "p99": 0.01323438493101344
+      },
+      "e2e": {
+        "count": 3552,
+        "mean": 1.5212831554822883,
+        "p50": 1.5218885459471494,
+        "p90": 1.7440477049094625,
+        "p99": 1.9191367691196497
+      }
+    }
+  },
+  "slo_grid": [
+    {
+      "ttft_slo_s": 2.0,
+      "tpot_slo_s": 0.05,
+      "arms": {
+        "8C-proxy": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 3552
+        },
+        "4P+4D": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 3552
+        },
+        "6P+2D": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 3552
+        }
+      }
+    }
+  ]
+}

analysis/crossover/d2_o2048_goodput.json

@@ -0,0 +1,121 @@
+{
+  "baseline": "8C-proxy",
+  "arms": {
+    "8C-proxy": {
+      "name": "8C-proxy",
+      "n_offered": 3552,
+      "n_success": 1050,
+      "completion_rate": 0.2956081081081081,
+      "offered_window_s": 299.74141899999995,
+      "offered_qps": 11.850214134070008,
+      "wall_clock_s": 414.92026689198974,
+      "amplification": 1.3842607013613617,
+      "ttft": {
+        "count": 1050,
+        "mean": 0.8393797435154972,
+        "p50": 0.8442945880015031,
+        "p90": 1.4299255040969001,
+        "p99": 2.1686189359705894
+      },
+      "tpot": {
+        "count": 1050,
+        "mean": 0.08418485046407832,
+        "p50": 0.08937374470713316,
+        "p90": 0.09085719287396188,
+        "p99": 0.09190563933350096
+      },
+      "e2e": {
+        "count": 1050,
+        "mean": 173.22149556513375,
+        "p50": 183.9983977669981,
+        "p90": 187.03014375649946,
+        "p99": 189.36978534828302
+      }
+    },
+    "4P+4D": {
+      "name": "4P+4D",
+      "n_offered": 3552,
+      "n_success": 1320,
+      "completion_rate": 0.3716216216216216,
+      "offered_window_s": 299.74141899999995,
+      "offered_qps": 11.850214134070008,
+      "wall_clock_s": 802.1317602040072,
+      "amplification": 2.6760791447511205,
+      "ttft": {
+        "count": 1320,
+        "mean": 19.92388417242484,
+        "p50": 8.059029546493548,
+        "p90": 64.71132232169766,
+        "p99": 87.07121913193099
+      },
+      "tpot": {
+        "count": 1320,
+        "mean": 0.05007250707090057,
+        "p50": 0.0492711045202749,
+        "p90": 0.07142825628519621,
+        "p99": 0.08267229383702501
+      },
+      "e2e": {
+        "count": 1320,
+        "mean": 122.44309290737698,
+        "p50": 118.35192646600626,
+        "p90": 167.89536100080443,
+        "p99": 187.19021692074878
+      }
+    },
+    "6P+2D": {
+      "name": "6P+2D",
+      "n_offered": 3552,
+      "n_success": 1041,
+      "completion_rate": 0.29307432432432434,
+      "offered_window_s": 299.74141899999995,
+      "offered_qps": 11.850214134070008,
+      "wall_clock_s": 879.8981197069952,
+      "amplification": 2.935523968100636,
+      "ttft": {
+        "count": 1041,
+        "mean": 86.06142147925078,
+        "p50": 92.45557322700915,
+        "p90": 153.47880471601093,
+        "p99": 159.16995789420034
+      },
+      "tpot": {
+        "count": 1041,
+        "mean": 0.04275836784091199,
+        "p50": 0.038143618996089604,
+        "p90": 0.06297858072350163,
+        "p99": 0.07100282797010407
+      },
+      "e2e": {
+        "count": 1041,
+        "mean": 173.5925440399473,
+        "p50": 199.81516771799943,
+        "p90": 230.71809448600106,
+        "p99": 234.9438766648003
+      }
+    }
+  },
+  "slo_grid": [
+    {
+      "ttft_slo_s": 2.0,
+      "tpot_slo_s": 0.05,
+      "arms": {
+        "8C-proxy": {
+          "attainment": 0.009009009009009009,
+          "pd_advantage": 1.0,
+          "n_slo": 32
+        },
+        "4P+4D": {
+          "attainment": 0.11486486486486487,
+          "pd_advantage": 12.75,
+          "n_slo": 408
+        },
+        "6P+2D": {
+          "attainment": 0.04954954954954955,
+          "pd_advantage": 5.5,
+          "n_slo": 176
+        }
+      }
+    }
+  ]
+}

analysis/crossover/d2_o256_goodput.json

@@ -0,0 +1,121 @@
+{
+  "baseline": "8C-proxy",
+  "arms": {
+    "8C-proxy": {
+      "name": "8C-proxy",
+      "n_offered": 3552,
+      "n_success": 3552,
+      "completion_rate": 1.0,
+      "offered_window_s": 299.74141899999995,
+      "offered_qps": 11.850214134070008,
+      "wall_clock_s": 301.78380814404227,
+      "amplification": 1.0068138369093473,
+      "ttft": {
+        "count": 3552,
+        "mean": 0.14506312318632342,
+        "p50": 0.14291562204016373,
+        "p90": 0.1491016250802204,
+        "p99": 0.1832571337954128
+      },
+      "tpot": {
+        "count": 3552,
+        "mean": 0.0089611879488046,
+        "p50": 0.008925844482420122,
+        "p90": 0.010698060123752072,
+        "p99": 0.011754058378410762
+      },
+      "e2e": {
+        "count": 3552,
+        "mean": 2.430673889741114,
+        "p50": 2.419925262918696,
+        "p90": 2.874893436266575,
+        "p99": 3.1480205075151746
+      }
+    },
+    "4P+4D": {
+      "name": "4P+4D",
+      "n_offered": 3552,
+      "n_success": 3551,
+      "completion_rate": 0.9997184684684685,
+      "offered_window_s": 299.74141899999995,
+      "offered_qps": 11.850214134070008,
+      "wall_clock_s": 744.8868488909211,
+      "amplification": 2.4850981601942745,
+      "ttft": {
+        "count": 3551,
+        "mean": 0.20500909556101396,
+        "p50": 0.20017725799698383,
+        "p90": 0.21613375598099083,
+        "p99": 0.29023518750909716
+      },
+      "tpot": {
+        "count": 3551,
+        "mean": 0.010307210080394247,
+        "p50": 0.010299421709450874,
+        "p90": 0.011793581666607482,
+        "p99": 0.012639003840969035
+      },
+      "e2e": {
+        "count": 3551,
+        "mean": 2.8339171693501317,
+        "p50": 2.8360355379991233,
+        "p90": 3.2198793930001557,
+        "p99": 3.438588996999897
+      }
+    },
+    "6P+2D": {
+      "name": "6P+2D",
+      "n_offered": 3552,
+      "n_success": 3552,
+      "completion_rate": 1.0,
+      "offered_window_s": 299.74141899999995,
+      "offered_qps": 11.850214134070008,
+      "wall_clock_s": 303.4300506779691,
+      "amplification": 1.0123060459587976,
+      "ttft": {
+        "count": 3552,
+        "mean": 0.2333820717117756,
+        "p50": 0.23283391550648957,
+        "p90": 0.24394672318594532,
+        "p99": 0.2734717815916518
+      },
+      "tpot": {
+        "count": 3552,
+        "mean": 0.016453822599812207,
+        "p50": 0.016576926972415737,
+        "p90": 0.017214638463623238,
+        "p99": 0.01769411424845092
+      },
+      "e2e": {
+        "count": 3552,
+        "mean": 4.430368151474328,
+        "p50": 4.463736370031256,
+        "p90": 4.628522685484495,
+        "p99": 4.75812664218829
+      }
+    }
+  },
+  "slo_grid": [
+    {
+      "ttft_slo_s": 2.0,
+      "tpot_slo_s": 0.05,
+      "arms": {
+        "8C-proxy": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 3552
+        },
+        "4P+4D": {
+          "attainment": 0.9997184684684685,
+          "pd_advantage": 0.9997184684684685,
+          "n_slo": 3551
+        },
+        "6P+2D": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 3552
+        }
+      }
+    }
+  ]
+}

analysis/crossover/d2_o4096_goodput.json

@@ -0,0 +1,121 @@
+{
+  "baseline": "8C-proxy",
+  "arms": {
+    "8C-proxy": {
+      "name": "8C-proxy",
+      "n_offered": 3552,
+      "n_success": 545,
+      "completion_rate": 0.15343468468468469,
+      "offered_window_s": 299.74141899999995,
+      "offered_qps": 11.850214134070008,
+      "wall_clock_s": 505.4139580530027,
+      "amplification": 1.6861665622961597,
+      "ttft": {
+        "count": 545,
+        "mean": 4.137136383750316,
+        "p50": 0.5702517530007754,
+        "p90": 1.6976309113961179,
+        "p99": 56.1814190310361
+      },
+      "tpot": {
+        "count": 545,
+        "mean": 0.08390926873267023,
+        "p50": 0.08933810225128375,
+        "p90": 0.09449654152820693,
+        "p99": 0.10591519025133134
+      },
+      "e2e": {
+        "count": 545,
+        "mean": 347.7765141811284,
+        "p50": 366.3568219640001,
+        "p90": 388.1241692415934,
+        "p99": 435.2848098113155
+      }
+    },
+    "4P+4D": {
+      "name": "4P+4D",
+      "n_offered": 3552,
+      "n_success": 771,
+      "completion_rate": 0.2170608108108108,
+      "offered_window_s": 299.74141899999995,
+      "offered_qps": 11.850214134070008,
+      "wall_clock_s": 850.1328976760124,
+      "amplification": 2.8362209684341706,
+      "ttft": {
+        "count": 771,
+        "mean": 63.210636654403324,
+        "p50": 1.1097561890055658,
+        "p90": 179.64081536799495,
+        "p99": 186.67013871119852
+      },
+      "tpot": {
+        "count": 771,
+        "mean": 0.05484690890217986,
+        "p50": 0.04795774376874187,
+        "p90": 0.08067800507863548,
+        "p99": 0.09099416300415113
+      },
+      "e2e": {
+        "count": 771,
+        "mean": 287.81779562259663,
+        "p50": 312.2160719559906,
+        "p90": 372.0004520520015,
+        "p99": 379.3279856524052
+      }
+    },
+    "6P+2D": {
+      "name": "6P+2D",
+      "n_offered": 3552,
+      "n_success": 627,
+      "completion_rate": 0.17652027027027026,
+      "offered_window_s": 299.74141899999995,
+      "offered_qps": 11.850214134070008,
+      "wall_clock_s": 867.8833550199925,
+      "amplification": 2.8954402028102515,
+      "ttft": {
+        "count": 627,
+        "mean": 179.58769048342904,
+        "p50": 238.1998468660022,
+        "p90": 378.29023678940143,
+        "p99": 385.40577973942356
+      },
+      "tpot": {
+        "count": 627,
+        "mean": 0.04188420961205498,
+        "p50": 0.03654626756630041,
+        "p90": 0.06031132874202571,
+        "p99": 0.06738955674930582
+      },
+      "e2e": {
+        "count": 627,
+        "mean": 351.1066520824709,
+        "p50": 387.2650127700035,
+        "p90": 507.9008203571953,
+        "p99": 570.6463984230224
+      }
+    }
+  },
+  "slo_grid": [
+    {
+      "ttft_slo_s": 2.0,
+      "tpot_slo_s": 0.05,
+      "arms": {
+        "8C-proxy": {
+          "attainment": 0.0,
+          "pd_advantage": NaN,
+          "n_slo": 0
+        },
+        "4P+4D": {
+          "attainment": 0.05855855855855856,
+          "pd_advantage": NaN,
+          "n_slo": 208
+        },
+        "6P+2D": {
+          "attainment": 0.036036036036036036,
+          "pd_advantage": NaN,
+          "n_slo": 128
+        }
+      }
+    }
+  ]
+}

analysis/crossover/d2_o512_goodput.json

@@ -0,0 +1,121 @@
+{
+  "baseline": "8C-proxy",
+  "arms": {
+    "8C-proxy": {
+      "name": "8C-proxy",
+      "n_offered": 3552,
+      "n_success": 3552,
+      "completion_rate": 1.0,
+      "offered_window_s": 299.74141899999995,
+      "offered_qps": 11.850214134070008,
+      "wall_clock_s": 304.97662343096454,
+      "amplification": 1.0174657357946404,
+      "ttft": {
+        "count": 3552,
+        "mean": 0.16242270423816507,
+        "p50": 0.15836620499612764,
+        "p90": 0.17363918052287775,
+        "p99": 0.24847999344696287
+      },
+      "tpot": {
+        "count": 3552,
+        "mean": 0.013691483182659388,
+        "p50": 0.013865661168213087,
+        "p90": 0.015242900696529327,
+        "p99": 0.01626683903372128
+      },
+      "e2e": {
+        "count": 3552,
+        "mean": 7.160272567887971,
+        "p50": 7.247483663493767,
+        "p90": 7.962273431208451,
+        "p99": 8.480892732607899
+      }
+    },
+    "4P+4D": {
+      "name": "4P+4D",
+      "n_offered": 3552,
+      "n_success": 3551,
+      "completion_rate": 0.9997184684684685,
+      "offered_window_s": 299.74141899999995,
+      "offered_qps": 11.850214134070008,
+      "wall_clock_s": 744.9637431279989,
+      "amplification": 2.4853546954349977,
+      "ttft": {
+        "count": 3551,
+        "mean": 0.2446379999283825,
+        "p50": 0.24067969399038702,
+        "p90": 0.2630795220611617,
+        "p99": 0.3426029055262916
+      },
+      "tpot": {
+        "count": 3551,
+        "mean": 0.016916919575073883,
+        "p50": 0.016998299133030737,
+        "p90": 0.01775875886689886,
+        "p99": 0.018166751548973206
+      },
+      "e2e": {
+        "count": 3551,
+        "mean": 8.89104466149889,
+        "p50": 8.933106195996515,
+        "p90": 9.333998591057025,
+        "p99": 9.536390463996213
+      }
+    },
+    "6P+2D": {
+      "name": "6P+2D",
+      "n_offered": 3552,
+      "n_success": 3551,
+      "completion_rate": 0.9997184684684685,
+      "offered_window_s": 299.74141899999995,
+      "offered_qps": 11.850214134070008,
+      "wall_clock_s": 312.45468625507783,
+      "amplification": 1.0424141157985172,
+      "ttft": {
+        "count": 3551,
+        "mean": 0.36231219612768273,
+        "p50": 0.3462264990666881,
+        "p90": 0.4059687410481274,
+        "p99": 0.9837204645154998
+      },
+      "tpot": {
+        "count": 3551,
+        "mean": 0.03268022218101953,
+        "p50": 0.03333031399418835,
+        "p90": 0.03557429400772957,
+        "p99": 0.038558618127279086
+      },
+      "e2e": {
+        "count": 3551,
+        "mean": 17.068018403084057,
+        "p50": 17.392655145958997,
+        "p90": 18.56302695896011,
+        "p99": 20.159411454980727
+      }
+    }
+  },
+  "slo_grid": [
+    {
+      "ttft_slo_s": 2.0,
+      "tpot_slo_s": 0.05,
+      "arms": {
+        "8C-proxy": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 3552
+        },
+        "4P+4D": {
+          "attainment": 0.9997184684684685,
+          "pd_advantage": 0.9997184684684685,
+          "n_slo": 3551
+        },
+        "6P+2D": {
+          "attainment": 0.9997184684684685,
+          "pd_advantage": 0.9997184684684685,
+          "n_slo": 3551
+        }
+      }
+    }
+  ]
+}

analysis/crossover/d2_o64_goodput.json

@@ -0,0 +1,121 @@
+{
+  "baseline": "8C-proxy",
+  "arms": {
+    "8C-proxy": {
+      "name": "8C-proxy",
+      "n_offered": 3552,
+      "n_success": 3552,
+      "completion_rate": 1.0,
+      "offered_window_s": 299.74141899999995,
+      "offered_qps": 11.850214134070008,
+      "wall_clock_s": 300.21845804993063,
+      "amplification": 1.001591501940313,
+      "ttft": {
+        "count": 3552,
+        "mean": 0.1369056966029782,
+        "p50": 0.13516780693316832,
+        "p90": 0.13835511771030723,
+        "p99": 0.15317223175661632
+      },
+      "tpot": {
+        "count": 3552,
+        "mean": 0.005430781936309906,
+        "p50": 0.005039893150780468,
+        "p90": 0.007063257358303028,
+        "p99": 0.0077793481296283135
+      },
+      "e2e": {
+        "count": 3552,
+        "mean": 0.4793568905444592,
+        "p50": 0.45412574551301077,
+        "p90": 0.5810393166844734,
+        "p99": 0.6301419002050533
+      }
+    },
+    "4P+4D": {
+      "name": "4P+4D",
+      "n_offered": 3552,
+      "n_success": 3551,
+      "completion_rate": 0.9997184684684685,
+      "offered_window_s": 299.74141899999995,
+      "offered_qps": 11.850214134070008,
+      "wall_clock_s": 735.1059510430787,
+      "amplification": 2.452467041410379,
+      "ttft": {
+        "count": 3551,
+        "mean": 0.17374673821534295,
+        "p50": 0.17102365801110864,
+        "p90": 0.18273873499128968,
+        "p99": 0.24688138795318082
+      },
+      "tpot": {
+        "count": 3551,
+        "mean": 0.005485491774206834,
+        "p50": 0.0053864502226046865,
+        "p90": 0.006091995366168992,
+        "p99": 0.007109403222178419
+      },
+      "e2e": {
+        "count": 3551,
+        "mean": 0.5196563635758822,
+        "p50": 0.5100997349945828,
+        "p90": 0.5655082209268585,
+        "p99": 0.6639982180204242
+      }
+    },
+    "6P+2D": {
+      "name": "6P+2D",
+      "n_offered": 3552,
+      "n_success": 3552,
+      "completion_rate": 1.0,
+      "offered_window_s": 299.74141899999995,
+      "offered_qps": 11.850214134070008,
+      "wall_clock_s": 300.38101644301787,
+      "amplification": 1.0021338307036503,
+      "ttft": {
+        "count": 3552,
+        "mean": 0.18293367427152893,
+        "p50": 0.1822461549891159,
+        "p90": 0.1938482352765277,
+        "p99": 0.21272844232735222
+      },
+      "tpot": {
+        "count": 3552,
+        "mean": 0.007192309629699456,
+        "p50": 0.007143509595484902,
+        "p90": 0.008732455453672816,
+        "p99": 0.009842920153335268
+      },
+      "e2e": {
+        "count": 3552,
+        "mean": 0.636424056327808,
+        "p50": 0.6324848984950222,
+        "p90": 0.7393875011475757,
+        "p99": 0.8261980937235056
+      }
+    }
+  },
+  "slo_grid": [
+    {
+      "ttft_slo_s": 2.0,
+      "tpot_slo_s": 0.05,
+      "arms": {
+        "8C-proxy": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 3552
+        },
+        "4P+4D": {
+          "attainment": 0.9997184684684685,
+          "pd_advantage": 0.9997184684684685,
+          "n_slo": 3551
+        },
+        "6P+2D": {
+          "attainment": 1.0,
+          "pd_advantage": 1.0,
+          "n_slo": 3552
+        }
+      }
+    }
+  ]
+}

analysis/lpwl_5policy_600s.md

@@ -0,0 +1,131 @@
+# LPWL vs 4 baselines — parameter-free routing for agentic workloads
+
+Date: 2026-05-29. Hardware: dash1, 8×H20, Qwen3-Coder-30B-A3B, TP=1,
+max_model_len=200000, fresh vLLM per arm (cold APC), `--policy` via
+`scripts/b3_isolated_policy.sh`. Analyzer: `scripts/bench_report.py`.
+
+## Motivation
+
+unified+A+B carries too many knobs (`overload_factor`, `lmetric_decode_weight`,
+the 0.5 `cache_ratio` gate). Goal: a policy derived from the agentic *pattern*
+with no tuned constants, that does not overfit.
+
+## LPWL (Least-Prefill-Work-Left)
+
+`scripts/cache_aware_proxy.py:pick_instance_leastwork`, `--policy leastwork`:
+
+```
+score_i = pending_prefill_tokens_i + max(0, input_len − cache_hit_i)   → argmin
+```
+
+Tie-break: fewest `num_requests`, then round-robin. **Zero hyperparameters.**
+
+Why this shape (straight from the workload):
+- Decode is cheap (I/O ~217×) ⇒ the only load worth modeling is outstanding
+  *prefill* token-work. No decode weight; dropping LMetric's `×num_requests`
+  also makes an idle-but-decoding host score `input` (its true marginal cost),
+  not 0 — fixing the empty-batch degeneracy for free.
+- Cache-awareness *is* the affinity mechanism: a returning session's owner has
+  `new_uncached ≈ 0`, so it sticks unless its prefill backlog exceeds the cache
+  saving (`input`). The stick-vs-spill crossover is computed from real
+  token-work — no `overload_factor`, no `cache_ratio` gate.
+- Session skew degrades gracefully: a heavy session inflates its owner's
+  `pending_prefill`, auto-diverting *other* sessions while the heavy one stays
+  put (no cold re-prefill).
+
+## Results — 600s trace (`w600_r0.0015_st30_first600s.jsonl`, 807 reqs)
+
+This is the colder regime (theoretical APC ceiling ≈ 70% vs 80% for full w600).
+
+| policy | knobs | TTFT mean | TTFT p90 | E2E mean | E2E p90 | E2E p99 | TPOT p90 | APC | req-bal |
+|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|
+| **LPWL** | **0** | **3398** | **7983** | **8116** | **19014** | 87024 | 26 | 0.648 | **1.55×** |
+| unified+A+B | 3 | 3876 | 11562 | 8199 | 22569 | **74266** | **25** | 0.661 | 1.56× |
+| unified default | 2 | 5066 | 16389 | 10481 | 28427 | 96361 | 34 | 0.689 | 2.28× |
+| LMetric | 0 | 4809 | 14037 | 10051 | 26726 | 97442 | 32 | 0.507 | 2.11× |
+| sticky | 0 | 5758 | 20356 | 10815 | 34734 | 82732 | 28 | **0.696** | 3.86× |
+
+(latencies ms; req-bal = max:min per-worker request count; this batch predates
+the GPU-capture harness change so per-worker GPU util reads N/A.)
+
+### Findings
+
+1. **LPWL is overall best with zero knobs:** TTFT mean −12% / p90 −31%, E2E
+   mean ~tie / p90 −16% vs the tuned unified+A+B; best request balance; TPOT
+   tied-best. Only loss is E2E p99 (+17%) from heavy-class decode concentration.
+2. **The baselines bracket the problem and explain why LPWL works:** sticky has
+   the highest APC (0.696) but worst latency (hot-pin, 3.86× imbalance); LMetric
+   has the worst APC (0.507) because `×num_requests` swallows the cache signal.
+   LPWL drops exactly that factor, so locality re-emerges (APC 0.648, beside the
+   explicit-affinity policies) while balance stays tight — the sweet spot, no
+   gate, no tuning.
+3. **Anti-overfit, demonstrated:** unified+A+B was tuned (of=1.3, lmw=0.01) on
+   the *full* w600; on the colder 600s regime the parameter-free policy beats it
+   by 31% TTFT p90. The tuning did not transfer; LPWL did.
+
+### Per-class TTFT (ms, mean / p50 / p90 / p99) — LPWL dominates except the floor
+
+| class | LPWL p90 | A+B p90 | LPWL p99 | A+B p99 |
+|---|---:|---:|---:|---:|
+| WARM<5k | 319 | 324 | 1032 | 2092 |
+| MED5-20k | 1618 | 1952 | 3013 | 33189 |
+| HEAVY20-50k | 4851 | 6198 | 14599 | 29044 |
+| HEAVY+>50k | 28942 | 33777 | 52651 | 50778 |
+
+LPWL's only weak class is the workload-inherent HEAVY+>50k floor (≈tied across
+all policies). Elsewhere it avoids the mid-class tails the unified gate creates
+when it pins a mid request behind a 50k-token turn on a barely-warm owner.
+
+## Full-w600 cross-check (1214 reqs, `outputs/lpwl_vs_ab_live/`)
+
+On the warmer full trace, LPWL vs unified+A+B is a wash: LPWL wins TTFT p90
+(−14%) but loses TPOT (+38%) and per-worker balance. Combined claim across both
+regimes: **LPWL ∈ [tied, clearly-better] vs a tuned baseline, at zero knobs.**
+
+## Ablation: derived-κ decode term (`leastwork_kappa`) — NET-NEGATIVE
+
+Tested the proposed knob-free fix for LPWL's E2E-p99: `--policy leastwork_kappa`,
+`score = (pending_prefill + new_uncached) × (1 + κ·ongoing_decode_tokens)`, with
+κ = 2.5e-6 *derived* from hardware (KV ~100 KB/tok ÷ HBM 4 TB/s ÷ TPOT 10 ms on
+H20+Qwen3-30B-A3B), not trace-tuned. Same 600s trace, fresh vLLM, cold APC.
+
+| metric | leastwork | leastwork_kappa | Δ |
+|---|---:|---:|---:|
+| TTFT p90 | 7983 | 9390 | +18% (worse) |
+| TTFT p99 | 44891 | 42370 | −6% |
+| E2E p90 | 19014 | 21674 | +14% (worse) |
+| E2E p99 | 87024 | 90155 | +4% (did NOT fix) |
+| APC | 0.648 | 0.647 | tie |
+| req-balance | 1.55× | 1.97× | worse |
+
+**Verdict: decode-awareness is the wrong lever for agentic.** The κ term is
+correct physics aimed at a negligible effect (decode is cheap, output p50≈80),
+so it mostly bounces heavy requests off their cache-owner → cold re-prefill
+elsewhere → new hotspots (balance degrades 1.55×→1.97×). It does NOT fix E2E-p99
+because that tail is the **structural HEAVY+>50k floor** (per-class p99 ≈51–52k
+for *all* policies), not decode interference — i.e. not routing-fixable. This is
+a negative result that *justifies* LPWL's omission of any decode term. The policy
+is kept in-tree as a documented ablation; do not revive without a decode-heavy
+regime. (First run on the GPU-capturing harness: per-worker GPU util mean 42–83%,
+1.95× spread — it even shows the κ-induced imbalance.)
+
+## Caveats / open work
+
+- n=1 per arm. The 600s −31% TTFT p90 is corroborated by mean/p50/per-class, but
+  repeat to bound run-to-run noise (no 3× repeats yet, by request — quick single
+  set first).
+- E2E-p99 deep tail is the one consistent LPWL weak spot (heavy-session decode
+  concentration). Proposed knob-free fix: add `+ κ·ongoing_decode` with
+  `κ = measured(TPOT/token) / prefill_throughput` (a derived hardware ratio, not
+  a tuned scalar). Not yet implemented.
+
+## Repro
+
+```bash
+# 5-policy, 600s trace (≈18 min/arm, ~90 min total)
+OUTROOT=.../outputs/policy5_600s \
+    bash microbench/connector_tax/cache_sweep/run_5policy_600s.sh
+# unified report
+.venv/bin/python scripts/bench_report.py --root .../outputs/policy5_600s \
+    leastwork unified_ab unified_def lmetric sticky
+```

analysis/mb1/README.md

@@ -0,0 +1,193 @@
+# MB1 — Prefill–Decode Interference (chunked-prefill on, vLLM 0.18.1 default)
+
+Persistent record of the phase-interference microbench used to put a
+quantitative upper bound on **what PD-disaggregation can buy** under the
+chunked-prefill-on baseline. Re-runs append a dated section at the
+bottom; the **Summary** block is what gets cited.
+
+---
+
+## Summary (latest)
+
+| Headline | Value |
+|---|---|
+| Baseline single-stream TPOT (D=1, idle GPU) | **4.8 ms** |
+| Effective per-stream TPOT during **8k-token** prefill burst (D=8) | **114 ms (≈15× baseline)** |
+| Effective per-stream TPOT during **32k-token** prefill burst (D=8) | **388 ms (≈52×)** |
+| Effective per-stream TPOT during **131k-token** prefill burst (D=8) | **1419 ms (≈183×)** |
+
+**What MB1 actually measures**:
+
+> During a prefill burst, every ongoing decode stream is essentially
+> halted (per-stream effective TPOT is 15×–2000× baseline, scaling with
+> prefill size). The **total decode time lost per prefill event is
+> `D × T_prefill`** (D concurrent decodes each lose ~T_prefill of useful
+> work). For the trace mean (P ≈ 33k tokens, T_prefill ≈ 4.5 s) at D=8
+> that's **~36 seconds of decode-equivalent work lost per request**.
+> This is the **upper bound on what PD-disaggregation's phase isolation
+> could recover** on the decode side.
+
+**⚠ Correction (2026-05-27)**: an earlier version of this README framed
+the §3.2 PD-disagg argument as "phase-isolation benefit is capped at
+the decode duration of the new request (~50–200 ms), so MB2 transfer
+cost dominates". That framing was wrong. The correct accounting is
+benefit-per-prefill-event = D × T_prefill (aggregate decode time saved
+across all stalled streams), which is **much larger than per-request
+transfer cost**. The actual reason static PD-disagg fails in agentic
+is **D-side KV pool capacity** (`figs/f4b_pdsep_kv_wall.png`), not a
+cost-vs-benefit imbalance on phase isolation. See `RESULTS_SUMMARY.md`
+section 4 for the corrected framing.
+
+## Setup
+
+| Component | Value |
+|---|---|
+| Host | dash1, H20 96 GiB, driver 570.133.20 |
+| Venv | `/home/admin/cpfs/wjh/agentic-kv-fresh/.venv` |
+| vLLM | 0.18.1 official wheel (chunked-prefill default-on, V1 engine) |
+| Model | `/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct` |
+| Launch flags | `--tensor-parallel-size 1 --enable-prefix-caching --gpu-memory-utilization 0.9 --max-model-len 200000 --max-num-batched-tokens 8192` |
+| kv_connector | **none** (this measures pure single-GPU phase interference; PD-disagg cost lives in MB2) |
+
+## Method
+
+Adapted from `microbench/interference/driver.py`:
+
+1. Start D streaming decode requests on `/v1/chat/completions` with a
+   long max_tokens cap. Discard the first 32 tokens as warmup.
+2. After 1 s, inject one prefill-only request with `max_tokens=1` and
+   an input of `P` synthetic tokens (uuid-seeded for zero prefix-cache
+   reuse). Measure the prefill's TTFT.
+3. Bin the *during-prefill* tokens from each decode stream by whether
+   their wall-clock falls inside `[prefill_inject_ts, prefill_inject_ts +
+   prefill_ttft]`. Report inter-token p50 / p90.
+4. Bin a baseline run (D streams, no prefill injection) the same way.
+
+We additionally compute the **effective per-stream TPOT during the
+prefill burst** as the single most informative summary:
+
+```
+eff_TPOT_during = prefill_ttft_ms / (num_tokens_during_prefill / D)
+```
+
+This is the average rate at which each decode stream produces tokens
+while a prefill is in flight. Compared to baseline TPOT it gives the
+real per-stream throughput penalty (chunked-prefill p50 looks deceptively
+fine because most decode-token intervals during the burst are at normal
+speed; p90 sees the stall but is itself noisy; the effective TPOT is
+the cleanest "average over the whole burst window" number).
+
+## Results — 2026-05-27, dash1 GPU 0, chunk_tokens=8192
+
+3 D × 5 P × 3 reps. Aggregated by `analyze_mb1.py`.
+
+| D | P (tok) | base TPOT (ms) | prefill_ttft (ms) | per-stream tokens during | effective TPOT during (ms) | penalty | max PD-disagg benefit per stream (ms) |
+|--:|--:|--:|--:|--:|--:|--:|--:|
+| 1 |   2 048 | 4.79 |   163 | 4.0 |     41 |     8× |    144 |
+| 1 |   8 192 | 4.78 |   584 | 5.0 |    117 |    24× |    560 |
+| 1 |  32 768 | 4.78 | 4 515 | 5.0 |    903 |   189× |  4 491 |
+| 1 |  65 536 | 4.78 | 15 568 | 5.3 |  2 919 |   610× | 15 542 |
+| 1 | 131 072 | 4.78 | 56 765 | 5.7 | 10 017 | 2 094× | 56 738 |
+| 4 |   2 048 | 5.62 |   138 | 3.9 |     36 |     6× |    117 |
+| 4 |   8 192 | 6.08 |   574 | 4.5 |    128 |    21× |    547 |
+| 4 |  32 768 | 6.09 | 4 529 | 11.9 |    381 |    63× |  4 457 |
+| 4 |  65 536 | 5.85 | 15 587 | 19.8 |    789 |   135× | 15 471 |
+| 4 | 131 072 | 6.27 | 56 697 | 37.4 |  1 517 |   242× | 56 463 |
+| 8 |   2 048 | 7.71 |   143 | 4.5 |     32 |     4× |    109 |
+| 8 |   8 192 | 7.69 |   583 | 5.1 |    114 |    15× |    544 |
+| 8 |  32 768 | 7.42 | 4 520 | 11.7 |    387 |    52× |  4 434 |
+| 8 |  65 536 | 7.67 | 15 615 | 20.6 |    757 |    99× | 15 457 |
+| 8 | 131 072 | 7.74 | 56 991 | 40.2 |  1 419 |   183× | 56 680 |
+
+**Reading the table**:
+
+- *Baseline TPOT* grows mildly with D (4.8 ms → 7.7 ms as D goes 1 → 8).
+  Multi-stream decoding has small but nonzero contention even without
+  prefill.
+- *Effective TPOT during* grows mostly with P: a single 8k prefill stalls
+  decode for ~580 ms regardless of D, so each stream emits only a handful
+  of tokens during that 580 ms window — effective per-stream TPOT
+  collapses to 100–130 ms. Larger prefill = more chunks = larger stall.
+- *Penalty* is the eff/baseline ratio. Above 50× for P ≥ 32k. Above
+  500× for D=1 at P ≥ 65k.
+- *Max PD-disagg benefit per stream* = `prefill_ttft − per_stream_tokens
+  × baseline_TPOT` ≈ `prefill_ttft` (since interference essentially
+  halts decode). This is the entire prefill duration's worth of decode
+  time that could in principle be recovered.
+
+**Connecting to the §3.2 PD-disagg argument** (corrected):
+
+PD-disagg's promised phase-isolation benefit is **per prefill event**,
+not per request. When a new prefill arrives, it stalls every concurrent
+decode stream on the same GPU. The aggregate decode time lost across
+those D streams is `D × T_prefill`. PD-disagg moving prefill off-decode-GPU
+recovers all of it.
+
+Plugging numbers per prefill event:
+
+| Prefill size | T_prefill | PD-disagg cost (MB2 T_transfer) | PD-disagg benefit (D=8 × T_prefill) | Ratio |
+|---:|---:|---:|---:|---:|
+| 2k tok (trace lower) | 0.14 s | 8 ms | 1.1 s | 0.7 % |
+| 33k tok (trace mean) | 4.5 s | 320 ms | 36 s | 0.9 % |
+| 125k tok (~p99) | 57 s | 1.9 s | 456 s | 0.4 % |
+
+On the **phase-isolation axis alone**, PD-disagg wins by 100×–250×.
+The reason static PD-disagg nonetheless **fails in agentic** is a
+*different* failure mode: the D-side KV pool cannot fit p90+ requests
+(p99 = 11.5 GiB; D-instance pool ≈ 38 GiB; 4P+4D halves system-wide
+decode capacity → TTFT p50 62×, success rate 99.5% → 52% in colleague's
+4P+4D experiment). The structural problem is **capacity** (see
+`figs/f4b_pdsep_kv_wall.png`), not transfer-cost vs phase-isolation
+trade-off.
+
+## Reproduction
+
+```bash
+# vllm pair-free single-instance launch
+ssh dash1 'GPU=0 PORT=8000 CHUNK_TOKENS=8192 \
+  bash /home/admin/cpfs/wjh/agentic-kv-fresh/scripts/mb1_launch.sh start'
+
+# sweep
+ssh dash1 'source /home/admin/cpfs/wjh/agentic-kv-fresh/.venv/bin/activate && \
+  python /home/admin/cpfs/wjh/agentic-kv-fresh/scripts/mb1_driver.py \
+    --host 127.0.0.1 --port 8000 \
+    --model /home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct \
+    --decode-batch-sizes 1,4,8 --prefill-tokens 2048,8192,32768,65536,131072 \
+    --reps 3 --output-dir /home/admin/cpfs/wjh/agentic-kv-fresh/mb1_results'
+
+# pull + analyze
+scp dash1:/home/admin/cpfs/wjh/agentic-kv-fresh/mb1_results/chunk8192/summary.csv \
+    analysis/mb1/summary.csv
+.venv/bin/python microbench/fresh_setup/analyze_mb1.py \
+  --summary analysis/mb1/summary.csv --out analysis/mb1/breakdown.json
+.venv/bin/python microbench/fresh_setup/plot_mb1.py \
+  --mb1 analysis/mb1/breakdown.json \
+  --mb2-intra analysis/mb2/intra_kvboth_breakdown.json \
+  --mb2-inter analysis/mb2/inter_kvboth_breakdown.json
+
+# teardown
+ssh dash1 'bash /home/admin/cpfs/wjh/agentic-kv-fresh/scripts/mb1_launch.sh stop'
+```
+
+## Open questions / next runs
+
+- **Chunk size sensitivity**: this run uses `--max-num-batched-tokens
+  8192`. Sarathi-Serve goes smaller (e.g. 1024) and recovers more
+  decode interleaving inside each prefill burst. Worth running
+  chunk_tokens ∈ {1024, 2048, 4096, 16384} to map the chunk-size axis.
+- **Higher D**: 12, 16 streams to see whether the penalty saturates or
+  keeps shrinking per-stream.
+- **Cross-validate effective_TPOT_during with token-time-series plot**:
+  raw per-token timestamps could reveal whether the stall is a few big
+  spikes or many small ones (currently inferred from p50/p90 spread).
+
+## Run log
+
+### 2026-05-27 — dash1 GPU 0, chunk_tokens=8192
+
+3 × 5 × 3 sweep. CSV: `analysis/mb1/summary.csv`. Per-config JSONs on
+dash1 at `/home/admin/cpfs/wjh/agentic-kv-fresh/mb1_results/chunk8192/`.
+Figure: `figs/mb1_interference.png`. The figure
+`figs/pd_cost_vs_benefit.png` from the original commit `029821c` was
+based on the wrong "benefit ≤ decode duration" accounting; **deleted in
+the correction commit**.

analysis/mb1/breakdown.json

@@ -0,0 +1,199 @@
+{
+  "summary": [
+    {
+      "decode_batch_size": 1,
+      "new_prefill_tokens": 2048,
+      "baseline_tpot_ms": 4.79,
+      "during_tpot_p50_ms_raw": 35.43,
+      "during_tpot_p90_ms_raw": 79.91,
+      "prefill_ttft_ms": 163.3,
+      "num_tokens_during_prefill_total": 4.0,
+      "per_stream_tokens_during": 4.0,
+      "effective_tpot_during_ms": 40.8,
+      "interference_penalty_x": 8.5,
+      "max_pd_disagg_benefit_ms_per_stream": 144.2
+    },
+    {
+      "decode_batch_size": 1,
+      "new_prefill_tokens": 8192,
+      "baseline_tpot_ms": 4.78,
+      "during_tpot_p50_ms_raw": 6.56,
+      "during_tpot_p90_ms_raw": 328.57,
+      "prefill_ttft_ms": 583.9,
+      "num_tokens_during_prefill_total": 5.0,
+      "per_stream_tokens_during": 5.0,
+      "effective_tpot_during_ms": 116.8,
+      "interference_penalty_x": 24.4,
+      "max_pd_disagg_benefit_ms_per_stream": 560.0
+    },
+    {
+      "decode_batch_size": 1,
+      "new_prefill_tokens": 32768,
+      "baseline_tpot_ms": 4.78,
+      "during_tpot_p50_ms_raw": 4.75,
+      "during_tpot_p90_ms_raw": 4.9,
+      "prefill_ttft_ms": 4515.3,
+      "num_tokens_during_prefill_total": 5.0,
+      "per_stream_tokens_during": 5.0,
+      "effective_tpot_during_ms": 903.1,
+      "interference_penalty_x": 188.8,
+      "max_pd_disagg_benefit_ms_per_stream": 4491.4
+    },
+    {
+      "decode_batch_size": 1,
+      "new_prefill_tokens": 65536,
+      "baseline_tpot_ms": 4.78,
+      "during_tpot_p50_ms_raw": 4.69,
+      "during_tpot_p90_ms_raw": 4.97,
+      "prefill_ttft_ms": 15567.6,
+      "num_tokens_during_prefill_total": 5.3,
+      "per_stream_tokens_during": 5.33,
+      "effective_tpot_during_ms": 2918.9,
+      "interference_penalty_x": 610.2,
+      "max_pd_disagg_benefit_ms_per_stream": 15542.0
+    },
+    {
+      "decode_batch_size": 1,
+      "new_prefill_tokens": 131072,
+      "baseline_tpot_ms": 4.78,
+      "during_tpot_p50_ms_raw": 4.71,
+      "during_tpot_p90_ms_raw": 4.9,
+      "prefill_ttft_ms": 56765.2,
+      "num_tokens_during_prefill_total": 5.7,
+      "per_stream_tokens_during": 5.67,
+      "effective_tpot_during_ms": 10017.4,
+      "interference_penalty_x": 2094.5,
+      "max_pd_disagg_benefit_ms_per_stream": 56738.1
+    },
+    {
+      "decode_batch_size": 4,
+      "new_prefill_tokens": 2048,
+      "baseline_tpot_ms": 5.62,
+      "during_tpot_p50_ms_raw": 22.18,
+      "during_tpot_p90_ms_raw": 84.85,
+      "prefill_ttft_ms": 138.3,
+      "num_tokens_during_prefill_total": 15.5,
+      "per_stream_tokens_during": 3.88,
+      "effective_tpot_during_ms": 35.7,
+      "interference_penalty_x": 6.3,
+      "max_pd_disagg_benefit_ms_per_stream": 116.6
+    },
+    {
+      "decode_batch_size": 4,
+      "new_prefill_tokens": 8192,
+      "baseline_tpot_ms": 6.08,
+      "during_tpot_p50_ms_raw": 8.45,
+      "during_tpot_p90_ms_raw": 515.39,
+      "prefill_ttft_ms": 574.1,
+      "num_tokens_during_prefill_total": 18.0,
+      "per_stream_tokens_during": 4.5,
+      "effective_tpot_during_ms": 127.6,
+      "interference_penalty_x": 21.0,
+      "max_pd_disagg_benefit_ms_per_stream": 546.8
+    },
+    {
+      "decode_batch_size": 4,
+      "new_prefill_tokens": 32768,
+      "baseline_tpot_ms": 6.09,
+      "during_tpot_p50_ms_raw": 9.83,
+      "during_tpot_p90_ms_raw": 1314.87,
+      "prefill_ttft_ms": 4529.1,
+      "num_tokens_during_prefill_total": 47.5,
+      "per_stream_tokens_during": 11.88,
+      "effective_tpot_during_ms": 381.4,
+      "interference_penalty_x": 62.7,
+      "max_pd_disagg_benefit_ms_per_stream": 4456.9
+    },
+    {
+      "decode_batch_size": 4,
+      "new_prefill_tokens": 65536,
+      "baseline_tpot_ms": 5.85,
+      "during_tpot_p50_ms_raw": 6.41,
+      "during_tpot_p90_ms_raw": 2077.47,
+      "prefill_ttft_ms": 15586.5,
+      "num_tokens_during_prefill_total": 79.0,
+      "per_stream_tokens_during": 19.75,
+      "effective_tpot_during_ms": 789.2,
+      "interference_penalty_x": 135.0,
+      "max_pd_disagg_benefit_ms_per_stream": 15471.0
+    },
+    {
+      "decode_batch_size": 4,
+      "new_prefill_tokens": 131072,
+      "baseline_tpot_ms": 6.27,
+      "during_tpot_p50_ms_raw": 6.3,
+      "during_tpot_p90_ms_raw": 4405.18,
+      "prefill_ttft_ms": 56697.1,
+      "num_tokens_during_prefill_total": 149.5,
+      "per_stream_tokens_during": 37.38,
+      "effective_tpot_during_ms": 1517.0,
+      "interference_penalty_x": 241.8,
+      "max_pd_disagg_benefit_ms_per_stream": 56462.6
+    },
+    {
+      "decode_batch_size": 8,
+      "new_prefill_tokens": 2048,
+      "baseline_tpot_ms": 7.71,
+      "during_tpot_p50_ms_raw": 8.38,
+      "during_tpot_p90_ms_raw": 98.98,
+      "prefill_ttft_ms": 143.1,
+      "num_tokens_during_prefill_total": 35.7,
+      "per_stream_tokens_during": 4.46,
+      "effective_tpot_during_ms": 32.1,
+      "interference_penalty_x": 4.2,
+      "max_pd_disagg_benefit_ms_per_stream": 108.8
+    },
+    {
+      "decode_batch_size": 8,
+      "new_prefill_tokens": 8192,
+      "baseline_tpot_ms": 7.69,
+      "during_tpot_p50_ms_raw": 9.34,
+      "during_tpot_p90_ms_raw": 519.29,
+      "prefill_ttft_ms": 583.3,
+      "num_tokens_during_prefill_total": 41.0,
+      "per_stream_tokens_during": 5.12,
+      "effective_tpot_during_ms": 113.8,
+      "interference_penalty_x": 14.8,
+      "max_pd_disagg_benefit_ms_per_stream": 543.9
+    },
+    {
+      "decode_batch_size": 8,
+      "new_prefill_tokens": 32768,
+      "baseline_tpot_ms": 7.42,
+      "during_tpot_p50_ms_raw": 11.61,
+      "during_tpot_p90_ms_raw": 1315.48,
+      "prefill_ttft_ms": 4520.3,
+      "num_tokens_during_prefill_total": 93.3,
+      "per_stream_tokens_during": 11.67,
+      "effective_tpot_during_ms": 387.5,
+      "interference_penalty_x": 52.2,
+      "max_pd_disagg_benefit_ms_per_stream": 4433.7
+    },
+    {
+      "decode_batch_size": 8,
+      "new_prefill_tokens": 65536,
+      "baseline_tpot_ms": 7.67,
+      "during_tpot_p50_ms_raw": 19.09,
+      "during_tpot_p90_ms_raw": 2471.4,
+      "prefill_ttft_ms": 15615.5,
+      "num_tokens_during_prefill_total": 165.0,
+      "per_stream_tokens_during": 20.62,
+      "effective_tpot_during_ms": 757.1,
+      "interference_penalty_x": 98.8,
+      "max_pd_disagg_benefit_ms_per_stream": 15457.4
+    },
+    {
+      "decode_batch_size": 8,
+      "new_prefill_tokens": 131072,
+      "baseline_tpot_ms": 7.74,
+      "during_tpot_p50_ms_raw": 11.51,
+      "during_tpot_p90_ms_raw": 4895.27,
+      "prefill_ttft_ms": 56991.4,
+      "num_tokens_during_prefill_total": 321.3,
+      "per_stream_tokens_during": 40.17,
+      "effective_tpot_during_ms": 1418.9,
+      "interference_penalty_x": 183.3,
+      "max_pd_disagg_benefit_ms_per_stream": 56680.4
+    }
+  ]
+}

analysis/mb1/summary.csv

@@ -0,0 +1,46 @@
+chunk_size,decode_batch_size,new_prefill_tokens,repetition,tpot_baseline_p50_ms,tpot_baseline_p90_ms,tpot_during_prefill_p50_ms,tpot_during_prefill_p90_ms,tpot_after_prefill_p50_ms,prefill_ttft_ms,num_tokens_during_prefill,tpot_penalty_p50_ms,tpot_penalty_ratio
+8192,1,131072,0,4.777565016411245,4.900234832894057,4.701301048044115,4.948397364933044,0.0,56719.25117995124,7,-0.07626396836712956,0.9840370632099913
+8192,1,131072,1,4.779465030878782,4.883405601140112,4.707481013610959,4.85471700085327,0.0,56696.089847013354,5,-0.07198401726782322,0.9849388965495606
+8192,1,131072,2,4.790953011251986,4.880544205661863,4.728371975943446,4.907831805758178,0.0,56880.19039196661,5,-0.06258103530853987,0.9869376645603573
+8192,1,2048,0,4.77885699365288,4.894876398611814,41.434570477576926,88.97331730695441,0.0,183.2046649651602,4,36.655713483924046,8.670393471202205
+8192,1,2048,1,4.788161953911185,4.949774022679776,41.68213551747613,83.5143867880106,0.0,175.55483896285295,4,36.89397356356494,8.705247633369687
+8192,1,2048,2,4.7893429873511195,4.874200583435595,23.186982492916286,67.25202781381086,0.0,131.23180496040732,4,18.397639505565166,4.841370215946989
+8192,1,32768,0,4.789774015080184,4.870833398308605,4.738486022688448,4.886626999359578,0.0,4500.839321000967,5,-0.051287992391735315,0.9892921895207875
+8192,1,32768,1,4.776834975928068,4.891659819986671,4.729953012429178,4.9245511763729155,0.0,4496.073378017172,5,-0.0468819634988904,0.9901855593221991
+8192,1,32768,2,4.784431017469615,4.866032593417913,4.782894975505769,4.8977664206177,0.0,4549.013931944501,5,-0.0015360419638454914,0.9996789499193871
+8192,1,65536,0,4.778854956384748,4.9255444086156785,4.633405013009906,4.895579582080245,0.0,15530.37424501963,5,-0.1454499433748424,0.9695638506080803
+8192,1,65536,1,4.784283053595573,4.8808404128067195,4.754905996378511,4.985795798711479,0.0,15584.887631004676,5,-0.02937705721706152,0.99385967408534
+8192,1,65536,2,4.787993966601789,4.9004736240021884,4.6836750116199255,5.0271204963792115,0.0,15587.390075030271,6,-0.1043189549818635,0.9782123879625725
+8192,1,8192,0,4.785028984770179,4.878618801012635,7.490115996915847,324.06569679733366,0.0,573.2795029762201,5,2.7050870121456683,1.565323014919123
+8192,1,8192,1,4.778591974172741,4.899543372448534,5.9131429879926145,336.8099076091312,0.0,606.6823820001446,5,1.1345510138198733,1.237423705550061
+8192,1,8192,2,4.78826800826937,4.90188361145556,6.276679981965572,324.8370993998833,0.0,571.7499859747477,5,1.488411973696202,1.310845585736994
+8192,4,131072,0,6.113810988608748,6.309205386787653,0.0,0.0,0.0,56702.702289039735,0,-6.113810988608748,0.0
+8192,4,131072,1,6.630807969486341,7.086459483252838,6.2820459716022015,4400.500871409893,0.0,56807.70832300186,150,-0.3487619978841394,0.9474027902045915
+8192,4,131072,2,6.073819473385811,6.344516028184444,6.326125003397465,4409.856556192978,0.0,56580.784838995896,149,0.2523055300116539,1.0415398467335428
+8192,4,2048,0,5.402160517405719,5.543816485442221,6.210724503034726,84.62208869168535,6.125201500253752,140.3041940066032,18,0.8085639856290072,1.1496741873966574
+8192,4,2048,1,6.067108013667166,6.381415005307645,0.0,0.0,0.0,140.06177097326145,0,-6.067108013667166,0.0
+8192,4,2048,2,5.400336522143334,5.536347016459331,38.15686801681295,85.07051098858938,5.25214200024493,134.67552902875468,13,32.756531494669616,7.065646346363043
+8192,4,32768,0,6.115561525803059,6.369604001520202,7.216634490760043,1314.6978712815326,5.17624247004278,4522.433568025008,50,1.101072964956984,1.1800444587649532
+8192,4,32768,1,6.070095987524837,6.3612310332246125,0.0,0.0,0.0,4508.074064040557,0,-6.070095987524837,0.0
+8192,4,32768,2,6.0734800063073635,6.312666402664036,12.442811043001711,1315.0411327951588,4.754714027512819,4556.892123946454,45,6.369331036694348,2.0487119460473635
+8192,4,65536,0,5.406292999396101,5.540905491216108,0.0,0.0,0.0,15581.590663990937,0,-5.406292999396101,0.0
+8192,4,65536,1,6.076910009142011,6.315114628523588,0.0,0.0,0.0,15574.196094006766,0,-6.076910009142011,0.0
+8192,4,65536,2,6.060379033442587,6.384042033459991,6.411670008674264,2077.4700703914277,4.8022730043157935,15603.720718005206,79,0.3512909752316773,1.0579651822589267
+8192,4,8192,0,6.110575021011755,6.416070973500609,8.451583969872445,515.3855616226792,5.358011490898207,574.6672929963097,18,2.34100894886069,1.3831077993169092
+8192,4,8192,1,6.051429023500532,6.398122606333345,0.0,0.0,0.0,573.6081749782898,0,-6.051429023500532,0.0
+8192,4,8192,2,6.064729997888207,6.366449000779539,0.0,0.0,0.0,574.1707819979638,0,-6.064729997888207,0.0
+8192,8,131072,0,7.737616979284212,7.99839201499708,10.740376019384712,4742.438135773409,7.792441989295185,57010.66731195897,335,3.0027590401005,1.388072845701685
+8192,8,131072,1,7.744895527139306,8.013638522243127,8.647068490972742,5123.228083999129,7.672236970392987,56970.40947602363,310,0.9021729638334364,1.116486137310966
+8192,8,131072,2,7.740180502878502,8.016240986762568,15.140031988266855,4820.136589207682,7.68946303287521,56993.02393599646,319,7.3998514853883535,1.9560308680962177
+8192,8,2048,0,7.741285488009453,8.022559515666217,8.103576023131609,124.87094267853536,7.6825070136692375,141.97922096354887,30,0.36229053512215614,1.046799789993963
+8192,8,2048,1,7.728310010861605,8.021069981623441,8.17067950265482,84.82906777062453,7.745136506855488,144.1582590341568,38,0.4423694917932153,1.0572401328584768
+8192,8,2048,2,7.662211020942777,8.034424972720444,8.87883099494502,87.23540699575096,7.592331967316568,143.27958395006135,39,1.216619974002242,1.1587818412566437
+8192,8,32768,0,7.295333489309996,7.422819995554164,11.429400008637458,1315.43214758276,7.8034960315562785,4523.641717038117,94,4.134066519327462,1.5666727265292526
+8192,8,32768,1,7.278127042809501,7.490781514206901,12.640403030673042,1315.491412486881,7.821676495950669,4519.993302994408,90,5.362275987863541,1.736765922925357
+8192,8,32768,2,7.684049021918327,8.047712198458612,10.752685484476388,1315.5166705255397,7.80402502277866,4517.200137954205,96,3.068636462558061,1.3993514947399404
+8192,8,65536,0,7.708174001891166,8.017168991500512,26.662671996746212,2496.8427699001018,7.768569514155388,15603.601168957539,160,18.954497994855046,3.459012729889679
+8192,8,65536,1,7.594842027174309,7.9874323040712625,13.054963492322713,2459.1690181812737,7.54699349636212,15620.474929979537,174,5.460121465148404,1.7189249553331216
+8192,8,65536,2,7.693717983784154,7.933055714238435,17.5579380011186,2458.176895044744,7.808708498487249,15622.32490995666,161,9.864220017334446,2.2821135422594123
+8192,8,8192,0,7.636573514901102,7.904737605713308,10.151655005756766,514.8188057704829,7.7977380133233964,575.7745200535282,37,2.515081490855664,1.3293468577167538
+8192,8,8192,1,7.687711506150663,7.965393498307094,9.002390026580542,524.0793236298487,7.753994490485638,592.1044679707848,45,1.3146785204298794,1.1710103870804793
+8192,8,8192,2,7.756220467854291,8.035426988499239,8.864110975991935,518.9726910321042,7.770269992761314,581.98908099439,41,1.1078905081376433,1.1428389655411813

analysis/mb2/A_inter_kvboth.jsonl

@@ -0,0 +1,51 @@
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 50331648, "duration_s": 0.06580113701056689, "t_start_unix": 1779885615.6732209, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885615.7390358}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 50331648, "duration_s": 0.0052392969955690205, "t_start_unix": 1779885616.0322638, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885616.0375087}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 201326592, "duration_s": 0.02050818904535845, "t_start_unix": 1779885616.2556505, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885616.2761638}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 201326592, "duration_s": 0.02001398801803589, "t_start_unix": 1779885616.4400308, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885616.46005}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 805306368, "duration_s": 0.08249958901433274, "t_start_unix": 1779885617.072654, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885617.1551628}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 805306368, "duration_s": 0.08082435996038839, "t_start_unix": 1779885617.7853239, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885617.866155}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 1572864, "duration_s": 0.00040365802124142647, "t_start_unix": 1779885642.3123364, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885642.3127441}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 1572864, "duration_s": 0.000374739000108093, "t_start_unix": 1779885642.3945863, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885642.3949661}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 50331648, "duration_s": 0.005158354004379362, "t_start_unix": 1779885642.4766958, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885642.481858}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 50331648, "duration_s": 0.00513053999748081, "t_start_unix": 1779885642.5614145, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885642.5665495}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 50331648, "duration_s": 0.0051341859507374465, "t_start_unix": 1779885642.6461189, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885642.6512585}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 100663296, "duration_s": 0.010125375993084162, "t_start_unix": 1779885642.7582293, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885642.76836}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 100663296, "duration_s": 0.010367848037276417, "t_start_unix": 1779885642.878179, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885642.8885527}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 100663296, "duration_s": 0.010208865976892412, "t_start_unix": 1779885642.997267, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885643.0074801}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 100663296, "duration_s": 0.01015280099818483, "t_start_unix": 1779885643.1138487, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885643.124007}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 100663296, "duration_s": 0.010063701018225402, "t_start_unix": 1779885643.2299926, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885643.2400591}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 1572864, "duration_s": 0.00040220399387180805, "t_start_unix": 1779885643.4041483, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885643.4045541}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 1572864, "duration_s": 0.0003751559997908771, "t_start_unix": 1779885643.569847, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885643.5702271}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 201326592, "duration_s": 0.020166583999525756, "t_start_unix": 1779885643.7347023, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885643.7548745}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 201326592, "duration_s": 0.019989027990959585, "t_start_unix": 1779885643.9190295, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885643.939023}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 201326592, "duration_s": 0.02000429102918133, "t_start_unix": 1779885644.1027336, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885644.122742}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 402653184, "duration_s": 0.040499016002286226, "t_start_unix": 1779885644.419112, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885644.4596438}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 402653184, "duration_s": 0.04015034798067063, "t_start_unix": 1779885644.761118, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885644.8012745}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 402653184, "duration_s": 0.04006708099041134, "t_start_unix": 1779885645.1008255, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885645.1408994}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 402653184, "duration_s": 0.04012463602703065, "t_start_unix": 1779885645.440819, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885645.4809544}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 402653184, "duration_s": 0.04009692999534309, "t_start_unix": 1779885645.7793777, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885645.8194828}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 1572864, "duration_s": 0.00046314700739458203, "t_start_unix": 1779885646.445756, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885646.4462252}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 1572864, "duration_s": 0.0004179630195721984, "t_start_unix": 1779885647.0757725, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885647.0761962}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 805306368, "duration_s": 0.08170936699025333, "t_start_unix": 1779885647.7075222, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885647.7892444}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 805306368, "duration_s": 0.08086105203256011, "t_start_unix": 1779885648.422354, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885648.503224}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 805306368, "duration_s": 0.08071460900828242, "t_start_unix": 1779885649.1382625, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885649.2189863}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 1610612736, "duration_s": 0.1638482889975421, "t_start_unix": 1779885650.8273866, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885650.991251}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 1610612736, "duration_s": 0.16476035403320566, "t_start_unix": 1779885652.5951493, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885652.7599196}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 1610612736, "duration_s": 0.1617715489701368, "t_start_unix": 1779885654.3595936, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885654.5214472}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 1610612736, "duration_s": 0.1616577100357972, "t_start_unix": 1779885656.125777, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885656.2874432}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 1610612736, "duration_s": 0.1617818950326182, "t_start_unix": 1779885657.8867118, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885658.0485108}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 3221225472, "duration_s": 0.30771408596774563, "t_start_unix": 1779885662.69878, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885663.0065253}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 3221225472, "duration_s": 0.3096057590446435, "t_start_unix": 1779885667.6915898, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885668.0012283}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 3221225472, "duration_s": 0.3080696280230768, "t_start_unix": 1779885672.684307, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885672.9923992}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 3221225472, "duration_s": 0.5257709489669651, "t_start_unix": 1779885677.7157974, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885678.2415998}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 3221225472, "duration_s": 0.5268570999614894, "t_start_unix": 1779885682.9683614, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885683.4952588}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 6442450944, "duration_s": 1.901308581000194, "t_start_unix": 1779885698.9342654, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885700.835605}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 6442450944, "duration_s": 1.9211939970264211, "t_start_unix": 1779885716.3570645, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885718.2782962}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 6442450944, "duration_s": 1.7335722800344229, "t_start_unix": 1779885733.838071, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885735.571683}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 6442450944, "duration_s": 0.9299940629862249, "t_start_unix": 1779885751.0288215, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885751.9588547}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 6442450944, "duration_s": 0.6535220990190282, "t_start_unix": 1779885767.3456392, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885767.9991918}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 12884901888, "duration_s": 1.2830275790183805, "t_start_unix": 1779885822.9884846, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885824.2715507}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 12884901888, "duration_s": 5.053741328010801, "t_start_unix": 1779885879.5277712, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885884.5815506}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 12884901888, "duration_s": 9.158571614010725, "t_start_unix": 1779885939.9274101, "ret": 0, "tp_rank": 0, "t_log_unix": 1779885949.086019}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 12884901888, "duration_s": 2.8184301540022716, "t_start_unix": 1779886004.3031003, "ret": 0, "tp_rank": 0, "t_log_unix": 1779886007.121565}
+{"event": "send_blocks", "remote_session": "172.27.123.133:16878", "total_bytes": 12884901888, "duration_s": 1.4530277770245448, "t_start_unix": 1779886062.159227, "ret": 0, "tp_rank": 0, "t_log_unix": 1779886063.6122832}

analysis/mb2/A_intra_kvboth.jsonl

@@ -0,0 +1,51 @@
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 50331648, "duration_s": 0.023202952987048775, "t_start_unix": 1779879143.174031, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879143.1972432}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 50331648, "duration_s": 0.005375694017857313, "t_start_unix": 1779879143.2982283, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879143.3036084}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 201326592, "duration_s": 0.021170366962905973, "t_start_unix": 1779879143.5159554, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879143.5371296}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 201326592, "duration_s": 0.020726953051052988, "t_start_unix": 1779879143.6974514, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879143.7181835}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 805306368, "duration_s": 0.08536655298667029, "t_start_unix": 1779879144.3294952, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879144.4148676}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 805306368, "duration_s": 0.08367906499188393, "t_start_unix": 1779879145.0419943, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879145.125678}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 1572864, "duration_s": 0.0004059679922647774, "t_start_unix": 1779879221.7078288, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879221.7082384}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 1572864, "duration_s": 0.000346789020113647, "t_start_unix": 1779879221.7838593, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879221.7842083}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 50331648, "duration_s": 0.005353622022084892, "t_start_unix": 1779879221.8607252, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879221.8660822}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 50331648, "duration_s": 0.005279594974126667, "t_start_unix": 1779879221.9432015, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879221.9484842}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 50331648, "duration_s": 0.0053006180096417665, "t_start_unix": 1779879222.0243337, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879222.0296378}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 100663296, "duration_s": 0.010396577999927104, "t_start_unix": 1779879222.130936, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879222.141335}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 100663296, "duration_s": 0.010438029014039785, "t_start_unix": 1779879222.2442062, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879222.2546473}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 100663296, "duration_s": 0.010436972021125257, "t_start_unix": 1779879222.3581295, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879222.3685696}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 100663296, "duration_s": 0.010396371013484895, "t_start_unix": 1779879222.4725878, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879222.4829867}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 100663296, "duration_s": 0.010352785000577569, "t_start_unix": 1779879222.5837166, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879222.5940716}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 1572864, "duration_s": 0.00034007197245955467, "t_start_unix": 1779879222.7521152, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879222.752458}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 1572864, "duration_s": 0.00041691696969792247, "t_start_unix": 1779879222.9143836, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879222.914805}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 201326592, "duration_s": 0.020633380976505578, "t_start_unix": 1779879223.0778644, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879223.098502}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 201326592, "duration_s": 0.020639199996367097, "t_start_unix": 1779879223.2603853, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879223.2810278}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 201326592, "duration_s": 0.020575353992171586, "t_start_unix": 1779879223.4418828, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879223.4624615}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 402653184, "duration_s": 0.041439525957684964, "t_start_unix": 1779879223.7544343, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879223.7958782}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 402653184, "duration_s": 0.04152030003024265, "t_start_unix": 1779879224.0914912, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879224.133016}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 402653184, "duration_s": 0.04148670402355492, "t_start_unix": 1779879224.4262393, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879224.4677298}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 402653184, "duration_s": 0.04146742797456682, "t_start_unix": 1779879224.7617002, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879224.8031723}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 402653184, "duration_s": 0.04143296502297744, "t_start_unix": 1779879225.0978234, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879225.1392617}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 1572864, "duration_s": 0.0003991159610450268, "t_start_unix": 1779879225.760789, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879225.7611918}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 1572864, "duration_s": 0.00041423802031204104, "t_start_unix": 1779879226.3864496, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879226.3868673}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 805306368, "duration_s": 0.08309489500243217, "t_start_unix": 1779879227.0107942, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879227.0938945}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 805306368, "duration_s": 0.08372796402545646, "t_start_unix": 1779879227.7207224, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879227.8044555}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 805306368, "duration_s": 0.08398396399570629, "t_start_unix": 1779879228.4314566, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879228.5154452}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 1610612736, "duration_s": 0.16950496198842302, "t_start_unix": 1779879230.1334376, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879230.3029544}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 1610612736, "duration_s": 0.16713789198547602, "t_start_unix": 1779879231.8981037, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879232.0652575}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 1610612736, "duration_s": 0.16713115200400352, "t_start_unix": 1779879233.6608078, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879233.827945}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 1610612736, "duration_s": 0.16709016199456528, "t_start_unix": 1779879235.419875, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879235.5869706}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 1610612736, "duration_s": 0.166486973001156, "t_start_unix": 1779879237.1821773, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879237.34867}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 3221225472, "duration_s": 0.31926770601421595, "t_start_unix": 1779879241.9880297, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879242.3073065}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 3221225472, "duration_s": 0.3197040680097416, "t_start_unix": 1779879246.9779432, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879247.2976692}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 3221225472, "duration_s": 0.32088329299585894, "t_start_unix": 1779879251.9643052, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879252.285209}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 3221225472, "duration_s": 0.5439103110111319, "t_start_unix": 1779879256.9989722, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879257.5428913}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 3221225472, "duration_s": 0.5193864739849232, "t_start_unix": 1779879262.2562187, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879262.7756212}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 6442450944, "duration_s": 1.9844180009677075, "t_start_unix": 1779879278.199048, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879280.1834915}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 6442450944, "duration_s": 2.1099297259934247, "t_start_unix": 1779879295.6967168, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879297.8066647}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 6442450944, "duration_s": 1.8950715209939517, "t_start_unix": 1779879313.3236735, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879315.2187643}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 6442450944, "duration_s": 0.9277855920372531, "t_start_unix": 1779879330.6715357, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879331.599329}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 6442450944, "duration_s": 0.6652462020283565, "t_start_unix": 1779879346.9950044, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879347.6602724}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 12884901888, "duration_s": 1.3330365709844045, "t_start_unix": 1779879402.7169023, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879404.04997}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 12884901888, "duration_s": 5.839069904992357, "t_start_unix": 1779879459.0566247, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879464.8957155}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 12884901888, "duration_s": 9.862486142024864, "t_start_unix": 1779879519.9567635, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879529.8192694}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 12884901888, "duration_s": 2.8350498770014383, "t_start_unix": 1779879584.9780834, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879587.813154}
+{"event": "send_blocks", "remote_session": "172.27.123.142:16428", "total_bytes": 12884901888, "duration_s": 1.485496642999351, "t_start_unix": 1779879642.639775, "ret": 0, "tp_rank": 0, "t_log_unix": 1779879644.1252885}

analysis/mb2/B_intra_kvboth.jsonl

@@ -0,0 +1,102 @@
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-ad00672f263a6643-0-9479211a"], "t_start_unix": 1779879143.1678784, "tp_rank": 0, "t_log_unix": 1779879143.167884}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-ad00672f263a6643-0-9479211a"], "duration_s": 0.03333390498301014, "t_start_unix": 1779879143.1678784, "tp_rank": 0, "t_log_unix": 1779879143.201217}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-ace77e2b02f9f141-0-b3c061bc"], "t_start_unix": 1779879143.2968972, "tp_rank": 0, "t_log_unix": 1779879143.2969005}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-ace77e2b02f9f141-0-b3c061bc"], "duration_s": 0.007019245007541031, "t_start_unix": 1779879143.2968972, "tp_rank": 0, "t_log_unix": 1779879143.3039184}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-a4a2366879c68ded-0-8ac4098e"], "t_start_unix": 1779879143.5146625, "tp_rank": 0, "t_log_unix": 1779879143.5146651}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-a4a2366879c68ded-0-8ac4098e"], "duration_s": 0.02278437599306926, "t_start_unix": 1779879143.5146625, "tp_rank": 0, "t_log_unix": 1779879143.537448}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-8690cafcace0d5e2-0-b89f33d2"], "t_start_unix": 1779879143.6958342, "tp_rank": 0, "t_log_unix": 1779879143.6958375}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-8690cafcace0d5e2-0-b89f33d2"], "duration_s": 0.022794076008722186, "t_start_unix": 1779879143.6958342, "tp_rank": 0, "t_log_unix": 1779879143.71863}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-b087e2ec4cfa8eb7-0-b908f425"], "t_start_unix": 1779879144.3279662, "tp_rank": 0, "t_log_unix": 1779879144.3279696}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-b087e2ec4cfa8eb7-0-b908f425"], "duration_s": 0.08753501297906041, "t_start_unix": 1779879144.3279662, "tp_rank": 0, "t_log_unix": 1779879144.415505}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-a115d16ff5575e08-0-9fa81984"], "t_start_unix": 1779879145.040141, "tp_rank": 0, "t_log_unix": 1779879145.0401456}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-a115d16ff5575e08-0-9fa81984"], "duration_s": 0.0860149699728936, "t_start_unix": 1779879145.040141, "tp_rank": 0, "t_log_unix": 1779879145.1261594}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-9e585ed083951df5-0-b03f812b"], "t_start_unix": 1779879221.7062025, "tp_rank": 0, "t_log_unix": 1779879221.7062056}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-9e585ed083951df5-0-b03f812b"], "duration_s": 0.002459956973325461, "t_start_unix": 1779879221.7062025, "tp_rank": 0, "t_log_unix": 1779879221.7086644}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-9271d403c044eadd-0-9c3c4639"], "t_start_unix": 1779879221.7826598, "tp_rank": 0, "t_log_unix": 1779879221.782662}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-9271d403c044eadd-0-9c3c4639"], "duration_s": 0.0020201010047458112, "t_start_unix": 1779879221.7826598, "tp_rank": 0, "t_log_unix": 1779879221.7846813}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-82a580cefd3e2440-0-a383c3c4"], "t_start_unix": 1779879221.859549, "tp_rank": 0, "t_log_unix": 1779879221.8595514}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-82a580cefd3e2440-0-a383c3c4"], "duration_s": 0.006836243963334709, "t_start_unix": 1779879221.859549, "tp_rank": 0, "t_log_unix": 1779879221.8663864}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-a31cb4bc9e7f63d2-0-8f48aacd"], "t_start_unix": 1779879221.9419758, "tp_rank": 0, "t_log_unix": 1779879221.9419782}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-a31cb4bc9e7f63d2-0-8f48aacd"], "duration_s": 0.00694335694424808, "t_start_unix": 1779879221.9419758, "tp_rank": 0, "t_log_unix": 1779879221.9489205}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-a9dfc1a5b425d994-0-a0930098"], "t_start_unix": 1779879222.0232244, "tp_rank": 0, "t_log_unix": 1779879222.0232272}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-a9dfc1a5b425d994-0-a0930098"], "duration_s": 0.006697195000015199, "t_start_unix": 1779879222.0232244, "tp_rank": 0, "t_log_unix": 1779879222.0299227}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-9712857755af2efc-0-90b2dc9b"], "t_start_unix": 1779879222.1297998, "tp_rank": 0, "t_log_unix": 1779879222.1298025}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-9712857755af2efc-0-90b2dc9b"], "duration_s": 0.01183948403922841, "t_start_unix": 1779879222.1297998, "tp_rank": 0, "t_log_unix": 1779879222.1416407}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-b4f0a10dee65acbe-0-a3c132fc"], "t_start_unix": 1779879222.243023, "tp_rank": 0, "t_log_unix": 1779879222.243025}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-b4f0a10dee65acbe-0-a3c132fc"], "duration_s": 0.01214482297655195, "t_start_unix": 1779879222.243023, "tp_rank": 0, "t_log_unix": 1779879222.2551687}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-b4c514b80b52a3f2-0-bcd24f8e"], "t_start_unix": 1779879222.3569698, "tp_rank": 0, "t_log_unix": 1779879222.356972}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-b4c514b80b52a3f2-0-bcd24f8e"], "duration_s": 0.011961110983975232, "t_start_unix": 1779879222.3569698, "tp_rank": 0, "t_log_unix": 1779879222.368932}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-ac7118d8090d181c-0-8af4adf0"], "t_start_unix": 1779879222.4715128, "tp_rank": 0, "t_log_unix": 1779879222.4715152}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-ac7118d8090d181c-0-8af4adf0"], "duration_s": 0.011788576026447117, "t_start_unix": 1779879222.4715128, "tp_rank": 0, "t_log_unix": 1779879222.4833028}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-85291bcb93aaf638-0-868db1a8"], "t_start_unix": 1779879222.5826046, "tp_rank": 0, "t_log_unix": 1779879222.5826073}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-85291bcb93aaf638-0-868db1a8"], "duration_s": 0.0118055299972184, "t_start_unix": 1779879222.5826046, "tp_rank": 0, "t_log_unix": 1779879222.5944116}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-a448cf2e059ba0c9-0-a1360796"], "t_start_unix": 1779879222.750828, "tp_rank": 0, "t_log_unix": 1779879222.7508304}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-a448cf2e059ba0c9-0-a1360796"], "duration_s": 0.0021119200391694903, "t_start_unix": 1779879222.750828, "tp_rank": 0, "t_log_unix": 1779879222.7529414}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-b486fd9e945a4658-0-8bb561cd"], "t_start_unix": 1779879222.913044, "tp_rank": 0, "t_log_unix": 1779879222.9130466}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-b486fd9e945a4658-0-8bb561cd"], "duration_s": 0.0022232600022107363, "t_start_unix": 1779879222.913044, "tp_rank": 0, "t_log_unix": 1779879222.9152684}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-82da2bfe65f276c6-0-88d9a9a2"], "t_start_unix": 1779879223.0765986, "tp_rank": 0, "t_log_unix": 1779879223.0766027}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-82da2bfe65f276c6-0-88d9a9a2"], "duration_s": 0.022250515001360327, "t_start_unix": 1779879223.0765986, "tp_rank": 0, "t_log_unix": 1779879223.0988505}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-93bd777652eba5f3-0-9ec3d058"], "t_start_unix": 1779879223.2591784, "tp_rank": 0, "t_log_unix": 1779879223.2591808}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-93bd777652eba5f3-0-9ec3d058"], "duration_s": 0.022157608007546514, "t_start_unix": 1779879223.2591784, "tp_rank": 0, "t_log_unix": 1779879223.2813375}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-81f950480a3cabf9-0-bbf8584f"], "t_start_unix": 1779879223.4402068, "tp_rank": 0, "t_log_unix": 1779879223.440209}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-81f950480a3cabf9-0-bbf8584f"], "duration_s": 0.022589912987314165, "t_start_unix": 1779879223.4402068, "tp_rank": 0, "t_log_unix": 1779879223.4627984}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-b109ed06b5882659-0-8d14993c"], "t_start_unix": 1779879223.7529812, "tp_rank": 0, "t_log_unix": 1779879223.752984}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-b109ed06b5882659-0-8d14993c"], "duration_s": 0.043345845013391227, "t_start_unix": 1779879223.7529812, "tp_rank": 0, "t_log_unix": 1779879223.796329}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-8a57776c81d64b2c-0-ace8fb2b"], "t_start_unix": 1779879224.0899644, "tp_rank": 0, "t_log_unix": 1779879224.0899673}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-8a57776c81d64b2c-0-ace8fb2b"], "duration_s": 0.04341953102266416, "t_start_unix": 1779879224.0899644, "tp_rank": 0, "t_log_unix": 1779879224.1333857}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-9b1a5dce18758450-0-b17b3649"], "t_start_unix": 1779879224.424807, "tp_rank": 0, "t_log_unix": 1779879224.42481}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-9b1a5dce18758450-0-b17b3649"], "duration_s": 0.04336977802449837, "t_start_unix": 1779879224.424807, "tp_rank": 0, "t_log_unix": 1779879224.4681823}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-8c7d412b85f43ed7-0-9dea4add"], "t_start_unix": 1779879224.7599711, "tp_rank": 0, "t_log_unix": 1779879224.7599735}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-8c7d412b85f43ed7-0-9dea4add"], "duration_s": 0.043769759009592235, "t_start_unix": 1779879224.7599711, "tp_rank": 0, "t_log_unix": 1779879224.8037443}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-8860308db3f010a5-0-ad51eb46"], "t_start_unix": 1779879225.0962389, "tp_rank": 0, "t_log_unix": 1779879225.0962446}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-8860308db3f010a5-0-ad51eb46"], "duration_s": 0.043612666020635515, "t_start_unix": 1779879225.0962389, "tp_rank": 0, "t_log_unix": 1779879225.1398532}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-86cca1a2b9427801-0-ba41ade7"], "t_start_unix": 1779879225.7592747, "tp_rank": 0, "t_log_unix": 1779879225.759278}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-86cca1a2b9427801-0-ba41ade7"], "duration_s": 0.002386144013144076, "t_start_unix": 1779879225.7592747, "tp_rank": 0, "t_log_unix": 1779879225.7616625}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-a208c6d804293be7-0-94d265ab"], "t_start_unix": 1779879226.384918, "tp_rank": 0, "t_log_unix": 1779879226.384921}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-a208c6d804293be7-0-94d265ab"], "duration_s": 0.0023903060355223715, "t_start_unix": 1779879226.384918, "tp_rank": 0, "t_log_unix": 1779879226.3873098}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-b53bea2317cc1211-0-8fcad8a8"], "t_start_unix": 1779879227.0092332, "tp_rank": 0, "t_log_unix": 1779879227.0092363}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-b53bea2317cc1211-0-8fcad8a8"], "duration_s": 0.08524628396844491, "t_start_unix": 1779879227.0092332, "tp_rank": 0, "t_log_unix": 1779879227.094482}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-9daf909593bbdf03-0-8fd7d50e"], "t_start_unix": 1779879227.7190688, "tp_rank": 0, "t_log_unix": 1779879227.719072}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-9daf909593bbdf03-0-8fd7d50e"], "duration_s": 0.08596085698809475, "t_start_unix": 1779879227.7190688, "tp_rank": 0, "t_log_unix": 1779879227.8050315}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-9ef40f3b6d736128-0-8e8e1c30"], "t_start_unix": 1779879228.4297745, "tp_rank": 0, "t_log_unix": 1779879228.4297774}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-9ef40f3b6d736128-0-8e8e1c30"], "duration_s": 0.0860762019874528, "t_start_unix": 1779879228.4297745, "tp_rank": 0, "t_log_unix": 1779879228.5158527}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-851e5d7e3e83d7ea-0-a66a5e0b"], "t_start_unix": 1779879230.131392, "tp_rank": 0, "t_log_unix": 1779879230.1313956}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-851e5d7e3e83d7ea-0-a66a5e0b"], "duration_s": 0.1721468890318647, "t_start_unix": 1779879230.131392, "tp_rank": 0, "t_log_unix": 1779879230.3035412}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-9be12af6a9ccccf5-0-af1230c7"], "t_start_unix": 1779879231.896075, "tp_rank": 0, "t_log_unix": 1779879231.896078}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-9be12af6a9ccccf5-0-af1230c7"], "duration_s": 0.16974544001277536, "t_start_unix": 1779879231.896075, "tp_rank": 0, "t_log_unix": 1779879232.0658224}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-b61b9b237366297b-0-9832f0e3"], "t_start_unix": 1779879233.6589305, "tp_rank": 0, "t_log_unix": 1779879233.6589334}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-b61b9b237366297b-0-9832f0e3"], "duration_s": 0.16975757898762822, "t_start_unix": 1779879233.6589305, "tp_rank": 0, "t_log_unix": 1779879233.8286898}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-bae0d0efe47ece8f-0-affbc685"], "t_start_unix": 1779879235.4181106, "tp_rank": 0, "t_log_unix": 1779879235.418114}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-bae0d0efe47ece8f-0-affbc685"], "duration_s": 0.1695251659839414, "t_start_unix": 1779879235.4181106, "tp_rank": 0, "t_log_unix": 1779879235.587638}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-a34bc73c9cd2efc1-0-90d647fc"], "t_start_unix": 1779879237.1803744, "tp_rank": 0, "t_log_unix": 1779879237.1803775}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-a34bc73c9cd2efc1-0-90d647fc"], "duration_s": 0.16962904302636161, "t_start_unix": 1779879237.1803744, "tp_rank": 0, "t_log_unix": 1779879237.3500054}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-89a36c12ee6b0ff3-0-9fddbc0f"], "t_start_unix": 1779879241.9859307, "tp_rank": 0, "t_log_unix": 1779879241.9859338}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-89a36c12ee6b0ff3-0-9fddbc0f"], "duration_s": 0.32203804596792907, "t_start_unix": 1779879241.9859307, "tp_rank": 0, "t_log_unix": 1779879242.3079708}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-8d65512eb7e3c36c-0-8b23597c"], "t_start_unix": 1779879246.9755645, "tp_rank": 0, "t_log_unix": 1779879246.9755676}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-8d65512eb7e3c36c-0-8b23597c"], "duration_s": 0.3227974839974195, "t_start_unix": 1779879246.9755645, "tp_rank": 0, "t_log_unix": 1779879247.2983644}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-a13c271ecbbca78b-0-b76a0370"], "t_start_unix": 1779879251.9618897, "tp_rank": 0, "t_log_unix": 1779879251.961893}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-a13c271ecbbca78b-0-b76a0370"], "duration_s": 0.3240378479822539, "t_start_unix": 1779879251.9618897, "tp_rank": 0, "t_log_unix": 1779879252.2859304}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-bada04ec8c556aca-0-a263d637"], "t_start_unix": 1779879256.9512377, "tp_rank": 0, "t_log_unix": 1779879256.9512408}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-bada04ec8c556aca-0-a263d637"], "duration_s": 0.5924434679909609, "t_start_unix": 1779879256.9512377, "tp_rank": 0, "t_log_unix": 1779879257.5436878}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-9641a077022e6123-0-8c3c0975"], "t_start_unix": 1779879262.2127163, "tp_rank": 0, "t_log_unix": 1779879262.2127194}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-9641a077022e6123-0-8c3c0975"], "duration_s": 0.5644763479940593, "t_start_unix": 1779879262.2127163, "tp_rank": 0, "t_log_unix": 1779879262.777195}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-bb3a4e5084af8c3a-0-bdfa0931"], "t_start_unix": 1779879278.1063075, "tp_rank": 0, "t_log_unix": 1779879278.106311}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-bb3a4e5084af8c3a-0-bdfa0931"], "duration_s": 2.0784930550144054, "t_start_unix": 1779879278.1063075, "tp_rank": 0, "t_log_unix": 1779879280.1848085}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-91b951f85c93a71b-0-8396bee5"], "t_start_unix": 1779879295.600993, "tp_rank": 0, "t_log_unix": 1779879295.6009963}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-91b951f85c93a71b-0-8396bee5"], "duration_s": 2.2067435560165904, "t_start_unix": 1779879295.600993, "tp_rank": 0, "t_log_unix": 1779879297.8077443}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-81d236ecb6aadadf-0-ac184d51"], "t_start_unix": 1779879313.2315958, "tp_rank": 0, "t_log_unix": 1779879313.2315989}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-81d236ecb6aadadf-0-ac184d51"], "duration_s": 1.9879729640088044, "t_start_unix": 1779879313.2315958, "tp_rank": 0, "t_log_unix": 1779879315.219571}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-a4c76c62b44c4295-0-b007a6ed"], "t_start_unix": 1779879330.6154163, "tp_rank": 0, "t_log_unix": 1779879330.6154196}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-a4c76c62b44c4295-0-b007a6ed"], "duration_s": 0.9849357060156763, "t_start_unix": 1779879330.6154163, "tp_rank": 0, "t_log_unix": 1779879331.6003594}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-a06d4b774a8af9a5-0-980e9d23"], "t_start_unix": 1779879346.990221, "tp_rank": 0, "t_log_unix": 1779879346.9902246}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-a06d4b774a8af9a5-0-980e9d23"], "duration_s": 0.6725030990201049, "t_start_unix": 1779879346.990221, "tp_rank": 0, "t_log_unix": 1779879347.6627269}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-bf0d435e06e3349f-0-8507c933"], "t_start_unix": 1779879402.7123013, "tp_rank": 0, "t_log_unix": 1779879402.7123044}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-bf0d435e06e3349f-0-8507c933"], "duration_s": 1.3384539679973386, "t_start_unix": 1779879402.7123013, "tp_rank": 0, "t_log_unix": 1779879404.0507588}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-9f87ae0fb0c7eec8-0-a8a1daea"], "t_start_unix": 1779879458.9232886, "tp_rank": 0, "t_log_unix": 1779879458.9232917}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-9f87ae0fb0c7eec8-0-a8a1daea"], "duration_s": 5.973284716019407, "t_start_unix": 1779879458.9232886, "tp_rank": 0, "t_log_unix": 1779879464.896582}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-a62e48e40e6c6ad7-0-acca9741"], "t_start_unix": 1779879519.7647448, "tp_rank": 0, "t_log_unix": 1779879519.7647479}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-a62e48e40e6c6ad7-0-acca9741"], "duration_s": 10.056511385017075, "t_start_unix": 1779879519.7647448, "tp_rank": 0, "t_log_unix": 1779879529.8212643}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-824479d53bab40e4-0-af951a11"], "t_start_unix": 1779879584.888362, "tp_rank": 0, "t_log_unix": 1779879584.8883653}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-824479d53bab40e4-0-af951a11"], "duration_s": 2.925714804965537, "t_start_unix": 1779879584.888362, "tp_rank": 0, "t_log_unix": 1779879587.814085}
+{"event": "receive_kv_enter", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-9f06f19c981c0b3f-0-b3afb370"], "t_start_unix": 1779879642.6076336, "tp_rank": 0, "t_log_unix": 1779879642.6076367}
+{"event": "receive_kv_finish", "worker_addr": "tcp://172.27.123.142:44435", "req_ids": ["cmpl-9f06f19c981c0b3f-0-b3afb370"], "duration_s": 1.5183607729850337, "t_start_unix": 1779879642.6076336, "tp_rank": 0, "t_log_unix": 1779879644.1259985}

analysis/mb2/README.md

@@ -0,0 +1,314 @@
+# MB2 — Mooncake KV Transfer Cost (vanilla vLLM 0.18.1)
+
+Persistent record of the per-stage KV transfer microbench used in §3.2 of
+the EAR paper. Re-runs append a dated section at the bottom; the
+**Summary** block at the top is what gets cited in the paper.
+
+---
+
+## Summary (latest)
+
+| Path | Steady-state BW | Agentic-tail p99 transfer (11.5 GiB KV) |
+|---|---|---|
+| **intra-node** (dash1 GPU 0↔1) | **~9.7 GB/s** (96 MiB – 3 GiB) | p50 **1.9 s** · min **1.5 s** · max **10 s** |
+| **inter-node** (dash1 GPU0 → dash2 GPU0, 200 Gbps RoCE) | **~10.0 GB/s** (essentially identical) | p50 **1.7 s** · min **1.3 s** · max **9.2 s** |
+
+**Cross-cutting finding** (2026-05-27): **Mooncake transfer cost is
+topology-independent** on this hardware. Intra-node and inter-node curves
+are statistically indistinguishable (see `figs/mb2_transfer_time_compare.png`,
+`figs/mb2_transfer_bw_compare.png`). Mechanism: Mooncake's
+`batch_transfer_sync_write` always goes through the RDMA NIC, including
+the intra-node case (RDMA loopback). The 200 Gbps NIC, not NVLink, is
+the bottleneck. **Implication for §3.2**: PD-disaggregation does not
+get cheaper by co-locating P and D on the same node — the ~9.7 GB/s
+ceiling applies regardless. Halving the transfer cost cannot be bought
+back by topology.
+
+**What MB2 actually measures**: the **per-request charge** that
+PD-disagg pays for every routed request — `T_transfer ≈ KV_size / 9.7
+GB/s`. For agentic this is **8 ms (192 MiB / trace lower) – 1.9 s
+(11.5 GiB / p99)**.
+
+**⚠ Correction (2026-05-27)**: an earlier version of this README
+framed §3.2 as "transfer cost (1.5–10 s) >> decode duration (50–200 ms),
+so PD-disagg loses on cost-vs-benefit." That accounting was wrong:
+PD-disagg's phase-isolation benefit is **per-prefill-event** and equals
+`D × T_prefill` (aggregate across stalled decode streams), not the
+single-request decode duration. With trace-mean `T_prefill = 4.5 s` and
+D = 8, the benefit is ~36 s — far larger than the ~0.32 s transfer
+cost. PD-disagg's phase-isolation axis is a *win*, not a loss.
+
+The actual reason static PD-disagg fails in agentic is **D-side KV
+capacity** (`figs/f4b_pdsep_kv_wall.png`), not a cost-vs-benefit
+imbalance. See `RESULTS_SUMMARY.md` section 4 for the corrected
+framing. MB2 still serves as the source of the per-request transfer
+cost number used in that analysis.
+
+---
+
+## Setup
+
+| Component | Value |
+|---|---|
+| Host | `dash1` (`ds-6348bee4-1-...-rwkv2`), 8× NVIDIA H20 96 GiB, driver 570.133.20 |
+| Venv  | `/home/admin/cpfs/wjh/agentic-kv-fresh/.venv` (shared via cpfs from any dash host) |
+| vLLM | 0.18.1 official wheel |
+| mooncake-transfer-engine | 0.3.11.post1 (`pip install mooncake-transfer-engine`) |
+| Model | `/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct` |
+| Per-token KV | 98304 B |
+| kv_role | `kv_both` on both instances (see *Known limitations* re kv_producer/kv_consumer) |
+| Per-instance config | `--tensor-parallel-size 1 --gpu-memory-utilization 0.9 --max-model-len 200000 --enable-prefix-caching` |
+
+## Method
+
+3-step black-box bench:
+
+1. `do_remote_decode` to A (producer) with a client-generated `transfer_id`.
+   `max_tokens=1`; A computes prefill and parks the KV for later pull.
+2. `do_remote_prefill` to B (consumer) with the same `transfer_id` plus
+   `remote_engine_id` (from A's `/query` on bootstrap port) and
+   `remote_bootstrap_addr` (`http://127.0.0.1:8998`). **This step
+   triggers the actual KV transfer; it is the measured step.**
+3. Plain `completion` on B (`--skip-verify` off): expect `cached_tokens
+   ≈ prompt_len`, confirming the KV landed on B.
+
+Per-stage breakdown is obtained by instrumenting the vLLM-shipped
+`MooncakeConnector` (NOT the mooncake-package's `mooncake_connector_v1`,
+which vLLM 0.18.1 does not load) at two sites:
+
+- **`_send_blocks`** (P-side, line 980): emits `send_blocks` event with
+  `total_bytes`, `duration_s`, `t_start_unix`. The `duration_s` is the
+  wall-time of a single `batch_transfer_sync_write` call — **this is
+  what we call `pure_transfer`**.
+- **`receive_kv_from_single_worker`** (D-side, line 1139, async):
+  emits `receive_kv_enter` at function start and `receive_kv_finish`
+  on FINISH-status response. The wall-time between them is
+  **`rx_total`** (= ZMQ round-trip + setup + pure_transfer + ack).
+
+Pairing across A's and B's logs is by **time window**: each B
+(enter, finish) pair is matched to the A send_blocks whose
+`t_start_unix` falls in `[rx_t_start, rx_t_end]`. With single-request
+benchmarks this is unambiguous.
+
+Scripts:
+- `microbench/fresh_setup/start_vllm_pair.sh` — bring up pair + apply/revert patch
+- `microbench/fresh_setup/instrument_mooncake.py` — apply/revert MB2 patches
+- `microbench/fresh_setup/mb2_kv_transfer.py` — client (3-step bench loop)
+- `microbench/fresh_setup/analyze_mb2.py` — pair A/B events into per-size table
+- `microbench/fresh_setup/plot_mb2.py` — log-log time + bandwidth curves
+
+## Results — intra-node (2026-05-27, dash1 GPU 0+1, kv_both)
+
+Raw events: `A_intra_kvboth.jsonl`, `B_intra_kvboth.jsonl`.
+Joined + aggregated: `intra_kvboth_breakdown.json`.
+Figures: `figs/mb2_transfer_time_intra.png`, `figs/mb2_transfer_bw_intra.png`.
+
+| input_tokens | KV (MiB) | n | pure_ms p50 | pure_ms max | rx_total_ms | overhead_ms | BW p50 (GB/s) | BW max (GB/s) |
+|---:|---:|---:|---:|---:|---:|---:|---:|---:|
+|    512 |    48 | 5 |   5.3 |   5.6 |  12.2 |  3.3 | 9.40 |  9.53 |
+|   1024 |    96 | 5 |  10.4 |  10.5 |  11.9 |  1.5 | 9.68 |  9.72 |
+|   2048 |   192 | 5 |  20.6 |  21.0 |  22.5 |  1.8 | 9.75 |  9.78 |
+|   4096 |   384 | 5 |  41.5 |  41.7 |  43.5 |  2.0 | 9.71 |  9.72 |
+|   8192 |   768 | 5 |  83.7 |  84.4 |  86.2 |  2.2 | 9.62 |  9.69 |
+|  16384 |  1536 | 5 | 167.1 | 167.7 | 170.2 |  2.7 | 9.64 |  9.67 |
+|  32768 |  3072 | 5 | 320.9 | 322.1 | 425.2 | 20.5 | 10.04 | 10.09 |
+|  65536 |  6144 | 5 | **1895.1** | 2375.2 | 1586.1 | 69.6 | 3.40 | 9.68 |
+| 131072 | 12288 | 5 | **2835.1** | 8923.6 | 4362.5 | 91.4 | 4.54 | 9.67 |
+
+**Three regimes** in the data:
+
+1. **<= 3 GiB** — linear in size, bandwidth ≈ **9.7 GB/s steady**.
+2. **6 GiB ± a bit** — onset of variance: max bandwidth still 9.7 GB/s,
+   but p50 collapses to ~3.4 GB/s. Some runs achieve full speed; others
+   take 2–3 × longer.
+3. **12 GiB** — wide spread (min 1.5 s, max 10 s for the same 11.5 GiB
+   transfer). This is the agentic-p99 size region.
+
+The bandwidth ceiling of ~10 GB/s is well below H20's NVLink p2p
+(claimed ~900 GB/s in IB) — likely the transfer is **PCIe-staged
+through host memory** rather than NVLink direct. To confirm we would
+need `nvidia-smi topo -m` and `mooncake_transfer_engine_topology_dump`
+analysis; not done yet.
+
+## Known limitations of this measurement
+
+- **kv_both, not strict PD-disagg.** vLLM 0.18.1 with
+  `kv_role=kv_consumer` raises `AttributeError: 'MooncakeConnectorWorker'
+  object has no attribute 'bootstrap_server'` (the attribute is only
+  assigned inside `if not self.is_kv_consumer`). The transfer mechanics
+  are identical — same `batch_transfer_sync_write` — so the cost
+  measurement is comparable. The role gate only affects which request
+  types each instance *accepts*. §5.2 strict PD-disagg baseline will
+  need either to fix that bug or front the pair with a role-aware proxy.
+- **Single in-flight request.** All measurements here are serial.
+  Real PD-disagg will have many concurrent transfers; bandwidth
+  contention is not characterized.
+- **Intra-node only.** Inter-node RDMA path will be slower; not yet
+  measured.
+- **Sanity preamble events.** The raw logs include 6 events from
+  earlier sanity runs in addition to the 45-event sweep. `analyze_mb2.py`
+  treats them as additional samples (same sizes); the per-size
+  aggregates use all of them.
+
+## Implications for §3.2 PD-disagg argument
+
+For each PD-disagg-routed request, transfer wall-time is:
+
+```
+T_transfer(KV_size) ≈ KV_size / 9.7 GB/s   for KV_size ≤ 3 GiB
+                    ≈ 0.3 – 10 s            for KV_size in [3, 12] GiB
+```
+
+This is the **per-request transfer charge** of PD-disagg. It's a
+real cost, but in the context of phase-isolation accounting it is
+*small* compared to the benefit:
+
+| Prefill | T_prefill (MB1) | T_transfer (MB2) | Phase-isolation benefit at D=8 = D × T_prefill |
+|---:|---:|---:|---:|
+| 2k tok (trace lower) | 0.14 s | 8 ms | 1.1 s |
+| 33k tok (trace mean) | 4.5 s | 320 ms | 36 s |
+| 125k tok (~p99) | 57 s | 1.9 s | 456 s |
+
+On the phase-isolation axis alone, PD-disagg recovers two orders of
+magnitude more decode time than it pays in transfer. **It is NOT this
+axis that defeats static PD-disagg in agentic** — see colleague's
+4P+4D experiment (TTFT p50 62×, success rate 99.5% → 52%) which is
+driven by **D-side KV-pool overflow** on long-context requests
+(`figs/f4b_pdsep_kv_wall.png`), not by transfer latency.
+
+What MB2 contributes to the paper is therefore:
+- The **per-request transfer cost number** (used as the cost input
+  to the cost-benefit accounting above).
+- The empirical observation that **Mooncake's transfer cost is
+  topology-independent** — intra-node and inter-node both go through
+  the RDMA NIC and hit the same 9.7 GB/s ceiling. PD-disagg's
+  transfer cost does not get cheaper by co-locating P and D.
+
+The dominant §3.2 failure mode of static PD-disagg in agentic is
+**capacity**, not transfer cost. MB3 / MB4 / MB5 will quantify the
+remaining axes (D-pool occupancy, cache reuse degradation under PD
+routing, static-partition mismatch).
+
+## Open questions / next runs
+
+- **Inter-node RDMA**: dash1 ↔ dash2. Expected lower bandwidth (~5–15
+  GB/s); want to see if the 6 GiB-onset variance moves.
+- **Bandwidth ceiling investigation**: is the 9.7 GB/s ceiling PCIe (so
+  the connector is not using NVLink direct) or some internal limit? If
+  PCIe, can it be lifted with NVLink-direct mooncake config?
+- **Variance at 6+ GiB**: investigate. Maybe related to chunking
+  inside `batch_transfer_sync_write`, or GPU memory pressure when KV
+  approaches HBM ceiling.
+- **Concurrent transfers**: measure aggregate bandwidth when N
+  simultaneous transfers happen. PD-disagg in practice does this.
+- **Strict kv_producer/kv_consumer**: patch the bootstrap_server bug
+  or use a proxy; verify transfer time is unchanged.
+
+## Reproduction
+
+```bash
+# On dash machine with cpfs mount + ssh access:
+bash microbench/fresh_setup/install.sh        # once (idempotent)
+bash microbench/fresh_setup/deploy.sh dash1    # push scripts to cpfs
+
+# bring up pair (intra-node)
+ssh dash1 'GPU_A=0 GPU_B=1 bash /home/admin/cpfs/wjh/agentic-kv-fresh/scripts/start_vllm_pair.sh start'
+
+# sweep
+ssh dash1 'source /home/admin/cpfs/wjh/agentic-kv-fresh/.venv/bin/activate && \
+  python /home/admin/cpfs/wjh/agentic-kv-fresh/scripts/mb2_kv_transfer.py \
+    --sizes 512,1024,2048,4096,8192,16384,32768,65536,131072 \
+    --repeats 5 --label intra-kvboth \
+    --out /home/admin/cpfs/wjh/agentic-kv-fresh/mb2_results/intra_kvboth.json'
+
+# pull logs
+scp dash1:/home/admin/cpfs/wjh/agentic-kv-fresh/mb2_transfer_logs/A/.efc_*_mb2_transfer_pid*.jsonl \
+    analysis/mb2/A_intra_kvboth.jsonl
+scp dash1:/home/admin/cpfs/wjh/agentic-kv-fresh/mb2_transfer_logs/B/.efc_*_mb2_transfer_pid*.jsonl \
+    analysis/mb2/B_intra_kvboth.jsonl
+
+# analyze
+.venv/bin/python microbench/fresh_setup/analyze_mb2.py \
+  --a-log analysis/mb2/A_intra_kvboth.jsonl \
+  --b-log analysis/mb2/B_intra_kvboth.jsonl \
+  --out analysis/mb2/intra_kvboth_breakdown.json
+
+.venv/bin/python microbench/fresh_setup/plot_mb2.py \
+  --breakdown analysis/mb2/intra_kvboth_breakdown.json \
+  --out-time figs/mb2_transfer_time_intra.png \
+  --out-bw figs/mb2_transfer_bw_intra.png
+
+# tear down
+ssh dash1 'bash /home/admin/cpfs/wjh/agentic-kv-fresh/scripts/start_vllm_pair.sh stop'
+```
+
+## Run log
+
+### 2026-05-27 — intra-node, kv_both, dash1 GPU 0+1
+
+Sweep: `512, 1024, 2048, 4096, 8192, 16384, 32768, 65536, 131072` tokens
+× 5 repeats. Sanity preamble of `512, 2048, 8192` × 2 included in the
+raw logs (counted as additional samples for those sizes).
+
+Result table above. **9.7 GB/s steady-state up to 3 GiB**, variance
+opens at 6 GiB, p99 agentic-tail transfer 1.5 – 10 s.
+
+Committed as `de164e5`.
+
+### 2026-05-27 — inter-node, kv_both, dash1 GPU 0 → dash2 GPU 0
+
+Same sweep config. 200 Gbps RoCE between hosts (RTT ~0.2 ms ping).
+Producer A on dash1 GPU 0, consumer B on dash2 GPU 0.
+remote_bootstrap_addr=`http://172.27.123.142:8998` (dash1's internal IP).
+
+Raw events: `A_inter_kvboth.jsonl` (45 send_blocks + 6 sanity).
+B's receive_kv events are **missing** for this run — the
+`MB2_LOG_DIR` env var did not propagate from the start-script through
+vLLM's EngineCore subprocess on dash2 (visible via
+`cat /proc/$ENGINE_PID/environ` shows empty for dash2 but contains
+MB2_LOG_DIR for dash1 — bookmark for future investigation, likely
+spawn-vs-fork difference in vLLM's multiproc executor across hosts).
+Pure-transfer numbers below come from A's send_blocks alone; full
+rx_total breakdown not available for this run.
+
+Per-size pure-transfer (analyzed by `analyze_mb2_send_only.py`):
+
+| input_tokens | KV (MiB) | n | pure_ms p50 | min | max | BW p50 (GB/s) | BW max |
+|---:|---:|---:|---:|---:|---:|---:|---:|
+|    512 |    48 | 5 |   5.2 |   5.1 |  65.8 |  9.76 |  9.81 |
+|   1024 |    96 | 5 |  10.2 |  10.1 |  10.4 |  9.91 | 10.00 |
+|   2048 |   192 | 5 |  20.0 |  20.0 |  20.5 | 10.06 | 10.07 |
+|   4096 |   384 | 5 |  40.1 |  40.1 |  40.5 | 10.04 | 10.05 |
+|   8192 |   768 | 5 |  80.9 |  80.7 |  82.5 |  9.96 |  9.98 |
+|  16384 |  1536 | 5 | 161.8 | 161.7 | 164.8 |  9.96 |  9.96 |
+|  32768 |  3072 | 5 | 309.6 | 307.7 | 526.9 | 10.40 | 10.47 |
+|  65536 |  6144 | 5 | 1733.6 | 653.5 | 1921.2 | 3.72 | 9.86 |
+| 131072 | 12288 | 5 | 2818.4 | 1283.0 | 9158.6 | 4.57 | 10.04 |
+
+Side-by-side comparison with the 2026-05-27 intra-node run:
+
+| Size | intra p50 ms | inter p50 ms | gap | intra GB/s | inter GB/s |
+|---|---:|---:|---:|---:|---:|
+|   512 |    5.3 |    5.2 | −2% | 9.40 |  9.76 |
+|  1024 |   10.4 |   10.2 | −2% | 9.68 |  9.91 |
+|  2048 |   20.6 |   20.0 | −3% | 9.75 | 10.06 |
+|  4096 |   41.5 |   40.1 | −3% | 9.71 | 10.04 |
+|  8192 |   83.7 |   80.9 | −3% | 9.62 |  9.96 |
+| 16384 |  167.1 |  161.8 | −3% | 9.64 |  9.96 |
+| 32768 |  320.9 |  309.6 | −3% | 10.04 | 10.40 |
+| 65536 | 1895.1 | 1733.6 | −9% | 3.40 |  3.72 |
+|131072 | 2835.1 | 2818.4 | −1% | 4.54 |  4.57 |
+
+The two paths produce essentially the same numbers — **mooncake intra-
+node is not using NVLink**, it's going through RDMA-loopback on the
+local NIC and gets the same ~10 GB/s ceiling as cross-node RDMA. The
+6+ GiB variance regime is also identical between paths.
+
+Figures: `figs/mb2_transfer_time_inter.png`, `figs/mb2_transfer_bw_inter.png`,
+`figs/mb2_transfer_time_compare.png` (overlay), `figs/mb2_transfer_bw_compare.png`.
+
+This collapses the §3.2 narrative to a single number: **PD-disagg
+across this cluster costs ~9.7–10 GB/s of transfer bandwidth no matter
+how you place P and D** (within-node or across-node). For p99 agentic
+KV (11.5 GiB), that's 1.3–10 s of transfer; for 6 GiB it's 0.7–2 s.
+Decode is 50–200 ms. So PD-disagg's cost dominates regardless of layout.

analysis/mb2/inter_kvboth_breakdown.json

@@ -0,0 +1,112 @@
+{
+  "summary": [
+    {
+      "input_tokens": 512,
+      "kv_mib": 48.0,
+      "n": 5,
+      "pure_transfer_ms_mean": 17.29,
+      "pure_transfer_ms_p50": 5.16,
+      "pure_transfer_ms_min": 5.13,
+      "pure_transfer_ms_max": 65.8,
+      "throughput_gbps_mean": 7.95,
+      "throughput_gbps_p50": 9.76,
+      "throughput_gbps_max": 9.81
+    },
+    {
+      "input_tokens": 1024,
+      "kv_mib": 96.0,
+      "n": 5,
+      "pure_transfer_ms_mean": 10.18,
+      "pure_transfer_ms_p50": 10.15,
+      "pure_transfer_ms_min": 10.06,
+      "pure_transfer_ms_max": 10.37,
+      "throughput_gbps_mean": 9.89,
+      "throughput_gbps_p50": 9.91,
+      "throughput_gbps_max": 10.0
+    },
+    {
+      "input_tokens": 2048,
+      "kv_mib": 192.0,
+      "n": 5,
+      "pure_transfer_ms_mean": 20.14,
+      "pure_transfer_ms_p50": 20.01,
+      "pure_transfer_ms_min": 19.99,
+      "pure_transfer_ms_max": 20.51,
+      "throughput_gbps_mean": 10.0,
+      "throughput_gbps_p50": 10.06,
+      "throughput_gbps_max": 10.07
+    },
+    {
+      "input_tokens": 4096,
+      "kv_mib": 384.0,
+      "n": 5,
+      "pure_transfer_ms_mean": 40.19,
+      "pure_transfer_ms_p50": 40.12,
+      "pure_transfer_ms_min": 40.07,
+      "pure_transfer_ms_max": 40.5,
+      "throughput_gbps_mean": 10.02,
+      "throughput_gbps_p50": 10.04,
+      "throughput_gbps_max": 10.05
+    },
+    {
+      "input_tokens": 8192,
+      "kv_mib": 768.0,
+      "n": 5,
+      "pure_transfer_ms_mean": 81.32,
+      "pure_transfer_ms_p50": 80.86,
+      "pure_transfer_ms_min": 80.71,
+      "pure_transfer_ms_max": 82.5,
+      "throughput_gbps_mean": 9.9,
+      "throughput_gbps_p50": 9.96,
+      "throughput_gbps_max": 9.98
+    },
+    {
+      "input_tokens": 16384,
+      "kv_mib": 1536.0,
+      "n": 5,
+      "pure_transfer_ms_mean": 162.76,
+      "pure_transfer_ms_p50": 161.78,
+      "pure_transfer_ms_min": 161.66,
+      "pure_transfer_ms_max": 164.76,
+      "throughput_gbps_mean": 9.9,
+      "throughput_gbps_p50": 9.96,
+      "throughput_gbps_max": 9.96
+    },
+    {
+      "input_tokens": 32768,
+      "kv_mib": 3072.0,
+      "n": 5,
+      "pure_transfer_ms_mean": 395.6,
+      "pure_transfer_ms_p50": 309.61,
+      "pure_transfer_ms_min": 307.71,
+      "pure_transfer_ms_max": 526.86,
+      "throughput_gbps_mean": 8.71,
+      "throughput_gbps_p50": 10.4,
+      "throughput_gbps_max": 10.47
+    },
+    {
+      "input_tokens": 65536,
+      "kv_mib": 6144.0,
+      "n": 5,
+      "pure_transfer_ms_mean": 1427.92,
+      "pure_transfer_ms_p50": 1733.57,
+      "pure_transfer_ms_min": 653.52,
+      "pure_transfer_ms_max": 1921.19,
+      "throughput_gbps_mean": 5.45,
+      "throughput_gbps_p50": 3.72,
+      "throughput_gbps_max": 9.86
+    },
+    {
+      "input_tokens": 131072,
+      "kv_mib": 12288.0,
+      "n": 5,
+      "pure_transfer_ms_mean": 3953.36,
+      "pure_transfer_ms_p50": 2818.43,
+      "pure_transfer_ms_min": 1283.03,
+      "pure_transfer_ms_max": 9158.57,
+      "throughput_gbps_mean": 5.49,
+      "throughput_gbps_p50": 4.57,
+      "throughput_gbps_max": 10.04
+    }
+  ]
+}

analysis/mb2/inter_kvboth_client.json

@@ -0,0 +1,679 @@
+{
+  "model": "/home/admin/cpfs/wjh/models/Qwen/Qwen3-Coder-30B-A3B-Instruct",
+  "kv_bytes_per_token": 98304,
+  "src_host": "172.27.123.142",
+  "src_port": 8000,
+  "dst_host": "172.27.123.133",
+  "dst_port": 8001,
+  "config_label": "inter-kvboth",
+  "raw": [
+    {
+      "input_tokens": 512,
+      "session": "677ca02530774bae990d549d0591d336",
+      "t_step1_client_unix": 1779885642.2549865,
+      "t_step2_client_unix": 1779885642.3058398,
+      "t_step2_end_unix": 1779885642.3249059,
+      "t_prefill_s": 0.05082751903682947,
+      "t_transfer_s": 0.019030610972549766,
+      "t_followup_s": 0.01642513304250315,
+      "cached_followup": 496,
+      "pull_completion_tokens": 1,
+      "ok": true
+    },
+    {
+      "input_tokens": 512,
+      "session": "6758b8079675495a9a13a73db1199a71",
+      "t_step1_client_unix": 1779885642.3416288,
+      "t_step2_client_unix": 1779885642.3896646,
+      "t_step2_end_unix": 1779885642.40659,
+      "t_prefill_s": 0.04800783301470801,
+      "t_transfer_s": 0.016899197013117373,
+      "t_followup_s": 0.017115428985562176,
+      "cached_followup": 496,
+      "pull_completion_tokens": 1,
+      "ok": true
+    },
+    {
+      "input_tokens": 512,
+      "session": "e9da485a88cc40eb927e18c37298d815",
+      "t_step1_client_unix": 1779885642.4239926,
+      "t_step2_client_unix": 1779885642.4724562,
+      "t_step2_end_unix": 1779885642.492528,
+      "t_prefill_s": 0.04843967000488192,
+      "t_transfer_s": 0.02004897501319647,
+      "t_followup_s": 0.016289777995552868,
+      "cached_followup": 496,
+      "pull_completion_tokens": 1,
+      "ok": true
+    },
+    {
+      "input_tokens": 512,
+      "session": "a70db16a01b04409b04ef30e94082720",
+      "t_step1_client_unix": 1779885642.509085,
+      "t_step2_client_unix": 1779885642.5566719,
+      "t_step2_end_unix": 1779885642.577205,
+      "t_prefill_s": 0.047563806001562625,
+      "t_transfer_s": 0.020512140006758273,
+      "t_followup_s": 0.016482150997035205,
+      "cached_followup": 496,
+      "pull_completion_tokens": 1,
+      "ok": true
+    },
+    {
+      "input_tokens": 512,
+      "session": "d13cddd850904d4591b15c112ab32a5f",
+      "t_step1_client_unix": 1779885642.5939884,
+      "t_step2_client_unix": 1779885642.6417143,
+      "t_step2_end_unix": 1779885642.6617038,
+      "t_prefill_s": 0.04769411502638832,
+      "t_transfer_s": 0.019964047998655587,
+      "t_followup_s": 0.01518680399749428,
+      "cached_followup": 496,
+      "pull_completion_tokens": 1,
+      "ok": true
+    },
+    {
+      "input_tokens": 1024,
+      "session": "64d9b1335bd04698ab4f681fe4fe1e40",
+      "t_step1_client_unix": 1779885642.6773622,
+      "t_step2_client_unix": 1779885642.7512894,
+      "t_step2_end_unix": 1779885642.7811818,
+      "t_prefill_s": 0.07390057999873534,
+      "t_transfer_s": 0.029867444012779742,
+      "t_followup_s": 0.017206061049364507,
+      "cached_followup": 1008,
+      "pull_completion_tokens": 1,
+      "ok": true
+    },
+    {
+      "input_tokens": 1024,
+      "session": "0fc46cc27a844482b9eab2fef48a56d2",
+      "t_step1_client_unix": 1779885642.7988534,
+      "t_step2_client_unix": 1779885642.8733544,
+      "t_step2_end_unix": 1779885642.9005985,
+      "t_prefill_s": 0.074467666039709,
+      "t_transfer_s": 0.027220223972108215,
+      "t_followup_s": 0.0172900699544698,
+      "cached_followup": 1008,
+      "pull_completion_tokens": 1,
+      "ok": true
+    },
+    {
+      "input_tokens": 1024,
+      "session": "dac7cceb94c0465abce6bff73119de06",
+      "t_step1_client_unix": 1779885642.9183617,
+      "t_step2_client_unix": 1779885642.9924595,
+      "t_step2_end_unix": 1779885643.0189831,
+      "t_prefill_s": 0.07407327799592167,
+      "t_transfer_s": 0.026499781000893563,
+      "t_followup_s": 0.01622009096900001,
+      "cached_followup": 1008,
+      "pull_completion_tokens": 1,
+      "ok": true
+    },
+    {
+      "input_tokens": 1024,
+      "session": "4137a7ebecf148db8aa61249ef0d0100",
+      "t_step1_client_unix": 1779885643.0356703,
+      "t_step2_client_unix": 1779885643.109194,
+      "t_step2_end_unix": 1779885643.135519,
+      "t_prefill_s": 0.07349911500932649,
+      "t_transfer_s": 0.02630024799145758,
+      "t_followup_s": 0.01637468097032979,
+      "cached_followup": 1008,
+      "pull_completion_tokens": 1,
+      "ok": true
+    },
+    {
+      "input_tokens": 1024,
+      "session": "e28432915bf64c4891dc5157d0bebe41",
+      "t_step1_client_unix": 1779885643.1523507,
+      "t_step2_client_unix": 1779885643.2252986,
+      "t_step2_end_unix": 1779885643.2520146,
+      "t_prefill_s": 0.07292484800564125,
+      "t_transfer_s": 0.026693789986893535,
+      "t_followup_s": 0.016969401971437037,
+      "cached_followup": 1008,
+      "pull_completion_tokens": 1,
+      "ok": true
+    },
+    {
+      "input_tokens": 2048,
+      "session": "717a42513a2c48358ebfbb73ca07c3fd",
+      "t_step1_client_unix": 1779885643.269825,
+      "t_step2_client_unix": 1779885643.3983366,
+      "t_step2_end_unix": 1779885643.417481,
+      "t_prefill_s": 0.12848620803561062,
+      "t_transfer_s": 0.019112227018922567,
+      "t_followup_s": 0.017094306997023523,
+      "cached_followup": 2032,
+      "pull_completion_tokens": 1,
+      "ok": true
+    },
+    {
+      "input_tokens": 2048,
+      "session": "f2d1287f0e564a60a0d8476bb917ec9a",
+      "t_step1_client_unix": 1779885643.4353812,
+      "t_step2_client_unix": 1779885643.5640473,
+      "t_step2_end_unix": 1779885643.5819182,
+      "t_prefill_s": 0.12862860300811008,
+      "t_transfer_s": 0.01783871802035719,
+      "t_followup_s": 0.016443474043626338,
+      "cached_followup": 2032,
+      "pull_completion_tokens": 1,
+      "ok": true
+    },
+    {
+      "input_tokens": 2048,
+      "session": "cff5a22ae9c341d8acc455188ecc63b5",
+      "t_step1_client_unix": 1779885643.5992126,
+      "t_step2_client_unix": 1779885643.7292078,
+      "t_step2_end_unix": 1779885643.766638,
+      "t_prefill_s": 0.12995763396611437,
+      "t_transfer_s": 0.037401642999611795,
+      "t_followup_s": 0.018064863979816437,
+      "cached_followup": 2032,
+      "pull_completion_tokens": 1,
+      "ok": true
+    },
+    {
+      "input_tokens": 2048,
+      "session": "cc86ca16270b493fab68af1e889007af",
+      "t_step1_client_unix": 1779885643.7855458,
+      "t_step2_client_unix": 1779885643.9140093,
+      "t_step2_end_unix": 1779885643.9510815,
+      "t_prefill_s": 0.12843843002337962,
+      "t_transfer_s": 0.03704743104754016,
+      "t_followup_s": 0.017415895010344684,
+      "cached_followup": 2032,
+      "pull_completion_tokens": 1,
+      "ok": true
+    },
+    {
+      "input_tokens": 2048,
+      "session": "85f348460077474ab5c7f6db23005f48",
+      "t_step1_client_unix": 1779885643.9692879,
+      "t_step2_client_unix": 1779885644.0971973,
+      "t_step2_end_unix": 1779885644.1345108,
+      "t_prefill_s": 0.12788102397462353,
+      "t_transfer_s": 0.037287415005266666,
+      "t_followup_s": 0.01738263398874551,
+      "cached_followup": 2032,
+      "pull_completion_tokens": 1,
+      "ok": true
+    },
+    {
+      "input_tokens": 4096,
+      "session": "f715284b17af4bc29ccb689f21ac61b7",
+      "t_step1_client_unix": 1779885644.1534681,
+      "t_step2_client_unix": 1779885644.4121828,
+      "t_step2_end_unix": 1779885644.4760094,
+      "t_prefill_s": 0.25868210894986987,
+      "t_transfer_s": 0.06378706597024575,
+      "t_followup_s": 0.020318155991844833,
+      "cached_followup": 4080,
+      "pull_completion_tokens": 1,
+      "ok": true
+    },
+    {
+      "input_tokens": 4096,
+      "session": "b99bf61a90d54373a57af6a38d67581d",
+      "t_step1_client_unix": 1779885644.497922,
+      "t_step2_client_unix": 1779885644.7539463,
+      "t_step2_end_unix": 1779885644.8172574,
+      "t_prefill_s": 0.255985789000988,
+      "t_transfer_s": 0.06327042501652613,
+      "t_followup_s": 0.021031456999480724,
+      "cached_followup": 4080,
+      "pull_completion_tokens": 1,
+      "ok": true
+    },
+    {
+      "input_tokens": 4096,
+      "session": "69925f5b50f74c48800999908730d9ec",
+      "t_step1_client_unix": 1779885644.839903,
+      "t_step2_client_unix": 1779885645.0941288,
+      "t_step2_end_unix": 1779885645.1562943,
+      "t_prefill_s": 0.2541897820192389,
+      "t_transfer_s": 0.06213490100344643,
+      "t_followup_s": 0.021715508948545903,
+      "cached_followup": 4080,
+      "pull_completion_tokens": 1,
+      "ok": true
+    },
+    {
+      "input_tokens": 4096,
+      "session": "312743112bc84c0699a91e3110a63f03",
+      "t_step1_client_unix": 1779885645.1795704,
+      "t_step2_client_unix": 1779885645.4336376,
+      "t_step2_end_unix": 1779885645.4955726,
+      "t_prefill_s": 0.2540306959999725,
+      "t_transfer_s": 0.06190510099986568,
+      "t_followup_s": 0.020819986995775253,
+      "cached_followup": 4080,
+      "pull_completion_tokens": 1,
+      "ok": true
+    },
+    {
+      "input_tokens": 4096,
+      "session": "e5975bd25e1b4aa586cbfe9b68370de2",
+      "t_step1_client_unix": 1779885645.5179617,
+      "t_step2_client_unix": 1779885645.7724028,
+      "t_step2_end_unix": 1779885645.8355522,
+      "t_prefill_s": 0.2544059020001441,
+      "t_transfer_s": 0.06311210000421852,
+      "t_followup_s": 0.02235932502662763,
+      "cached_followup": 4080,
+      "pull_completion_tokens": 1,
+      "ok": true
+    },
+    {
+      "input_tokens": 8192,
+      "session": "f8989efa9261470e81c2891945b460a5",
+      "t_step1_client_unix": 1779885645.8609917,
+      "t_step2_client_unix": 1779885646.4364054,
+      "t_step2_end_unix": 1779885646.4636223,
+      "t_prefill_s": 0.575372110994067,
+      "t_transfer_s": 0.02718952501891181,
+      "t_followup_s": 0.024976509041152894,
+      "cached_followup": 8176,
+      "pull_completion_tokens": 1,
+      "ok": true
+    },
+    {
+      "input_tokens": 8192,
+      "session": "3a8632e8562c4cbcaa32b57ba23bbf68",
+      "t_step1_client_unix": 1779885646.4916308,
+      "t_step2_client_unix": 1779885647.0670328,
+      "t_step2_end_unix": 1779885647.0934644,
+      "t_prefill_s": 0.5753633370040916,
+      "t_transfer_s": 0.026399013004265726,
+      "t_followup_s": 0.027285586984362453,
+      "cached_followup": 8176,
+      "pull_completion_tokens": 1,
+      "ok": true
+    },
+    {
+      "input_tokens": 8192,
+      "session": "4d2cfc02defd492d89e90f652032f630",
+      "t_step1_client_unix": 1779885647.1237462,
+      "t_step2_client_unix": 1779885647.6985745,
+      "t_step2_end_unix": 1779885647.8096595,
+      "t_prefill_s": 0.574782071984373,
+      "t_transfer_s": 0.11104170099133626,
+      "t_followup_s": 0.02618999598780647,
+      "cached_followup": 8176,
+      "pull_completion_tokens": 1,
+      "ok": true
+    },
+    {
+      "input_tokens": 8192,
+      "session": "3485a775304947ada42d61cfd39b7520",
+      "t_step1_client_unix": 1779885647.8389955,
+      "t_step2_client_unix": 1779885648.4133987,
+      "t_step2_end_unix": 1779885648.5235367,
+      "t_prefill_s": 0.574362020008266,
+      "t_transfer_s": 0.11010575300315395,
+      "t_followup_s": 0.028998883964959532,
+      "cached_followup": 8176,
+      "pull_completion_tokens": 1,
+      "ok": true
+    },
+    {
+      "input_tokens": 8192,
+      "session": "6f87692fb5884094b7b141116ba739f8",
+      "t_step1_client_unix": 1779885648.5556417,
+      "t_step2_client_unix": 1779885649.1294918,
+      "t_step2_end_unix": 1779885649.2386518,
+      "t_prefill_s": 0.5738097460125573,
+      "t_transfer_s": 0.1091307660099119,
+      "t_followup_s": 0.025612210971303284,
+      "cached_followup": 8176,
+      "pull_completion_tokens": 1,
+      "ok": true
+    },
+    {
+      "input_tokens": 16384,
+      "session": "c4e007ebdcf244cf95a8ccb3ab59ea0f",
+      "t_step1_client_unix": 1779885649.270192,
+      "t_step2_client_unix": 1779885650.7972357,
+      "t_step2_end_unix": 1779885651.0219116,
+      "t_prefill_s": 1.5270042870542966,
+      "t_transfer_s": 0.22463428904302418,
+      "t_followup_s": 0.03729638300137594,
+      "cached_followup": 16368,
+      "pull_completion_tokens": 1,
+      "ok": true
+    },
+    {
+      "input_tokens": 16384,
+      "session": "6f34ab6672e14f41b7780323b30ce21e",
+      "t_step1_client_unix": 1779885651.0651574,
+      "t_step2_client_unix": 1779885652.5826244,
+      "t_step2_end_unix": 1779885652.7871268,
+      "t_prefill_s": 1.5174251759890467,
+      "t_transfer_s": 0.2044643560075201,
+      "t_followup_s": 0.03547715302556753,
+      "cached_followup": 16368,
+      "pull_completion_tokens": 1,
+      "ok": true
+    },
+    {
+      "input_tokens": 16384,
+      "session": "ca02bd5aa32544f2b1b09096e5f63c8b",
+      "t_step1_client_unix": 1779885652.8284712,
+      "t_step2_client_unix": 1779885654.3465412,
+      "t_step2_end_unix": 1779885654.5522587,
+      "t_prefill_s": 1.5180304029490799,
+      "t_transfer_s": 0.20567574101733044,
+      "t_followup_s": 0.03611186001216993,
+      "cached_followup": 16368,
+      "pull_completion_tokens": 1,
+      "ok": true
+    },
+    {
+      "input_tokens": 16384,
+      "session": "59047ab966574f72ab12693ad29b090f",
+      "t_step1_client_unix": 1779885654.5942106,
+      "t_step2_client_unix": 1779885656.1123874,
+      "t_step2_end_unix": 1779885656.3158703,
+      "t_prefill_s": 1.5181243289844133,
+      "t_transfer_s": 0.20344976399792358,
+      "t_followup_s": 0.035486461012624204,
+      "cached_followup": 16368,
+      "pull_completion_tokens": 1,
+      "ok": true
+    },
+    {
+      "input_tokens": 16384,
+      "session": "9a6e2228fd40414c8fd2b78bf399552c",
+      "t_step1_client_unix": 1779885656.357184,
+      "t_step2_client_unix": 1779885657.8739264,
+      "t_step2_end_unix": 1779885658.079835,
+      "t_prefill_s": 1.5166968539706431,
+      "t_transfer_s": 0.20585927896900102,
+      "t_followup_s": 0.03941280301660299,
+      "cached_followup": 16368,
+      "pull_completion_tokens": 1,
+      "ok": true
+    },
+    {
+      "input_tokens": 32768,
+      "session": "5cf9fb856b674c0a9237c06980287cb3",
+      "t_step1_client_unix": 1779885658.1312225,
+      "t_step2_client_unix": 1779885662.6780548,
+      "t_step2_end_unix": 1779885663.0500648,
+      "t_prefill_s": 4.54679255298106,
+      "t_transfer_s": 0.3719712099991739,
+      "t_followup_s": 0.05820785299874842,
+      "cached_followup": 32752,
+      "pull_completion_tokens": 1,
+      "ok": true
+    },
+    {
+      "input_tokens": 32768,
+      "session": "524f192c92834453bc48052bc6516126",
+      "t_step1_client_unix": 1779885663.1199641,
+      "t_step2_client_unix": 1779885667.6679316,
+      "t_step2_end_unix": 1779885668.047607,
+      "t_prefill_s": 4.547927976993378,
+      "t_transfer_s": 0.3796397229889408,
+      "t_followup_s": 0.058314190013334155,
+      "cached_followup": 32752,
+      "pull_completion_tokens": 1,
+      "ok": true
+    },
+    {
+      "input_tokens": 32768,
+      "session": "fb7aa2c99bf24febb83536c54e30a36d",
+      "t_step1_client_unix": 1779885668.117419,
+      "t_step2_client_unix": 1779885672.6644971,
+      "t_step2_end_unix": 1779885673.0345452,
+      "t_prefill_s": 4.547034470015205,
+      "t_transfer_s": 0.3700092720100656,
+      "t_followup_s": 0.05658108199713752,
+      "cached_followup": 32752,
+      "pull_completion_tokens": 1,
+      "ok": true
+    },
+    {
+      "input_tokens": 32768,
+      "session": "85612444c3224b65afb09d65a829b5cc",
+      "t_step1_client_unix": 1779885673.1025856,
+      "t_step2_client_unix": 1779885677.6487892,
+      "t_step2_end_unix": 1779885678.2856307,
+      "t_prefill_s": 4.546165039995685,
+      "t_transfer_s": 0.6368027949938551,
+      "t_followup_s": 0.05732736398931593,
+      "cached_followup": 32752,
+      "pull_completion_tokens": 1,
+      "ok": true
+    },
+    {
+      "input_tokens": 32768,
+      "session": "fe51fbacc4d544b3a500b099f852b19b",
+      "t_step1_client_unix": 1779885678.3544915,
+      "t_step2_client_unix": 1779885682.9011996,
+      "t_step2_end_unix": 1779885683.542081,
+      "t_prefill_s": 4.546669405011926,
+      "t_transfer_s": 0.6408427829737775,
+      "t_followup_s": 0.057976288022473454,
+      "cached_followup": 32752,
+      "pull_completion_tokens": 1,
+      "ok": true
+    },
+    {
+      "input_tokens": 65536,
+      "session": "9628b04d4d9e4a9fb26decccef84d32f",
+      "t_step1_client_unix": 1779885683.6233714,
+      "t_step2_client_unix": 1779885698.8039112,
+      "t_step2_end_unix": 1779885700.9075115,
+      "t_prefill_s": 15.18050062697148,
+      "t_transfer_s": 2.103562032978516,
+      "t_followup_s": 0.0921451430185698,
+      "cached_followup": 65520,
+      "pull_completion_tokens": 1,
+      "ok": true
+    },
+    {
+      "input_tokens": 65536,
+      "session": "b8bd7328d2274613b473e932db11081c",
+      "t_step1_client_unix": 1779885701.0228417,
+      "t_step2_client_unix": 1779885716.2263126,
+      "t_step2_end_unix": 1779885718.3809934,
+      "t_prefill_s": 15.200471161981113,
+      "t_transfer_s": 2.1520405350020155,
+      "t_followup_s": 0.11590708000585437,
+      "cached_followup": 65520,
+      "pull_completion_tokens": 1,
+      "ok": true
+    },
+    {
+      "input_tokens": 65536,
+      "session": "8ca1dfe708c945e59c194ee59b7de87d",
+      "t_step1_client_unix": 1779885718.5226529,
+      "t_step2_client_unix": 1779885733.706992,
+      "t_step2_end_unix": 1779885735.6409419,
+      "t_prefill_s": 15.184306180977728,
+      "t_transfer_s": 1.9339170460007153,
+      "t_followup_s": 0.09697375196265057,
+      "cached_followup": 65520,
+      "pull_completion_tokens": 1,
+      "ok": true
+    },
+    {
+      "input_tokens": 65536,
+      "session": "aa7555475c87427ab1a1d95277545655",
+      "t_step1_client_unix": 1779885735.7609437,
+      "t_step2_client_unix": 1779885750.9334345,
+      "t_step2_end_unix": 1779885752.0260077,
+      "t_prefill_s": 15.172441756993067,
+      "t_transfer_s": 1.09253996796906,
+      "t_followup_s": 0.08802600798662752,
+      "cached_followup": 65520,
+      "pull_completion_tokens": 1,
+      "ok": true
+    },
+    {
+      "input_tokens": 65536,
+      "session": "dece39eeadd645c7a8e563763f93c35f",
+      "t_step1_client_unix": 1779885752.1370633,
+      "t_step2_client_unix": 1779885767.3073487,
+      "t_step2_end_unix": 1779885768.0632908,
+      "t_prefill_s": 15.17024433100596,
+      "t_transfer_s": 0.7559080219944008,
+      "t_followup_s": 0.08695380098652095,
+      "cached_followup": 65520,
+      "pull_completion_tokens": 1,
+      "ok": true
+    },
+    {
+      "input_tokens": 131072,
+      "session": "98a339dad05a463e8ac1425413221d27",
+      "t_step1_client_unix": 1779885768.196416,
+      "t_step2_client_unix": 1779885822.9193504,
+      "t_step2_end_unix": 1779885824.3877244,
+      "t_prefill_s": 54.72287795698503,
+      "t_transfer_s": 1.4683381259674206,
+      "t_followup_s": 0.16380483901593834,
+      "cached_followup": 131056,
+      "pull_completion_tokens": 1,
+      "ok": true
+    },
+    {
+      "input_tokens": 131072,
+      "session": "4ce2151ad3d4493397bb77c7a62871e7",
+      "t_step1_client_unix": 1779885824.5981278,
+      "t_step2_client_unix": 1779885879.3268123,
+      "t_step2_end_unix": 1779885884.7123976,
+      "t_prefill_s": 54.72864995297277,
+      "t_transfer_s": 5.38555136800278,
+      "t_followup_s": 0.17249851900851354,
+      "cached_followup": 131056,
+      "pull_completion_tokens": 1,
+      "ok": true
+    },
+    {
+      "input_tokens": 131072,
+      "session": "6a214438dcf74f36bbefaaed3583c04c",
+      "t_step1_client_unix": 1779885884.9482107,
+      "t_step2_client_unix": 1779885939.6747935,
+      "t_step2_end_unix": 1779885949.2077906,
+      "t_prefill_s": 54.72654917498585,
+      "t_transfer_s": 9.532963149016723,
+      "t_followup_s": 0.1658564520184882,
+      "cached_followup": 131056,
+      "pull_completion_tokens": 1,
+      "ok": true
+    },
+    {
+      "input_tokens": 131072,
+      "session": "4cd33910256e43768a2f475650d7714c",
+      "t_step1_client_unix": 1779885949.4215908,
+      "t_step2_client_unix": 1779886004.145101,
+      "t_step2_end_unix": 1779886007.2428164,
+      "t_prefill_s": 54.72347703698324,
+      "t_transfer_s": 3.0976800689822994,
+      "t_followup_s": 0.16629640298197046,
+      "cached_followup": 131056,
+      "pull_completion_tokens": 1,
+      "ok": true
+    },
+    {
+      "input_tokens": 131072,
+      "session": "7e3626253b5d4fe29b3cec1e3598bb39",
+      "t_step1_client_unix": 1779886007.4549844,
+      "t_step2_client_unix": 1779886062.0564816,
+      "t_step2_end_unix": 1779886063.7279255,
+      "t_prefill_s": 54.60145856201416,
+      "t_transfer_s": 1.6714086790452711,
+      "t_followup_s": 0.1628041280200705,
+      "cached_followup": 131056,
+      "pull_completion_tokens": 1,
+      "ok": true
+    }
+  ],
+  "summary": [
+    {
+      "input_tokens": 512,
+      "n_ok": 5,
+      "transfer_s_mean": 0.019290994200855494,
+      "transfer_s_p50": 0.019964047998655587,
+      "transfer_s_p90": 0.020512140006758273,
+      "transfer_s_min": 0.016899197013117373,
+      "transfer_s_max": 0.020512140006758273
+    },
+    {
+      "input_tokens": 1024,
+      "n_ok": 5,
+      "transfer_s_mean": 0.027316297392826528,
+      "transfer_s_p50": 0.026693789986893535,
+      "transfer_s_p90": 0.029867444012779742,
+      "transfer_s_min": 0.02630024799145758,
+      "transfer_s_max": 0.029867444012779742
+    },
+    {
+      "input_tokens": 2048,
+      "n_ok": 5,
+      "transfer_s_mean": 0.029737486818339674,
+      "transfer_s_p50": 0.03704743104754016,
+      "transfer_s_p90": 0.037401642999611795,
+      "transfer_s_min": 0.01783871802035719,
+      "transfer_s_max": 0.037401642999611795
+    },
+    {
+      "input_tokens": 4096,
+      "n_ok": 5,
+      "transfer_s_mean": 0.0628419185988605,
+      "transfer_s_p50": 0.06311210000421852,
+      "transfer_s_p90": 0.06378706597024575,
+      "transfer_s_min": 0.06190510099986568,
+      "transfer_s_max": 0.06378706597024575
+    },
+    {
+      "input_tokens": 8192,
+      "n_ok": 5,
+      "transfer_s_mean": 0.07677335160551593,
+      "transfer_s_p50": 0.1091307660099119,
+      "transfer_s_p90": 0.11104170099133626,
+      "transfer_s_min": 0.026399013004265726,
+      "transfer_s_max": 0.11104170099133626
+    },
+    {
+      "input_tokens": 16384,
+      "n_ok": 5,
+      "transfer_s_mean": 0.20881668580695986,
+      "transfer_s_p50": 0.20567574101733044,
+      "transfer_s_p90": 0.22463428904302418,
+      "transfer_s_min": 0.20344976399792358,
+      "transfer_s_max": 0.22463428904302418
+    },
+    {
+      "input_tokens": 32768,
+      "n_ok": 5,
+      "transfer_s_mean": 0.4798531565931626,
+      "transfer_s_p50": 0.3796397229889408,
+      "transfer_s_p90": 0.6408427829737775,
+      "transfer_s_min": 0.3700092720100656,
+      "transfer_s_max": 0.6408427829737775
+    },
+    {
+      "input_tokens": 65536,
+      "n_ok": 5,
+      "transfer_s_mean": 1.6075935207889416,
+      "transfer_s_p50": 1.9339170460007153,
+      "transfer_s_p90": 2.1520405350020155,
+      "transfer_s_min": 0.7559080219944008,
+      "transfer_s_max": 2.1520405350020155
+    },
+    {
+      "input_tokens": 131072,
+      "n_ok": 5,
+      "transfer_s_mean": 4.2311882782028984,
+      "transfer_s_p50": 3.0976800689822994,
+      "transfer_s_p90": 9.532963149016723,
+      "transfer_s_min": 1.4683381259674206,
+      "transfer_s_max": 9.532963149016723
+    }
+  ]
+}

analysis/mb2/intra_kvboth_breakdown.json

@@ -0,0 +1,758 @@
+{
+  "rows": [
+    {
+      "input_tokens_est": 512,
+      "total_bytes": 50331648,
+      "pure_transfer_s": 0.023202952987048775,
+      "rx_total_s": 0.03333390498301014,
+      "rx_overhead_s": 0.010130951995961368,
+      "rx_t_start_unix": 1779879143.1678784,
+      "send_t_start_unix": 1779879143.174031,
+      "req_ids": [
+        "cmpl-ad00672f263a6643-0-9479211a"
+      ]
+    },
+    {
+      "input_tokens_est": 512,
+      "total_bytes": 50331648,
+      "pure_transfer_s": 0.005375694017857313,
+      "rx_total_s": 0.007019245007541031,
+      "rx_overhead_s": 0.0016435509896837175,
+      "rx_t_start_unix": 1779879143.2968972,
+      "send_t_start_unix": 1779879143.2982283,
+      "req_ids": [
+        "cmpl-ace77e2b02f9f141-0-b3c061bc"
+      ]
+    },
+    {
+      "input_tokens_est": 2048,
+      "total_bytes": 201326592,
+      "pure_transfer_s": 0.021170366962905973,
+      "rx_total_s": 0.02278437599306926,
+      "rx_overhead_s": 0.0016140090301632881,
+      "rx_t_start_unix": 1779879143.5146625,
+      "send_t_start_unix": 1779879143.5159554,
+      "req_ids": [
+        "cmpl-a4a2366879c68ded-0-8ac4098e"
+      ]
+    },
+    {
+      "input_tokens_est": 2048,
+      "total_bytes": 201326592,
+      "pure_transfer_s": 0.020726953051052988,
+      "rx_total_s": 0.022794076008722186,
+      "rx_overhead_s": 0.0020671229576691985,
+      "rx_t_start_unix": 1779879143.6958342,
+      "send_t_start_unix": 1779879143.6974514,
+      "req_ids": [
+        "cmpl-8690cafcace0d5e2-0-b89f33d2"
+      ]
+    },
+    {
+      "input_tokens_est": 8192,
+      "total_bytes": 805306368,
+      "pure_transfer_s": 0.08536655298667029,
+      "rx_total_s": 0.08753501297906041,
+      "rx_overhead_s": 0.002168459992390126,
+      "rx_t_start_unix": 1779879144.3279662,
+      "send_t_start_unix": 1779879144.3294952,
+      "req_ids": [
+        "cmpl-b087e2ec4cfa8eb7-0-b908f425"
+      ]
+    },
+    {
+      "input_tokens_est": 8192,
+      "total_bytes": 805306368,
+      "pure_transfer_s": 0.08367906499188393,
+      "rx_total_s": 0.0860149699728936,
+      "rx_overhead_s": 0.002335904981009662,
+      "rx_t_start_unix": 1779879145.040141,
+      "send_t_start_unix": 1779879145.0419943,
+      "req_ids": [
+        "cmpl-a115d16ff5575e08-0-9fa81984"
+      ]
+    },
+    {
+      "input_tokens_est": 16,
+      "total_bytes": 1572864,
+      "pure_transfer_s": 0.0004059679922647774,
+      "rx_total_s": 0.002459956973325461,
+      "rx_overhead_s": 0.0020539889810606837,
+      "rx_t_start_unix": 1779879221.7062025,
+      "send_t_start_unix": 1779879221.7078288,
+      "req_ids": [
+        "cmpl-9e585ed083951df5-0-b03f812b"
+      ]
+    },
+    {
+      "input_tokens_est": 16,
+      "total_bytes": 1572864,
+      "pure_transfer_s": 0.000346789020113647,
+      "rx_total_s": 0.0020201010047458112,
+      "rx_overhead_s": 0.0016733119846321642,
+      "rx_t_start_unix": 1779879221.7826598,
+      "send_t_start_unix": 1779879221.7838593,
+      "req_ids": [
+        "cmpl-9271d403c044eadd-0-9c3c4639"
+      ]
+    },
+    {
+      "input_tokens_est": 512,
+      "total_bytes": 50331648,
+      "pure_transfer_s": 0.005353622022084892,
+      "rx_total_s": 0.006836243963334709,
+      "rx_overhead_s": 0.0014826219412498176,
+      "rx_t_start_unix": 1779879221.859549,
+      "send_t_start_unix": 1779879221.8607252,
+      "req_ids": [
+        "cmpl-82a580cefd3e2440-0-a383c3c4"
+      ]
+    },
+    {
+      "input_tokens_est": 512,
+      "total_bytes": 50331648,
+      "pure_transfer_s": 0.005279594974126667,
+      "rx_total_s": 0.00694335694424808,
+      "rx_overhead_s": 0.0016637619701214135,
+      "rx_t_start_unix": 1779879221.9419758,
+      "send_t_start_unix": 1779879221.9432015,
+      "req_ids": [
+        "cmpl-a31cb4bc9e7f63d2-0-8f48aacd"
+      ]
+    },
+    {
+      "input_tokens_est": 512,
+      "total_bytes": 50331648,
+      "pure_transfer_s": 0.0053006180096417665,
+      "rx_total_s": 0.006697195000015199,
+      "rx_overhead_s": 0.0013965769903734326,
+      "rx_t_start_unix": 1779879222.0232244,
+      "send_t_start_unix": 1779879222.0243337,
+      "req_ids": [
+        "cmpl-a9dfc1a5b425d994-0-a0930098"
+      ]
+    },
+    {
+      "input_tokens_est": 1024,
+      "total_bytes": 100663296,
+      "pure_transfer_s": 0.010396577999927104,
+      "rx_total_s": 0.01183948403922841,
+      "rx_overhead_s": 0.001442906039301306,
+      "rx_t_start_unix": 1779879222.1297998,
+      "send_t_start_unix": 1779879222.130936,
+      "req_ids": [
+        "cmpl-9712857755af2efc-0-90b2dc9b"
+      ]
+    },
+    {
+      "input_tokens_est": 1024,
+      "total_bytes": 100663296,
+      "pure_transfer_s": 0.010438029014039785,
+      "rx_total_s": 0.01214482297655195,
+      "rx_overhead_s": 0.0017067939625121653,
+      "rx_t_start_unix": 1779879222.243023,
+      "send_t_start_unix": 1779879222.2442062,
+      "req_ids": [
+        "cmpl-b4f0a10dee65acbe-0-a3c132fc"
+      ]
+    },
+    {
+      "input_tokens_est": 1024,
+      "total_bytes": 100663296,
+      "pure_transfer_s": 0.010436972021125257,
+      "rx_total_s": 0.011961110983975232,
+      "rx_overhead_s": 0.0015241389628499746,
+      "rx_t_start_unix": 1779879222.3569698,
+      "send_t_start_unix": 1779879222.3581295,
+      "req_ids": [
+        "cmpl-b4c514b80b52a3f2-0-bcd24f8e"
+      ]
+    },
+    {
+      "input_tokens_est": 1024,
+      "total_bytes": 100663296,
+      "pure_transfer_s": 0.010396371013484895,
+      "rx_total_s": 0.011788576026447117,
+      "rx_overhead_s": 0.001392205012962222,
+      "rx_t_start_unix": 1779879222.4715128,
+      "send_t_start_unix": 1779879222.4725878,
+      "req_ids": [
+        "cmpl-ac7118d8090d181c-0-8af4adf0"
+      ]
+    },
+    {
+      "input_tokens_est": 1024,
+      "total_bytes": 100663296,
+      "pure_transfer_s": 0.010352785000577569,
+      "rx_total_s": 0.0118055299972184,
+      "rx_overhead_s": 0.0014527449966408312,
+      "rx_t_start_unix": 1779879222.5826046,
+      "send_t_start_unix": 1779879222.5837166,
+      "req_ids": [
+        "cmpl-85291bcb93aaf638-0-868db1a8"
+      ]
+    },
+    {
+      "input_tokens_est": 16,
+      "total_bytes": 1572864,
+      "pure_transfer_s": 0.00034007197245955467,
+      "rx_total_s": 0.0021119200391694903,
+      "rx_overhead_s": 0.0017718480667099357,
+      "rx_t_start_unix": 1779879222.750828,
+      "send_t_start_unix": 1779879222.7521152,
+      "req_ids": [
+        "cmpl-a448cf2e059ba0c9-0-a1360796"
+      ]
+    },
+    {
+      "input_tokens_est": 16,
+      "total_bytes": 1572864,
+      "pure_transfer_s": 0.00041691696969792247,
+      "rx_total_s": 0.0022232600022107363,
+      "rx_overhead_s": 0.0018063430325128138,
+      "rx_t_start_unix": 1779879222.913044,
+      "send_t_start_unix": 1779879222.9143836,
+      "req_ids": [
+        "cmpl-b486fd9e945a4658-0-8bb561cd"
+      ]
+    },
+    {
+      "input_tokens_est": 2048,
+      "total_bytes": 201326592,
+      "pure_transfer_s": 0.020633380976505578,
+      "rx_total_s": 0.022250515001360327,
+      "rx_overhead_s": 0.0016171340248547494,
+      "rx_t_start_unix": 1779879223.0765986,
+      "send_t_start_unix": 1779879223.0778644,
+      "req_ids": [
+        "cmpl-82da2bfe65f276c6-0-88d9a9a2"
+      ]
+    },
+    {
+      "input_tokens_est": 2048,
+      "total_bytes": 201326592,
+      "pure_transfer_s": 0.020639199996367097,
+      "rx_total_s": 0.022157608007546514,
+      "rx_overhead_s": 0.0015184080111794174,
+      "rx_t_start_unix": 1779879223.2591784,
+      "send_t_start_unix": 1779879223.2603853,
+      "req_ids": [
+        "cmpl-93bd777652eba5f3-0-9ec3d058"
+      ]
+    },
+    {
+      "input_tokens_est": 2048,
+      "total_bytes": 201326592,
+      "pure_transfer_s": 0.020575353992171586,
+      "rx_total_s": 0.022589912987314165,
+      "rx_overhead_s": 0.002014558995142579,
+      "rx_t_start_unix": 1779879223.4402068,
+      "send_t_start_unix": 1779879223.4418828,
+      "req_ids": [
+        "cmpl-81f950480a3cabf9-0-bbf8584f"
+      ]
+    },
+    {
+      "input_tokens_est": 4096,
+      "total_bytes": 402653184,
+      "pure_transfer_s": 0.041439525957684964,
+      "rx_total_s": 0.043345845013391227,
+      "rx_overhead_s": 0.0019063190557062626,
+      "rx_t_start_unix": 1779879223.7529812,
+      "send_t_start_unix": 1779879223.7544343,
+      "req_ids": [
+        "cmpl-b109ed06b5882659-0-8d14993c"
+      ]
+    },
+    {
+      "input_tokens_est": 4096,
+      "total_bytes": 402653184,
+      "pure_transfer_s": 0.04152030003024265,
+      "rx_total_s": 0.04341953102266416,
+      "rx_overhead_s": 0.0018992309924215078,
+      "rx_t_start_unix": 1779879224.0899644,
+      "send_t_start_unix": 1779879224.0914912,
+      "req_ids": [
+        "cmpl-8a57776c81d64b2c-0-ace8fb2b"
+      ]
+    },
+    {
+      "input_tokens_est": 4096,
+      "total_bytes": 402653184,
+      "pure_transfer_s": 0.04148670402355492,
+      "rx_total_s": 0.04336977802449837,
+      "rx_overhead_s": 0.0018830740009434521,
+      "rx_t_start_unix": 1779879224.424807,
+      "send_t_start_unix": 1779879224.4262393,
+      "req_ids": [
+        "cmpl-9b1a5dce18758450-0-b17b3649"
+      ]
+    },
+    {
+      "input_tokens_est": 4096,
+      "total_bytes": 402653184,
+      "pure_transfer_s": 0.04146742797456682,
+      "rx_total_s": 0.043769759009592235,
+      "rx_overhead_s": 0.002302331035025418,
+      "rx_t_start_unix": 1779879224.7599711,
+      "send_t_start_unix": 1779879224.7617002,
+      "req_ids": [
+        "cmpl-8c7d412b85f43ed7-0-9dea4add"
+      ]
+    },
+    {
+      "input_tokens_est": 4096,
+      "total_bytes": 402653184,
+      "pure_transfer_s": 0.04143296502297744,
+      "rx_total_s": 0.043612666020635515,
+      "rx_overhead_s": 0.002179700997658074,
+      "rx_t_start_unix": 1779879225.0962389,
+      "send_t_start_unix": 1779879225.0978234,
+      "req_ids": [
+        "cmpl-8860308db3f010a5-0-ad51eb46"
+      ]
+    },
+    {
+      "input_tokens_est": 16,
+      "total_bytes": 1572864,
+      "pure_transfer_s": 0.0003991159610450268,
+      "rx_total_s": 0.002386144013144076,
+      "rx_overhead_s": 0.001987028052099049,
+      "rx_t_start_unix": 1779879225.7592747,
+      "send_t_start_unix": 1779879225.760789,
+      "req_ids": [
+        "cmpl-86cca1a2b9427801-0-ba41ade7"
+      ]
+    },
+    {
+      "input_tokens_est": 16,
+      "total_bytes": 1572864,
+      "pure_transfer_s": 0.00041423802031204104,
+      "rx_total_s": 0.0023903060355223715,
+      "rx_overhead_s": 0.0019760680152103305,
+      "rx_t_start_unix": 1779879226.384918,
+      "send_t_start_unix": 1779879226.3864496,
+      "req_ids": [
+        "cmpl-a208c6d804293be7-0-94d265ab"
+      ]
+    },
+    {
+      "input_tokens_est": 8192,
+      "total_bytes": 805306368,
+      "pure_transfer_s": 0.08309489500243217,
+      "rx_total_s": 0.08524628396844491,
+      "rx_overhead_s": 0.002151388966012746,
+      "rx_t_start_unix": 1779879227.0092332,
+      "send_t_start_unix": 1779879227.0107942,
+      "req_ids": [
+        "cmpl-b53bea2317cc1211-0-8fcad8a8"
+      ]
+    },
+    {
+      "input_tokens_est": 8192,
+      "total_bytes": 805306368,
+      "pure_transfer_s": 0.08372796402545646,
+      "rx_total_s": 0.08596085698809475,
+      "rx_overhead_s": 0.0022328929626382887,
+      "rx_t_start_unix": 1779879227.7190688,
+      "send_t_start_unix": 1779879227.7207224,
+      "req_ids": [
+        "cmpl-9daf909593bbdf03-0-8fd7d50e"
+      ]
+    },
+    {
+      "input_tokens_est": 8192,
+      "total_bytes": 805306368,
+      "pure_transfer_s": 0.08398396399570629,
+      "rx_total_s": 0.0860762019874528,
+      "rx_overhead_s": 0.002092237991746515,
+      "rx_t_start_unix": 1779879228.4297745,
+      "send_t_start_unix": 1779879228.4314566,
+      "req_ids": [
+        "cmpl-9ef40f3b6d736128-0-8e8e1c30"
+      ]
+    },
+    {
+      "input_tokens_est": 16384,
+      "total_bytes": 1610612736,
+      "pure_transfer_s": 0.16950496198842302,
+      "rx_total_s": 0.1721468890318647,
+      "rx_overhead_s": 0.002641927043441683,
+      "rx_t_start_unix": 1779879230.131392,
+      "send_t_start_unix": 1779879230.1334376,
+      "req_ids": [
+        "cmpl-851e5d7e3e83d7ea-0-a66a5e0b"
+      ]
+    },
+    {
+      "input_tokens_est": 16384,
+      "total_bytes": 1610612736,
+      "pure_transfer_s": 0.16713789198547602,
+      "rx_total_s": 0.16974544001277536,
+      "rx_overhead_s": 0.0026075480272993445,
+      "rx_t_start_unix": 1779879231.896075,
+      "send_t_start_unix": 1779879231.8981037,
+      "req_ids": [
+        "cmpl-9be12af6a9ccccf5-0-af1230c7"
+      ]
+    },
+    {
+      "input_tokens_est": 16384,
+      "total_bytes": 1610612736,
+      "pure_transfer_s": 0.16713115200400352,
+      "rx_total_s": 0.16975757898762822,
+      "rx_overhead_s": 0.0026264269836246967,
+      "rx_t_start_unix": 1779879233.6589305,
+      "send_t_start_unix": 1779879233.6608078,
+      "req_ids": [
+        "cmpl-b61b9b237366297b-0-9832f0e3"
+      ]
+    },
+    {
+      "input_tokens_est": 16384,
+      "total_bytes": 1610612736,
+      "pure_transfer_s": 0.16709016199456528,
+      "rx_total_s": 0.1695251659839414,
+      "rx_overhead_s": 0.0024350039893761277,
+      "rx_t_start_unix": 1779879235.4181106,
+      "send_t_start_unix": 1779879235.419875,
+      "req_ids": [
+        "cmpl-bae0d0efe47ece8f-0-affbc685"
+      ]
+    },
+    {
+      "input_tokens_est": 16384,
+      "total_bytes": 1610612736,
+      "pure_transfer_s": 0.166486973001156,
+      "rx_total_s": 0.16962904302636161,
+      "rx_overhead_s": 0.003142070025205612,
+      "rx_t_start_unix": 1779879237.1803744,
+      "send_t_start_unix": 1779879237.1821773,
+      "req_ids": [
+        "cmpl-a34bc73c9cd2efc1-0-90d647fc"
+      ]
+    },
+    {
+      "input_tokens_est": 32768,
+      "total_bytes": 3221225472,
+      "pure_transfer_s": 0.31926770601421595,
+      "rx_total_s": 0.32203804596792907,
+      "rx_overhead_s": 0.002770339953713119,
+      "rx_t_start_unix": 1779879241.9859307,
+      "send_t_start_unix": 1779879241.9880297,
+      "req_ids": [
+        "cmpl-89a36c12ee6b0ff3-0-9fddbc0f"
+      ]
+    },
+    {
+      "input_tokens_est": 32768,
+      "total_bytes": 3221225472,
+      "pure_transfer_s": 0.3197040680097416,
+      "rx_total_s": 0.3227974839974195,
+      "rx_overhead_s": 0.003093415987677872,
+      "rx_t_start_unix": 1779879246.9755645,
+      "send_t_start_unix": 1779879246.9779432,
+      "req_ids": [
+        "cmpl-8d65512eb7e3c36c-0-8b23597c"
+      ]
+    },
+    {
+      "input_tokens_est": 32768,
+      "total_bytes": 3221225472,
+      "pure_transfer_s": 0.32088329299585894,
+      "rx_total_s": 0.3240378479822539,
+      "rx_overhead_s": 0.003154554986394942,
+      "rx_t_start_unix": 1779879251.9618897,
+      "send_t_start_unix": 1779879251.9643052,
+      "req_ids": [
+        "cmpl-a13c271ecbbca78b-0-b76a0370"
+      ]
+    },
+    {
+      "input_tokens_est": 32768,
+      "total_bytes": 3221225472,
+      "pure_transfer_s": 0.5439103110111319,
+      "rx_total_s": 0.5924434679909609,
+      "rx_overhead_s": 0.04853315697982907,
+      "rx_t_start_unix": 1779879256.9512377,
+      "send_t_start_unix": 1779879256.9989722,
+      "req_ids": [
+        "cmpl-bada04ec8c556aca-0-a263d637"
+      ]
+    },
+    {
+      "input_tokens_est": 32768,
+      "total_bytes": 3221225472,
+      "pure_transfer_s": 0.5193864739849232,
+      "rx_total_s": 0.5644763479940593,
+      "rx_overhead_s": 0.04508987400913611,
+      "rx_t_start_unix": 1779879262.2127163,
+      "send_t_start_unix": 1779879262.2562187,
+      "req_ids": [
+        "cmpl-9641a077022e6123-0-8c3c0975"
+      ]
+    },
+    {
+      "input_tokens_est": 65536,
+      "total_bytes": 6442450944,
+      "pure_transfer_s": 1.9844180009677075,
+      "rx_total_s": 2.0784930550144054,
+      "rx_overhead_s": 0.09407505404669791,
+      "rx_t_start_unix": 1779879278.1063075,
+      "send_t_start_unix": 1779879278.199048,
+      "req_ids": [
+        "cmpl-bb3a4e5084af8c3a-0-bdfa0931"
+      ]
+    },
+    {
+      "input_tokens_est": 65536,
+      "total_bytes": 6442450944,
+      "pure_transfer_s": 2.1099297259934247,
+      "rx_total_s": 2.2067435560165904,
+      "rx_overhead_s": 0.09681383002316579,
+      "rx_t_start_unix": 1779879295.600993,
+      "send_t_start_unix": 1779879295.6967168,
+      "req_ids": [
+        "cmpl-91b951f85c93a71b-0-8396bee5"
+      ]
+    },
+    {
+      "input_tokens_est": 65536,
+      "total_bytes": 6442450944,
+      "pure_transfer_s": 1.8950715209939517,
+      "rx_total_s": 1.9879729640088044,
+      "rx_overhead_s": 0.0929014430148527,
+      "rx_t_start_unix": 1779879313.2315958,
+      "send_t_start_unix": 1779879313.3236735,
+      "req_ids": [
+        "cmpl-81d236ecb6aadadf-0-ac184d51"
+      ]
+    },
+    {
+      "input_tokens_est": 65536,
+      "total_bytes": 6442450944,
+      "pure_transfer_s": 0.9277855920372531,
+      "rx_total_s": 0.9849357060156763,
+      "rx_overhead_s": 0.05715011397842318,
+      "rx_t_start_unix": 1779879330.6154163,
+      "send_t_start_unix": 1779879330.6715357,
+      "req_ids": [
+        "cmpl-a4c76c62b44c4295-0-b007a6ed"
+      ]
+    },
+    {
+      "input_tokens_est": 65536,
+      "total_bytes": 6442450944,
+      "pure_transfer_s": 0.6652462020283565,
+      "rx_total_s": 0.6725030990201049,
+      "rx_overhead_s": 0.007256896991748363,
+      "rx_t_start_unix": 1779879346.990221,
+      "send_t_start_unix": 1779879346.9950044,
+      "req_ids": [
+        "cmpl-a06d4b774a8af9a5-0-980e9d23"
+      ]
+    },
+    {
+      "input_tokens_est": 131072,
+      "total_bytes": 12884901888,
+      "pure_transfer_s": 1.3330365709844045,
+      "rx_total_s": 1.3384539679973386,
+      "rx_overhead_s": 0.005417397012934089,
+      "rx_t_start_unix": 1779879402.7123013,
+      "send_t_start_unix": 1779879402.7169023,
+      "req_ids": [
+        "cmpl-bf0d435e06e3349f-0-8507c933"
+      ]
+    },
+    {
+      "input_tokens_est": 131072,
+      "total_bytes": 12884901888,
+      "pure_transfer_s": 5.839069904992357,
+      "rx_total_s": 5.973284716019407,
+      "rx_overhead_s": 0.13421481102705002,
+      "rx_t_start_unix": 1779879458.9232886,
+      "send_t_start_unix": 1779879459.0566247,
+      "req_ids": [
+        "cmpl-9f87ae0fb0c7eec8-0-a8a1daea"
+      ]
+    },
+    {
+      "input_tokens_est": 131072,
+      "total_bytes": 12884901888,
+      "pure_transfer_s": 9.862486142024864,
+      "rx_total_s": 10.056511385017075,
+      "rx_overhead_s": 0.19402524299221113,
+      "rx_t_start_unix": 1779879519.7647448,
+      "send_t_start_unix": 1779879519.9567635,
+      "req_ids": [
+        "cmpl-a62e48e40e6c6ad7-0-acca9741"
+      ]
+    },
+    {
+      "input_tokens_est": 131072,
+      "total_bytes": 12884901888,
+      "pure_transfer_s": 2.8350498770014383,
+      "rx_total_s": 2.925714804965537,
+      "rx_overhead_s": 0.09066492796409875,
+      "rx_t_start_unix": 1779879584.888362,
+      "send_t_start_unix": 1779879584.9780834,
+      "req_ids": [
+        "cmpl-824479d53bab40e4-0-af951a11"
+      ]
+    },
+    {
+      "input_tokens_est": 131072,
+      "total_bytes": 12884901888,
+      "pure_transfer_s": 1.485496642999351,
+      "rx_total_s": 1.5183607729850337,
+      "rx_overhead_s": 0.032864129985682666,
+      "rx_t_start_unix": 1779879642.6076336,
+      "send_t_start_unix": 1779879642.639775,
+      "req_ids": [
+        "cmpl-9f06f19c981c0b3f-0-b3afb370"
+      ]
+    }
+  ],
+  "summary": [
+    {
+      "input_tokens": 16,
+      "kv_mib": 1.5,
+      "n": 6,
+      "pure_transfer_ms_mean": 0.39,
+      "pure_transfer_ms_p50": 0.4,
+      "pure_transfer_ms_max": 0.42,
+      "pure_transfer_ms_min": 0.34,
+      "rx_total_ms_mean": 2.27,
+      "rx_overhead_ms_mean": 1.88,
+      "throughput_gbps_mean": 4.09,
+      "throughput_gbps_p50": 3.91,
+      "throughput_gbps_max": 4.63
+    },
+    {
+      "input_tokens": 512,
+      "kv_mib": 48.0,
+      "n": 5,
+      "pure_transfer_ms_mean": 8.9,
+      "pure_transfer_ms_p50": 5.35,
+      "pure_transfer_ms_max": 23.2,
+      "pure_transfer_ms_min": 5.28,
+      "rx_total_ms_mean": 12.17,
+      "rx_overhead_ms_mean": 3.26,
+      "throughput_gbps_mean": 7.99,
+      "throughput_gbps_p50": 9.4,
+      "throughput_gbps_max": 9.53
+    },
+    {
+      "input_tokens": 1024,
+      "kv_mib": 96.0,
+      "n": 5,
+      "pure_transfer_ms_mean": 10.4,
+      "pure_transfer_ms_p50": 10.4,
+      "pure_transfer_ms_max": 10.44,
+      "pure_transfer_ms_min": 10.35,
+      "rx_total_ms_mean": 11.91,
+      "rx_overhead_ms_mean": 1.5,
+      "throughput_gbps_mean": 9.68,
+      "throughput_gbps_p50": 9.68,
+      "throughput_gbps_max": 9.72
+    },
+    {
+      "input_tokens": 2048,
+      "kv_mib": 192.0,
+      "n": 5,
+      "pure_transfer_ms_mean": 20.75,
+      "pure_transfer_ms_p50": 20.64,
+      "pure_transfer_ms_max": 21.17,
+      "pure_transfer_ms_min": 20.58,
+      "rx_total_ms_mean": 22.52,
+      "rx_overhead_ms_mean": 1.77,
+      "throughput_gbps_mean": 9.7,
+      "throughput_gbps_p50": 9.75,
+      "throughput_gbps_max": 9.78
+    },
+    {
+      "input_tokens": 4096,
+      "kv_mib": 384.0,
+      "n": 5,
+      "pure_transfer_ms_mean": 41.47,
+      "pure_transfer_ms_p50": 41.47,
+      "pure_transfer_ms_max": 41.52,
+      "pure_transfer_ms_min": 41.43,
+      "rx_total_ms_mean": 43.5,
+      "rx_overhead_ms_mean": 2.03,
+      "throughput_gbps_mean": 9.71,
+      "throughput_gbps_p50": 9.71,
+      "throughput_gbps_max": 9.72
+    },
+    {
+      "input_tokens": 8192,
+      "kv_mib": 768.0,
+      "n": 5,
+      "pure_transfer_ms_mean": 83.97,
+      "pure_transfer_ms_p50": 83.73,
+      "pure_transfer_ms_max": 85.37,
+      "pure_transfer_ms_min": 83.09,
+      "rx_total_ms_mean": 86.17,
+      "rx_overhead_ms_mean": 2.2,
+      "throughput_gbps_mean": 9.59,
+      "throughput_gbps_p50": 9.62,
+      "throughput_gbps_max": 9.69
+    },
+    {
+      "input_tokens": 16384,
+      "kv_mib": 1536.0,
+      "n": 5,
+      "pure_transfer_ms_mean": 167.47,
+      "pure_transfer_ms_p50": 167.13,
+      "pure_transfer_ms_max": 169.5,
+      "pure_transfer_ms_min": 166.49,
+      "rx_total_ms_mean": 170.16,
+      "rx_overhead_ms_mean": 2.69,
+      "throughput_gbps_mean": 9.62,
+      "throughput_gbps_p50": 9.64,
+      "throughput_gbps_max": 9.67
+    },
+    {
+      "input_tokens": 32768,
+      "kv_mib": 3072.0,
+      "n": 5,
+      "pure_transfer_ms_mean": 404.63,
+      "pure_transfer_ms_p50": 320.88,
+      "pure_transfer_ms_max": 543.91,
+      "pure_transfer_ms_min": 319.27,
+      "rx_total_ms_mean": 425.16,
+      "rx_overhead_ms_mean": 20.53,
+      "throughput_gbps_mean": 8.47,
+      "throughput_gbps_p50": 10.04,
+      "throughput_gbps_max": 10.09
+    },
+    {
+      "input_tokens": 65536,
+      "kv_mib": 6144.0,
+      "n": 5,
+      "pure_transfer_ms_mean": 1516.49,
+      "pure_transfer_ms_p50": 1895.07,
+      "pure_transfer_ms_max": 2109.93,
+      "pure_transfer_ms_min": 665.25,
+      "rx_total_ms_mean": 1586.13,
+      "rx_overhead_ms_mean": 69.64,
+      "throughput_gbps_mean": 5.27,
+      "throughput_gbps_p50": 3.4,
+      "throughput_gbps_max": 9.68
+    },
+    {
+      "input_tokens": 131072,
+      "kv_mib": 12288.0,
+      "n": 5,
+      "pure_transfer_ms_mean": 4271.03,
+      "pure_transfer_ms_p50": 2835.05,
+      "pure_transfer_ms_max": 9862.49,
+      "pure_transfer_ms_min": 1333.04,
+      "rx_total_ms_mean": 4362.47,
+      "rx_overhead_ms_mean": 91.44,
+      "throughput_gbps_mean": 5.28,
+      "throughput_gbps_p50": 4.54,
+      "throughput_gbps_max": 9.67
+    }
+  ]
+}

analysis/mb5_pd_ablation/README.md

@@ -0,0 +1,107 @@
+# PD-disagg vs colocation — controlled reuse & concurrency axes (v2)
+
+Self-contained results for the **controlled-variable** redo of the MB5 PD-vs-colo
+ablation. Supersedes the confounded first cut (held input fixed and sliced the
+prefix, so "more reuse" was entangled with "less prefill"). All arms route through
+the proxy at fair **APC parity** (session-routed producers reach the same prefix-cache
+hit rate as colo), so PD loses on *structure*, not on broken cache.
+
+- **Config arms:** `colo` = 8×kv_both (8C-proxy, session-affinity); PD = `6P+2D / 4P+4D / 2P+6D`.
+- **Driver:** closed-loop N (`REPLAY_MAX_INFLIGHT`) + fixed think-time; `gen_synthetic_trace.py --mode regular`.
+- **PD-arm wall-cap:** collapsed PD arms drain pathologically slowly, so PD arms run with a
+  wall-deadline (`REPLAY_MAX_DURATION`; un-run turns counted as failures → honest completion%);
+  **colo is uncapped** so the reference is always fully measured.
+- **Hardware:** run on **dash2** (8×H20). dash0's RDMA NICs were faulted for Mooncake during
+  this work (could not init the transfer engine; needs an admin reset — no sudo); dash2's NICs
+  are healthy. cpfs/venv/data are shared across the boxes.
+
+---
+
+## 1. Reuse / APC axis — fixed real prefill, vary cached prefix
+
+N=8. Hold the **real new-prefill work per turn constant** (`--delta-len`) and grow the cached
+prefix → reuse = prefix/(prefix+delta). Three shapes isolate output vs delta:
+
+| | delta (real prefill/turn) | output | role |
+|---|---|---|---|
+| **A** | 2048 | 256 | original |
+| **C** | 2048 | 128 | A vs C = pure **output** 256→128 |
+| **B** | 1024 | 128 | C vs B = pure **delta** 2048→1024 |
+
+**PD-best advantage** = colo E2E-p90 / best-PD E2E-p90 (>1 ⇒ PD wins):
+
+| reuse% | A d2048/o256 | C d2048/o128 | B d1024/o128 |
+|---|---|---|---|
+| 20 | 1.34 | 1.41 | — |
+| 50 | 1.36 | 1.37 | — |
+| 67 | **1.47** | **1.49** | **1.27** |
+| 80 | 1.31 | 1.23 | 1.25 |
+| 90 | 1.15 | 1.01 | — |
+| 95 | 0.87 | 0.89 | 0.89 |
+
+![reuse 3-way](../../figs/mb5_pd_ablation/reuse_compare_ABC.png)
+
+**Findings:**
+1. **Output length is ~negligible.** A and C (same delta) track each other across the whole
+   range — halving output barely moves PD's advantage.
+2. **Delta (real prefill/turn) is the dominant shape factor.** B (delta=1024) sits clearly
+   below A/C at mid reuse (67%: 1.27 vs ~1.48). More real prefill per turn → bigger PD win,
+   because PD-disagg's benefit is isolating prefill from decode — more prefill to isolate.
+3. **Crossover to colo at reuse ~90–95% is robust** across all three shapes: PD always loses
+   the high-reuse / large-resident-context corner (it must KV-transfer the whole resident
+   context every turn for a few hundred new tokens; colo keeps it local).
+
+*Caveat:* the clean, uncapped, 100%-completion comparison region is reuse **20–80%** (carries
+findings 1–2). At reuse 90/95% the PD arms collapse and C's points are capped-completion, while
+A/B are full-drain — comparable in direction, not in exact PD completion%.
+
+Data: `fig1_reuse_fixed.json` (A), `fig1_reuse_d2048_o128.json` (C), `fig1_reuse_d1024_o128.json` (B).
+
+---
+
+## 2. Concurrency axis — agentic corner, sweep N
+
+in=32768 (prefix 32256 + delta 512, **reuse 0.984**), out=128; closed-loop N ∈ {8,16,32,48,64,96,128};
+PD arms capped 600s, colo uncapped.
+
+| N | **colo** completion · E2E-mean · TPS | best PD-arm completion |
+|---|---|---|
+| 8 | **256/256** · 2.4s · 326 | 6P+2D 256/256 |
+| 16 | **512/512** · 3.5s · 462 | 6P+2D 439/512 (86%) |
+| 32 | **1024/1024** · 13.3s · 190 | all PD **<27%** |
+| 48 | **1536/1536** · 24.9s · 168 | all PD <32% |
+| 64 | **2048/2048** · 38.4s · 166 | all PD <31% |
+| 96 | **3072/3072** · 60.0s · 171 | PD **2–7%** |
+| 128 | **4096/4096** · 80.8s · 181 | 4P+4D 6%, 2P+6D <1% |
+
+![concurrency](../../figs/mb5_pd_ablation/fig3_concurrency_axis.png)
+
+**Finding:** **colo completes 100% of requests at every concurrency level** — it degrades
+*gracefully* (latency rises 2.4s→81s, nothing dropped). **Every static PD split collapses, and
+progressively earlier as N rises**: PD is viable only at N≤8–16; by N≥32 it drops 70–99% of
+requests while its prefix-cache hit-rate craters to ~0%. colo's elastic pool absorbs the
+time-varying P/D demand; the static partition + per-turn 32k KV-transfer cannot. (Latency
+percentiles count successes only, so they *understate* PD — read them with the completion column.)
+
+Data: `fig3_conc32k.json`.
+
+*Caveat:* N=128 6P+2D is missing (one transient vLLM/Mooncake startup flake at the end); does
+not change the picture (all PD arms are already collapsed by N=128). The SLO auto-stop in the
+driver is a no-op (a stdout-capture bug), so the full grid ran — more points, not fewer.
+
+---
+
+## 3. Reproduce
+
+```bash
+# on a box with healthy Mooncake/RDMA NICs (dash2), cpfs mounted:
+R=/home/admin/cpfs/wjh/agentic-kv-fresh
+# reuse axis (three shapes): DELTA/OL pick the shape; tag carries _o${OL}
+ssh dash2 "cd $R && DELTA=2048 OL=256 bash microbench/fresh_setup/run_reuse_fixed.sh"
+ssh dash2 "cd $R && DELTA=2048 OL=128 bash microbench/fresh_setup/run_reuse_fixed.sh"
+ssh dash2 "cd $R && DELTA=1024 OL=128 bash microbench/fresh_setup/run_reuse_fixed.sh"
+# concurrency axis (capped):
+ssh dash2 "cd $R && NLIST='8 16 32 48 64 96 128' CONC_PD_MAXDUR=600 bash microbench/fresh_setup/run_conc.sh"
+# render (reads the *.json in this dir):
+python microbench/fresh_setup/plot_pd_crossover.py
+```

analysis/migration_trigger_validation/README.md

@@ -0,0 +1,147 @@
+# Migration Trigger Validation (unified_v4) — 2026-05-30
+
+Hardware: dash2, 8×H20, Qwen3-Coder-30B-A3B, TP=1, kv_both + DR-fix substrate.
+Trace: `w600_r0.0015_st30_first600s.jsonl` (807 reqs, 600s span).
+Policy: `unified_v4` = unified hybrid routing + pending-prefill-queue-triggered
+session migration (commit `3a6bf5d` on `kzlin-dev` branch).
+
+## Research Question
+
+Does Pillar 2 (hot-triggered session migration) provide measurable benefit on
+top of Pillar 1 (affinity-default routing)?
+
+## Experiment Design
+
+| Arm | Policy | Substrate | Trace QPS |
+|---|---|---|---|
+| unified_1x | unified (affinity-only) | kv_both + DR-fix | ~1.3 (original) |
+| unified_v4_1x | unified_v4 (affinity + migration) | kv_both + DR-fix | ~1.3 |
+| unified_v4_2x | unified_v4 (affinity + migration) | kv_both + DR-fix | ~2.7 (2× compressed) |
+
+The 2× trace was generated by halving inter-request intervals:
+`ts_new = ts_min + (ts_orig - ts_min) / 2`.
+
+## Results
+
+### 1x QPS: unified vs unified_v4
+
+| Metric | unified | unified_v4 | Delta |
+|---|---:|---:|---:|
+| OK/total | 807/807 | 807/807 | — |
+| TTFT mean | 3.990s | 4.142s | +3.8% |
+| TTFT p50 | 0.719s | 0.711s | −1.0% |
+| TTFT p90 | 11.499s | 12.293s | +6.9% |
+| TPOT p90 | 0.024s | 0.022s | −9.3% |
+| E2E p50 | 2.265s | 2.293s | +1.2% |
+| E2E p90 | 24.507s | 23.955s | −2.3% |
+| **Migrations** | **0** | **0** | — |
+
+**Conclusion**: At 1x QPS (~1.3 req/s, ~0.16 req/instance/s), the migration
+trigger NEVER fires. The two arms produce statistically identical results.
+
+### 2x QPS: unified_v4 under higher load
+
+| Metric | unified_v4 @ 2x |
+|---|---:|
+| OK/total | 807/807 |
+| TTFT mean | 5.227s |
+| TTFT p90 | 15.000s |
+| E2E p90 | 39.401s |
+| **Migrations** | **4/807 (0.5%)** |
+
+4 migrated requests (all verified via `v3_decode_target_url` in breakdown):
+
+| Session | Input | new_local | src_pp | fleet_median | proj_prefill | Target |
+|---|---:|---:|---:|---:|---:|---|
+| 1313181 | 22,686 | 22,686 | 13,360 | 6,680 | 5.1s | inst_5 |
+| 1310590 | 32,440 | 14,520 | 57,051 | 12,630 | 10.2s | inst_4 |
+| 1373431 | 126,340 | 126,340 | 73,385 | 33,294 | 28.5s | inst_4 |
+| 1313181 | 60,004 | 17,508 | 19,503 | 3,806 | 5.3s | inst_5 |
+
+## Root Cause Analysis: Why Zero Migrations at 1x
+
+The unified_v4 trigger requires BOTH arms to fire simultaneously:
+- **Absolute SLO arm**: `proj_prefill_s(src) > 2.5s` — fires for 41% of eligible requests
+- **Relative arm**: `src_pending_prefill > fleet_median × 1.5` — NEVER fires at 1x
+
+The relative arm fails because `pending_prefill_tokens` (the proxy's shadow
+counter) is 0 for **95% of routing decisions** at 1x QPS:
+
+| QPS | src_pp > 0 (% of eligible) | Migrations |
+|---:|---:|---:|
+| 1.3 (1×) | 5% (8/241) | 0 |
+| 2.7 (2×) | 24% (62/257) | 4 |
+
+**Mechanism**: `pending_prefill_tokens` reflects previously-dispatched requests
+that haven't finished their prefill yet. At 0.16 req/instance/s, each instance
+completes its prefill before the next request arrives — the counter is almost
+always 0 at decision time. Only under genuine queueing pressure (2× and above)
+does the counter become non-zero and the relative arm can fire.
+
+The high TTFT at 1x (~11.5s p90) comes from **compute-bound large prefills**
+(single 60k+ token requests inherently need ~9s), NOT from queue depth.
+
+## Interpretation for Paper
+
+1. **The migration mechanism is functionally correct.** At 2x it fires on the
+   right signal (src genuinely overloaded relative to fleet) and selects valid
+   targets (cooler instances with load gap).
+
+2. **At benchmark scale (8 instances, ~1 QPS), migration is not needed.** The
+   affinity-default routing (Pillar 1) already achieves APC ~79% and the
+   remaining hot-pin issue is mild (max-worker/median-worker ≈ 3.7×). The
+   "dispatch coupling" feedback loop is present but not yet at the catastrophic
+   amplification regime.
+
+3. **Migration becomes relevant under scale-out + higher load.** With more
+   instances (16–32), session skew concentrates more load per hot instance
+   while cold instances sit idle — exactly the condition where `src_pp >
+   fleet_median × 1.5` naturally fires. The 1x→2x progression (0%→0.5%
+   migration rate) shows the correct scaling direction.
+
+4. **Paper §3.3 framing**: Migration is a **scale-out insurance mechanism**
+   that gracefully degrades to no-op under low load. Its value is NOT
+   demonstrable at 8-instance single-node benchmark; the argument must rely on
+   (a) the mechanism's correctness (this experiment), (b) the substrate's
+   net-positive property (commit `ef9e010`), and (c) scale-out projection
+   (future: 16+ GPU, multi-node).
+
+## Next Steps
+
+- [ ] **Scale-out validation** (16 GPU, 2 nodes): With more instances and the
+  same trace, more sessions compete per-instance → higher pending_prefill →
+  migration triggers naturally. This is the strongest evidence path.
+- [ ] **3–4× QPS on 8 instances**: Push to saturation to measure migration's
+  effect in the catastrophic regime. Risk: may exceed serving capacity (errors).
+- [ ] **Threshold sensitivity**: Ablate `v4_rel_hi` (1.5→1.2→1.0) and
+  `v4_ttft_slo_s` (2.5→1.5→1.0) to characterize the trigger landscape.
+
+## Reproduction
+
+```bash
+# On dash2 (local /tmp, does NOT modify shared NAS):
+# 1x QPS
+bash /tmp/migration_exp/run_migration_ab.sh   # interleaved unified vs unified_v4
+
+# 2x QPS
+python3 -c "
+import json
+trace_in = '/home/admin/cpfs/wjh/agentic-kv/traces/w600_r0.0015_st30_first600s.jsonl'
+rows = [json.loads(l) for l in open(trace_in)]
+ts_min = min(r['timestamp'] for r in rows)
+for r in rows: r['timestamp'] = ts_min + (r['timestamp'] - ts_min) / 2.0
+with open('/tmp/migration_exp/trace_2x.jsonl', 'w') as f:
+    for r in rows: f.write(json.dumps(r) + '\n')
+"
+bash /tmp/migration_exp/run_2x.sh
+```
+
+## Data Locations (dash2 /tmp, ephemeral)
+
+| Path | Content |
+|---|---|
+| `/tmp/migration_exp/outputs/unified_run1/` | Baseline arm (1x) |
+| `/tmp/migration_exp/outputs/unified_v4_run1/` | Migration arm (1x) |
+| `/tmp/migration_exp/outputs/unified_v4_2x/` | Migration arm (2x) |
+| `/tmp/migration_exp/outputs/*/breakdown.json` | Per-request routing decisions with v4_* fields |
+| `/tmp/migration_exp/outputs/*/metrics.jsonl` | Per-request latency metrics |

analysis/migration_trigger_validation/results.json

@@ -0,0 +1,110 @@
+{
+  "experiment": "migration_trigger_validation",
+  "date": "2026-05-30",
+  "hardware": "dash2, 8xH20, Qwen3-Coder-30B-A3B, TP=1",
+  "substrate": "kv_both + DR-fix (delay_free_blocks + VLLM_EVICT_SENT_BLOCKS gate)",
+  "arms": {
+    "unified_1x": {
+      "metrics": {
+        "ok": 807,
+        "total": 807,
+        "ttft_mean": 3.99,
+        "ttft_p50": 0.719,
+        "ttft_p90": 11.499,
+        "ttft_p99": 45.982,
+        "tpot_p90": 0.0239,
+        "e2e_p50": 2.265,
+        "e2e_p90": 24.507,
+        "e2e_p99": 71.233
+      },
+      "migrations": 0
+    },
+    "unified_v4_1x": {
+      "metrics": {
+        "ok": 807,
+        "total": 807,
+        "ttft_mean": 4.142,
+        "ttft_p50": 0.711,
+        "ttft_p90": 12.293,
+        "ttft_p99": 46.148,
+        "tpot_p90": 0.0217,
+        "e2e_p50": 2.293,
+        "e2e_p90": 23.955,
+        "e2e_p99": 75.915
+      },
+      "trigger_summary": {
+        "trace": "w600_r0.0015_st30_first600s.jsonl",
+        "qps_factor": 1,
+        "total_requests": 807,
+        "migrations_triggered": 13,
+        "size_floor_filtered": 552,
+        "eligible_requests": 255,
+        "slo_arm_true": 100,
+        "src_pp_nonzero": 22
+      }
+    },
+    "unified_v4_2x": {
+      "metrics": {
+        "ok": 807,
+        "total": 807,
+        "ttft_mean": 5.227,
+        "ttft_p50": 0.942,
+        "ttft_p90": 15.0,
+        "ttft_p99": 59.227,
+        "tpot_p90": 0.1087,
+        "e2e_p50": 5.035,
+        "e2e_p90": 39.401,
+        "e2e_p99": 163.032
+      },
+      "trigger_summary": {
+        "trace": "trace_2x.jsonl (timestamps / 2)",
+        "qps_factor": 2,
+        "total_requests": 807,
+        "migrations_triggered": 4,
+        "size_floor_filtered": 550,
+        "eligible_requests": 257,
+        "slo_arm_true": 133,
+        "src_pp_nonzero": 62,
+        "pending_prefill_p90_when_nonzero": 68131,
+        "migrated_details": [
+          {
+            "session_id": "1313181",
+            "input_length": 22686,
+            "new_local": 22686,
+            "src_pending_prefill": 13360,
+            "fleet_median_pp": 6680.0,
+            "proj_prefill_s": 5.15,
+            "target_idx": 5
+          },
+          {
+            "session_id": "1310590",
+            "input_length": 32440,
+            "new_local": 14520,
+            "src_pending_prefill": 57051,
+            "fleet_median_pp": 12630.5,
+            "proj_prefill_s": 10.22,
+            "target_idx": 4
+          },
+          {
+            "session_id": "1373431",
+            "input_length": 126340,
+            "new_local": 126340,
+            "src_pending_prefill": 73385,
+            "fleet_median_pp": 33294.5,
+            "proj_prefill_s": 28.53,
+            "target_idx": 4
+          },
+          {
+            "session_id": "1313181",
+            "input_length": 60004,
+            "new_local": 17508,
+            "src_pending_prefill": 19503,
+            "fleet_median_pp": 3806.5,
+            "proj_prefill_s": 5.29,
+            "target_idx": 5
+          }
+        ]
+      }
+    }
+  }
+}

analysis/pd_separation_analysis.md

@@ -23,6 +23,22 @@ Per-request breakdown shows **87.7% of TTFT** is spent waiting for KV cache memo
 > Earlier cross-machine comparison (commit `1e86285`) was invalidated — baseline used warm instances. See REPORT.md §3.5.
 | **Delta** | **-45%** | **-36%** | **-44%** | **+30pp** |
 
+> ✅⚠️ **2026-05-30 — confirmed + refined by the clean MB5 ablation; one caveat.**
+> A producer-side contamination (`e13391e`: evicts a producer's prefix-cache on every
+> KV transfer) was found in the *agentic-kv-fresh* MB5 stack and gated off; everything
+> was re-run clean.
+> - **Confirmed:** this doc's central thesis — PD's failure is a **decode-side KV memory
+>   wall**, not transfer/prefill cost — is reproduced on the clean stack (concurrency
+>   axis: at N=64 the split collapses, APC 71%→1.4%, TPS −30%; colo scales). Fig 3.
+> - **Refined:** "PD separation is net negative" is **regime-dependent**, not universal.
+>   Clean ablation shows PD *wins* at low load / decode-heavy / low-reuse and loses the
+>   **agentic corner** (high reuse + short output + large context + high concurrency).
+> - **Caveat (cross-check):** if this study's runs used the fork vLLM that contains
+>   `e13391e`, any **producer prefix-cache / APC reuse** figures here (e.g. §5.3.1) may be
+>   understated. The decode-memory-wall result is *not* reuse-dependent and is unaffected.
+>   On the clean stack, session-routed producers reach APC parity with colo (71–82%).
+> Figures: [`figs/mb5_pd_ablation/`](../figs/mb5_pd_ablation/).
+
 ---
 
 ## 1. Workload Characterization

analysis/v2/PD_DISAGG_LMETRIC.md

@@ -0,0 +1,165 @@
+# PD-colo vs PD-disagg on the real agentic trace — LMetric (cache-aware) clean-stack anchor
+
+**Figure:** [`figs/v2/fig4_lmetric_pd_vs_colo.png`](../../figs/v2/fig4_lmetric_pd_vs_colo.png)
+**Data:**   [`analysis/v2/fig4l_lmetric.json`](fig4l_lmetric.json)
+**Date:**   2026-05-31 · Hardware: dash1, 8×H20 · Model: Qwen3-Coder-30B-A3B-Instruct
+· vLLM 0.18.1 (V1, chunked-prefill on, `e13391e` eviction gated **off**)
+· Mooncake 0.3.11 · Routing: cache-aware proxy with **`--policy lmetric`**
+· Replayer per-request timeout 600 s.
+
+## TL;DR
+
+On the production agentic trace with the *right* routing baseline (LMetric, cache-aware),
+**PD-colo (8× kv_both) keeps 100 % completion on both traces** and matches the daily-bench
+expectation (~17 min for the high-load first600s, ~50 min for the full trace, with E2E p50
+~3 s and TTFT p50 ~1 s — **3.5–7× better than the original §3 round-robin baseline at the
+same wall-clock**). Every static **PD-disagg ratio fails** (14–65 % completion), and the
+failure mode rotates predictably with the split — **no static partition has a working
+operating point on this workload**. LMetric improves colo dramatically; it does *not*
+rescue PD-disagg, confirming the bottleneck is structural (D-pool admission capacity +
+multi-turn KV accumulation), not routing. A follow-up linear-policy run with PD-disagg
+**wall-capped at 2× the colo wall** (see end of doc) hits the **identical** success-rate
+ceiling — confirming the cap is structural, not policy-driven.
+
+## Setup
+
+- Trace: `w600_r0.0015_st30.jsonl` (1214 reqs, 274 sessions, agentic multi-turn,
+  contexts up to ~112 k tokens; "first600s" variant = same heavy sessions compressed
+  into 600 s → 807 reqs at 3.2× higher arrival rate).
+- 8 instances on 8 GPUs.
+- `--mode baseline` for colo (plain vLLM); `--mode pdsep --pd-ratio P:D` for the three PD
+  splits, all with Mooncake KV transfer.
+- Cache-aware proxy with LMetric scoring (`P_tokens × num_requests`) + session affinity
+  for multi-turn (the colleague's canonical baseline).
+
+## Results
+
+### first600s (1.35 req/s, high-load stress)
+
+| arm | success | E2E mean / p50 / p90 / p99 | TTFT p90 | TPOT p99 | TPS | wall |
+|---|---|---|---|---|---|---|
+| **colo (8C)** | **807/807 = 100 %** | 11.1 / 3.27 / 28.6 / 95.9 s | 14.5 s | 388 ms | 226 | 17.0 min |
+| pd6 (6:2) | 474/807 = **58.7 %** | 83.2 / 6.75 / 382 / 542 s | 380 s | 19 ms | 40 | 55 min |
+| pd4 (4:4) | 348/807 = **43.1 %** | 203 / 215 / 477 / 575 s | 475 s | 25 ms | 15 | 114 min |
+| pd2 (2:6) | 180/807 = **22.3 %** | 380 / 536 / 579 / 602 s | 577 s | 18 ms | 34 | 321 min* |
+
+### Full trace (0.42 req/s, original §3 anchor load)
+
+| arm | success | E2E mean / p50 / p90 / p99 | TTFT p90 | TPOT p99 | TPS | wall |
+|---|---|---|---|---|---|---|
+| **colo (8C)** | **1214/1214 = 100 %** | 10.9 / 3.13 / 29.6 / 93.7 s | 16.9 s | 254 ms | 125 | 49.9 min |
+| pd6 (6:2) | 793/1214 = **65.3 %** | 61.9 / 3.70 / 307 / 477 s | 300 s | 18 ms | 46 | 94 min |
+| pd4 (4:4) | 533/1214 = **43.9 %** | 131 / 8.22 / 468 / 531 s | 467 s | 21 ms | 13 | 231 min |
+| pd2 (2:6) | 169/1214 = **13.9 %** | 195 / 6.82 / 552 / 593 s | 549 s | 13 ms | 1 | 563 min |
+
+\* The pd2 wall-clock is dominated by per-request timeouts (`request_timeout=600 s`)
+draining concurrently behind the multi-turn session causality.
+
+## Five clean findings
+
+1. **LMetric+colo is the right baseline.** Full-trace colo wall **49.9 min ≈ the original
+   §3 RR's 49.9 min**, but E2E p50 **3.13 s vs §3's 10.8 s (3.5×)** and TTFT p50
+   **1.02 s vs §3's 7.0 s (7×)**. Same throughput envelope, far better latency — by virtue
+   of cache-aware routing concentrating each session's turns onto one instance for
+   prefix-cache reuse. The original §3 RR was an *unfairly weak* colo baseline.
+
+2. **Every static PD-disagg ratio fails on the agentic workload.** Completion drops to
+   14–65 %, on *both* traces. The drop is not a high-load artifact (it holds at the
+   original §3 arrival rate of 0.42 req/s); it is structural.
+
+3. **Failure mode rotates predictably with the P:D split:**
+   - **pd2 (2 producers)** → prefill-bound → 78–86 % TTFT timeouts.
+   - **pd6 (2 decode)** → decode-admission-bound → 35–41 % TTFT timeouts.
+   - **pd4 (4P+4D)** → both bottlenecks hit → 57 % TTFT timeouts.
+   - **No static ratio works.** Colo's elastic 8-GPU pool absorbs whichever phase is
+     hot at the moment.
+
+4. **Decode isolation works, but doesn't matter under failure.** TPOT p99 on every PD
+   arm is **13–25 ms** — an order of magnitude better than colo's 254–388 ms — but the
+   win applies only to the 14–65 % of requests that get admitted. The other 35–86 %
+   time out before ever seeing a first token, so the TPOT win is invisible to the end user.
+
+5. **The §3 RR "100 % PD completion" was a measurement artifact.** Original §3
+   (contaminated stack, RR routing) reported 100 % completion for pd6/pd4. LMetric on
+   the clean stack shows 44–65 %. Most plausible mechanism: `e13391e`'s eviction of
+   producer KV on every transfer acted as a **relief valve**, reducing producer-pool
+   pressure and letting more requests squeeze under the 600 s timeout — at the (uncosted)
+   price of cross-turn re-prefill. With the eviction off, producers retain prefix
+   correctly → cache works on PD too → but the cache itself contends for producer
+   pool capacity, and the decode-pool admission ceiling tips earlier. **PD-disagg is
+   worse on agentic than §3 advertised, not better.**
+
+## Linear-policy + wall-cap follow-up (2026-06-01) — the success ceiling is policy-invariant
+
+To check whether the LMetric routing was secretly handicapping PD-disagg, we re-ran
+first600s with the **default `--policy linear`** (cache-aware load score + sticky
+session affinity — the original baseline of the cache_aware_proxy stack) and
+**wall-capped each PD-disagg arm at 2 × the colo wall** (kill bench.sh + cleanup
+GPUs once cap is exceeded, record `records_at_cap`).
+
+| arm | linear success | linear wall | linear @-cap? | lmetric success | lmetric wall |
+|---|---|---|---|---|---|
+| **colo** | 807/807 = **100 %** | 964 s | — | 807/807 = **100 %** | 1021 s |
+| **pd6 (6:2)** | **472/807 = 58 %** | 2280 s | ⊗ cap (706 dispatched) | 474/807 = 59 % | 3325 s |
+| **pd4 (4:4)** | **349/807 = 43 %** | 2281 s | ⊗ cap (577 dispatched) | 348/807 = 43 % | 6850 s |
+| **pd2 (2:6)** | **176/807 = 22 %** | 2280 s | ⊗ cap (521 dispatched) | 180/807 = 22 % | 19275 s |
+
+→ Figure: [`figs/v2/fig4_linear_vs_lmetric.png`](../../figs/v2/fig4_linear_vs_lmetric.png) ·
+data: [`fig4r_linear.json`](fig4r_linear.json)
+
+**Three clean conclusions from the wall-cap experiment:**
+
+1. **The success-rate ceiling is structural, not a routing artifact.** Linear and
+   LMetric — two very different scoring policies (one session-sticky cache-aware,
+   the other non-sticky pure load) — converge on **identical success rates**
+   (58 / 43 / 22 %) for every PD-disagg ratio. Routing has *zero* effect on the
+   completion ceiling. The bottleneck is the static P:D split itself.
+
+2. **LMetric's longer wall was wall *wasted on requests that will never succeed*.**
+   When the cap is enforced, linear hits the same ceiling in 2280 s as LMetric did
+   in 3300–19000 s — the extra wall just slowly times out the unreachable
+   requests at 600 s each.
+
+3. **The wall-cap is the right way to bench PD-disagg.** Reporting "completion %"
+   without a wall budget is misleading (the bench eventually completes if you wait
+   forever, but only by counting timeouts as failures over hours). The honest
+   metric is **success-in-2×-colo-wall**, which gives the same answer for both
+   routings and matches what an end user would observe on a real SLO.
+
+This **strengthens** the §5 D-pool capacity-ceiling thesis: even with
+session-affinity routing serving every request to a warm prefix cache (which
+*should* maximise PD's throughput), the static D-pool can't admit more than
+~22 / 43 / 58 % of the agentic trace within 2× the colo budget. Colo wins not
+because routing is smarter, but because its **elastic pool** absorbs whichever
+phase is hot — there's no cap to hit.
+
+---
+
+## Reproduce
+
+```bash
+# On dash1, from the main repo /home/admin/cpfs/wjh/agentic-kv:
+for TR in w600_r0.0015_st30.jsonl w600_r0.0015_st30_first600s.jsonl; do
+  TRACE=traces/$TR bash scripts/bench.sh --tag fig4l_lmetric_colo_${TR%.*} \
+    --mode baseline --policy lmetric
+  for r in 6:2 4:4 2:6; do
+    TRACE=traces/$TR bash scripts/bench.sh --tag fig4l_lmetric_${r/:/p}_${TR%.*} \
+      --mode pdsep --pd-ratio $r --policy lmetric
+  done
+done
+
+python microbench/fresh_setup/plot_fig4l_lmetric.py
+python microbench/fresh_setup/plot_fig4_linear_vs_lmetric.py
+```
+
+For the linear + 2× wall-cap variant, run colo first to get `wall_clock_s`,
+compute `CAP=2*wall`, then launch each PD-disagg arm in the background and
+`SIGTERM` it (so bench.sh's cleanup trap fires) once `date +%s` minus the
+arm's start time exceeds `CAP`. The capped runs lack `metrics.summary.json`
+(replayer was killed before it could write); compute the summary directly from
+`metrics.jsonl` (see the inline collector used to build
+`analysis/v2/fig4r_linear.json`).
+
+Source `bench.sh` cleans GPUs before each arm and writes `metrics.jsonl` +
+`metrics.summary.json` per tag. Aggregation script: see the inline JSON dump used
+to build `analysis/v2/fig4l_lmetric.json`.

analysis/v2/fig4l_lmetric.json

@@ -0,0 +1 @@
+[{"tag": "fig4l_lmetric_colo_first600s", "arm": "colo", "trace": "first600s", "n": 807, "req": 807, "e2e": {"count": 807.0, "mean": 11.066699584425269, "p50": 3.27055042097345, "p90": 28.745733462180937, "p99": 97.40008939541167}, "ttft": {"count": 807.0, "mean": 5.119651803458883, "p50": 1.2114678020589054, "p90": 14.777630288852365, "p99": 50.68302261995841}, "tpot": {"count": 807.0, "mean": 0.03004899278845205, "p50": 0.009643197803618922, "p90": 0.042092699501536976, "p99": 0.3919741264067197}, "wall": 1020.5351374909515, "tps": 226.12940164644368}, {"tag": "fig4l_lmetric_colo_full", "arm": "colo", "trace": "full", "n": 1214, "req": 1214, "e2e": {"count": 1214.0, "mean": 10.928977524270508, "p50": 3.1279119075043127, "p90": 30.011970606888667, "p99": 94.77313101590481}, "ttft": {"count": 1214.0, "mean": 5.533819193267678, "p50": 1.017395684029907, "p90": 17.36427243486981, "p99": 51.49416554694993}, "tpot": {"count": 1214.0, "mean": 0.02049970290344434, "p50": 0.009544484575988867, "p90": 0.032480608771520716, "p99": 0.26057810739537074}, "wall": 2993.276069591986, "tps": 125.38402448497122}, {"tag": "fig4l_lmetric_pd2_first600s", "arm": "2P+6D", "trace": "first600s", "n": 180, "req": 807, "e2e": {"count": 180.0, "mean": 380.2505690135715, "p50": 535.6594606440049, "p90": 579.5011055286858, "p99": 601.5567972306756}, "ttft": {"count": 180.0, "mean": 378.7133691522933, "p50": 534.4269686369807, "p90": 577.3534130641376, "p99": 596.404559875431}, "tpot": {"count": 180.0, "mean": 0.007975266077679418, "p50": 0.007166497974743372, "p90": 0.012511071875514153, "p99": 0.017508981961061446}, "wall": 19275.367093455978, "tps": 1.8895100582735462}, {"tag": "fig4l_lmetric_pd2_full", "arm": "2P+6D", "trace": "full", "n": 169, "req": 1214, "e2e": {"count": 169.0, "mean": 194.88523891245458, "p50": 6.817620265996084, "p90": 552.1569225640735, "p99": 595.3934216396092}, "ttft": {"count": 169.0, "mean": 193.4153314989016, "p50": 5.60239192598965, "p90": 549.3611521873856, "p99": 582.4436428000824}, "tpot": {"count": 169.0, "mean": 0.007747395842651413, "p50": 0.007691574401794991, "p90": 0.011201243427351017, "p99": 0.013311375577245894}, "wall": 33770.57413210906, "tps": 0.9869539045920406}, {"tag": "fig4l_lmetric_pd4_first600s", "arm": "4P+4D", "trace": "first600s", "n": 348, "req": 807, "e2e": {"count": 348.0, "mean": 202.63302869595395, "p50": 214.03008900902933, "p90": 477.40967412578175, "p99": 576.6393926549597}, "ttft": {"count": 348.0, "mean": 199.96385804087797, "p50": 213.50966987549327, "p90": 475.7766476540827, "p99": 559.6153268160638}, "tpot": {"count": 348.0, "mean": 0.008873619369764751, "p50": 0.007645836479973812, "p90": 0.013845969236959285, "p99": 0.02567216653158788}, "wall": 6850.181333696004, "tps": 15.00296050477674}, {"tag": "fig4l_lmetric_pd4_full", "arm": "4P+4D", "trace": "full", "n": 533, "req": 1214, "e2e": {"count": 533.0, "mean": 130.94711188977982, "p50": 8.219856544979848, "p90": 473.44134307731883, "p99": 533.2597587251009}, "ttft": {"count": 533.0, "mean": 127.83193208824007, "p50": 4.8246813879814, "p90": 467.54664219671395, "p99": 528.8304683346115}, "tpot": {"count": 533.0, "mean": 0.008886429490232585, "p50": 0.007981476340708988, "p90": 0.013570741891233497, "p99": 0.023050950961825044}, "wall": 13884.384965199977, "tps": 12.621372890425038}, {"tag": "fig4l_lmetric_pd6_first600s", "arm": "6P+2D", "trace": "first600s", "n": 474, "req": 807, "e2e": {"count": 474.0, "mean": 83.15809065495806, "p50": 6.7270191764691845, "p90": 391.6558471220078, "p99": 544.7372293809171}, "ttft": {"count": 474.0, "mean": 80.70155321074382, "p50": 4.1273433425230905, "p90": 390.00296151017517, "p99": 539.0574236416071}, "tpot": {"count": 474.0, "mean": 0.008519881756330928, "p50": 0.00803907146806204, "p90": 0.012583933303093976, "p99": 0.018606097790947705}, "wall": 3325.2749515309697, "tps": 39.705588838364164}, {"tag": "fig4l_lmetric_pd6_full", "arm": "6P+2D", "trace": "full", "n": 793, "req": 1214, "e2e": {"count": 793.0, "mean": 61.907526705667, "p50": 3.69814173609484, "p90": 308.2633092067672, "p99": 477.48038318102715}, "ttft": {"count": 793.0, "mean": 59.25069201986225, "p50": 1.402295546955429, "p90": 302.5604081378088, "p99": 475.3738951798529}, "tpot": {"count": 793.0, "mean": 0.009137289999448822, "p50": 0.008635683270933276, "p90": 0.013065757584108427, "p99": 0.01816783740464599}, "wall": 5662.029295974993, "tps": 39.24494000021532}]

analysis/v2/fig4r_linear.json

@@ -0,0 +1 @@
+[{"tag": "fig4r_linear_colo_first600s", "arm": "colo", "trace": "first600s", "policy": "linear", "n": 807, "req": 807, "dispatched": 807, "e2e": {"count": 807.0, "mean": 8.436370009274967, "p50": 2.5224755640374497, "p90": 22.65510415879542, "p99": 75.54369598095519}, "ttft": {"count": 807.0, "mean": 4.2332503390957195, "p50": 0.8872958200518042, "p90": 11.684667797433207, "p99": 44.98891795879462}, "tpot": {"count": 807.0, "mean": 0.020958194728517718, "p50": 0.00851320761584622, "p90": 0.026440129078245465, "p99": 0.30344440533287176}, "wall": 963.6191155100241, "tps": 239.4857016486815, "capped": false}, {"tag": "fig4r_linear_pd2_first600s", "arm": "2P+6D", "trace": "first600s", "policy": "linear", "n": 176, "req": 807, "dispatched": 521, "e2e": {"count": 176, "mean": 378.5561210460834, "p50": 536.7719694490079, "p90": 583.832092280034, "p99": 601.3415494390065}, "ttft": {"count": 176, "mean": 377.12570991374446, "p50": 536.1157373189926, "p90": 580.3465002350276, "p99": 598.0943597999867}, "tpot": {"count": 176, "mean": 0.007864906140929698, "p50": 0.007212154543958604, "p90": 0.011962352272927423, "p99": 0.017870794738764347}, "wall": 2280, "tps": 14.419736842105262, "capped": true}, {"tag": "fig4r_linear_pd4_first600s", "arm": "4P+4D", "trace": "first600s", "policy": "linear", "n": 349, "req": 807, "dispatched": 577, "e2e": {"count": 349, "mean": 264.8537863784421, "p50": 306.6853819829412, "p90": 488.64622142596636, "p99": 596.5830293919425}, "ttft": {"count": 349, "mean": 262.3163347712099, "p50": 299.75751709297765, "p90": 485.475125996978, "p99": 596.4081599479541}, "tpot": {"count": 349, "mean": 0.010442244895290958, "p50": 0.008213572105774598, "p90": 0.019443845545703716, "p99": 0.028178529054794}, "wall": 2281, "tps": 38.306882946076286, "capped": true}, {"tag": "fig4r_linear_pd6_first600s", "arm": "6P+2D", "trace": "first600s", "policy": "linear", "n": 472, "req": 807, "dispatched": 706, "e2e": {"count": 472, "mean": 118.632779156234, "p50": 12.702161715948023, "p90": 458.1609142010566, "p99": 526.5488834320568}, "ttft": {"count": 472, "mean": 115.80202843308507, "p50": 9.745031949947588, "p90": 455.81679951993283, "p99": 516.5850186559837}, "tpot": {"count": 472, "mean": 0.00950947083585719, "p50": 0.008435572332624966, "p90": 0.015233499645638644, "p99": 0.023447183093280886}, "wall": 2280, "tps": 61.69210526315789, "capped": true}]

analysis/working_set/README.md

@@ -0,0 +1,67 @@
+# KV-cache Working-Set Sizing — GLM-5.1-FP8 · TP=8 · 1× B300 node
+
+工具：`scripts/working_set_analysis.py`（可配置 GPU 型号 / 并行度 TP·PP·EP / 模型 config.json /
+KV dtype / 权重大小）。图：`figs/working_set/glm5_fp8_tp8_b300.png`。
+
+## 复现
+
+```bash
+.venv/bin/python scripts/working_set_analysis.py \
+  /home/gahow/phd/kvcache-simulator/bailian-traces/glm_coder_blksz_512_040915-040917.jsonl \
+  --model-config /home/gahow/phd/kvcache-simulator/models/GLM-5/config.json \
+  --gpu B300 --tp 8 --ep 8 --kv-dtype-bytes 1 --weight-gb 744 --min-ts 0 \
+  --out figs/working_set/glm5_fp8_tp8_b300.png
+```
+
+## 方法
+
+`hash_ids` 是全局内容寻址 block id（同内容=同 id，复用=同 id 再现）。vLLM prefix cache 是
+block 级，所以**集群级 KV footprint = 任一时刻必须常驻的 distinct block 数**，与 placement 无关
+（affinity 只搬运 block，不改总量）。三种 working set：
+- `W_all` 永不淘汰（真上界）
+- `W_oracle` 每 block 只在 `[首次, 末次复用]` 常驻（Belady 完美预知 → 满 APC 上界的最小 HBM）
+- `W_denning(T)` 滑窗 T 内被访问的 distinct block（现实 TTL-LRU）
+
+KV/token：MLA → `L·(kv_lora_rank+qk_rope_head_dim)·dtype`；GQA → `2·L·kv_heads·head_dim·dtype`
+（与 `kvcache-simulator/src/config.rs::kv_block_bytes` 一致）。
+
+## 配置
+
+| 项 | 值 |
+|---|---|
+| 模型 | GLM-5.1-FP8（MLA, L=78, kv_lora=512+rope=64） |
+| KV/token · KV/block(512) | **43.9 KiB** · **23.0 MB**（≈ Qwen3 GQA 96 KiB 的一半） |
+| 硬件 | 8× B300 (288 GB) = 2304 GB HBM/replica |
+| 预算 | FP8 权重 744 GB + act 32 GB → **KV pool = 1528 GB/node** |
+| trace | dash0 glm_coder，475k req，**1.25h active @ 106 QPS**，~40k tok/req（剔除 77 条负 ts 暖机） |
+| APC 上界 | **80.4%** |
+
+## 结果
+
+| 保留窗口 T | peak footprint | = 节点 (GPU) | APC@T |
+|---:|---:|---:|---:|
+| 2s（在飞下限）| 533 GB | 0.3 (3) | 1.7% |
+| 10s | 2,068 GB | 1.4 (11) | 15% |
+| 30s | 4,906 GB | 3.2 (26) | 42% |
+| 60s | 7,698 GB | 5.0 (40) | 56% |
+| 300s | 21,960 GB | 14.4 (115) | 74% |
+| **oracle（满 80.4%）** | **21,399 GB** | **14.0 (112)** | 80.4% |
+| retain-forever | 167,018 GB | 109 (874) | — |
+
+## 结论
+
+1. **Serving：1 节点绰绰有余。** 在飞 KV（τ≈2-5s）仅 533–1157 GB ≪ 单节点 1528 GB。
+   MLA + B300 大 HBM 让 live footprint 微不足道——跑起来根本不缺显存。
+2. **缓存全部复用（80.4%）：1 节点差 ~14×。** oracle 下限 21.4 TB = 14 节点（112 GPU），
+   真实 LRU ~2× → ~28 节点。单节点（1528 GB）只能 hold ~10s 窗口 → cache 侧 APC 仅 ~10-15%。
+   要 ~56% 需 5 节点，~74% 需 ~14 节点。
+3. **瓶颈在长尾，不在 live。** 把 APC 50%→80% 装进 GPU HBM 要 5→14 节点，极不经济
+   → offload/migration 到 CPU DRAM（每节点 ~1.5 TB）是定量动机。与 Qwen 结论方向一致。
+
+## 注意
+
+- footprint 是 TTL-LRU（最浪费）+ shared-cache 下限：真实 capacity-LRU 同容量下 APC 更高，
+  但分区/affinity 不均衡又抬高需求；oracle / retain-forever 给出下/上界。
+- GLM trace mean ~40k tok/req，是 Qwen trace（11k）的 ~3.5×（tokenizer + 抽取不同），
+  **绝对 GB 不可跨模型横比**，方法与定性结论可比。
+- EP 不改变 KV 总量（只影响 expert 权重分布），`--ep` 仅作标注。

analysis/workload_chars/README.md

@@ -0,0 +1,81 @@
+# Agentic workload characterization C1–C3 (full 051315 production trace)
+
+Date 2026-05-29. Source: `trace-glm5.1-formatted/051315-051317.jsonl` on dash1
+(release file, 2,114,220 requests / 1,307,276 sessions / 2h, type=100% `coder`).
+This release file **is the full cluster-level production trace** — session skew
+reproduces 46.5/66.5/74.6/87.5/96.0 exactly. Compute: `compute_chars.py`
+(2-pass, ~65s, `~/ali-trace/.venv` python). Numbers: `chars.json`.
+
+> ⚠️ **Cluster-level, not per-instance.** This is one cluster's aggregate stream.
+> Concurrent-session counts have NO denominator of "8 instances" — do not compare
+> them to a single deployment's instance count.
+
+These three are NOT in the existing 13 analyzer figures (which are single-variable
+marginals on the older 041x traces). C1–C3 are joint/temporal and argument-bearing.
+
+## C1 — the workload is a MIXTURE, not "multi-turn agentic" (`c1_session_mixture.png`)
+
+- **90.3%** of sessions are single-turn; mean 1.62 turns, p99=18, max=3091.
+- But multi-turn sessions (9.7%) = **44.2% of requests** and **66.9% of input
+  (prefill) mass**. Single-turn = **60.2% of output (decode) mass**.
+- Continuation hazard P(reach k+1 | reached k): turn1→2 only **10.2%**, but
+  turn2→3 50.6%, turn5→6 87%, turn12→13 **94.3%** (Lindy / Pareto).
+- Predictability of heaviness at cold-start is near-zero:
+  corr(turn1_input, session_mass)=0.15, corr(turn1_input, n_turns)=**0.04**.
+
+**Routing:** heaviness is unpredictable at session start → proactive placement
+cannot pre-empt hot-pin → a REACTIVE mechanism (observable-load routing /
+migration) is required. But once a session has shown depth, it almost surely
+continues → "observed accumulated load" is the signal that works (not turn-1
+features, not cost-model prediction). The single/multi optimal strategies are
+opposite (load-balance the 90% one-shot sea vs affinity-pin the deep tail) and
+you can't tell them apart at turn 1 → the only viable policy starts everyone
+load-balanced and becomes sticky as turns accrue. This is exactly LPWL's
+emergent behavior (`new_uncached≈input`→by-load; `new_uncached≈0`→sticks), so
+C1 explains *why* a cache-aware-load score is the right shape — it auto-segments
+the mixture with no classifier.
+
+## C2 — marginal work collapses while resident state explodes (`c2_work_amortization.png`)
+
+Per turn: resident context grows 11k→56k+ tokens while new prefill collapses
+2.7k→~200 tokens; per-turn reuse climbs 83%→**99.6%**; resident/new ratio
+("the PD tax") grows to ~250× by turn 12, ~450× by turn 30.
+
+**PD-colocation:** the dominant cost is keeping ~50k+ resident KV available for
+the next turn's tiny delta. Disaggregation physically splits a turn's prefill-KV
+(P) and decode-KV (D), and the next turn's prefix = [prevPrompt + prevAnswer]
+spans both → must be gathered/transferred; colocation keeps it local for free.
+**Routing:** route on delta (`input − cache_hit`), never total input — C2 is the
+trace-level justification for LPWL's score function.
+
+## C3 — prefill/decode BALANCE (honest reframe) (`c3_prefill_decode_balance.png`)
+
+- Token mass: 98.7% input / **1.3% output**; of input, 60% reused-prefix, 40%
+  new-prefill (28.6B new-prefill tokens vs 0.94B decode tokens).
+- **But tokens ≠ time.** Under a per-request latency model (prefill@7k tok/s,
+  TPOT 10ms), aggregate decode-time share ≈ **70% (robust 68–71% across
+  constants)** — each decode token costs ~70–140× a prefill token. So this is
+  NOT a "decode is negligible" workload.
+- Per-request the bottleneck FLIPS within a session: turn-1 (and the 90%
+  single-turn) is prefill-bound; turns ≥3 are strongly decode-bound.
+
+**PD-colocation (correct argument):** the workload has *substantial* work on both
+sides of the roofline — compute-bound prefill (~30% of time) and memory-bound
+decode (~70%). Colocation interleaves them on one GPU (chunked prefill +
+continuous batching) so compute and HBM bandwidth are both used; static
+disaggregation strands P-instances bandwidth-idle and D-instances compute-idle.
+The earlier "decode is 1.3% so nothing to isolate" instinct was WRONG (token vs
+time confusion) — C3b is the correction.
+
+**Caveat:** C3b's 70% is a per-request-latency-weighted estimate; batched decode
+throughput will shift it. Ground-truth needs `-raw.jsonl` (`usage.cached_tokens`
+for exact reuse; `backend_first_response_time_ms` / `total_cost_time_ms` for real
+prefill vs decode wall time). Sampling that 522GB file is the next step.
+
+## Goal mapping
+
+| | argue PD-colocation | guide routing |
+|---|---|---|
+| C1 mixture + hazard | both segments favor colo (diff reasons) | reactive + auto-segment ⇒ LPWL shape |
+| C2 resident/delta | the PD tax (transfer/split resident KV) | route on delta, not total |
+| C3 prefill/decode | roofline complementarity (interleave) | per-req bottleneck flips within session |

analysis/workload_chars/chars.json

@@ -0,0 +1,964 @@
+{
+  "mixture": {
+    "single_sessions": 1179990,
+    "multi_sessions": 127286,
+    "req_single_pct": 55.81207253738968,
+    "req_multi_pct": 44.187927462610325,
+    "in_single_pct": 33.12487590117447,
+    "in_multi_pct": 66.87512409882554,
+    "out_single_pct": 60.24502960903973,
+    "out_multi_pct": 39.75497039096027
+  },
+  "turns": {
+    "mean": 1.6172713336739908,
+    "p99": 18.0,
+    "max": 3091,
+    "single_turn_pct": 90.26326498765371
+  },
+  "hazard": {
+    "1": 0.102101621998721,
+    "2": 0.5062146469376287,
+    "3": 0.7351961756478754,
+    "4": 0.8113739305485657,
+    "5": 0.8723731546954472,
+    "6": 0.8669264241631353,
+    "7": 0.9093235352011023,
+    "8": 0.9240204920989971,
+    "9": 0.901725753553022,
+    "10": 0.9346178826585841,
+    "11": 0.9260597637248089,
+    "12": 0.9427685226874781,
+    "13": 0.91950119395065,
+    "14": 0.936865189289012,
+    "15": 0.9382160896883085,
+    "16": 0.9308646838684262,
+    "17": 0.9371561574269995,
+    "18": 0.9312862196131557,
+    "19": 0.9333279456925813,
+    "20": 0.9351459000779289,
+    "21": 0.9399074074074074,
+    "22": 0.9404984730568416,
+    "23": 0.9473132921336546,
+    "24": 0.9193940734188413,
+    "25": 0.9497294046903187,
+    "26": 0.9323793845764214,
+    "27": 0.9483906016569333,
+    "28": 0.9368466275239868,
+    "29": 0.9472638336900031
+  },
+  "token_mass": {
+    "total_input": 71116829368,
+    "total_output": 940765734,
+    "out_in_ratio_pct": 1.3228454394837104,
+    "new_prefill": 28616906067,
+    "reused_prefix": 42499923301,
+    "new_prefill_pct_of_input": 40.23928839532401
+  },
+  "decode_time_fraction": {
+    "optimistic_for_prefill": 0.6812079219496285,
+    "mid": 0.6970810590484581,
+    "pessimistic": 0.711448473592609
+  },
+  "per_turn": {
+    "turn": [
+      1,
+      2,
+      3,
+      4,
+      5,
+      6,
+      7,
+      8,
+      9,
+      10,
+      11,
+      12,
+      13,
+      14,
+      15,
+      16,
+      17,
+      18,
+      19,
+      20,
+      21,
+      22,
+      23,
+      24,
+      25,
+      26,
+      27,
+      28,
+      29,
+      30,
+      31,
+      32,
+      33,
+      34,
+      35,
+      36,
+      37,
+      38,
+      39,
+      40,
+      41,
+      42,
+      43,
+      44,
+      45,
+      46,
+      47,
+      48,
+      49,
+      50,
+      51,
+      52,
+      53,
+      54,
+      55,
+      56,
+      57,
+      58,
+      59,
+      60,
+      61,
+      62,
+      63,
+      64,
+      65,
+      66,
+      67,
+      68,
+      69,
+      70,
+      71,
+      72,
+      73,
+      74,
+      75,
+      76,
+      77,
+      78,
+      79,
+      80,
+      81,
+      82,
+      83,
+      84,
+      85,
+      86,
+      87,
+      88,
+      89,
+      90,
+      91,
+      92,
+      93,
+      94,
+      95,
+      96,
+      97,
+      98,
+      99,
+      100,
+      101,
+      102,
+      103,
+      104,
+      105,
+      106,
+      107,
+      108,
+      109,
+      110,
+      111,
+      112,
+      113,
+      114,
+      115,
+      116,
+      117,
+      118,
+      119,
+      120,
+      121,
+      122,
+      123,
+      124,
+      125,
+      126,
+      127,
+      128,
+      129,
+      130,
+      131,
+      132,
+      133,
+      134,
+      135,
+      136,
+      137,
+      138,
+      139,
+      140,
+      141,
+      142,
+      143,
+      144,
+      145,
+      146,
+      147,
+      148
+    ],
+    "med_resident_input": [
+      11035.0,
+      19505.0,
+      28059.0,
+      35089.0,
+      41215.0,
+      44750.0,
+      47419.5,
+      49874.0,
+      51905.0,
+      53068.0,
+      54782.0,
+      56414.0,
+      58229.0,
+      59123.5,
+      60434.5,
+      61320.0,
+      62243.0,
+      63411.0,
+      64510.5,
+      65423.0,
+      66942.5,
+      67965.0,
+      68826.0,
+      70165.5,
+      70052.0,
+      70936.0,
+      71547.0,
+      72648.0,
+      73406.0,
+      73844.0,
+      73604.0,
+      74937.5,
+      74778.0,
+      75460.0,
+      75029.0,
+      74978.0,
+      75933.0,
+      76590.0,
+      74695.0,
+      76813.0,
+      77079.5,
+      78310.0,
+      77848.0,
+      77549.0,
+      78203.0,
+      79102.0,
+      79202.0,
+      78821.0,
+      79868.0,
+      80229.5,
+      80912.0,
+      81620.0,
+      81612.5,
+      81836.5,
+      82506.0,
+      82948.0,
+      82633.0,
+      84107.5,
+      84176.0,
+      84441.0,
+      84101.0,
+      85192.0,
+      84127.0,
+      84783.5,
+      85087.0,
+      85771.5,
+      86110.0,
+      85374.5,
+      87137.0,
+      87677.0,
+      88587.0,
+      88656.0,
+      88882.0,
+      89284.0,
+      91512.0,
+      89850.0,
+      90596.0,
+      91244.0,
+      92102.0,
+      93431.0,
+      92333.5,
+      96682.0,
+      94999.0,
+      95226.5,
+      95173.0,
+      95910.0,
+      96528.0,
+      96508.0,
+      97270.0,
+      97301.0,
+      97076.5,
+      97105.0,
+      98032.0,
+      97962.5,
+      97968.5,
+      98310.0,
+      97061.0,
+      97631.0,
+      100126.0,
+      97765.0,
+      101076.0,
+      98198.5,
+      98678.0,
+      98307.0,
+      99174.0,
+      99882.0,
+      99974.0,
+      99757.0,
+      100065.5,
+      99943.0,
+      100612.0,
+      101138.0,
+      106738.0,
+      99621.0,
+      101980.0,
+      102252.0,
+      103018.0,
+      101238.0,
+      102005.0,
+      101897.0,
+      103576.0,
+      102159.5,
+      102695.5,
+      100590.5,
+      103236.0,
+      101812.0,
+      103074.0,
+      99966.0,
+      102183.5,
+      101882.0,
+      102572.5,
+      105622.5,
+      106066.0,
+      103974.0,
+      105443.5,
+      104716.0,
+      105041.0,
+      106628.0,
+      108320.0,
+      108022.5,
+      107621.5,
+      107664.0,
+      107913.0,
+      108630.0,
+      108382.0,
+      107216.5,
+      105731.0,
+      103986.0
+    ],
+    "med_new_prefill": [
+      11035.0,
+      2920.0,
+      1249.0,
+      767.0,
+      628.0,
+      485.0,
+      400.0,
+      359.0,
+      314.0,
+      274.0,
+      263.0,
+      258.0,
+      244.0,
+      231.0,
+      227.0,
+      222.0,
+      201.0,
+      200.0,
+      198.0,
+      189.0,
+      182.5,
+      184.0,
+      179.0,
+      188.0,
+      173.0,
+      180.0,
+      164.0,
+      167.0,
+      159.5,
+      168.0,
+      156.0,
+      174.0,
+      156.0,
+      159.0,
+      166.0,
+      165.0,
+      153.0,
+      158.0,
+      182.0,
+      149.0,
+      184.0,
+      172.0,
+      149.0,
+      167.0,
+      163.0,
+      152.0,
+      153.0,
+      171.0,
+      151.0,
+      146.0,
+      162.0,
+      153.0,
+      156.0,
+      164.0,
+      148.0,
+      143.0,
+      143.0,
+      149.0,
+      170.5,
+      159.0,
+      144.0,
+      168.0,
+      148.0,
+      144.5,
+      142.5,
+      146.5,
+      147.0,
+      157.0,
+      168.0,
+      153.0,
+      155.0,
+      127.5,
+      145.0,
+      143.0,
+      146.0,
+      123.0,
+      139.0,
+      137.0,
+      115.0,
+      139.5,
+      117.0,
+      154.0,
+      111.0,
+      124.0,
+      118.0,
+      90.0,
+      104.0,
+      116.0,
+      112.0,
+      76.5,
+      110.0,
+      101.0,
+      123.0,
+      114.0,
+      86.0,
+      92.0,
+      108.0,
+      85.0,
+      146.0,
+      77.5,
+      101.0,
+      102.0,
+      85.0,
+      77.0,
+      114.0,
+      66.0,
+      105.0,
+      90.0,
+      89.0,
+      100.0,
+      108.5,
+      100.0,
+      169.0,
+      89.0,
+      106.5,
+      78.0,
+      75.0,
+      90.0,
+      77.0,
+      88.0,
+      102.0,
+      83.5,
+      123.5,
+      116.5,
+      108.0,
+      119.0,
+      82.0,
+      80.0,
+      105.0,
+      90.0,
+      91.0,
+      113.0,
+      122.0,
+      102.0,
+      101.5,
+      64.0,
+      78.0,
+      52.5,
+      98.5,
+      72.0,
+      87.0,
+      102.0,
+      97.0,
+      123.0,
+      80.0,
+      132.5,
+      86.5,
+      111.0
+    ],
+    "med_output": [
+      63.0,
+      67.0,
+      111.0,
+      142.0,
+      158.0,
+      162.0,
+      164.0,
+      164.0,
+      159.0,
+      160.0,
+      159.0,
+      161.0,
+      160.0,
+      158.0,
+      154.0,
+      154.0,
+      154.0,
+      149.0,
+      146.0,
+      147.0,
+      142.0,
+      144.0,
+      143.0,
+      142.0,
+      140.0,
+      136.0,
+      137.0,
+      139.0,
+      136.0,
+      133.0,
+      130.0,
+      131.0,
+      125.0,
+      123.0,
+      122.0,
+      122.0,
+      118.0,
+      122.0,
+      114.0,
+      112.0,
+      115.0,
+      111.0,
+      109.0,
+      112.0,
+      109.0,
+      107.0,
+      111.0,
+      105.0,
+      108.0,
+      107.0,
+      100.0,
+      100.0,
+      95.0,
+      105.0,
+      103.0,
+      102.0,
+      100.0,
+      100.0,
+      98.0,
+      98.0,
+      101.0,
+      99.0,
+      101.0,
+      102.0,
+      97.0,
+      91.0,
+      100.0,
+      97.0,
+      94.0,
+      98.5,
+      92.5,
+      97.0,
+      102.0,
+      92.0,
+      95.0,
+      91.0,
+      91.0,
+      92.0,
+      85.0,
+      98.0,
+      96.0,
+      99.0,
+      94.0,
+      96.0,
+      90.0,
+      85.0,
+      99.0,
+      86.0,
+      99.0,
+      93.0,
+      92.0,
+      93.0,
+      87.0,
+      83.0,
+      87.5,
+      82.0,
+      80.0,
+      90.0,
+      92.0,
+      80.0,
+      77.0,
+      82.0,
+      87.0,
+      74.0,
+      83.0,
+      79.0,
+      84.0,
+      80.5,
+      79.0,
+      76.0,
+      78.5,
+      71.5,
+      81.0,
+      87.0,
+      82.0,
+      85.0,
+      87.0,
+      75.0,
+      75.0,
+      82.0,
+      86.0,
+      76.5,
+      77.5,
+      70.0,
+      78.0,
+      85.0,
+      77.0,
+      67.0,
+      76.5,
+      107.0,
+      92.0,
+      80.5,
+      85.0,
+      83.0,
+      77.0,
+      70.0,
+      84.0,
+      69.0,
+      97.0,
+      72.0,
+      81.0,
+      87.0,
+      89.0,
+      102.0,
+      83.0,
+      82.5,
+      91.0,
+      79.5
+    ],
+    "resident_over_new": [
+      1.0,
+      6.679794520547945,
+      22.46517213771017,
+      45.748370273794,
+      65.62898089171975,
+      92.26804123711341,
+      118.54875,
+      138.92479108635098,
+      165.30254777070064,
+      193.67883211678833,
+      208.29657794676805,
+      218.65891472868216,
+      238.64344262295083,
+      255.94588744588745,
+      266.23127753303964,
+      276.2162162162162,
+      309.6666666666667,
+      317.055,
+      325.81060606060606,
+      346.15343915343914,
+      366.8082191780822,
+      369.375,
+      384.5027932960894,
+      373.22074468085106,
+      404.9248554913295,
+      394.0888888888889,
+      436.2621951219512,
+      435.0179640718563,
+      460.2257053291536,
+      439.54761904761904,
+      471.8205128205128,
+      430.67528735632186,
+      479.34615384615387,
+      474.59119496855345,
+      451.98192771084337,
+      454.41212121212124,
+      496.29411764705884,
+      484.746835443038,
+      410.4120879120879,
+      515.5234899328859,
+      418.9103260869565,
+      455.2906976744186,
+      522.4697986577181,
+      464.36526946107784,
+      479.7730061349693,
+      520.4078947368421,
+      517.6601307189543,
+      460.94152046783626,
+      528.9271523178808,
+      549.5171232876712,
+      499.4567901234568,
+      533.4640522875817,
+      523.1570512820513,
+      499.0030487804878,
+      557.472972972973,
+      580.0559440559441,
+      577.8531468531469,
+      564.4798657718121,
+      493.7008797653959,
+      531.0754716981132,
+      584.0347222222222,
+      507.0952380952381,
+      568.4256756756756,
+      586.7370242214533,
+      597.1017543859649,
+      585.4709897610921,
+      585.7823129251701,
+      543.7866242038217,
+      518.672619047619,
+      573.0522875816994,
+      571.5290322580645,
+      695.3411764705883,
+      612.9793103448276,
+      624.3636363636364,
+      626.7945205479452,
+      730.4878048780488,
+      651.7697841726618,
+      666.014598540146,
+      800.8869565217391,
+      669.7562724014336,
+      789.1752136752136,
+      627.8051948051948,
+      855.8468468468468,
+      767.9556451612904,
+      806.5508474576271,
+      1065.6666666666667,
+      928.1538461538462,
+      831.9655172413793,
+      868.4821428571429,
+      1271.9084967320262,
+      882.5136363636364,
+      961.4356435643564,
+      797.0081300813008,
+      859.3201754385965,
+      1139.1686046511627,
+      1068.5869565217392,
+      898.7129629629629,
+      1148.6,
+      685.7945205479452,
+      1261.483870967742,
+      1000.7524752475248,
+      962.7303921568628,
+      1160.9176470588236,
+      1276.7142857142858,
+      869.9473684210526,
+      1513.3636363636363,
+      952.1333333333333,
+      1108.411111111111,
+      1124.3314606741574,
+      999.43,
+      927.2995391705069,
+      1011.38,
+      631.5857988165681,
+      1119.3370786516855,
+      957.5586854460093,
+      1310.923076923077,
+      1373.5733333333333,
+      1124.8666666666666,
+      1324.7402597402597,
+      1157.9204545454545,
+      1015.4509803921569,
+      1223.4670658682635,
+      831.5425101214574,
+      863.4377682403433,
+      955.8888888888889,
+      855.563025210084,
+      1257.0,
+      1249.575,
+      973.1761904761905,
+      1132.0222222222221,
+      1127.1703296703297,
+      934.712389380531,
+      869.3934426229508,
+      1019.3529411764706,
+      1038.8522167487686,
+      1636.1875,
+      1346.679487179487,
+      2031.009523809524,
+      1099.6954314720813,
+      1500.3125,
+      1237.028735632184,
+      1055.5294117647059,
+      1112.5051546391753,
+      883.170731707317,
+      1354.775,
+      809.1811320754717,
+      1222.3236994219653,
+      936.8108108108108
+    ],
+    "reuse_pct": [
+      0.0,
+      85.02947962061009,
+      95.5486653123775,
+      97.81412978426287,
+      98.47628290670872,
+      98.91620111731844,
+      99.1564651672835,
+      99.28018606889361,
+      99.39504864656584,
+      99.48368131453984,
+      99.5199153006462,
+      99.54266671393626,
+      99.5809648113483,
+      99.60929241333818,
+      99.62438673274372,
+      99.63796477495107,
+      99.67707212055974,
+      99.68459730961506,
+      99.69307322063851,
+      99.71111077144124,
+      99.72737797363409,
+      99.72927241962775,
+      99.73992386598088,
+      99.73206205328829,
+      99.75304059841261,
+      99.74625014097215,
+      99.7707800466826,
+      99.77012443563484,
+      99.78271530937526,
+      99.77249336438979,
+      99.78805499701103,
+      99.76780650542119,
+      99.79138249217684,
+      99.78929234031276,
+      99.77875221580989,
+      99.77993544773133,
+      99.7985065781676,
+      99.7937067502285,
+      99.75634245933462,
+      99.80602241808027,
+      99.76128542608606,
+      99.78036010726599,
+      99.80860137704244,
+      99.78465228436214,
+      99.79156809841055,
+      99.8078430381027,
+      99.80682306002375,
+      99.7830527397521,
+      99.81093804777883,
+      99.81802204924622,
+      99.79978247973106,
+      99.8125459446214,
+      99.8088528105376,
+      99.79960042279423,
+      99.82061910648923,
+      99.82760283551141,
+      99.82694565125313,
+      99.822845762863,
+      99.79744820376354,
+      99.81170284577398,
+      99.82877730348034,
+      99.80279838482487,
+      99.82407550489143,
+      99.82956589430727,
+      99.8325243574224,
+      99.82919734410615,
+      99.82928811984671,
+      99.81610434028896,
+      99.80720015607606,
+      99.82549585410084,
+      99.8250307607211,
+      99.85618570655116,
+      99.83686235683265,
+      99.83983692486895,
+      99.84045808200017,
+      99.86310517529216,
+      99.84657159256479,
+      99.84985314102846,
+      99.87513843347593,
+      99.85069195449047,
+      99.87328542728262,
+      99.84071492108149,
+      99.88315666480699,
+      99.8697841462198,
+      99.87601525642776,
+      99.90616202690022,
+      99.89225924084204,
+      99.87980271065611,
+      99.88485658476407,
+      99.9213779920042,
+      99.88668730331234,
+      99.89598887801864,
+      99.87453076546434,
+      99.88362893964528,
+      99.91221668189266,
+      99.90641847217984,
+      99.88872976787793,
+      99.91293748911718,
+      99.8541837285021,
+      99.92072827699074,
+      99.90007519094543,
+      99.89612875960428,
+      99.91386124566772,
+      99.92167393980083,
+      99.88505051727267,
+      99.93392202799302,
+      99.89497269290015,
+      99.90978076726445,
+      99.91105825684177,
+      99.89994296749147,
+      99.89215998091679,
+      99.90112519527774,
+      99.84166838426802,
+      99.91066140673152,
+      99.89556775838399,
+      99.92371787348903,
+      99.9271971888408,
+      99.91110057488295,
+      99.92451350423998,
+      99.91363828179436,
+      99.90152158801267,
+      99.91826506590185,
+      99.87974156608614,
+      99.8841838941053,
+      99.89538533069859,
+      99.883117903587,
+      99.92044550517105,
+      99.91997279074886,
+      99.89724368415645,
+      99.91166251153295,
+      99.91128226376466,
+      99.89301521929514,
+      99.88497727829841,
+      99.90189855156096,
+      99.9037399175862,
+      99.93888231024867,
+      99.92574328119497,
+      99.95076340173313,
+      99.90906573116692,
+      99.9333472193293,
+      99.91916113415999,
+      99.90526081141329,
+      99.91011277603255,
+      99.88677161005248,
+      99.92618700522226,
+      99.8764182751722,
+      99.91818861071965,
+      99.89325486123131
+    ]
+  }
+}

analysis/workload_chars/compute_chars.py

@@ -0,0 +1,180 @@
+import json, sys, math, statistics as st
+from collections import defaultdict, Counter
+import matplotlib; matplotlib.use("Agg")
+import matplotlib.pyplot as plt
+import numpy as np
+
+PATH="/home/admin/cpfs/wjh/ali-trace/trace-glm5.1-formatted/051315-051317.jsonl"
+OUT="/tmp/wlc_out"; import os; os.makedirs(OUT, exist_ok=True)
+BLOCK=512
+# --- transparent cost model for C3 (clearly-labeled estimate; raw-timing validation pending) ---
+PREFILL_TOK_S=7000.0     # MB1: 32k->4.5s ~7100 tok/s effective on H20 / 30B-A3B
+TPOT_S=0.010             # ~10ms/token decode (crossover unloaded ~5ms, loaded ~25ms)
+
+def pct(v,p):
+    if not v: return float('nan')
+    s=sorted(v);k=(len(s)-1)*p;f=int(k)
+    return s[f] if f+1>=len(s) else s[f]+(s[f+1]-s[f])*(k-f)
+
+# ---------- Pass A: structure (scalars only) ----------
+parents={}; recs={}; childcount=Counter()
+for line in open(PATH):
+    if not line.strip(): continue
+    d=json.loads(line); cid=d["chat_id"]; pid=d["parent_chat_id"]
+    parents[cid]=pid
+    recs[cid]=(float(d["timestamp"]),int(d["input_length"]),int(d["output_length"]),int(d["turn"]))
+    if pid!="-1": childcount[pid]+=1
+print(f"[A] records={len(recs)}", file=sys.stderr)
+
+root_of={}
+def root(cid):
+    path=[];c=cid
+    while True:
+        if c in root_of:r=root_of[c];break
+        p=parents.get(c,"-1")
+        if p=="-1" or p not in recs:r=c;break
+        path.append(c);c=p
+    for x in path:root_of[x]=r
+    root_of[cid]=r;return r
+sessions=defaultdict(list)
+for cid in recs: sessions[root(cid)].append(cid)
+seq={r:sorted(m,key=lambda c:(recs[c][3],recs[c][0])) for r,m in sessions.items()}
+print(f"[A] sessions={len(seq)}", file=sys.stderr)
+
+# ---------- C1: mixture + turn tail + hazard ----------
+sr=mr=sm=mm=so=mo=0
+turns_per=[]
+for r,s in seq.items():
+    multi=len(s)>1; turns_per.append(len(s))
+    for c in s:
+        _,inl,outl,_=recs[c]
+        if multi: mr+=1;mm+=inl;mo+=outl
+        else: sr+=1;sm+=inl;so+=outl
+tot_r=sr+mr; tot_in=sm+mm; tot_out=so+mo
+cnt_turn=Counter()
+for r,s in seq.items():
+    for c in s: cnt_turn[recs[c][3]]+=1
+hazard={k: (cnt_turn[k+1]/cnt_turn[k] if cnt_turn[k] else 0) for k in range(1,30)}
+
+# ---------- C2/C3: per-turn resident vs new-prefill (scalar) + hash_ids reuse ----------
+by_in=defaultdict(list); by_new=defaultdict(list); by_out=defaultdict(list)
+by_reuse_hash=defaultdict(list)  # hash-block prefix stability: reused/parent_blocks
+store={}  # cid -> (blockset, in, out) for chats with pending children
+tot_new_prefill=0; tot_reused=0
+for line in open(PATH):
+    if not line.strip(): continue
+    d=json.loads(line); cid=d["chat_id"]; pid=d["parent_chat_id"]
+    inl=int(d["input_length"]); outl=int(d["output_length"]); turn=int(d["turn"])
+    blocks=set(d["hash_ids"])
+    if pid in store:
+        pblk,pin,pout=store[pid]
+        new_prefill=max(0, inl - pin - pout)              # actual recompute (accounts for cached answer)
+        reused_blk=len(blocks & pblk)
+        by_reuse_hash[turn].append(reused_blk/len(pblk) if pblk else 0)
+        childcount[pid]-=1
+        if childcount[pid]<=0: del store[pid]
+        tot_reused += (inl-new_prefill)
+    else:
+        new_prefill=inl                                    # session start: all new (intra-session)
+    tot_new_prefill+=new_prefill
+    by_in[turn].append(inl); by_new[turn].append(new_prefill); by_out[turn].append(outl)
+    if childcount[cid]>0: store[cid]=(blocks,inl,outl)
+print(f"[B] done; store residual={len(store)}", file=sys.stderr)
+
+TURNS=[t for t in sorted(by_in) if len(by_in[t])>=50]
+med_in=[pct(by_in[t],.5) for t in TURNS]
+med_new=[max(pct(by_new[t],.5),1) for t in TURNS]
+med_out=[pct(by_out[t],.5) for t in TURNS]
+ratio=[med_in[i]/med_new[i] for i in range(len(TURNS))]
+reuse_pct=[(1-med_new[i]/med_in[i])*100 for i in range(len(TURNS))]
+# C3 time per turn (cost model)
+t_pref=[med_new[i]/PREFILL_TOK_S for i in range(len(TURNS))]
+t_dec=[med_out[i]*TPOT_S for i in range(len(TURNS))]
+
+# aggregate decode/prefill time fraction over a RANGE of constants
+def agg_time(prate,tpot):
+    tp=tot_new_prefill/prate; td=tot_out*tpot; return td/(tp+td)
+frac_lo=agg_time(13000,0.005); frac_mid=agg_time(7000,0.010); frac_hi=agg_time(3000,0.025)
+
+chars={
+ "mixture":{"single_sessions":sr if False else sum(1 for s in seq.values() if len(s)==1),
+   "multi_sessions":sum(1 for s in seq.values() if len(s)>1),
+   "req_single_pct":sr/tot_r*100,"req_multi_pct":mr/tot_r*100,
+   "in_single_pct":sm/tot_in*100,"in_multi_pct":mm/tot_in*100,
+   "out_single_pct":so/tot_out*100,"out_multi_pct":mo/tot_out*100},
+ "turns":{"mean":st.mean(turns_per),"p99":pct(turns_per,.99),"max":max(turns_per),
+   "single_turn_pct":sum(1 for x in turns_per if x==1)/len(turns_per)*100},
+ "hazard":hazard,
+ "token_mass":{"total_input":tot_in,"total_output":tot_out,"out_in_ratio_pct":tot_out/tot_in*100,
+   "new_prefill":tot_new_prefill,"reused_prefix":tot_reused,
+   "new_prefill_pct_of_input":tot_new_prefill/tot_in*100},
+ "decode_time_fraction":{"optimistic_for_prefill":frac_lo,"mid":frac_mid,"pessimistic":frac_hi},
+ "per_turn":{"turn":TURNS,"med_resident_input":med_in,"med_new_prefill":med_new,
+   "med_output":med_out,"resident_over_new":ratio,"reuse_pct":reuse_pct},
+}
+json.dump(chars, open(f"{OUT}/chars.json","w"), indent=2)
+
+# ================= FIGURES =================
+plt.rcParams.update({"figure.dpi":140,"font.size":10,"axes.grid":True,"grid.alpha":.3})
+
+# ---- C1 ----
+fig,ax=plt.subplots(1,3,figsize=(15,4.2))
+cats=["% sessions","% requests","% input\ntokens","% output\ntokens"];
+singv=[chars["mixture"]["single_sessions"]/len(seq)*100, chars["mixture"]["req_single_pct"],
+       chars["mixture"]["in_single_pct"], chars["mixture"]["out_single_pct"]]
+multv=[100-x for x in singv]
+x=np.arange(len(cats))
+ax[0].bar(x,singv,label="single-turn",color="#7fb3d5")
+ax[0].bar(x,multv,bottom=singv,label="multi-turn",color="#e74c3c")
+for i in range(len(cats)):
+    ax[0].text(i,singv[i]/2,f"{singv[i]:.0f}",ha="center",va="center",fontsize=9)
+    ax[0].text(i,singv[i]+multv[i]/2,f"{multv[i]:.0f}",ha="center",va="center",color="white",fontsize=9)
+ax[0].set_xticks(x);ax[0].set_xticklabels(cats);ax[0].set_ylabel("%");ax[0].set_ylim(0,100)
+ax[0].set_title("C1a Mixture: 90% sessions single-turn,\nbut multi-turn carries 2/3 prefill mass");ax[0].legend(loc="center right")
+# turn CCDF log-log
+tc=sorted(turns_per); n=len(tc); xs=sorted(set(tc))
+ccdf=[sum(1 for v in tc if v>=xx)/n for xx in xs]
+ax[1].loglog(xs,ccdf,marker=".",ms=3,color="#34495e")
+ax[1].set_xlabel("turns per session (k)");ax[1].set_ylabel("P(turns >= k)")
+ax[1].set_title(f"C1b Heavy-tailed session length\n(p99={chars['turns']['p99']:.0f}, max={chars['turns']['max']})")
+# hazard
+hk=list(range(1,20)); hv=[hazard[k]*100 for k in hk]
+ax[2].plot(hk,hv,marker="o",color="#16a085")
+ax[2].set_xlabel("reached turn k");ax[2].set_ylabel("P(continue to k+1) %");ax[2].set_ylim(0,100)
+ax[2].set_title("C1c Continuation hazard rises 10%->94%\n(unpredictable at start, Lindy after)")
+fig.tight_layout(); fig.savefig(f"{OUT}/c1_session_mixture.png"); plt.close(fig)
+
+# ---- C2 ----
+fig,ax=plt.subplots(1,3,figsize=(15,4.2))
+ax[0].semilogy(TURNS,med_in,marker="o",label="resident context (input)",color="#e74c3c")
+ax[0].semilogy(TURNS,med_new,marker="s",label="new prefill this turn",color="#2980b9")
+ax[0].set_xlabel("turn");ax[0].set_ylabel("tokens (median, log)");ax[0].legend()
+ax[0].set_xlim(1,30)
+ax[0].set_title("C2a Resident state explodes,\nmarginal work collapses")
+ax[1].plot(TURNS,ratio,marker="o",color="#8e44ad")
+ax[1].set_xlabel("turn");ax[1].set_ylabel("resident / new-prefill");ax[1].set_xlim(1,30)
+ax[1].set_title("C2b The PD tax = resident/delta\n(grows to ~250x by deep turns)")
+ax[2].plot(TURNS,reuse_pct,marker="o",color="#27ae60")
+ax[2].set_xlabel("turn");ax[2].set_ylabel("per-turn reuse %");ax[2].set_ylim(50,100);ax[2].set_xlim(1,30)
+ax[2].set_title("C2c Per-turn reuse climbs to 99.6%\n(deep turns are near-pure cache hits)")
+fig.tight_layout(); fig.savefig(f"{OUT}/c2_work_amortization.png"); plt.close(fig)
+
+# ---- C3 ----
+fig,ax=plt.subplots(1,2,figsize=(11,4.4))
+# token mass decomposition
+vals=[tot_reused/1e9, tot_new_prefill/1e9, tot_out/1e9]
+labs=[f"reused prefix\n{tot_reused/tot_in*100:.0f}% of input",
+      f"new prefill\n{tot_new_prefill/tot_in*100:.0f}% of input",
+      f"decode output\n{tot_out/tot_in*100:.1f}% of input"]
+ax[0].bar(range(3),vals,color=["#95a5a6","#2980b9","#e67e22"])
+ax[0].set_xticks(range(3));ax[0].set_xticklabels(labs,fontsize=8.5)
+ax[0].set_ylabel("tokens (billions)")
+ax[0].set_title("C3a Token mass: prefill-dominated\n(but tokens != time, see C3b)")
+# per-turn prefill vs decode TIME (cost model)
+ax[1].semilogy(TURNS,t_pref,marker="o",label="prefill time (new tok / 7k·s⁻¹)",color="#2980b9")
+ax[1].semilogy(TURNS,t_dec,marker="s",label="decode time (out·10ms)",color="#e67e22")
+ax[1].set_xlabel("turn");ax[1].set_ylabel("seconds (median, log)");ax[1].legend(fontsize=8);ax[1].set_xlim(1,30)
+ax[1].set_title(f"C3b Prefill→decode bottleneck flips within a session\n(agg decode-time share ≈ {frac_mid*100:.0f}%, range {frac_lo*100:.0f}–{frac_hi*100:.0f}%)")
+fig.tight_layout(); fig.savefig(f"{OUT}/c3_prefill_decode_balance.png"); plt.close(fig)
+print("FIGURES + chars.json written to", OUT)
+print(json.dumps({k:chars[k] for k in ["mixture","turns","token_mass","decode_time_fraction"]}, indent=2))

figs/mb5/CORRECTION.md

@@ -0,0 +1,19 @@
+# ⚠️ Correction notice for figs/mb5/ (2026-05-30)
+
+These figures back `microbench/fresh_setup/PD_DISAGG_RESULTS.md`. A producer-side
+contamination was later found in the stack that produced them: commit **`e13391e`**
+(deployed over the "fresh" pip vLLM by `scripts/deploy_vllm_patches.sh`) evicts a
+producer's prefix-cache blocks on every KV transfer, so a disaggregated producer
+could never keep a session's prefix warm. It is now gated behind
+`VLLM_EVICT_SENT_BLOCKS` (default off) and everything was re-run clean.
+
+| figure | section | status |
+|---|---|---|
+| `mb5_producer_hotspot.png` | §6.3 session-affinity hot-pinning | 🛑 **RETRACTED** — pure `e13391e` artifact. On the clean stack, session-routed producers reach APC parity with colo (71–82%); there is no 0%-util stall / hot-pin pathology. |
+| `mb5_latency_compare.png` | §3 round-robin headline | ✅ stands — RR's ~0% prefix-hit is a *routing* artifact (consecutive turns → different producers), not the eviction bug; reproduced clean. |
+| `mb5_kv_timeline.png`, `mb5_role_split.png`, `mb5_peak_utilization.png` | §5 per-role KV pool occupancy | ✅ D-pool capacity-ceiling mechanism stands (decode pegs while prefill strands). P-pool occupancy may read slightly low under eviction; the qualitative split is unaffected. |
+| `mb5_summary.csv` | aggregate | mixed — §3/§5 rows valid; any session-affinity rows superseded. |
+
+**Superseded by the corrected three-axis ablation:** [`../mb5_pd_ablation/`](../mb5_pd_ablation/)
+(reuse / shape / concurrency), data in [`../../analysis/mb5_pd_ablation/`](../../analysis/mb5_pd_ablation/).
+Raw §6 data `analysis/mb5/session_prod.json` is contaminated; `analysis/mb5/rr_prod.json` (round-robin) stands.

figs/mb5/mb5_summary.csv

@@ -0,0 +1,5 @@
+config,rep,n_requests,n_success,wall_clock_s,peak_pool_frac,steady_pool_frac,p_pool_peak_frac,p_pool_steady_frac,d_pool_peak_frac,d_pool_steady_frac,peak_waiting,latency_p50_s,latency_p90_s,latency_p99_s,ttft_p50_s,ttft_p90_s,ttft_p99_s,prefix_cache_hit_ratio
+8C,1,1214,1214,2994.218414353032,0.7174957362137578,0.3439702956225128,,,,,29,10.82550932947197,83.34998885790122,194.10265863158946,6.967104309005663,53.12018221841427,114.12611859919207,0.1937163528742694
+6P+2D,1,1214,1214,3419.065942236979,0.7726478112563957,0.42145750426378625,0.743272692817889,0.3082291074474133,0.9959636156907333,0.7434906196702672,128,44.48975181748392,91.82252187062406,147.70196208347772,40.95952733900049,86.68752026481089,142.84028979733685,0.0
+4P+4D,1,1214,1214,4170.666486939997,0.6997939169982945,0.45876918703808983,0.6438459351904491,0.28540363843092664,0.9753411028993746,0.5977686185332576,152,59.52004547297838,157.08703426021387,224.03997302683115,56.419772224500775,153.07864206891392,219.73412787001706,0.0
+2P+6D,1,1214,109,5761.816568834998,0.9698692438885731,0.9435119386014781,0.9969869243888573,0.9198408186469585,0.9620238772029562,0.9494504453287853,872,26.293884326005355,499.3484142678091,577.7122636228032,23.580788671970367,498.0334587502061,576.5306194114453,0.0