Paper section: system analysis + workload figures + KV-wall model

Adds the system-level argument resolving the roofline/PD-sep paradox.
Even at 95% cache reuse prefill stays compute-bound (the C6 roofline
fact), yet PD separation regresses TTFT 72%. The new system_analysis.md
walks through six layers showing why the roofline claim is necessary
but not sufficient, with the falsifiable condition being decode-side
KV memory budget: concurrent_decode * KV_per_req / (N_D * HBM_pool).

For chatbot this ratio is << 1 at any layout; for agentic at p90+
context it goes >> 1 under 4P+4D and 6P+2D, predicting the empirical
97% decode KV occupancy. fig_kv_memory_wall.pdf visualizes the model
with audit-able constants; fig_c1a/b ground the per-request KV-size
inputs in the actual sampled trace (input p50=33.5k, p90=101k,
intra-session reuse 79.2%).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

analysis/pd_sep_paper_section/README.md

@@ -9,28 +9,46 @@ not the whole paper.
 ```
 analysis/pd_sep_paper_section/
 ├── README.md                       # this file
+├── system_analysis.md              # why PD-sep loses despite compute-bound prefill (6 layers)
 ├── scripts/
 │   ├── plot_workload.py            # C1: input/output CDF + KV reuse decomposition
 │   ├── plot_roofline.py            # C6: prefill roofline at varying cache reuse
-│   └── plot_routing_lever.py       # C7: routing vs PD-sep as design levers
+│   ├── plot_routing_lever.py       # C7: routing vs PD-sep as design levers
+│   └── plot_kv_memory_wall.py      # KV mem-wall: the system-level explanation
 └── figures/
-    ├── fig_c6_roofline.pdf         # rendered locally (analytical, no trace needed)
-    ├── fig_c7_routing_lever.pdf    # rendered locally (from REPORT.md §3.1)
-    └── (fig_c1a_io_cdf.pdf,        # produced on dash0 when trace is available
-          fig_c1b_reuse.pdf)
+    ├── fig_c1a_io_cdf.pdf          # input/output token CDF (from traces/w600_r0.0015_st30.jsonl)
+    ├── fig_c1b_reuse.pdf           # KV reuse decomposition: 79% intra-session
+    ├── fig_c6_roofline.pdf         # analytical roofline
+    ├── fig_c7_routing_lever.pdf    # routing vs PD-sep (legacy data, footer caveat)
+    └── fig_kv_memory_wall.pdf      # the explanatory figure for system_analysis.md
 ```
 
 ## Candidate claims -> figures (status)
 
 | Claim | Figure | Status |
 |---|---|---|
-| C1: 98% prefill share + 91% intra-session KV reuse | `figures/fig_c1a_io_cdf.pdf`, `figures/fig_c1b_reuse.pdf` | **needs trace on dash0** |
+| C1a: agentic input distribution (p50=33.5k, p90=101k, p99=132k); I/O = 142x | `figures/fig_c1a_io_cdf.pdf` | **rendered** |
+| C1b: 79% intra-session reuse + 0.8% cross-session | `figures/fig_c1b_reuse.pdf` | **rendered** |
 | C2: PD-sep vs Combined headline numbers | (not yet) | **needs re-run without --enforce-eager on `traces/w600_r0.0015_st30.jsonl`** |
 | C3: decode KV cache memory wall (time-series) | (not yet) | needs step-level vLLM telemetry during PD-sep run |
 | C4: TTFT stacked breakdown (prefill / KV pull / decode wait) | (not yet) | needs per-request breakdown.json from PD-sep run |
 | C5: cuda-graph ablation (eager vs cudagraph × Combined vs PD-sep) | (not yet) | needs the 2×2 matrix |
 | C6: prefill stays compute-bound at 95% reuse | `figures/fig_c6_roofline.pdf` | **rendered** |
 | C7: cache-aware routing is a larger lever than PD-sep | `figures/fig_c7_routing_lever.pdf` | **rendered** (legacy data, footer caveat) |
+| KV-WALL: per-D-instance KV demand vs PD layout (system mechanism) | `figures/fig_kv_memory_wall.pdf` | **rendered** (analytical, audit constants in script) |
+
+## System-level argument (`system_analysis.md`)
+
+The doc answers: *if prefill stays compute-bound even at 95% reuse, why
+does PD separation not help?* Six layers, each pointing to a figure in
+this directory:
+
+1. compute-bound is a *kernel* property, not a system claim
+2. absolute prefill work after cache hit is small (~hundreds of ms savings ceiling)
+3. PD separation relocates compute; it doesn't accelerate it
+4. PD separation's costs (KV transfer, decode-side concentration) scale with workload size
+5. **decode-side KV memory wall** — quantified in `fig_kv_memory_wall.pdf`
+6. the DistServe / Splitwise assumption that silently breaks: `concurrent × KV/req / (N_D × HBM)` is ≪ 1 for chatbot but ≥ 1 for agentic at p90+ context
 
 ## In-place edits made for this task
 
@@ -58,7 +76,7 @@ old `--enforce-eager` setup. Update them once the new runs land.
 From repo root:
 
 ```bash
-# C1 (needs sampled trace on dash0)
+# C1 (needs traces/w600_r0.0015_st30.jsonl; ~1.2 MB, pull from dash0 if missing)
 .venv/bin/python analysis/pd_sep_paper_section/scripts/plot_workload.py \
     --trace traces/w600_r0.0015_st30.jsonl
 
@@ -67,9 +85,12 @@ From repo root:
 
 # C7 (hardcoded REPORT.md §3.1 numbers; no inputs)
 .venv/bin/python analysis/pd_sep_paper_section/scripts/plot_routing_lever.py
+
+# KV mem-wall (analytical; audit constants at top of the script)
+.venv/bin/python analysis/pd_sep_paper_section/scripts/plot_kv_memory_wall.py
 ```
 
-All three default `--outdir` to `analysis/pd_sep_paper_section/figures`.
+All four default `--outdir` to `analysis/pd_sep_paper_section/figures`.
 
 ## Caveats / open items
 

analysis/pd_sep_paper_section/scripts/plot_kv_memory_wall.py

@@ -0,0 +1,127 @@
+"""Decode-side KV cache memory budget as a function of per-request KV
+footprint and prefill/decode split.
+
+Idea: PD separation is equivalent to multiplying the per-D-instance KV
+demand by (N_total / N_D). For workloads with large per-request KV
+footprint (agentic), this concentration breaches the memory wall.
+
+The plot fixes the system-wide concurrent decode count to a steady-state
+estimate from the trace (QPS x avg_decode_seconds) and shows per-D-instance
+KV pool occupancy as a function of per-request KV footprint, one line per
+PD layout.
+
+All constants are documented at the top so they can be audited.
+"""
+import argparse
+from pathlib import Path
+
+import matplotlib
+matplotlib.use("Agg")
+import matplotlib.pyplot as plt
+import numpy as np
+
+# ---- Cluster constants (8x H20, vLLM 0.18.1) ----
+N_TOTAL_GPUS = 8
+HBM_PER_GPU_GB = 96
+MODEL_GB = 50            # Qwen3-30B-A3B MoE weights bf16
+ACTIVATION_OVERHEAD_GB = 18
+KV_POOL_PER_GPU_GB = HBM_PER_GPU_GB - MODEL_GB - ACTIVATION_OVERHEAD_GB  # ~28
+
+# ---- Workload steady-state ----
+# Peak QPS on the sampled trace = 1.6; mean E2E ~5s under Combined; both
+# numbers from REPORT.md. So at any instant ~8 decodes are alive.
+CONCURRENT_DECODE = 8
+
+# ---- KV footprint constants for Qwen3-30B-A3B ----
+# 2 (K+V) * 4 kv-heads * 128 head_dim * 2 bytes * 48 layers = 96 KB / token.
+KV_BYTES_PER_TOKEN = 2 * 4 * 128 * 2 * 48  # = 98304 bytes
+
+
+def kv_mb(seqlen_tokens):
+    return seqlen_tokens * KV_BYTES_PER_TOKEN / 1e6
+
+
+# Reference operating points (per-request KV size, MB)
+POINTS = [
+    ("chatbot avg (~2k input)",   kv_mb(2_000)),
+    ("agentic avg (33.6k input)", kv_mb(33_600)),
+    ("agentic p90 (101k input)",  kv_mb(101_000)),
+    ("agentic p99 (132k input)",  kv_mb(132_000)),
+]
+
+# PD layouts: (label, N_D, color, linestyle)
+LAYOUTS = [
+    ("Combined 8C  (N_D=8)",    8, "#2ca02c", "-"),
+    ("PD-sep 4P+4D (N_D=4)",    4, "#ff7f0e", "--"),
+    ("PD-sep 6P+2D (N_D=2)",    2, "#d62728", "-."),
+]
+
+
+def occupancy(kv_per_req_mb, n_d):
+    """Per-D-instance KV pool occupancy (fraction)."""
+    pool_mb = KV_POOL_PER_GPU_GB * 1024
+    demand_mb = CONCURRENT_DECODE * kv_per_req_mb / n_d
+    return demand_mb / pool_mb
+
+
+def plot(out_path):
+    fig, ax = plt.subplots(figsize=(8.5, 4.6))
+
+    kv_range_mb = np.logspace(0.0, 4.5, 400)  # 1 MB .. ~30 GB
+    for label, n_d, color, ls in LAYOUTS:
+        y = occupancy(kv_range_mb, n_d) * 100
+        ax.plot(kv_range_mb, y, color=color, lw=1.8, ls=ls, label=label)
+
+    # memory-wall threshold (vLLM starts queuing aggressively above ~90%)
+    ax.axhline(90, color="#888", ls=":", lw=1)
+    ax.text(1.2, 93, "memory wall (~90%, vLLM stops admitting new reqs)",
+            fontsize=8.5, color="#666")
+
+    # operating-point markers, labelled along the top edge
+    for name, kv in POINTS:
+        ax.axvline(kv, color="#777", lw=0.7, ls=(0, (1, 2)))
+        ax.text(kv, 198, name, fontsize=8, color="#333",
+                rotation=90, ha="right", va="top")
+
+    ax.set_xscale("log")
+    ax.set_xlim(50, 3e4)
+    ax.set_ylim(0, 200)
+    ax.set_xlabel("Per-request KV footprint (MB)")
+    ax.set_ylabel("Per-D-instance KV pool occupancy (%)")
+    ax.set_title(
+        "Decode-side KV concentration explains the PD-sep memory wall  "
+        f"(8x H20, KV pool ≈ {KV_POOL_PER_GPU_GB} GB/GPU, {CONCURRENT_DECODE} concurrent decodes)",
+        fontsize=10,
+    )
+    ax.grid(True, alpha=0.25, which="both")
+    ax.legend(loc="upper left", fontsize=9, framealpha=0.95)
+    fig.tight_layout()
+    fig.savefig(out_path, bbox_inches="tight")
+    plt.close(fig)
+    print(f"wrote {out_path}")
+    print(f"  KV/token: {KV_BYTES_PER_TOKEN/1024:.1f} KB  "
+          f"pool/GPU: {KV_POOL_PER_GPU_GB} GB  "
+          f"concurrent decodes: {CONCURRENT_DECODE}")
+    print()
+    print(f"  per-D occupancy at each operating point:")
+    print(f"  {'workload':28s}  {'KV/req':>10s}  "
+          f"{'Combined':>10s}  {'4P+4D':>10s}  {'6P+2D':>10s}")
+    for name, kv in POINTS:
+        c8 = occupancy(kv, 8) * 100
+        c4 = occupancy(kv, 4) * 100
+        c2 = occupancy(kv, 2) * 100
+        print(f"  {name:28s}  {kv:>7.0f} MB  "
+              f"{c8:>9.1f}%  {c4:>9.1f}%  {c2:>9.1f}%")
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--outdir", default="analysis/pd_sep_paper_section/figures")
+    args = ap.parse_args()
+    out = Path(args.outdir)
+    out.mkdir(parents=True, exist_ok=True)
+    plot(out / "fig_kv_memory_wall.pdf")
+
+
+if __name__ == "__main__":
+    main()

analysis/pd_sep_paper_section/scripts/plot_workload.py

@@ -160,29 +160,39 @@ def plot_reuse(rows, out_path):
     d = reuse_decomposition(rows)
     total = sum(d.values())
     parts = [
-        ("intra-session reuse",          d["intra_session_reuse_tokens"], "#2ca02c"),
-        ("cross-session reuse",          d["cross_session_reuse_tokens"], "#1f77b4"),
+        ("intra-session reuse",           d["intra_session_reuse_tokens"], "#2ca02c"),
+        ("cross-session reuse",           d["cross_session_reuse_tokens"], "#1f77b4"),
         ("first emission (reused later)", d["first_emission_will_reuse_tokens"], "#ff7f0e"),
         ("unique (never reused)",        d["unique_no_reuse_tokens"], "#d62728"),
     ]
 
-    fig, ax = plt.subplots(figsize=(8.5, 1.9))
+    fig, ax = plt.subplots(figsize=(9.0, 2.2))
     left = 0
+    handles = []
     for label, val, color in parts:
         frac = val / total
-        ax.barh(0, frac, left=left, color=color, edgecolor="white", height=0.6, label=label)
-        if frac > 0.025:
-            ax.text(left + frac / 2, 0,
-                    f"{label}\n{frac*100:.1f}%",
-                    ha="center", va="center", fontsize=8.5, color="white")
+        b = ax.barh(0, frac, left=left, color=color, edgecolor="white",
+                    height=0.55, label=f"{label}  ({frac*100:.1f}%)")
+        handles.append(b)
+        if frac > 0.04:
+            ax.text(left + frac / 2, 0, f"{frac*100:.1f}%",
+                    ha="center", va="center", fontsize=10,
+                    color="white", fontweight="bold")
         left += frac
 
     ax.set_xlim(0, 1)
+    ax.set_ylim(-0.6, 0.6)
     ax.set_yticks([])
-    ax.set_xlabel("share of total cacheable tokens (block-aligned, 512 tok blocks)")
-    ax.set_title("Where do prefix cache hits come from?  "
-                 f"(N={len(rows)} requests, sampled trace)")
-    ax.legend(loc="upper center", bbox_to_anchor=(0.5, -0.45), ncol=4, fontsize=8, frameon=False)
+    ax.set_xticks([0, 0.25, 0.5, 0.75, 1.0])
+    ax.set_xticklabels(["0%", "25%", "50%", "75%", "100%"])
+    ax.set_title(
+        f"Where do prefix cache hits come from?  (N={len(rows)} requests; "
+        "block-aligned 512-tok blocks)",
+        pad=8,
+    )
+    ax.legend(loc="upper center", bbox_to_anchor=(0.5, -0.20),
+              ncol=2, fontsize=9, frameon=False, handlelength=1.5,
+              columnspacing=2.5)
     for spine in ("top", "right", "left"):
         ax.spines[spine].set_visible(False)
     fig.tight_layout()

analysis/pd_sep_paper_section/system_analysis.md

@@ -0,0 +1,226 @@
+# Why does PD separation fail on agentic workloads even though prefill stays compute-bound?
+
+This is the central paradox of the section. The roofline in
+`figures/fig_c6_roofline.pdf` shows prefill is compute-bound at every
+realistic reuse level — at 95 % cache reuse, arithmetic intensity is still
+≈ 4,500 FLOP/byte, more than two orders of magnitude above the H20 ridge of
+37. The DistServe / Splitwise argument follows directly from that fact:
+"prefill is compute-heavy, decode is memory-bound, so isolate them onto
+different GPUs and specialize each."
+
+Yet on this workload, single-machine PD separation regresses TTFT by 72 %
+(REPORT.md §3.1) and saturates the decode-side KV pool at 97 % occupancy.
+This document explains, layer by layer, why a true premise (compute-bound)
+does not imply the conclusion (PD separation pays). All five layers are
+backed by either figures in this directory or measurements in
+`analysis/pd_separation_analysis.md`.
+
+The short answer: **the roofline tells you about the per-kernel efficiency
+of prefill. PD separation is a decision about a whole serving system. The
+gap between those two scales is where DistServe's argument loses force —
+and where agentic workloads, with their very large per-request KV
+footprint, push the system past a memory-capacity wall that chatbot
+workloads never reach.**
+
+---
+
+## Layer 1: compute-bound ≠ "needs dedicated GPUs"
+
+Roofline analysis classifies a *kernel*. It answers the question, "given
+that this kernel is running, is it bottlenecked by FLOP rate or by HBM
+bandwidth?" — it does **not** answer:
+
+- how long the kernel takes in wall-clock terms,
+- whether two kernels can profitably share a GPU,
+- whether moving the kernel to a different GPU makes it faster.
+
+PD separation needs the second and third answers, not the first. A 50 ms
+compute-bound prefill burst can perfectly well coexist with decode steps
+on the same GPU; you lose at most a fraction of a decode step's latency
+per chunk. Co-location only fails when prefill bursts grow long enough
+that decode requests starve.
+
+The DistServe paper's roofline argument is a *necessary* condition
+("prefill *can* be compute-bound, so dedicating GPUs is *not wasted*"). It
+is **not** a *sufficient* condition ("therefore dedicated GPUs pay").
+
+## Layer 2: in agentic, absolute prefill work after cache hit is small
+
+The roofline is computed in `figures/fig_c6_roofline.pdf` at a full 64 k
+context. But the operating point on the trace is shifted by prefix cache
+hits:
+
+| reuse | new tokens | prefill time @ ~7,000 tok/s |
+|---|---|---|
+| 0 % (turn 1 cold) | 64,000 | ~9 s |
+| 71 % (trace average) | 18,600 | ~2.6 s |
+| 95 % (deep multi-turn) | 3,200 | ~0.5 s |
+
+Average-case prefill is ~2.6 s of compute. With 8 GPUs and peak QPS 1.6,
+each GPU sees ~0.3 s of prefill work per second of wall-clock. Chunked
+prefill in vLLM slices this into ~8 k-token chunks of ~50–100 ms each, then
+yields to decode. The decode-side disturbance per HEAVY request is on the
+order of "a few hundred ms of stretched decode," not "seconds of stalled
+decode."
+
+PD separation, in its best case, eliminates this disturbance. The
+*ceiling* on the benefit is therefore: hundreds of ms per HEAVY request.
+This budget has to absorb everything PD separation costs.
+
+## Layer 3: PD separation relocates compute; it does not accelerate it
+
+A prefill kernel does the same FLOPs no matter which GPU runs it. PD
+separation's potential acceleration vectors are only two:
+
+1. **Larger prefill batch** → better SM utilization for the prefill MMA
+   kernels.
+2. **No chunked-prefill yield to decode** → no overhead per chunk handoff.
+
+Both are quantitatively negligible in this regime:
+
+1. At peak QPS 1.6 with ~2.6 s of prefill per request, *system-wide*
+   prefill concurrency averages to ~4 active prefills. A 4P split sees ~1
+   prefill per GPU at any moment, so batching gains are zero. A 6P split
+   makes it worse, not better. The roofline ceiling of 148 TFLOPS is
+   already reachable at batch=1 for sequences this long.
+
+2. Chunked prefill's per-chunk overhead is dominated by scheduler tick
+   time (≈ 1–2 ms), not the chunk transition. Removing it saves single
+   percentages of prefill time.
+
+So the *speedup side* of PD separation is bounded by the few-hundred-ms
+budget from Layer 2 and contains no hidden upside.
+
+## Layer 4: the costs of PD separation are workload-scaled
+
+PD separation adds two costs, both of which scale *up* with workload size:
+
+1. **KV transfer over the network.** Mooncake transfers KV block-by-block
+   after the full prefill completes (no layer-wise pipeline; see
+   `analysis/elastic_hypotheses.md` H5). Empirically, transfer takes
+   ~1.1 s p50 for HEAVY requests (~40 k tokens of KV ≈ 3.8 GB at
+   96 KB/token), with tail extending to 18–30 s. Transfer time grows with
+   context length.
+
+2. **Decode-side KV concentration.** All decode work is funneled onto a
+   subset of GPUs (4 of 8 in 4P+4D, 2 of 8 in 6P+2D). Per-D-instance KV
+   *demand* therefore scales by N_total / N_D. This is the killer cost;
+   Layer 5 quantifies it.
+
+Both costs scale linearly or worse with per-request KV footprint. KV
+footprint, in turn, scales linearly with input length. So PD separation
+gets *worse* exactly along the axis (long context) where the workload is
+moving.
+
+## Layer 5: the decode-side KV memory wall (the actual mechanism)
+
+Visualized in `figures/fig_kv_memory_wall.pdf`. The model is simple and
+its constants are auditable in
+`scripts/plot_kv_memory_wall.py`:
+
+```
+per-D occupancy = (concurrent_decode × KV_per_req) / (N_D × KV_pool_per_GPU)
+```
+
+with:
+
+- `KV_per_req` = `seqlen × 96 KB/token` for Qwen3-30B-A3B
+  (2 × 4 kv-heads × 128 head-dim × 2 bytes × 48 layers = 96 KB/tok)
+- `KV_pool_per_GPU` ≈ 28 GB (96 GB H20 HBM minus weights and activations)
+- `concurrent_decode` ≈ 8 at steady state (peak QPS 1.6 × mean decode
+  duration ~5 s under Combined)
+
+Plug in the trace's input distribution from
+`figures/fig_c1a_io_cdf.pdf`:
+
+| operating point | KV/req | Combined (N_D=8) | 4P+4D (N_D=4) | 6P+2D (N_D=2) |
+|---|---|---|---|---|
+| chatbot avg (2 k) | 197 MB | 0.7 % | 1.4 % | 2.7 % |
+| **agentic avg (33.6 k)** | **3.3 GB** | **12 %** | **23 %** | **46 %** |
+| **agentic p90 (101 k)** | **9.9 GB** | **35 %** | **69 %** | **138 %** ⚠ |
+| **agentic p99 (132 k)** | **13.0 GB** | **45 %** | **90 %** ⚠ | **181 %** ⚠ |
+
+vLLM's scheduler stops admitting new requests at ~90 % KV pool occupancy,
+so anything above the wall translates directly to queueing.
+
+Two consequences fall out of this table:
+
+1. **PD-sep with even a 4P+4D split breaches the wall at p99 context.**
+   p99 alone is ~1 % of requests but holds the GPU for tens of seconds of
+   decode, so its KV stays resident; over a long enough window the wall
+   gets hit even from the tail. With 6P+2D the wall is breached well
+   before p90.
+
+2. **For chatbot, the entire table sits under 3 %.** PD separation never
+   approaches the wall because chatbot per-request KV is 15× smaller. This
+   is the assumption DistServe inherited from its target workload, and
+   the assumption that silently breaks under agentic.
+
+The empirical KV occupancy on the 6P+2D run was 97 %
+(`analysis/pd_separation_analysis.md` §3.3) — the model and the
+measurement agree to within the resolution of the steady-state assumption.
+
+## Layer 6: the DistServe / Splitwise assumption that silently breaks
+
+To formalize: the regime in which PD separation pays is bounded by both a
+roofline condition *and* a memory-capacity condition:
+
+| Condition | Form | Chatbot | Agentic |
+|---|---|---|---|
+| Prefill is compute-bound | AI_prefill ≫ ridge | ✓ | ✓ |
+| Decode is memory-bound | AI_decode ≪ ridge | ✓ | ✓ |
+| Per-D-instance KV demand fits | concurrent × KV/req / (N_D × pool) < 1 | ✓ (≪ 1) | ✗ (>1 at p90+) |
+| KV transfer time ≪ saved interference | transfer_s ≪ saved_decode_stall_s | ✓ (KV is MB) | ✗ (KV is GB) |
+
+DistServe and Splitwise hold all four conditions implicitly in their
+short-context regime. Agentic violates the bottom two. Both violations
+have the same root cause: per-request KV footprint is 15–60× larger.
+
+This is the falsifiable claim of the section: **PD separation pays iff
+per-request KV footprint × decode concurrency stays well below the
+per-D-instance HBM pool. When that condition fails — and it fails
+unavoidably for long-context agentic workloads — PD separation is net
+negative regardless of how compute-bound prefill is.**
+
+The roofline doesn't tell you whether you're inside this regime; only the
+memory budget does.
+
+---
+
+## What this means for the paper section
+
+The figures we already have support this argument:
+
+- `fig_c1a_io_cdf.pdf` — establishes the input-length distribution
+  responsible for the large KV footprint (p50 33.5 k, p90 101 k, p99
+  132 k).
+- `fig_c1b_reuse.pdf` — establishes that 79 % of reuse is intra-session,
+  i.e. the request set has long-lived sessions whose KV must sit in the
+  pool through many decode steps.
+- `fig_c6_roofline.pdf` — establishes the prefill compute-bound fact.
+  This is the apparent contradiction the section resolves.
+- `fig_kv_memory_wall.pdf` — establishes the resolution: the memory budget
+  is what governs PD separation's viability, not the roofline.
+- `fig_c7_routing_lever.pdf` — establishes that cache-aware routing
+  recovers most of the benefit PD separation promises, without paying the
+  memory-wall cost.
+
+Missing for a rigorous re-grounding (deferred until the cudagraph re-run
+matrix lands):
+
+- Per-step decode KV utilization time-series from a live PD-sep run
+  (currently inferred from a single log snapshot of "Running: 0, Waiting:
+  6, KV cache: 97.1 %"). This would *directly* show the memory wall being
+  hit instead of relying on the steady-state model.
+- Per-request TTFT stacked breakdown (prefill, KV transfer, decode-side
+  wait) on the new trace; currently `analysis/pd_separation_analysis.md`
+  §3.3 has it on the old methodology.
+- CUDA-graph ablation: with `--enforce-eager` removed, PD-sep's D-node
+  could in principle close some of the per-step decode latency gap. The
+  Layer 5 model is gate-independent — wall demand grows with concurrency,
+  not per-step latency — so this should not change the conclusion. But
+  the section needs the measurement to say so honestly.
+
+The 4 h cudagraph experiment matrix (Combined / PD-sep × eager /
+cudagraph × 3 seeds) on `traces/w600_r0.0015_st30.jsonl` would settle
+those three items.