diff --git a/analysis/pd_sep_paper_section/README.md b/analysis/pd_sep_paper_section/README.md index aeb7e45..0fd3fa5 100644 --- a/analysis/pd_sep_paper_section/README.md +++ b/analysis/pd_sep_paper_section/README.md @@ -9,28 +9,46 @@ not the whole paper. ``` analysis/pd_sep_paper_section/ ├── README.md # this file +├── system_analysis.md # why PD-sep loses despite compute-bound prefill (6 layers) ├── scripts/ │ ├── plot_workload.py # C1: input/output CDF + KV reuse decomposition │ ├── plot_roofline.py # C6: prefill roofline at varying cache reuse -│ └── plot_routing_lever.py # C7: routing vs PD-sep as design levers +│ ├── plot_routing_lever.py # C7: routing vs PD-sep as design levers +│ └── plot_kv_memory_wall.py # KV mem-wall: the system-level explanation └── figures/ - ├── fig_c6_roofline.pdf # rendered locally (analytical, no trace needed) - ├── fig_c7_routing_lever.pdf # rendered locally (from REPORT.md §3.1) - └── (fig_c1a_io_cdf.pdf, # produced on dash0 when trace is available - fig_c1b_reuse.pdf) + ├── fig_c1a_io_cdf.pdf # input/output token CDF (from traces/w600_r0.0015_st30.jsonl) + ├── fig_c1b_reuse.pdf # KV reuse decomposition: 79% intra-session + ├── fig_c6_roofline.pdf # analytical roofline + ├── fig_c7_routing_lever.pdf # routing vs PD-sep (legacy data, footer caveat) + └── fig_kv_memory_wall.pdf # the explanatory figure for system_analysis.md ``` ## Candidate claims -> figures (status) | Claim | Figure | Status | |---|---|---| -| C1: 98% prefill share + 91% intra-session KV reuse | `figures/fig_c1a_io_cdf.pdf`, `figures/fig_c1b_reuse.pdf` | **needs trace on dash0** | +| C1a: agentic input distribution (p50=33.5k, p90=101k, p99=132k); I/O = 142x | `figures/fig_c1a_io_cdf.pdf` | **rendered** | +| C1b: 79% intra-session reuse + 0.8% cross-session | `figures/fig_c1b_reuse.pdf` | **rendered** | | C2: PD-sep vs Combined headline numbers | (not yet) | **needs re-run without --enforce-eager on `traces/w600_r0.0015_st30.jsonl`** | | C3: decode KV cache memory wall (time-series) | (not yet) | needs step-level vLLM telemetry during PD-sep run | | C4: TTFT stacked breakdown (prefill / KV pull / decode wait) | (not yet) | needs per-request breakdown.json from PD-sep run | | C5: cuda-graph ablation (eager vs cudagraph × Combined vs PD-sep) | (not yet) | needs the 2×2 matrix | | C6: prefill stays compute-bound at 95% reuse | `figures/fig_c6_roofline.pdf` | **rendered** | | C7: cache-aware routing is a larger lever than PD-sep | `figures/fig_c7_routing_lever.pdf` | **rendered** (legacy data, footer caveat) | +| KV-WALL: per-D-instance KV demand vs PD layout (system mechanism) | `figures/fig_kv_memory_wall.pdf` | **rendered** (analytical, audit constants in script) | + +## System-level argument (`system_analysis.md`) + +The doc answers: *if prefill stays compute-bound even at 95% reuse, why +does PD separation not help?* Six layers, each pointing to a figure in +this directory: + +1. compute-bound is a *kernel* property, not a system claim +2. absolute prefill work after cache hit is small (~hundreds of ms savings ceiling) +3. PD separation relocates compute; it doesn't accelerate it +4. PD separation's costs (KV transfer, decode-side concentration) scale with workload size +5. **decode-side KV memory wall** — quantified in `fig_kv_memory_wall.pdf` +6. the DistServe / Splitwise assumption that silently breaks: `concurrent × KV/req / (N_D × HBM)` is ≪ 1 for chatbot but ≥ 1 for agentic at p90+ context ## In-place edits made for this task @@ -58,7 +76,7 @@ old `--enforce-eager` setup. Update them once the new runs land. From repo root: ```bash -# C1 (needs sampled trace on dash0) +# C1 (needs traces/w600_r0.0015_st30.jsonl; ~1.2 MB, pull from dash0 if missing) .venv/bin/python analysis/pd_sep_paper_section/scripts/plot_workload.py \ --trace traces/w600_r0.0015_st30.jsonl @@ -67,9 +85,12 @@ From repo root: # C7 (hardcoded REPORT.md §3.1 numbers; no inputs) .venv/bin/python analysis/pd_sep_paper_section/scripts/plot_routing_lever.py + +# KV mem-wall (analytical; audit constants at top of the script) +.venv/bin/python analysis/pd_sep_paper_section/scripts/plot_kv_memory_wall.py ``` -All three default `--outdir` to `analysis/pd_sep_paper_section/figures`. +All four default `--outdir` to `analysis/pd_sep_paper_section/figures`. ## Caveats / open items diff --git a/analysis/pd_sep_paper_section/figures/fig_c1a_io_cdf.pdf b/analysis/pd_sep_paper_section/figures/fig_c1a_io_cdf.pdf new file mode 100644 index 0000000..d7bebf2 Binary files /dev/null and b/analysis/pd_sep_paper_section/figures/fig_c1a_io_cdf.pdf differ diff --git a/analysis/pd_sep_paper_section/figures/fig_c1b_reuse.pdf b/analysis/pd_sep_paper_section/figures/fig_c1b_reuse.pdf new file mode 100644 index 0000000..4f33de1 Binary files /dev/null and b/analysis/pd_sep_paper_section/figures/fig_c1b_reuse.pdf differ diff --git a/analysis/pd_sep_paper_section/figures/fig_kv_memory_wall.pdf b/analysis/pd_sep_paper_section/figures/fig_kv_memory_wall.pdf new file mode 100644 index 0000000..323ca7c Binary files /dev/null and b/analysis/pd_sep_paper_section/figures/fig_kv_memory_wall.pdf differ diff --git a/analysis/pd_sep_paper_section/scripts/plot_kv_memory_wall.py b/analysis/pd_sep_paper_section/scripts/plot_kv_memory_wall.py new file mode 100644 index 0000000..77d2d82 --- /dev/null +++ b/analysis/pd_sep_paper_section/scripts/plot_kv_memory_wall.py @@ -0,0 +1,127 @@ +"""Decode-side KV cache memory budget as a function of per-request KV +footprint and prefill/decode split. + +Idea: PD separation is equivalent to multiplying the per-D-instance KV +demand by (N_total / N_D). For workloads with large per-request KV +footprint (agentic), this concentration breaches the memory wall. + +The plot fixes the system-wide concurrent decode count to a steady-state +estimate from the trace (QPS x avg_decode_seconds) and shows per-D-instance +KV pool occupancy as a function of per-request KV footprint, one line per +PD layout. + +All constants are documented at the top so they can be audited. +""" +import argparse +from pathlib import Path + +import matplotlib +matplotlib.use("Agg") +import matplotlib.pyplot as plt +import numpy as np + +# ---- Cluster constants (8x H20, vLLM 0.18.1) ---- +N_TOTAL_GPUS = 8 +HBM_PER_GPU_GB = 96 +MODEL_GB = 50 # Qwen3-30B-A3B MoE weights bf16 +ACTIVATION_OVERHEAD_GB = 18 +KV_POOL_PER_GPU_GB = HBM_PER_GPU_GB - MODEL_GB - ACTIVATION_OVERHEAD_GB # ~28 + +# ---- Workload steady-state ---- +# Peak QPS on the sampled trace = 1.6; mean E2E ~5s under Combined; both +# numbers from REPORT.md. So at any instant ~8 decodes are alive. +CONCURRENT_DECODE = 8 + +# ---- KV footprint constants for Qwen3-30B-A3B ---- +# 2 (K+V) * 4 kv-heads * 128 head_dim * 2 bytes * 48 layers = 96 KB / token. +KV_BYTES_PER_TOKEN = 2 * 4 * 128 * 2 * 48 # = 98304 bytes + + +def kv_mb(seqlen_tokens): + return seqlen_tokens * KV_BYTES_PER_TOKEN / 1e6 + + +# Reference operating points (per-request KV size, MB) +POINTS = [ + ("chatbot avg (~2k input)", kv_mb(2_000)), + ("agentic avg (33.6k input)", kv_mb(33_600)), + ("agentic p90 (101k input)", kv_mb(101_000)), + ("agentic p99 (132k input)", kv_mb(132_000)), +] + +# PD layouts: (label, N_D, color, linestyle) +LAYOUTS = [ + ("Combined 8C (N_D=8)", 8, "#2ca02c", "-"), + ("PD-sep 4P+4D (N_D=4)", 4, "#ff7f0e", "--"), + ("PD-sep 6P+2D (N_D=2)", 2, "#d62728", "-."), +] + + +def occupancy(kv_per_req_mb, n_d): + """Per-D-instance KV pool occupancy (fraction).""" + pool_mb = KV_POOL_PER_GPU_GB * 1024 + demand_mb = CONCURRENT_DECODE * kv_per_req_mb / n_d + return demand_mb / pool_mb + + +def plot(out_path): + fig, ax = plt.subplots(figsize=(8.5, 4.6)) + + kv_range_mb = np.logspace(0.0, 4.5, 400) # 1 MB .. ~30 GB + for label, n_d, color, ls in LAYOUTS: + y = occupancy(kv_range_mb, n_d) * 100 + ax.plot(kv_range_mb, y, color=color, lw=1.8, ls=ls, label=label) + + # memory-wall threshold (vLLM starts queuing aggressively above ~90%) + ax.axhline(90, color="#888", ls=":", lw=1) + ax.text(1.2, 93, "memory wall (~90%, vLLM stops admitting new reqs)", + fontsize=8.5, color="#666") + + # operating-point markers, labelled along the top edge + for name, kv in POINTS: + ax.axvline(kv, color="#777", lw=0.7, ls=(0, (1, 2))) + ax.text(kv, 198, name, fontsize=8, color="#333", + rotation=90, ha="right", va="top") + + ax.set_xscale("log") + ax.set_xlim(50, 3e4) + ax.set_ylim(0, 200) + ax.set_xlabel("Per-request KV footprint (MB)") + ax.set_ylabel("Per-D-instance KV pool occupancy (%)") + ax.set_title( + "Decode-side KV concentration explains the PD-sep memory wall " + f"(8x H20, KV pool ≈ {KV_POOL_PER_GPU_GB} GB/GPU, {CONCURRENT_DECODE} concurrent decodes)", + fontsize=10, + ) + ax.grid(True, alpha=0.25, which="both") + ax.legend(loc="upper left", fontsize=9, framealpha=0.95) + fig.tight_layout() + fig.savefig(out_path, bbox_inches="tight") + plt.close(fig) + print(f"wrote {out_path}") + print(f" KV/token: {KV_BYTES_PER_TOKEN/1024:.1f} KB " + f"pool/GPU: {KV_POOL_PER_GPU_GB} GB " + f"concurrent decodes: {CONCURRENT_DECODE}") + print() + print(f" per-D occupancy at each operating point:") + print(f" {'workload':28s} {'KV/req':>10s} " + f"{'Combined':>10s} {'4P+4D':>10s} {'6P+2D':>10s}") + for name, kv in POINTS: + c8 = occupancy(kv, 8) * 100 + c4 = occupancy(kv, 4) * 100 + c2 = occupancy(kv, 2) * 100 + print(f" {name:28s} {kv:>7.0f} MB " + f"{c8:>9.1f}% {c4:>9.1f}% {c2:>9.1f}%") + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("--outdir", default="analysis/pd_sep_paper_section/figures") + args = ap.parse_args() + out = Path(args.outdir) + out.mkdir(parents=True, exist_ok=True) + plot(out / "fig_kv_memory_wall.pdf") + + +if __name__ == "__main__": + main() diff --git a/analysis/pd_sep_paper_section/scripts/plot_workload.py b/analysis/pd_sep_paper_section/scripts/plot_workload.py index e5fc37e..f1be864 100644 --- a/analysis/pd_sep_paper_section/scripts/plot_workload.py +++ b/analysis/pd_sep_paper_section/scripts/plot_workload.py @@ -160,29 +160,39 @@ def plot_reuse(rows, out_path): d = reuse_decomposition(rows) total = sum(d.values()) parts = [ - ("intra-session reuse", d["intra_session_reuse_tokens"], "#2ca02c"), - ("cross-session reuse", d["cross_session_reuse_tokens"], "#1f77b4"), + ("intra-session reuse", d["intra_session_reuse_tokens"], "#2ca02c"), + ("cross-session reuse", d["cross_session_reuse_tokens"], "#1f77b4"), ("first emission (reused later)", d["first_emission_will_reuse_tokens"], "#ff7f0e"), ("unique (never reused)", d["unique_no_reuse_tokens"], "#d62728"), ] - fig, ax = plt.subplots(figsize=(8.5, 1.9)) + fig, ax = plt.subplots(figsize=(9.0, 2.2)) left = 0 + handles = [] for label, val, color in parts: frac = val / total - ax.barh(0, frac, left=left, color=color, edgecolor="white", height=0.6, label=label) - if frac > 0.025: - ax.text(left + frac / 2, 0, - f"{label}\n{frac*100:.1f}%", - ha="center", va="center", fontsize=8.5, color="white") + b = ax.barh(0, frac, left=left, color=color, edgecolor="white", + height=0.55, label=f"{label} ({frac*100:.1f}%)") + handles.append(b) + if frac > 0.04: + ax.text(left + frac / 2, 0, f"{frac*100:.1f}%", + ha="center", va="center", fontsize=10, + color="white", fontweight="bold") left += frac ax.set_xlim(0, 1) + ax.set_ylim(-0.6, 0.6) ax.set_yticks([]) - ax.set_xlabel("share of total cacheable tokens (block-aligned, 512 tok blocks)") - ax.set_title("Where do prefix cache hits come from? " - f"(N={len(rows)} requests, sampled trace)") - ax.legend(loc="upper center", bbox_to_anchor=(0.5, -0.45), ncol=4, fontsize=8, frameon=False) + ax.set_xticks([0, 0.25, 0.5, 0.75, 1.0]) + ax.set_xticklabels(["0%", "25%", "50%", "75%", "100%"]) + ax.set_title( + f"Where do prefix cache hits come from? (N={len(rows)} requests; " + "block-aligned 512-tok blocks)", + pad=8, + ) + ax.legend(loc="upper center", bbox_to_anchor=(0.5, -0.20), + ncol=2, fontsize=9, frameon=False, handlelength=1.5, + columnspacing=2.5) for spine in ("top", "right", "left"): ax.spines[spine].set_visible(False) fig.tight_layout() diff --git a/analysis/pd_sep_paper_section/system_analysis.md b/analysis/pd_sep_paper_section/system_analysis.md new file mode 100644 index 0000000..1a03059 --- /dev/null +++ b/analysis/pd_sep_paper_section/system_analysis.md @@ -0,0 +1,226 @@ +# Why does PD separation fail on agentic workloads even though prefill stays compute-bound? + +This is the central paradox of the section. The roofline in +`figures/fig_c6_roofline.pdf` shows prefill is compute-bound at every +realistic reuse level — at 95 % cache reuse, arithmetic intensity is still +≈ 4,500 FLOP/byte, more than two orders of magnitude above the H20 ridge of +37. The DistServe / Splitwise argument follows directly from that fact: +"prefill is compute-heavy, decode is memory-bound, so isolate them onto +different GPUs and specialize each." + +Yet on this workload, single-machine PD separation regresses TTFT by 72 % +(REPORT.md §3.1) and saturates the decode-side KV pool at 97 % occupancy. +This document explains, layer by layer, why a true premise (compute-bound) +does not imply the conclusion (PD separation pays). All five layers are +backed by either figures in this directory or measurements in +`analysis/pd_separation_analysis.md`. + +The short answer: **the roofline tells you about the per-kernel efficiency +of prefill. PD separation is a decision about a whole serving system. The +gap between those two scales is where DistServe's argument loses force — +and where agentic workloads, with their very large per-request KV +footprint, push the system past a memory-capacity wall that chatbot +workloads never reach.** + +--- + +## Layer 1: compute-bound ≠ "needs dedicated GPUs" + +Roofline analysis classifies a *kernel*. It answers the question, "given +that this kernel is running, is it bottlenecked by FLOP rate or by HBM +bandwidth?" — it does **not** answer: + +- how long the kernel takes in wall-clock terms, +- whether two kernels can profitably share a GPU, +- whether moving the kernel to a different GPU makes it faster. + +PD separation needs the second and third answers, not the first. A 50 ms +compute-bound prefill burst can perfectly well coexist with decode steps +on the same GPU; you lose at most a fraction of a decode step's latency +per chunk. Co-location only fails when prefill bursts grow long enough +that decode requests starve. + +The DistServe paper's roofline argument is a *necessary* condition +("prefill *can* be compute-bound, so dedicating GPUs is *not wasted*"). It +is **not** a *sufficient* condition ("therefore dedicated GPUs pay"). + +## Layer 2: in agentic, absolute prefill work after cache hit is small + +The roofline is computed in `figures/fig_c6_roofline.pdf` at a full 64 k +context. But the operating point on the trace is shifted by prefix cache +hits: + +| reuse | new tokens | prefill time @ ~7,000 tok/s | +|---|---|---| +| 0 % (turn 1 cold) | 64,000 | ~9 s | +| 71 % (trace average) | 18,600 | ~2.6 s | +| 95 % (deep multi-turn) | 3,200 | ~0.5 s | + +Average-case prefill is ~2.6 s of compute. With 8 GPUs and peak QPS 1.6, +each GPU sees ~0.3 s of prefill work per second of wall-clock. Chunked +prefill in vLLM slices this into ~8 k-token chunks of ~50–100 ms each, then +yields to decode. The decode-side disturbance per HEAVY request is on the +order of "a few hundred ms of stretched decode," not "seconds of stalled +decode." + +PD separation, in its best case, eliminates this disturbance. The +*ceiling* on the benefit is therefore: hundreds of ms per HEAVY request. +This budget has to absorb everything PD separation costs. + +## Layer 3: PD separation relocates compute; it does not accelerate it + +A prefill kernel does the same FLOPs no matter which GPU runs it. PD +separation's potential acceleration vectors are only two: + +1. **Larger prefill batch** → better SM utilization for the prefill MMA + kernels. +2. **No chunked-prefill yield to decode** → no overhead per chunk handoff. + +Both are quantitatively negligible in this regime: + +1. At peak QPS 1.6 with ~2.6 s of prefill per request, *system-wide* + prefill concurrency averages to ~4 active prefills. A 4P split sees ~1 + prefill per GPU at any moment, so batching gains are zero. A 6P split + makes it worse, not better. The roofline ceiling of 148 TFLOPS is + already reachable at batch=1 for sequences this long. + +2. Chunked prefill's per-chunk overhead is dominated by scheduler tick + time (≈ 1–2 ms), not the chunk transition. Removing it saves single + percentages of prefill time. + +So the *speedup side* of PD separation is bounded by the few-hundred-ms +budget from Layer 2 and contains no hidden upside. + +## Layer 4: the costs of PD separation are workload-scaled + +PD separation adds two costs, both of which scale *up* with workload size: + +1. **KV transfer over the network.** Mooncake transfers KV block-by-block + after the full prefill completes (no layer-wise pipeline; see + `analysis/elastic_hypotheses.md` H5). Empirically, transfer takes + ~1.1 s p50 for HEAVY requests (~40 k tokens of KV ≈ 3.8 GB at + 96 KB/token), with tail extending to 18–30 s. Transfer time grows with + context length. + +2. **Decode-side KV concentration.** All decode work is funneled onto a + subset of GPUs (4 of 8 in 4P+4D, 2 of 8 in 6P+2D). Per-D-instance KV + *demand* therefore scales by N_total / N_D. This is the killer cost; + Layer 5 quantifies it. + +Both costs scale linearly or worse with per-request KV footprint. KV +footprint, in turn, scales linearly with input length. So PD separation +gets *worse* exactly along the axis (long context) where the workload is +moving. + +## Layer 5: the decode-side KV memory wall (the actual mechanism) + +Visualized in `figures/fig_kv_memory_wall.pdf`. The model is simple and +its constants are auditable in +`scripts/plot_kv_memory_wall.py`: + +``` +per-D occupancy = (concurrent_decode × KV_per_req) / (N_D × KV_pool_per_GPU) +``` + +with: + +- `KV_per_req` = `seqlen × 96 KB/token` for Qwen3-30B-A3B + (2 × 4 kv-heads × 128 head-dim × 2 bytes × 48 layers = 96 KB/tok) +- `KV_pool_per_GPU` ≈ 28 GB (96 GB H20 HBM minus weights and activations) +- `concurrent_decode` ≈ 8 at steady state (peak QPS 1.6 × mean decode + duration ~5 s under Combined) + +Plug in the trace's input distribution from +`figures/fig_c1a_io_cdf.pdf`: + +| operating point | KV/req | Combined (N_D=8) | 4P+4D (N_D=4) | 6P+2D (N_D=2) | +|---|---|---|---|---| +| chatbot avg (2 k) | 197 MB | 0.7 % | 1.4 % | 2.7 % | +| **agentic avg (33.6 k)** | **3.3 GB** | **12 %** | **23 %** | **46 %** | +| **agentic p90 (101 k)** | **9.9 GB** | **35 %** | **69 %** | **138 %** ⚠ | +| **agentic p99 (132 k)** | **13.0 GB** | **45 %** | **90 %** ⚠ | **181 %** ⚠ | + +vLLM's scheduler stops admitting new requests at ~90 % KV pool occupancy, +so anything above the wall translates directly to queueing. + +Two consequences fall out of this table: + +1. **PD-sep with even a 4P+4D split breaches the wall at p99 context.** + p99 alone is ~1 % of requests but holds the GPU for tens of seconds of + decode, so its KV stays resident; over a long enough window the wall + gets hit even from the tail. With 6P+2D the wall is breached well + before p90. + +2. **For chatbot, the entire table sits under 3 %.** PD separation never + approaches the wall because chatbot per-request KV is 15× smaller. This + is the assumption DistServe inherited from its target workload, and + the assumption that silently breaks under agentic. + +The empirical KV occupancy on the 6P+2D run was 97 % +(`analysis/pd_separation_analysis.md` §3.3) — the model and the +measurement agree to within the resolution of the steady-state assumption. + +## Layer 6: the DistServe / Splitwise assumption that silently breaks + +To formalize: the regime in which PD separation pays is bounded by both a +roofline condition *and* a memory-capacity condition: + +| Condition | Form | Chatbot | Agentic | +|---|---|---|---| +| Prefill is compute-bound | AI_prefill ≫ ridge | ✓ | ✓ | +| Decode is memory-bound | AI_decode ≪ ridge | ✓ | ✓ | +| Per-D-instance KV demand fits | concurrent × KV/req / (N_D × pool) < 1 | ✓ (≪ 1) | ✗ (>1 at p90+) | +| KV transfer time ≪ saved interference | transfer_s ≪ saved_decode_stall_s | ✓ (KV is MB) | ✗ (KV is GB) | + +DistServe and Splitwise hold all four conditions implicitly in their +short-context regime. Agentic violates the bottom two. Both violations +have the same root cause: per-request KV footprint is 15–60× larger. + +This is the falsifiable claim of the section: **PD separation pays iff +per-request KV footprint × decode concurrency stays well below the +per-D-instance HBM pool. When that condition fails — and it fails +unavoidably for long-context agentic workloads — PD separation is net +negative regardless of how compute-bound prefill is.** + +The roofline doesn't tell you whether you're inside this regime; only the +memory budget does. + +--- + +## What this means for the paper section + +The figures we already have support this argument: + +- `fig_c1a_io_cdf.pdf` — establishes the input-length distribution + responsible for the large KV footprint (p50 33.5 k, p90 101 k, p99 + 132 k). +- `fig_c1b_reuse.pdf` — establishes that 79 % of reuse is intra-session, + i.e. the request set has long-lived sessions whose KV must sit in the + pool through many decode steps. +- `fig_c6_roofline.pdf` — establishes the prefill compute-bound fact. + This is the apparent contradiction the section resolves. +- `fig_kv_memory_wall.pdf` — establishes the resolution: the memory budget + is what governs PD separation's viability, not the roofline. +- `fig_c7_routing_lever.pdf` — establishes that cache-aware routing + recovers most of the benefit PD separation promises, without paying the + memory-wall cost. + +Missing for a rigorous re-grounding (deferred until the cudagraph re-run +matrix lands): + +- Per-step decode KV utilization time-series from a live PD-sep run + (currently inferred from a single log snapshot of "Running: 0, Waiting: + 6, KV cache: 97.1 %"). This would *directly* show the memory wall being + hit instead of relying on the steady-state model. +- Per-request TTFT stacked breakdown (prefill, KV transfer, decode-side + wait) on the new trace; currently `analysis/pd_separation_analysis.md` + §3.3 has it on the old methodology. +- CUDA-graph ablation: with `--enforce-eager` removed, PD-sep's D-node + could in principle close some of the per-step decode latency gap. The + Layer 5 model is gate-independent — wall demand grows with concurrency, + not per-step latency — so this should not change the conclusion. But + the section needs the measurement to say so honestly. + +The 4 h cudagraph experiment matrix (Combined / PD-sep × eager / +cudagraph × 3 seeds) on `traces/w600_r0.0015_st30.jsonl` would settle +those three items.