# Paper section: PD separation under agentic workloads

This directory collects everything produced for the "PD-sep is net negative
on agentic workloads" paper section. It is one section of a larger paper,
not the whole paper.

## Layout

```
analysis/pd_sep_paper_section/
├── README.md                       # this file
├── system_analysis.md              # why PD-sep loses despite compute-bound prefill (6 layers)
├── scripts/
│   ├── plot_workload.py            # C1: input/output CDF + KV reuse decomposition
│   ├── plot_roofline.py            # C6: prefill roofline at varying cache reuse
│   ├── plot_routing_lever.py       # C7: routing vs PD-sep as design levers
│   └── plot_kv_memory_wall.py      # KV mem-wall: the system-level explanation
└── figures/
    ├── fig_c1a_io_cdf.pdf          # input/output token CDF (from traces/w600_r0.0015_st30.jsonl)
    ├── fig_c1b_reuse.pdf           # KV reuse decomposition: 79% intra-session
    ├── fig_c6_roofline.pdf         # analytical roofline
    ├── fig_c7_routing_lever.pdf    # routing vs PD-sep (legacy data, footer caveat)
    └── fig_kv_memory_wall.pdf      # the explanatory figure for system_analysis.md
```

## Candidate claims -> figures (status)

| Claim | Figure | Status |
|---|---|---|
| C1a: agentic input distribution (p50=33.5k, p90=101k, p99=132k); I/O = 142x | `figures/fig_c1a_io_cdf.pdf` | **rendered** |
| C1b: 79% intra-session reuse + 0.8% cross-session | `figures/fig_c1b_reuse.pdf` | **rendered** |
| C2: PD-sep vs Combined headline numbers | (not yet) | **needs re-run without --enforce-eager on `traces/w600_r0.0015_st30.jsonl`** |
| C3: decode KV cache memory wall (time-series) | (not yet) | needs step-level vLLM telemetry during PD-sep run |
| C4: TTFT stacked breakdown (prefill / KV pull / decode wait) | (not yet) | needs per-request breakdown.json from PD-sep run |
| C5: cuda-graph ablation (eager vs cudagraph × Combined vs PD-sep) | (not yet) | needs the 2×2 matrix |
| C6: prefill stays compute-bound at 95% reuse | `figures/fig_c6_roofline.pdf` | **rendered** |
| C7: cache-aware routing is a larger lever than PD-sep | `figures/fig_c7_routing_lever.pdf` | **rendered** (legacy data, footer caveat) |
| KV-WALL: per-D-instance KV demand vs PD layout (system mechanism) | `figures/fig_kv_memory_wall.pdf` | **rendered** (analytical, audit constants in script) |

## System-level argument (`system_analysis.md`)

The doc answers: *if prefill stays compute-bound even at 95% reuse, why
does PD separation not help?* Six layers, each pointing to a figure in
this directory:

1. compute-bound is a *kernel* property, not a system claim
2. absolute prefill work after cache hit is small (~hundreds of ms savings ceiling)
3. PD separation relocates compute; it doesn't accelerate it
4. PD separation's costs (KV transfer, decode-side concentration) scale with workload size
5. **decode-side KV memory wall** — quantified in `fig_kv_memory_wall.pdf`
6. the DistServe / Splitwise assumption that silently breaks: `concurrent × KV/req / (N_D × HBM)` is ≪ 1 for chatbot but ≥ 1 for agentic at p90+ context

## In-place edits made for this task

These edits are in the repo, not in this directory, because they modify
existing launch scripts. `--enforce-eager` was removed so cuda graphs can be
captured — PD-sep's D-node is a particularly clean case for cuda-graph
benefit and the prior methodology suppressed it.

| File | Lines | Change |
|---|---|---|
| `scripts/bench.sh` | 150, 161 | drop `--enforce-eager` (elastic + baseline modes) |
| `scripts/launch_pd_mooncake.sh` | 47, 64 | drop `--enforce-eager` (P and D instances) |
| `scripts/launch_pd_separated.sh` | 52, 68 | drop `--enforce-eager` (P and D instances) |
| `scripts/launch_phase1_ps.sh` | 32, 43 | drop `--enforce-eager` (C and PS instances) |
| `scripts/launch_elastic_p2p.sh` | 57 | drop `--enforce-eager` (kv_both instances) |

`scripts/legacy/*.sh` are intentionally left as-is — they record the
configuration of past experiments.

`REPORT.md` and `analysis/pd_separation_analysis.md` still describe the
old `--enforce-eager` setup. Update them once the new runs land.

## Reproducing the figures

From repo root:

```bash
# C1 (needs traces/w600_r0.0015_st30.jsonl; ~1.2 MB, pull from dash0 if missing)
.venv/bin/python analysis/pd_sep_paper_section/scripts/plot_workload.py \
    --trace traces/w600_r0.0015_st30.jsonl

# C6 (analytical, runs anywhere with matplotlib)
.venv/bin/python analysis/pd_sep_paper_section/scripts/plot_roofline.py

# C7 (hardcoded REPORT.md §3.1 numbers; no inputs)
.venv/bin/python analysis/pd_sep_paper_section/scripts/plot_routing_lever.py

# KV mem-wall (analytical; audit constants at top of the script)
.venv/bin/python analysis/pd_sep_paper_section/scripts/plot_kv_memory_wall.py
```

All four default `--outdir` to `analysis/pd_sep_paper_section/figures`.

## Caveats / open items

- **C7 uses legacy data**. The footer of `fig_c7_routing_lever.pdf` says so:
  PD-sep numbers come from the random-sampled trace + `--enforce-eager`. Re-run
  on `traces/w600_r0.0015_st30.jsonl` with cuda-graphs on before paper-grade
  citation. The plotting code keeps the source numbers in a single `ROWS`
  table (top of `plot_routing_lever.py`) for a one-line swap.
- **C2/C3/C4/C5 figures are not produced** because the experiments have not
  been re-run. The 4h matrix proposed in the prior conversation turn
  (Combined + RR, Combined + cache-aware, PD-sep 4P+4D, PD-sep 6P+2D, plus
  eager-vs-cudagraph ablation, ×3 seeds) is the prerequisite.
- **C6 is analytical**, so it is independent of any re-run. The numbers
  match `scripts/compute_roofline.py` (constants are duplicated; if one
  changes, the other must change too).