agentic-kvc/analysis/mb5_pd_ablation/README.md

PD-disagg vs colocation — controlled reuse & concurrency axes (v2)

Self-contained results for the controlled-variable redo of the MB5 PD-vs-colo ablation. Supersedes the confounded first cut (held input fixed and sliced the prefix, so "more reuse" was entangled with "less prefill"). All arms route through the proxy at fair APC parity (session-routed producers reach the same prefix-cache hit rate as colo), so PD loses on structure, not on broken cache.

• Config arms: colo = 8×kv_both (8C-proxy, session-affinity); PD = 6P+2D / 4P+4D / 2P+6D.
• Driver: closed-loop N (REPLAY_MAX_INFLIGHT) + fixed think-time; gen_synthetic_trace.py --mode regular.
• PD-arm wall-cap: collapsed PD arms drain pathologically slowly, so PD arms run with a wall-deadline (REPLAY_MAX_DURATION; un-run turns counted as failures → honest completion%); colo is uncapped so the reference is always fully measured.
• Hardware: run on dash2 (8×H20). dash0's RDMA NICs were faulted for Mooncake during this work (could not init the transfer engine; needs an admin reset — no sudo); dash2's NICs are healthy. cpfs/venv/data are shared across the boxes.

---

1. Reuse / APC axis — fixed real prefill, vary cached prefix

N=8. Hold the real new-prefill work per turn constant (--delta-len) and grow the cached prefix → reuse = prefix/(prefix+delta). Three shapes isolate output vs delta:

	delta (real prefill/turn)	output	role
A	2048	256	original
C	2048	128	A vs C = pure output 256→128
B	1024	128	C vs B = pure delta 2048→1024

PD-best advantage = colo E2E-p90 / best-PD E2E-p90 (>1 ⇒ PD wins):

reuse%	A d2048/o256	C d2048/o128	B d1024/o128
20	1.34	1.41	—
50	1.36	1.37	—
67	1.47	1.49	1.27
80	1.31	1.23	1.25
90	1.15	1.01	—
95	0.87	0.89	0.89

Findings:

1. Output length is ~negligible. A and C (same delta) track each other across the whole range — halving output barely moves PD's advantage.
2. Delta (real prefill/turn) is the dominant shape factor. B (delta=1024) sits clearly below A/C at mid reuse (67%: 1.27 vs ~1.48). More real prefill per turn → bigger PD win, because PD-disagg's benefit is isolating prefill from decode — more prefill to isolate.
3. Crossover to colo at reuse ~90–95% is robust across all three shapes: PD always loses the high-reuse / large-resident-context corner (it must KV-transfer the whole resident context every turn for a few hundred new tokens; colo keeps it local).

Caveat: the clean, uncapped, 100%-completion comparison region is reuse 20–80% (carries findings 1–2). At reuse 90/95% the PD arms collapse and C's points are capped-completion, while A/B are full-drain — comparable in direction, not in exact PD completion%.

Data: fig1_reuse_fixed.json (A), fig1_reuse_d2048_o128.json (C), fig1_reuse_d1024_o128.json (B).

---

2. Concurrency axis — agentic corner, sweep N

in=32768 (prefix 32256 + delta 512, reuse 0.984), out=128; closed-loop N ∈ {8,16,32,48,64,96,128}; PD arms capped 600s, colo uncapped.

N	colo completion · E2E-mean · TPS	best PD-arm completion
8	256/256 · 2.4s · 326	6P+2D 256/256
16	512/512 · 3.5s · 462	6P+2D 439/512 (86%)
32	1024/1024 · 13.3s · 190	all PD <27%
48	1536/1536 · 24.9s · 168	all PD <32%
64	2048/2048 · 38.4s · 166	all PD <31%
96	3072/3072 · 60.0s · 171	PD 2–7%
128	4096/4096 · 80.8s · 181	4P+4D 6%, 2P+6D <1%

Finding: colo completes 100% of requests at every concurrency level — it degrades gracefully (latency rises 2.4s→81s, nothing dropped). Every static PD split collapses, and progressively earlier as N rises: PD is viable only at N≤8–16; by N≥32 it drops 70–99% of requests while its prefix-cache hit-rate craters to ~0%. colo's elastic pool absorbs the time-varying P/D demand; the static partition + per-turn 32k KV-transfer cannot. (Latency percentiles count successes only, so they understate PD — read them with the completion column.)

Data: fig3_conc32k.json.

Caveat: N=128 6P+2D is missing (one transient vLLM/Mooncake startup flake at the end); does not change the picture (all PD arms are already collapsed by N=128). The SLO auto-stop in the driver is a no-op (a stdout-capture bug), so the full grid ran — more points, not fewer.

---

3. Reproduce

# on a box with healthy Mooncake/RDMA NICs (dash2), cpfs mounted:
R=/home/admin/cpfs/wjh/agentic-kv-fresh
# reuse axis (three shapes): DELTA/OL pick the shape; tag carries _o${OL}
ssh dash2 "cd $R && DELTA=2048 OL=256 bash microbench/fresh_setup/run_reuse_fixed.sh"
ssh dash2 "cd $R && DELTA=2048 OL=128 bash microbench/fresh_setup/run_reuse_fixed.sh"
ssh dash2 "cd $R && DELTA=1024 OL=128 bash microbench/fresh_setup/run_reuse_fixed.sh"
# concurrency axis (capped):
ssh dash2 "cd $R && NLIST='8 16 32 48 64 96 128' CONC_PD_MAXDUR=600 bash microbench/fresh_setup/run_conc.sh"
# render (reads the *.json in this dir):
python microbench/fresh_setup/plot_pd_crossover.py