gahow/agentic-kvc
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

PD-disagg docs: annotated corrections for e13391e contamination

Browse Source 
Adds dated, non-destructive correction notes to the contaminated PD-vs-colo
artifacts after the producer-eviction bug (`evict_blocks(sent_block_ids)` on
`finished_sending`, deployed over the "fresh" pip vLLM by
`scripts/deploy_vllm_patches.sh`) was found and gated behind
`VLLM_EVICT_SENT_BLOCKS` (default off).

  PD_DISAGG_RESULTS.md  top CORRECTION banner + §6 RETRACTED marker.
                        §6 (session-affinity hot-pin) was an `e13391e`
                        artifact under controlled concurrency; §3 RR, §4
                        TPOT win, §5 D-pool ceiling, §5.1 consumer crash
                        stand.
  RESULTS_SUMMARY.md    §4 confirm+refine note: clean ablation confirms
                        the D-pool capacity thesis and adds regime-
                        dependence.
  pd_separation_analysis.md  scoped caution: thesis confirmed; flags
                        only reuse-dependent figures for cross-check
                        (this study used a different stack).
  figs/mb5/CORRECTION.md  flags mb5_producer_hotspot.png as retracted;
                        §3 RR and §5 D-pool figures stand.
This commit is contained in:
Gahow Wang
2026-05-31 20:14:14 +08:00
parent 0b180c191e
commit a2111b6e18
4 changed files with 101 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

16
analysis/pd_separation_analysis.md
View File

@@ -23,6 +23,22 @@ Per-request breakdown shows **87.7% of TTFT** is spent waiting for KV cache memo
> Earlier cross-machine comparison (commit `1e86285`) was invalidated — baseline used warm instances. See REPORT.md §3.5.
| **Delta** | **-45%** | **-36%** | **-44%** | **+30pp** |
> ✅⚠️ **2026-05-30 — confirmed + refined by the clean MB5 ablation; one caveat.**
> A producer-side contamination (`e13391e`: evicts a producer's prefix-cache on every
> KV transfer) was found in the *agentic-kv-fresh* MB5 stack and gated off; everything
> was re-run clean.
> - **Confirmed:** this doc's central thesis — PD's failure is a **decode-side KV memory
> wall**, not transfer/prefill cost — is reproduced on the clean stack (concurrency
> axis: at N=64 the split collapses, APC 71%→1.4%, TPS −30%; colo scales). Fig 3.
> - **Refined:** "PD separation is net negative" is **regime-dependent**, not universal.
> Clean ablation shows PD *wins* at low load / decode-heavy / low-reuse and loses the
> **agentic corner** (high reuse + short output + large context + high concurrency).
> - **Caveat (cross-check):** if this study's runs used the fork vLLM that contains
> `e13391e`, any **producer prefix-cache / APC reuse** figures here (e.g. §5.3.1) may be
> understated. The decode-memory-wall result is *not* reuse-dependent and is unaffected.
> On the clean stack, session-routed producers reach APC parity with colo (71–82%).
> Figures: [`figs/mb5_pd_ablation/`](../figs/mb5_pd_ablation/).
---
## 1. Workload Characterization