Correct PD-disagg cost/benefit framing across repo

The §3.2 cost-vs-benefit math in commits 029821c (MB1 plot +
pd_cost_vs_benefit.png) and abde010 (RESULTS_SUMMARY.md) was wrong.

What was wrong:
  I framed PD-disagg's max phase-isolation benefit as "≤ decode duration
  of the new request (~50–200 ms)" — implicitly treating the benefit as
  per-request and bounded by that request's own decode. The correct
  accounting is per-prefill-event across all stalled streams:

      benefit_per_prefill = D × T_prefill × (1 − TPOT_baseline/TPOT_during)
                          ≈ D × T_prefill

  which the user pointed out follows from the chunked-prefill math
  (each of L/N chunks slows D ongoing decode steps from ~10 ms to t ms,
  summing to D × T_prefill).

Plug MB1 + MB2 numbers in:

  prefill size | T_prefill | T_transfer | D=8 benefit | cost/benefit
   2k tok      | 0.14 s    |     8 ms   |   1.1 s     |    0.7 %
  33k tok      | 4.5  s    |  320 ms    |  36   s     |    0.9 %
 125k tok      | 57   s    |  1.9 s     | 456   s     |    0.4 %

On the phase-isolation axis alone, PD-disagg WINS by 100×–250× — the
opposite of what the deleted figure showed.

The actual dominant reason static PD-disagg fails in agentic is the
D-side KV pool capacity wall (figs/f4b_pdsep_kv_wall.png) — p99
single-request KV is 11.5 GiB, per-D-instance pool is 38 GiB, so 4P+4D
halves system decode capacity. Colleague's 4P+4D experiment showed
TTFT p50 62× worse and success rate 99.5% → 52%, driven by pool
overflow + queueing, not by transfer latency.

Changes:
- figs/pd_cost_vs_benefit.png : DELETED (figure built on wrong math)
- microbench/fresh_setup/plot_mb1.py : drop the pd_cost_vs_benefit
  function; keep mb1_interference.png and update its title to note
  per-prefill aggregate stall = D × T_prefill (not capped by decode)
- figs/mb1_interference.png : regenerated, no misleading band annotation
- analysis/mb1/README.md : Summary block rewritten ("what MB1 measures";
  no more "max benefit = decode duration" claim); §3.2 implications
  section replaced with the corrected per-prefill-event table; explicit
  ⚠ Correction note documents what was wrong
- analysis/mb2/README.md : Summary block + §3.2 implications section
  rewritten the same way; ⚠ Correction note links to RESULTS_SUMMARY §4
- RESULTS_SUMMARY.md §4 + §6 : §4 reordered to lead with the D-side
  capacity argument (the real failure mode), MB1/MB2 demoted from
  "kill-shot for PD-disagg" to "supporting context inputs to a
  cost-benefit table that actually favors PD-disagg on this axis";
  §6 paper-claims list reordered to remove the wrong "PD-disagg loses
  on cost-vs-benefit" claim and replace with the corrected ones

PAPER_OUTLINE.md and MEETING.md were checked and never picked up this
specific wrong claim — they already (correctly) frame §3.2 around the
D-side KV memory wall.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
2026-05-27 22:02:52 +08:00
parent abde010b64
commit 4936c6b861
8 changed files with 438 additions and 215 deletions

View File

@@ -32,10 +32,14 @@ RUNDIR="${3:?usage: $0 <policy> <trace> <rundir>}"
# Auto-enable kv_both when the policy requires it.
# KV_CONNECTOR (Mooncake|Nixl) selects the underlying connector when KV_BOTH=1.
_KV_CONNECTOR_EXPLICIT="${KV_CONNECTOR:-}"
KV_CONNECTOR="${KV_CONNECTOR:-Mooncake}"
if [ "$POLICY" = "unified_v2" ] || [ "$POLICY" = "unified_kv_both" ]; then
if [ "$POLICY" = "unified_v2" ] || [ "$POLICY" = "unified_v3" ] || [ "$POLICY" = "unified_kv_both" ]; then
ENABLE_KV_BOTH=1
KV_CONNECTOR="Mooncake"
# honor explicit KV_CONNECTOR override (e.g. =Nixl); otherwise default Mooncake.
if [ -z "$_KV_CONNECTOR_EXPLICIT" ]; then
KV_CONNECTOR="Mooncake"
fi
fi
if [ "$POLICY" = "unified_nixl_both" ]; then
ENABLE_KV_BOTH=1
@@ -143,7 +147,10 @@ for i in $(seq 0 $((N_INSTANCES - 1))); do
combined_args="$combined_args http://127.0.0.1:$((BASE_PORT + i))"
done
proxy_extra=""
if [ "$ENABLE_KV_BOTH" = "1" ]; then
# Bootstrap ports only needed for Mooncake handshake. Nixl uses its own
# UCX side-channel and the proxy forwards kv_transfer_params from src's
# response body instead of pre-baking engine_id/bootstrap_addr.
if [ "$ENABLE_KV_BOTH" = "1" ] && [ "$KV_CONNECTOR" = "Mooncake" ]; then
bp_list=""
for i in $(seq 0 $((N_INSTANCES - 1))); do
if [ -z "$bp_list" ]; then
@@ -154,11 +161,15 @@ if [ "$ENABLE_KV_BOTH" = "1" ]; then
done
proxy_extra="--bootstrap-ports $bp_list"
fi
if [ "$ENABLE_KV_BOTH" = "1" ] && [ "$KV_CONNECTOR" = "Nixl" ]; then
proxy_extra="--connector-type nixl"
fi
nohup "$VENV/python" "$ROOT/scripts/cache_aware_proxy.py" \
--port "$PROXY_PORT" \
--combined $combined_args \
--policy "$POLICY" \
$proxy_extra \
${EXTRA_PROXY_ARGS:-} \
> "$RUNDIR/proxy.log" 2>&1 &
disown
tries=0