v2: linear (default cache-aware) baseline + 2x wall-cap on first600s

Follow-up to the LMetric sweep: rerun with --policy linear (cache-aware
load + sticky session affinity, the cache_aware_proxy default) and cap
each PD-disagg arm at 2x the colo bench wall (SIGTERM bench.sh once cap
is exceeded; the cleanup trap clears vLLM and proxy; capped runs lack
metrics.summary.json so the analysis script computes from raw
metrics.jsonl).

Headline: the success-rate ceiling is policy-invariant.

  arm        linear (capped at 2x)    lmetric (uncapped)
  colo       807/807 = 100%, 964s     807/807 = 100%, 1021s
  pd6 (6:2)  472/807 =  58%, 2280s ⊗  474/807 =  59%, 3325s
  pd4 (4:4)  349/807 =  43%, 2281s ⊗  348/807 =  43%, 6850s
  pd2 (2:6)  176/807 =  22%, 2280s ⊗  180/807 =  22%, 19275s

Routing affects only how much wall is wasted timing out unreachable
requests at 600s each. Linear hits the same ceiling in 2280s as
LMetric does in 3300-19000s. This *strengthens* the §5 D-pool
capacity-ceiling thesis -- the cap is structural, not a routing
artifact.

Artifacts:
  analysis/v2/fig4r_linear.json          -- 4-arm linear summary
  analysis/v2/PD_DISAGG_LMETRIC.md       -- extended with wall-cap section
  figs/v2/fig4_linear_vs_lmetric.png     -- 3-panel side-by-side comparison
  microbench/fresh_setup/plot_fig4_linear_vs_lmetric.py
This commit is contained in:
2026-06-01 00:55:40 +08:00
parent 7529284cee
commit 32f7f55990
4 changed files with 163 additions and 1 deletions

View File

@@ -17,7 +17,9 @@ same wall-clock**). Every static **PD-disagg ratio fails** (1465 % completion
failure mode rotates predictably with the split — **no static partition has a working
operating point on this workload**. LMetric improves colo dramatically; it does *not*
rescue PD-disagg, confirming the bottleneck is structural (D-pool admission capacity +
multi-turn KV accumulation), not routing.
multi-turn KV accumulation), not routing. A follow-up linear-policy run with PD-disagg
**wall-capped at 2× the colo wall** (see end of doc) hits the **identical** success-rate
ceiling — confirming the cap is structural, not policy-driven.
## Setup
@@ -87,6 +89,52 @@ draining concurrently behind the multi-turn session causality.
pool capacity, and the decode-pool admission ceiling tips earlier. **PD-disagg is
worse on agentic than §3 advertised, not better.**
## Linear-policy + wall-cap follow-up (2026-06-01) — the success ceiling is policy-invariant
To check whether the LMetric routing was secretly handicapping PD-disagg, we re-ran
first600s with the **default `--policy linear`** (cache-aware load score + sticky
session affinity — the original baseline of the cache_aware_proxy stack) and
**wall-capped each PD-disagg arm at 2 × the colo wall** (kill bench.sh + cleanup
GPUs once cap is exceeded, record `records_at_cap`).
| arm | linear success | linear wall | linear @-cap? | lmetric success | lmetric wall |
|---|---|---|---|---|---|
| **colo** | 807/807 = **100 %** | 964 s | — | 807/807 = **100 %** | 1021 s |
| **pd6 (6:2)** | **472/807 = 58 %** | 2280 s | ⊗ cap (706 dispatched) | 474/807 = 59 % | 3325 s |
| **pd4 (4:4)** | **349/807 = 43 %** | 2281 s | ⊗ cap (577 dispatched) | 348/807 = 43 % | 6850 s |
| **pd2 (2:6)** | **176/807 = 22 %** | 2280 s | ⊗ cap (521 dispatched) | 180/807 = 22 % | 19275 s |
→ Figure: [`figs/v2/fig4_linear_vs_lmetric.png`](../../figs/v2/fig4_linear_vs_lmetric.png) ·
data: [`fig4r_linear.json`](fig4r_linear.json)
**Three clean conclusions from the wall-cap experiment:**
1. **The success-rate ceiling is structural, not a routing artifact.** Linear and
LMetric — two very different scoring policies (one session-sticky cache-aware,
the other non-sticky pure load) — converge on **identical success rates**
(58 / 43 / 22 %) for every PD-disagg ratio. Routing has *zero* effect on the
completion ceiling. The bottleneck is the static P:D split itself.
2. **LMetric's longer wall was wall *wasted on requests that will never succeed*.**
When the cap is enforced, linear hits the same ceiling in 2280 s as LMetric did
in 330019000 s — the extra wall just slowly times out the unreachable
requests at 600 s each.
3. **The wall-cap is the right way to bench PD-disagg.** Reporting "completion %"
without a wall budget is misleading (the bench eventually completes if you wait
forever, but only by counting timeouts as failures over hours). The honest
metric is **success-in-2×-colo-wall**, which gives the same answer for both
routings and matches what an end user would observe on a real SLO.
This **strengthens** the §5 D-pool capacity-ceiling thesis: even with
session-affinity routing serving every request to a warm prefix cache (which
*should* maximise PD's throughput), the static D-pool can't admit more than
~22 / 43 / 58 % of the agentic trace within 2× the colo budget. Colo wins not
because routing is smarter, but because its **elastic pool** absorbs whichever
phase is hot — there's no cap to hit.
---
## Reproduce
```bash
@@ -101,8 +149,17 @@ for TR in w600_r0.0015_st30.jsonl w600_r0.0015_st30_first600s.jsonl; do
done
python microbench/fresh_setup/plot_fig4l_lmetric.py
python microbench/fresh_setup/plot_fig4_linear_vs_lmetric.py
```
For the linear + 2× wall-cap variant, run colo first to get `wall_clock_s`,
compute `CAP=2*wall`, then launch each PD-disagg arm in the background and
`SIGTERM` it (so bench.sh's cleanup trap fires) once `date +%s` minus the
arm's start time exceeds `CAP`. The capped runs lack `metrics.summary.json`
(replayer was killed before it could write); compute the summary directly from
`metrics.jsonl` (see the inline collector used to build
`analysis/v2/fig4r_linear.json`).
Source `bench.sh` cleans GPUs before each arm and writes `metrics.jsonl` +
`metrics.summary.json` per tag. Aggregation script: see the inline JSON dump used
to build `analysis/v2/fig4l_lmetric.json`.