v2: linear (default cache-aware) baseline + 2x wall-cap on first600s
Follow-up to the LMetric sweep: rerun with --policy linear (cache-aware load + sticky session affinity, the cache_aware_proxy default) and cap each PD-disagg arm at 2x the colo bench wall (SIGTERM bench.sh once cap is exceeded; the cleanup trap clears vLLM and proxy; capped runs lack metrics.summary.json so the analysis script computes from raw metrics.jsonl). Headline: the success-rate ceiling is policy-invariant. arm linear (capped at 2x) lmetric (uncapped) colo 807/807 = 100%, 964s 807/807 = 100%, 1021s pd6 (6:2) 472/807 = 58%, 2280s ⊗ 474/807 = 59%, 3325s pd4 (4:4) 349/807 = 43%, 2281s ⊗ 348/807 = 43%, 6850s pd2 (2:6) 176/807 = 22%, 2280s ⊗ 180/807 = 22%, 19275s Routing affects only how much wall is wasted timing out unreachable requests at 600s each. Linear hits the same ceiling in 2280s as LMetric does in 3300-19000s. This *strengthens* the §5 D-pool capacity-ceiling thesis -- the cap is structural, not a routing artifact. Artifacts: analysis/v2/fig4r_linear.json -- 4-arm linear summary analysis/v2/PD_DISAGG_LMETRIC.md -- extended with wall-cap section figs/v2/fig4_linear_vs_lmetric.png -- 3-panel side-by-side comparison microbench/fresh_setup/plot_fig4_linear_vs_lmetric.py
This commit is contained in:
@@ -17,7 +17,9 @@ same wall-clock**). Every static **PD-disagg ratio fails** (14–65 % completion
|
||||
failure mode rotates predictably with the split — **no static partition has a working
|
||||
operating point on this workload**. LMetric improves colo dramatically; it does *not*
|
||||
rescue PD-disagg, confirming the bottleneck is structural (D-pool admission capacity +
|
||||
multi-turn KV accumulation), not routing.
|
||||
multi-turn KV accumulation), not routing. A follow-up linear-policy run with PD-disagg
|
||||
**wall-capped at 2× the colo wall** (see end of doc) hits the **identical** success-rate
|
||||
ceiling — confirming the cap is structural, not policy-driven.
|
||||
|
||||
## Setup
|
||||
|
||||
@@ -87,6 +89,52 @@ draining concurrently behind the multi-turn session causality.
|
||||
pool capacity, and the decode-pool admission ceiling tips earlier. **PD-disagg is
|
||||
worse on agentic than §3 advertised, not better.**
|
||||
|
||||
## Linear-policy + wall-cap follow-up (2026-06-01) — the success ceiling is policy-invariant
|
||||
|
||||
To check whether the LMetric routing was secretly handicapping PD-disagg, we re-ran
|
||||
first600s with the **default `--policy linear`** (cache-aware load score + sticky
|
||||
session affinity — the original baseline of the cache_aware_proxy stack) and
|
||||
**wall-capped each PD-disagg arm at 2 × the colo wall** (kill bench.sh + cleanup
|
||||
GPUs once cap is exceeded, record `records_at_cap`).
|
||||
|
||||
| arm | linear success | linear wall | linear @-cap? | lmetric success | lmetric wall |
|
||||
|---|---|---|---|---|---|
|
||||
| **colo** | 807/807 = **100 %** | 964 s | — | 807/807 = **100 %** | 1021 s |
|
||||
| **pd6 (6:2)** | **472/807 = 58 %** | 2280 s | ⊗ cap (706 dispatched) | 474/807 = 59 % | 3325 s |
|
||||
| **pd4 (4:4)** | **349/807 = 43 %** | 2281 s | ⊗ cap (577 dispatched) | 348/807 = 43 % | 6850 s |
|
||||
| **pd2 (2:6)** | **176/807 = 22 %** | 2280 s | ⊗ cap (521 dispatched) | 180/807 = 22 % | 19275 s |
|
||||
|
||||
→ Figure: [`figs/v2/fig4_linear_vs_lmetric.png`](../../figs/v2/fig4_linear_vs_lmetric.png) ·
|
||||
data: [`fig4r_linear.json`](fig4r_linear.json)
|
||||
|
||||
**Three clean conclusions from the wall-cap experiment:**
|
||||
|
||||
1. **The success-rate ceiling is structural, not a routing artifact.** Linear and
|
||||
LMetric — two very different scoring policies (one session-sticky cache-aware,
|
||||
the other non-sticky pure load) — converge on **identical success rates**
|
||||
(58 / 43 / 22 %) for every PD-disagg ratio. Routing has *zero* effect on the
|
||||
completion ceiling. The bottleneck is the static P:D split itself.
|
||||
|
||||
2. **LMetric's longer wall was wall *wasted on requests that will never succeed*.**
|
||||
When the cap is enforced, linear hits the same ceiling in 2280 s as LMetric did
|
||||
in 3300–19000 s — the extra wall just slowly times out the unreachable
|
||||
requests at 600 s each.
|
||||
|
||||
3. **The wall-cap is the right way to bench PD-disagg.** Reporting "completion %"
|
||||
without a wall budget is misleading (the bench eventually completes if you wait
|
||||
forever, but only by counting timeouts as failures over hours). The honest
|
||||
metric is **success-in-2×-colo-wall**, which gives the same answer for both
|
||||
routings and matches what an end user would observe on a real SLO.
|
||||
|
||||
This **strengthens** the §5 D-pool capacity-ceiling thesis: even with
|
||||
session-affinity routing serving every request to a warm prefix cache (which
|
||||
*should* maximise PD's throughput), the static D-pool can't admit more than
|
||||
~22 / 43 / 58 % of the agentic trace within 2× the colo budget. Colo wins not
|
||||
because routing is smarter, but because its **elastic pool** absorbs whichever
|
||||
phase is hot — there's no cap to hit.
|
||||
|
||||
---
|
||||
|
||||
## Reproduce
|
||||
|
||||
```bash
|
||||
@@ -101,8 +149,17 @@ for TR in w600_r0.0015_st30.jsonl w600_r0.0015_st30_first600s.jsonl; do
|
||||
done
|
||||
|
||||
python microbench/fresh_setup/plot_fig4l_lmetric.py
|
||||
python microbench/fresh_setup/plot_fig4_linear_vs_lmetric.py
|
||||
```
|
||||
|
||||
For the linear + 2× wall-cap variant, run colo first to get `wall_clock_s`,
|
||||
compute `CAP=2*wall`, then launch each PD-disagg arm in the background and
|
||||
`SIGTERM` it (so bench.sh's cleanup trap fires) once `date +%s` minus the
|
||||
arm's start time exceeds `CAP`. The capped runs lack `metrics.summary.json`
|
||||
(replayer was killed before it could write); compute the summary directly from
|
||||
`metrics.jsonl` (see the inline collector used to build
|
||||
`analysis/v2/fig4r_linear.json`).
|
||||
|
||||
Source `bench.sh` cleans GPUs before each arm and writes `metrics.jsonl` +
|
||||
`metrics.summary.json` per tag. Aggregation script: see the inline JSON dump used
|
||||
to build `analysis/v2/fig4l_lmetric.json`.
|
||||
|
||||
Reference in New Issue
Block a user