PD_DISAGG_RESULTS §6.3: producer hot-pinning figure
Direct per-producer KV-pool evidence for the session-affinity backfire. At the same 4P+4D ratio: - round-robin: 4 producers within 1pp of each other (spread 0pp, CV 0.01) - session-affinity: spread 49pp (one producer ~93%, another 45%; CV 0.25) A 25x jump in producer load imbalance — heavy multi-turn sessions concentrate onto single producers, the same hot-pinning pathology as sticky routing in the colocated §3.3 study. plot_producer_hotspot.py: reduce (numpy, per-producer KV timeline from snapshots, runs on the serving host) + plot (matplotlib, 2-panel rr vs session comparison) — same two-stage pattern as aggregate_mb5.py. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -274,6 +274,14 @@ failed transfer also pins the producer's KV (it is not freed on
|
||||
`kv_load_failure_policy=fail`), compounding the stall until the pipeline
|
||||
deadlocks at ~0% utilization.
|
||||
|
||||
The per-producer KV-pool timelines make the hot-pinning direct. At the **same
|
||||
4P+4D ratio**, round-robin holds all four producers within **1 percentage
|
||||
point** of each other (spread 0pp, CV 0.01); session-affinity blows the spread
|
||||
open to **49 percentage points** (one producer pegged at ~93% while another
|
||||
sits at 45%, CV 0.25 — a 25× jump in load imbalance):
|
||||
|
||||

|
||||
|
||||
Producer-side prefix-cache hit in the degraded state is ~0.2% (vs round-robin's
|
||||
~5%) — session-affinity never even gets to *collect* the cache-reuse benefit it
|
||||
was supposed to provide, because the producers it concentrates load onto are
|
||||
|
||||
Reference in New Issue
Block a user