Gahow Wang
baf7ffb08c
Key finding: at 16 concurrent sessions (2 per GPU), TPOT p90 degrades from 0.073 to 0.106 (+45%), with MEDIUM TPOT at 0.197 (+149%). This is the first time we've reproduced real prefill-decode interference in controlled experiments. Elastic RDMA at 16 sessions doesn't help: only 13/500 offloaded (cache-gate correct for cold turn-1), kv_both adds ~16% TPOT overhead at high concurrency. Load scaling: 1000req_ts20, 200req_ts10, 200req_ts5, 500req_ts10 all show ~30% GPU util at 8 sessions. The bottleneck is max_inflight_sessions, not arrival rate. Updated elastic_hypotheses.md with H8, H9, and comprehensive final analysis. The real bottleneck is vLLM's chunked prefill scheduling, not routing or PD disaggregation. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>