Balanced session-sticky routing improves APC from 44.7% to 48.7% (+4pp,
close to simulated 49.2%) but TTFT worsens by 30% and E2E by 23%.
Root cause: session-sticky creates load hotspots — some instances get
multiple heavy concurrent sessions, causing queue delays, despite higher
per-instance APC.
Key finding: APC optimization and latency optimization are in tension.
- Cache affinity (sticky) -> higher APC, worse load balance -> worse latency
- Load-based routing (old) -> lower APC, better load balance -> better latency
The optimal design must balance both dimensions, not optimize one at
the expense of the other.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
