agentic-kvc

Go to file

Gahow Wang e45f00eb68 Cache policy simulation: routing quality dominates, not eviction policy

With balanced session-sticky routing:
  LRU APC = 49.2% (only 1.8pp below infinite 51.0%)
  LFU APC = 43.5% (worse than LRU!)
  SessionProtLRU = 49.0% (no improvement)

The previous 10.1pp gap was from routing imbalance (all traffic to inst_0),
not from cache eviction policy. Balanced routing recovers 5.9pp of the gap.

Multi-turn sessions get 80.1% APC with simple LRU + session-sticky routing
because inter-turn gap is only 2 requests (LRU naturally keeps it warm).

Conclusion: fix routing balance, not cache policy.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

2026-05-22 01:28:53 +08:00

analysis

KV cache lifecycle design + eviction loss analysis

2026-05-22 01:27:22 +08:00

patches

Add vLLM patches directory for version-controlled patch management

2026-05-22 00:26:14 +08:00

replayer

Agentic workload PD separation analysis with trace-driven benchmarks

2026-05-21 21:21:57 +08:00

scripts

Cache policy simulation: routing quality dominates, not eviction policy