agentic-kvc

Go to file

Gahow Wang 1e1e2e774d Fix sampler: window+thin preserves cross-session KV cache sharing

Random session sampling destroys cross-session hash block sharing
(52% -> 16%) because sessions sharing system prompts get scattered.

New approach: take a contiguous time window from the trace (preserving
temporal locality of shared-prefix sessions), then thin within the
window to hit target QPS. This preserves both intra-session reuse
(62% of reusable tokens) and cross-session sharing (38%).

Results (block sharing rate):
  Old random r=0.002:  16.0%  ->  Window+thin: 29.7%
  Old random r=0.016:  19.5%  ->  Window+thin: 42.7%
  Full trace baseline: 52%

Also corrected the "91% intra-session" claim: actual split is
62% intra / 38% cross (token-level), making cross-session sharing
preservation critical for valid APC benchmarks.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

2026-05-23 14:03:12 +08:00

analysis

Add comprehensive research findings document

2026-05-23 07:16:31 +08:00

patches

Add vLLM patches directory for version-controlled patch management

2026-05-22 00:26:14 +08:00

replayer

Fix replay methodology: trace-driven dispatch, no artificial limits

2026-05-23 12:43:41 +08:00

scripts

Fix sampler: window+thin preserves cross-session KV cache sharing