Gahow Wang
0c3220cbb8
analysis/characterization/window_1_results.md is the headline write-up for Window 1: workload characterization (KV per request, real reuse decomposition, APC theoretical ceilings), B3 5-policy sweep with per-policy interpretation, B2 same-vs-different-worker interference microbench with causal reading, and an explicit list of what Window 1 does *not* answer (deferred to B4 SRR sweep + B5 attribution). Under window_1_results/: - 5 raw result JSONs from the B3 sweep, the B2 microbench, the APC upper bound, and the KV footprint - per-policy hotspot_index.json snapshots so render_window1_figures.py can plot per-worker TTFT p90 distributions - 8 PNG figures (figures/) covering the headline claims Three takeaways the figures pin down: 1) intra-session reuse dominates (93.2%), so session-affinity routing is the right primary lever 2) unified hybrid affinity hits 79.4% APC (97% of the 79.6% intra- session ceiling) AND cuts TTFT p90 from lmetric's 15.6s to 7.24s 3) B2 different-worker control sits at idx ≈ 1.0 across 32× prefill- size variation; same-worker TTFT idx scales 2.15× -> 218×, which is the cleanest causal evidence for same-worker prefill-decode interference Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
18 lines
615 B
JSON
18 lines
615 B
JSON
{
|
|
"trace": "/home/admin/cpfs/wjh/agentic-kv/traces/w600_r0.0015_st30.jsonl",
|
|
"n_requests": 1214,
|
|
"n_sessions": 274,
|
|
"block_size": 512,
|
|
"shared_prefix_min_sessions": 8,
|
|
"total_input_tokens": 53335690,
|
|
"apc_upper_any_session": 0.8030439654947747,
|
|
"apc_upper_intra_session": 0.7956783534627564,
|
|
"apc_upper_shared_prefix_only": 0.0010271546126055554,
|
|
"cached_tokens_any_session": 42830904,
|
|
"cached_tokens_intra_session": 42438054,
|
|
"cached_tokens_shared_prefix_only": 54784,
|
|
"n_requests_any_hit": 961,
|
|
"n_requests_intra_hit": 914,
|
|
"n_requests_shared_hit": 107,
|
|
"n_shared_pos0_blocks": 1
|
|
} |