Window 1 results: combined B1' + B2 + B3 report and artifacts
analysis/characterization/window_1_results.md is the headline write-up for Window 1: workload characterization (KV per request, real reuse decomposition, APC theoretical ceilings), B3 5-policy sweep with per-policy interpretation, B2 same-vs-different-worker interference microbench with causal reading, and an explicit list of what Window 1 does *not* answer (deferred to B4 SRR sweep + B5 attribution). Under window_1_results/: - 5 raw result JSONs from the B3 sweep, the B2 microbench, the APC upper bound, and the KV footprint - per-policy hotspot_index.json snapshots so render_window1_figures.py can plot per-worker TTFT p90 distributions - 8 PNG figures (figures/) covering the headline claims Three takeaways the figures pin down: 1) intra-session reuse dominates (93.2%), so session-affinity routing is the right primary lever 2) unified hybrid affinity hits 79.4% APC (97% of the 79.6% intra- session ceiling) AND cuts TTFT p90 from lmetric's 15.6s to 7.24s 3) B2 different-worker control sits at idx ≈ 1.0 across 32× prefill- size variation; same-worker TTFT idx scales 2.15× -> 218×, which is the cleanest causal evidence for same-worker prefill-decode interference Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,26 @@
|
||||
{
|
||||
"formula": "kv_bytes_per_request = input_tokens * kv_bytes_per_token",
|
||||
"kv_bytes_per_request": {
|
||||
"count": 2114220,
|
||||
"max": 19893878784.0,
|
||||
"mean": 3306689367.3278427,
|
||||
"min": 0.0,
|
||||
"p50": 1969029120.0,
|
||||
"p90": 8636507750.40001,
|
||||
"p95": 10296164352.0,
|
||||
"p99": 12339806208.0
|
||||
},
|
||||
"kv_bytes_per_token": 98304.0,
|
||||
"kv_mib_per_request": {
|
||||
"count": 2114220,
|
||||
"max": 18972.28125,
|
||||
"mean": 3153.5047219541957,
|
||||
"min": 0.0,
|
||||
"p50": 1877.8125,
|
||||
"p90": 8236.415625000009,
|
||||
"p95": 9819.1875,
|
||||
"p99": 11768.15625
|
||||
},
|
||||
"status": "available",
|
||||
"total_kv_gib": 6510940.188720703
|
||||
}
|
||||
Reference in New Issue
Block a user