Gahow Wang
0c3220cbb8
analysis/characterization/window_1_results.md is the headline write-up for Window 1: workload characterization (KV per request, real reuse decomposition, APC theoretical ceilings), B3 5-policy sweep with per-policy interpretation, B2 same-vs-different-worker interference microbench with causal reading, and an explicit list of what Window 1 does *not* answer (deferred to B4 SRR sweep + B5 attribution). Under window_1_results/: - 5 raw result JSONs from the B3 sweep, the B2 microbench, the APC upper bound, and the KV footprint - per-policy hotspot_index.json snapshots so render_window1_figures.py can plot per-worker TTFT p90 distributions - 8 PNG figures (figures/) covering the headline claims Three takeaways the figures pin down: 1) intra-session reuse dominates (93.2%), so session-affinity routing is the right primary lever 2) unified hybrid affinity hits 79.4% APC (97% of the 79.6% intra- session ceiling) AND cuts TTFT p90 from lmetric's 15.6s to 7.24s 3) B2 different-worker control sits at idx ≈ 1.0 across 32× prefill- size variation; same-worker TTFT idx scales 2.15× -> 218×, which is the cleanest causal evidence for same-worker prefill-decode interference Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
25 lines
921 B
JSON
25 lines
921 B
JSON
{
|
|
"hotspot_index_ttft_p90": 3.3497107140827365,
|
|
"per_worker_latency_p90_s": {
|
|
"http://127.0.0.1:8000": 41.42001512600109,
|
|
"http://127.0.0.1:8001": 12.4878579101933,
|
|
"http://127.0.0.1:8002": 22.462878945574648,
|
|
"http://127.0.0.1:8003": 15.501050900109117,
|
|
"http://127.0.0.1:8004": 39.956250199786155,
|
|
"http://127.0.0.1:8005": 36.69850301651168,
|
|
"http://127.0.0.1:8006": 10.116177947795954,
|
|
"http://127.0.0.1:8007": 20.35038618039107
|
|
},
|
|
"per_worker_ttft_p90_s": {
|
|
"http://127.0.0.1:8000": 11.264844838529825,
|
|
"http://127.0.0.1:8001": 3.6063860427122614,
|
|
"http://127.0.0.1:8002": 16.175747957825664,
|
|
"http://127.0.0.1:8003": 9.314684258581842,
|
|
"http://127.0.0.1:8004": 37.73397144810297,
|
|
"http://127.0.0.1:8005": 18.328030522551852,
|
|
"http://127.0.0.1:8006": 3.6328767628350773,
|
|
"http://127.0.0.1:8007": 7.772977900883419
|
|
},
|
|
"status": "supported"
|
|
}
|