agentic-kvc/analysis/characterization/current_results/characterization_claim_matrix.md

Characterization Claim Matrix

Updated 2026-05-25 after Window 1 (B1' KV-footprint + reuse, B3 5-policy sweep, B2 PD-colo interference microbench).

Claim	Status	Supporting Data	Needed Next	Reviewer Risk
Per-session sequentiality is enforced when replayer + proxy carry the new join fields.	supported	A1 unix timestamps (t_dispatch/t_first_token/t_finish_unix) and X-Request-Id passthrough; smoke validation 2026-05-25 confirmed 30/30 join coverage.	Use this stack for all Window 2 B4/B5 SRR runs.	Legacy outputs/ without these fields still cannot be re-classified as online_realistic.
Agentic workload is long-input / short-output / heavy-tail session mass.	supported	Full trace CPU summary (full_trace_summary.json): input p50/p90/p99 = 20k/87.9k/125.5k; top 1% sessions hold 46.5% of input mass. Full trace 2.11M requests, 1.31M sessions.	—	Sample trace (w600) percentiles inherit from this full trace but should not be extrapolated.
KV per request for Qwen3-Coder-30B-A3B is 98304 B/token; p50/p90/p99 footprint = 1.83/8.04/11.49 GiB.	supported	window_1_results/kv_footprint_summary.json; computed from model config and full trace input lengths.	—	Assumes bf16; would scale for fp8/int8 quant.
Workload reuse is overwhelmingly intra-session.	supported	Real reuse decomposition from w600 lmetric run: intra 93.2%, cross 5.7%, shared 1.1% (window_1_results/lmetric_reuse.json). Theoretical any-vs-intra ceiling gap 0.7 pp.	—	Trace-specific; ChatGPT-style workloads with long system prompts would shift toward shared-prefix.
Theoretical APC ceiling on w600 trace is 79.6% (intra) / 80.3% (any-session).	supported	window_1_results/apc_upper_w600.json from block-level trie walk on hash_ids.	—	Assumes infinite per-worker cache (no eviction); achieved values must be read as fraction of this ceiling.
Cache-aware LMetric leaves a measurable locality gap (22.7 pp).	supported	lmetric achieved 56.9% vs intra-session ceiling 79.6%; B3 sweep window_1_results/b3_policy_comparison.json.	—	sticky data shows the gap can be recovered by harder affinity.
Hybrid affinity (unified) breaks the locality-vs-latency tradeoff.	supported	unified APC 79.4% (97% of intra ceiling) AND TTFT p90 7.24 s (lmetric is 15.6 s).	—	unified concentrates a single very hot worker (engine_4 at 37.7 s p90); hotspot_index 3.35.
Same-worker prefill-decode interference is causal, not correlation.	supported	B2 microbench: different-worker control idx 0.92-1.02 across 32× prefill-size variation; same-worker TTFT idx scales 2.15× (2k) → 218× (65k). window_1_results/b2_sweep_summary.json.	—	Synthetic decode load (256-token prompts at 4 req/s) bounds the realism; production behavior is layered on top of B3.
The cost of same-worker prefill interference migrates from TPOT to TTFT as prefill size grows past the chunked-prefill horizon.	supported	B2 same-worker TPOT p90 idx peaks at 32k (7.89×) and drops at 65k (2.26×), while TTFT idx grows monotonically (94.6× → 218×) and TPOT p99 grows monotonically (59 → 169.5 ms). See window_1_results.md "TPOT idx peaks at 32k, not 65k".	—	SLO thresholds for TTFT and TPOT cannot be the same under PD-colo; this should be reflected in B4 SRR sweep design.
Hard session affinity (sticky) inflates same-worker prefill-decode interference.	supported	sticky interference_index 13.65 vs lmetric 6.53; sticky's slow-request breakdown 57% same-worker overlap vs lmetric 23%.	—	Confirms the B2 causal claim observed at the system level.
Heavy-tail sessions are a contributor to hot-spot but not the sole cause.	supported	Cap-8 trace (37% requests dropped) reduces hotspot_index only ~10% (2.253 → 2.020 after fixing the joined_analysis.hotspot_index median bug).	Run capped under unified to see whether unified's hotspot also persists.	Reviewer might counter that cap=8 is too soft; a stricter cap could be tried.
B3 saturated-replay latency gaps include an agentic dispatch-coupling feedback term, which is intentional and matches production.	supported, framed as feature	replayer/replay.py:282-287 fires turn N+1 immediately when turn N is behind schedule (no human think-time). Under saturation, slow policies have longer mean session lifetime, more concurrent in-flight, higher worker pressure — so B3 latency gaps reflect "policy + feedback amplification", which is what a production operator switching policies on agentic workload experiences. See analysis/characterization/agentic_dispatch_coupling.md.	Run B4 open-loop Poisson at fixed λ to get the orthogonal "controlled-load" measurement; both are needed, not "B4 fixes B3".	Some reviewers will read "non-Poisson arrivals" as benchmark crime; the rebuttal is the agentic-vs-chat workload distinction.
SRR per policy under SLO is not yet measured.	not_yet_supported	B3 was driven by trace timestamps with strict session sequentiality; saturation is reached but not parameterized.	Run B4 with the A4 open-loop Poisson loadgen, per-class SLO, 5 policies × λ binary search.	Without B4 the paper cannot claim "policy X sustains higher load than Y".
Failure attribution near SRR boundary is not yet measured.	not_yet_supported	B5 protocol exists; no runs.	After B4, rerun each policy at 0.9× / 1.0× / 1.1× of its SRR_max with the same instrumentation, label slow requests.	The current joined_analysis.label_slow_requests is the labeler; needs SRR boundaries to point at.