Files
agentic-kvc/analysis/characterization/current_results/characterization_claim_matrix.md

2.5 KiB

Characterization Claim Matrix

Claim Status Supporting Data Needed Next Reviewer Risk
Batch 0 substrate audit is only partially complete for existing runs. partially_supported metrics.jsonl lacks actual dispatch/finish timestamps in current artifacts. Add request dispatch and finish/error timestamps to future replayer/proxy metrics. Cannot use these runs to prove online per-session sequentiality.
Batch 1 workload shape can be characterized from formatted traces and metrics. supported_for_trace_shape Full compact trace CPU summary in full_trace_summary.json: input p50/p90/p99 = 20k/87.9k/125.5k, output p50/p90/p99 = 80/811/6.6k, top 1% sessions hold 46.5% of input-token mass. Add cache-hit joined records for actual reuse decomposition. Actual cache reuse decomposition needs cached_tokens joined with hash_ids.
Static PD separation is worse than combined in existing 200-request GPU A/B. supported_by_existing_artifact outputs/gpu_ab_combined vs outputs/gpu_ab_pdsep metrics.summary.json. Refresh with PD matrix, multiple seeds, cudagraph-enabled methodology. Legacy run has no per-stage TTFT breakdown and no step-level KV occupancy.
Elastic transfer-based migration does not improve high-contention 500-request run. supported_by_existing_artifact outputs/contention_16s_ts10 vs outputs/contention_16s_elastic metrics.summary.json and gpu_util.csv. Attribute whether failure is trigger quality, transfer overhead, or wrong load regime. Existing metrics lack actual sequentiality proof and per-request transfer waterfall.
PD-colo prefill/decode interference is not yet directly proven by step-level data in this package. not_yet_supported No decode-step and prefill-overlap timestamp artifact found in summarized runs. Run Batch 2 controlled same-worker/different-worker injection with step timestamps. Cannot claim interference as causal without Batch 2.
Session hot-spot residual imbalance is suggested but not fully attributed. partially_supported gpu_util.csv shows per-GPU mean-util imbalance in existing runs. Collect per-worker queue delay, session-to-worker map, and per-session token mass per worker. GPU util imbalance alone is not enough to prove session hot-spot.
SRR is not measured by existing fixed-request runs. not_yet_supported No arrival-rate sweep artifacts found. Implement Batch 4 Poisson session-arrival SRR sweep. Latency-at-one-load cannot support sustainable throughput claim.