agentic-kvc/microbench/connector_tax/cache_sweep/REPORT_TRACE_REPLAY.md

Trace-replay re-test with DR-fix

Run: results/trace_replay_20260526_1652/ Trace: traces/w600_r0.0015_st30.jsonl (1214 reqs, 274 sessions, 53.3 M tokens) Topology: 8 × TP1 vLLM + cache_aware_proxy, Qwen3-Coder-30B-A3B-Instruct Same trace, same proxy, same machine that produced the original analysis/characterization/elastic_migration_v2/ paper.

TL;DR

The original elastic_migration_v2 paper claimed kv_role=kv_both (Mooncake) cost TTFT p90 +45 % vs plain unified. That gap no longer exists. In a same-day re-run on the same trace with the same 8-instance topology:

metric	unified (plain)	unified_kv_both (baseline)	unified_kv_both_drfix
TTFT p90	11 971 ms	9 744 ms (−18.6 % vs plain)	7 584 ms (−36.6 % vs plain)
TPOT p90	20 ms	22 ms (+10 %)	18 ms (−10 %)
E2E p90	23 475 ms	21 254 ms (−9.5 %)	17 931 ms (−23.6 %)

Two findings:

1. The +45 % is gone. kv_both without any fix is now faster than plain unified at p90 (−18.6 %). Likely culprits in the commit chain since the elastic_migration_v2 paper: a7df84b (direct RDMA read), 0500350 (token-based lookup), 08d5e12 (NONE_HASH import fix), and especially e3a1d70 (switch from RDMA READ to bootstrap-triggered PUSH) which restructured the producer-side critical path.
2. DR-fix still helps. Disabling the O(|cache|) hash sync removes another 22 % from TTFT p90 (9.7 s → 7.6 s) and 16 % from E2E p90 (21.3 s → 17.9 s). The cache-sweep finding (+85 μs/1k blocks slope) translates into measurable p90/p99 wins under high APC + agentic session coupling.

How this changes the elastic_migration_v2 narrative

Original paper's four claims, re-checked today:

original claim	today's status
"kv_role=kv_both costs TTFT p90 +45 % even without PD-sep"	OBSOLETE (now −18.6 % vs plain)
"Mooncake−NIXL gap of 7 pp is implementation cost"	NOT TESTED (NIXL not re-run here)
"PD-sep rarely fires (0.41 % trigger rate)"	unchanged — trace property
"When PD-sep fires, mechanism is 10-20× slower than model predicts"	NOT TESTED (v2 policy not re-run)

The elastic_migration_v2 README should be marked as containing historical data that is no longer reproducible on the current codebase. The story ought to be re-cast as: "+45 % was a transient bug we fixed (whether intentionally as part of e3a1d70 or accidentally), and the remaining headroom (15-20 % p90) is recovered by the DR-fix."

Full per-metric A/B/C table

(10 s warmup discarded by the replayer; n=1214 each)

metric	unified	unified_kv_both	drfix	mc vs plain	drfix vs plain	drfix vs mc
TTFT mean	4 018 ms	3 552 ms	3 103 ms	−11.6 %	−22.8 %	−12.6 %
TTFT p50	500 ms	501 ms	485 ms	+0.2 %	−3.0 %	−3.2 %
TTFT p90	11 971 ms	9 744 ms	7 584 ms	−18.6 %	−36.6 %	−22.2 %
TTFT p99	46 695 ms	42 432 ms	41 883 ms	−9.1 %	−10.3 %	−1.3 %
TPOT mean	15.3 ms	14.4 ms	14.0 ms	−5.9 %	−8.5 %	−2.8 %
TPOT p50	8.4 ms	8.3 ms	8.0 ms	−0.9 %	−3.9 %	−3.1 %
TPOT p90	19.6 ms	21.6 ms	17.7 ms	+10.0 %	−9.7 %	−17.9 %
TPOT p99	151.6 ms	127.8 ms	112.4 ms	−15.7 %	−25.9 %	−12.1 %
E2E mean	8 180 ms	7 967 ms	7 184 ms	−2.6 %	−12.2 %	−9.8 %
E2E p50	1 942 ms	1 995 ms	1 806 ms	+2.7 %	−7.0 %	−9.5 %
E2E p90	23 475 ms	21 254 ms	17 931 ms	−9.5 %	−23.6 %	−15.6 %
E2E p99	73 709 ms	76 630 ms	71 958 ms	+4.0 %	−2.4 %	−6.1 %

Why kv_both already beats plain (without DR-fix)

A connector-loaded vLLM has delay_free_blocks=True by default — block eviction is deferred until the connector's bookkeeping signals it is safe. On a 93 %-intra-session-reuse trace, this extends prefix-cache hit windows across session turns, which more than compensates for the per-step connector cost on the codebase as it exists today. With the DR-fix removing the remaining O(|cache|) tax, the net swings strongly positive.

This was also one of the explanations proposed in the cache_sweep report ("connector mode has higher effective cache utilisation") and is now confirmed at the trace-replay scale.

Reproducibility

bash microbench/connector_tax/cache_sweep/run_trace_replay_drfix.sh

Runtime: ~2.5 h on 8 × H20. The orchestrator applies CT_DR_FIX, runs the three policies serially (plain → mc baseline → mc drfix via env var), reverts the patch, and emits per-policy metrics.jsonl. Analyse with:

python microbench/connector_tax/cache_sweep/analyze_trace_replay.py \\
       --root microbench/connector_tax/cache_sweep/results/trace_replay_20260526_1652

Files

trace_replay_20260526_1652/
├── trace_replay_summary.json       — machine-readable per-config TTFT/TPOT/E2E
├── unified/                        — plain control
│   ├── metrics.jsonl               — per-request timings (1214 rows)
│   ├── metrics.summary.json        — replayer's own summary
│   ├── breakdown.json              — proxy per-decision metadata
│   ├── stats.json                  — proxy aggregate counters
│   └── run_window.json             — t_start/t_end + policy + trace
├── unified_kv_both/                — Mooncake kv_both, hash sync ON
└── unified_kv_both_drfix/          — Mooncake kv_both, hash sync OFF (env-gated)

Heavy artifacts (engine_state/, vllm logs, replayer.log, proxy.log) are .gitignored — re-derive with run_trace_replay_drfix.sh.