Connector tax: trace-replay confirms +45% kv_both penalty is gone; DR-fix adds 22% more
Re-runs the elastic_migration_v2 trace (w600 r0.0015 st30, 1214 reqs,
274 sessions, 8×TP1 vLLM + cache_aware_proxy) with three configs:
- plain unified
- unified + Mooncake kv_both
- unified + Mooncake kv_both + DR-fix (env-gated O(|cache|) hash sync removal)
TTFT p90: 11.97 s → 9.74 s (−18.6%) → 7.58 s (−36.6% vs plain)
E2E p90: 23.48 s → 21.25 s (−9.5%) → 17.93 s (−23.6% vs plain)
Two findings:
1. The "+45% kv_both penalty" claim from elastic_migration_v2 is OBSOLETE
on current codebase — kv_both is now *faster* than plain at p90.
Likely fixed by e3a1d70 (RDMA-READ → bootstrap PUSH refactor) and
the connector-mode delay_free_blocks extending cross-turn prefix
cache hits on a 93%-intra-session-reuse trace.
2. DR-fix removes another 22% from TTFT p90 by skipping the
O(|cache|) hash sync in build_connector_meta. Cache-sweep with
DR-fix shows slope drops from +94.5 to +2.3 μs/1k blocks.
Adds:
- run_trace_replay_drfix.sh: A/B/C harness (env CT_DR_FIX gates patch)
- analyze_trace_replay.py: TTFT/TPOT/E2E delta analysis
- REPORT_TRACE_REPLAY.md: summary + reproduction
- results/20260526_1627_drfix/: cache-sweep with DR-fix
- results/trace_replay_20260526_1652/: full trace-replay A/B/C
Implication for EAR paper: the kv_both substrate is no longer the
bottleneck blocking session migration. The prior 4 migration reverts
were dominated by transfer overhead that has now been characterized
and (partially) removed.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -25,6 +25,7 @@ PORT="${PORT:-8000}"
|
||||
GPU_ID="${GPU_ID:-0}"
|
||||
MODEL_PATH="${MODEL_PATH:-$HOME/models/Qwen/Qwen3-Coder-30B-A3B-Instruct}"
|
||||
CONFIGS="${CONFIGS:-plain mooncake_both mooncake_both_drfix}"
|
||||
SEED="${SEED:-12345}" # shared seed across configs → identical Poisson + content
|
||||
|
||||
DATE="$(date +%Y%m%d_%H%M)"
|
||||
RUN_ROOT="$HERE/results/${DATE}_drfix"
|
||||
@@ -33,7 +34,7 @@ mkdir -p "$RUN_ROOT"
|
||||
echo "=== Cache-size sweep + DR-fix A/B ==="
|
||||
echo "Run dir : $RUN_ROOT"
|
||||
echo "Configs : $CONFIGS"
|
||||
echo "Rate : $RATE Duration: ${DURATION}s"
|
||||
echo "Rate : $RATE Duration: ${DURATION}s Seed: $SEED"
|
||||
echo ""
|
||||
|
||||
kill_all_vllm() {
|
||||
@@ -114,6 +115,7 @@ for cfg in $CONFIGS; do
|
||||
--url "http://127.0.0.1:$PORT/v1/chat/completions" \
|
||||
--model "$MODEL_PATH" \
|
||||
--rate "$RATE" --duration "$DURATION" \
|
||||
--seed "$SEED" \
|
||||
--output-dir "$cfg_dir" 2>&1 | tail -8
|
||||
|
||||
curl -s "http://127.0.0.1:$PORT/metrics" > "$cfg_dir/metrics_final.txt" 2>&1 || true
|
||||
|
||||
Reference in New Issue
Block a user