feat(experiments): E4 protocol + sweep script — KVC + D→P vs naive PD
Pre-registers the E4 experiment that tests whether KVC + D→P RDMA snapshot push beats the naive PD-disagg E1 baseline on the inferact_50sess subset. Compared to E3 the only changed flag is --enable-d-to-p-sync. Three hypotheses (see docs/E4_PROTOCOL_ZH.md §2.3): H1 (main): E4 TTFT p99 ≤ E1 TTFT p99 H2: E4 reseed-mode TTFT < E3 reseed-mode TTFT H3: E4 success count ≥ E3 success count The full reseed → snapshot-push orchestration is wired inb9b0cf0(_attempt_d_to_p_sync); the SGLang scheduler RPCs and the runtime mem-leak fix are in86412bb/a369722.
This commit is contained in:
Claude Code Agent
157
docs/E4_PROTOCOL_ZH.md
Normal file
157
docs/E4_PROTOCOL_ZH.md
Normal file
@@ -0,0 +1,157 @@
|
||||
# E4 — KVC + D→P RDMA snapshot vs naive PD-disagg (实验协议)
|
||||
|
||||
**Status**: 协议事前定稿（preregistration）
|
||||
**Date**: 2026-05-13
|
||||
**Branch**: `h200-cu130`
|
||||
**Prereq**: `docs/D_TO_P_SYNC_DESIGN_ZH.md`, `docs/D_TO_P_PHASE1_LINK_ZH.md`
|
||||
**Companion**: `docs/E1_E2_RESULTS_ZH.md`, `docs/E3_FINDINGS_ZH.md`
|
||||
|
||||
---
|
||||
|
||||
## 0. 一句话
|
||||
|
||||
E4 在 E3 配置（KVC v2 + RDMA + load-floor bonus K=200）之上加 `--enable-d-to-p-sync`，验证 D→P RDMA snapshot push 能否让 reseed 路径跳过 P 端 re-prefill，从而让 KVC 在保持 session-affinity 设计独特性的前提下 latency 优于 naive PD-disagg（E1 基线）。
|
||||
|
||||
---
|
||||
|
||||
## 1. 实验目的
|
||||
|
||||
回答 ProJEctGoal 设定的核心问题：**KVC 如何在保持自身独特性的情况下胜过 naive PD-disagg？**
|
||||
|
||||
历史结论：
|
||||
- E1（naive 1P3D + kv-aware + RDMA）：成功 1200/1285，TTFT p99 = 88.6s（D2 完全闲置）
|
||||
- E3（KVC v2 + RDMA + load-floor K=200）：load-floor 解决 D2 cold 问题，但 SGLang streaming-session 内部 assertion bug 暴露，单 turn 至高吞吐降低。即使在已经 patched 的版本 reseed 路径仍有 P 端完整 re-prefill 长尾。
|
||||
|
||||
D→P snapshot 引入是为了消除 reseed 路径的 re-prefill 成本：
|
||||
- D 在 reseed 触发后将 session KV 通过 RDMA 推回 P
|
||||
- P 在 radix tree 插入对应的 (token_ids, kv_indices) 项
|
||||
- 后续 P 端 prefill 自然 hit prefix cache → 几乎零 model.forward → 直接 mooncake P→D' 传输
|
||||
|
||||
预期效果（参考 `docs/D_TO_P_SYNC_DESIGN_ZH.md §3.2`）：
|
||||
- reseed re-prefill 段 1.5-3s → ~0
|
||||
- reseed transfer 段 0.2-0.4s 不变
|
||||
- reseed 总耗时 3-7s → 0.3-0.5s
|
||||
- TTFT p99 显著下降
|
||||
|
||||
---
|
||||
|
||||
## 2. 实验设置
|
||||
|
||||
### 2.1 配置
|
||||
|
||||
| 维度 | 值 |
|
||||
|---|---|
|
||||
| Trace | `outputs/inferact_50sess.jsonl` (1285 reqs / 50 sessions, md5 7bb263a32600ef5a6ef5099ba340a487) |
|
||||
| Model | Qwen3-30B-A3B-Instruct-2507 (TP=1) |
|
||||
| Topology | 1P + 3D = 4 GPU |
|
||||
| Hardware | 4× H200 80GB, mlx5_60 NDR 400Gb RoCE v2, GID Index 3 |
|
||||
| Time scale | ts=1 |
|
||||
| Concurrency | 32 |
|
||||
| Request timeout | 300 s |
|
||||
| Mooncake transfer timeout | 1800 s (MC_TRANSFER_TIMEOUT) |
|
||||
| KVC migration reject threshold | 3 |
|
||||
| Load-floor bonus | K=200 |
|
||||
| **D→P sync** | **on** (--enable-d-to-p-sync) |
|
||||
|
||||
### 2.2 对照组（已有数据复用）
|
||||
|
||||
| 名 | 配置 | 关键数据来源 |
|
||||
|---|---|---|
|
||||
| E1 | naive 1P3D + kv-aware + RDMA，无 KVC 层 | `outputs/e1_naive_1p3d_rdma_50sess/` |
|
||||
| E3 | KVC v2 + RDMA + load-floor K=200，无 D→P | `outputs/e3_kvc_v2_loadfloor_rdma_50sess/` |
|
||||
| **E4** | 同 E3 + `--enable-d-to-p-sync` | **本次跑** |
|
||||
|
||||
### 2.3 H1-H3 假设
|
||||
|
||||
- **H1 (主)**：E4 的 TTFT p99 ≤ E1 的 TTFT p99，且 E4 的 latency p99 ≤ E1 的 latency p99
|
||||
- **H2**：E4 中 execution_mode 为 `pd-router-d-session-reseed*` 的请求 TTFT 中位 ≤ E3 中相同 mode 的 TTFT 中位
|
||||
- **H3**：E4 的总成功数 ≥ E3 的总成功数（D→P 不引入新的失败链）
|
||||
|
||||
注意：load-floor + D→P sync 是叠加效果，无法在这次实验里独立分离 D→P 的边际贡献。后续可单独做 E4-ablate（K=200，--enable-d-to-p-sync 但人为关闭 D 端 dump）。
|
||||
|
||||
### 2.4 度量
|
||||
|
||||
每个 run 收集（来自 `request-metrics.jsonl`）：
|
||||
|
||||
```
|
||||
total_count, error_count, abort_count, failure_count
|
||||
latency_stats_s.{mean, p50, p90, p99}
|
||||
ttft_stats_s.{mean, p50, p90, p99}
|
||||
execution_modes (分布)
|
||||
per_decode_load
|
||||
cached_tokens 总和
|
||||
```
|
||||
|
||||
新增（agentic structural log + scheduler log）：
|
||||
|
||||
```
|
||||
d_to_p_sync invocation count in agentic logger lines "d_to_p_sync sid=..."
|
||||
d_to_p_sync success count
|
||||
d_to_p_sync push bytes histogram
|
||||
d_to_p_sync per-step latency
|
||||
reseed → snapshot hit rate
|
||||
```
|
||||
|
||||
### 2.5 失败模式
|
||||
|
||||
`_attempt_d_to_p_sync` 任何失败（prepare_receive ok=false / dump ok=false / finalize ok=false / 网络）都 fallback 到原 seeded_router 路径。所以 E4 即使 D→P 全失败，理论上仍应等于 E3 baseline。
|
||||
|
||||
---
|
||||
|
||||
## 3. 验收
|
||||
|
||||
### 3.1 必须
|
||||
|
||||
- [ ] E4 总成功请求数 ≥ 0.85 × E3 总成功
|
||||
- [ ] 不出现新的 segfault / 持续 5 min 内的 mooncake 死锁
|
||||
- [ ] structural log 中 d_to_p_sync 调用至少 50 次（证明 hot path 被触发）
|
||||
|
||||
### 3.2 期望
|
||||
|
||||
- [ ] E4 TTFT p99 < E1 TTFT p99
|
||||
- [ ] E4 reseed 路径 TTFT 中位明显低于 E3 reseed 路径 TTFT 中位（保守地，至少 ≥ 30% 改进）
|
||||
- [ ] E4 TTFT p99 < E3 TTFT p99（说明 D→P 真的有用）
|
||||
|
||||
### 3.3 探索
|
||||
|
||||
- [ ] D→P push 占链路带宽多少？（看 nvidia-smi DCGM 或 mooncake metrics）
|
||||
- [ ] D→P push 失败率？如失败，主要 reason 是什么？
|
||||
- [ ] P 端 radix insert 的 prefix_len 分布？
|
||||
|
||||
---
|
||||
|
||||
## 4. 报告交付物
|
||||
|
||||
跑完后产出 `docs/E4_RESULTS_ZH.md`，包含：
|
||||
|
||||
1. 三组 lat/ttft 全分位数对比表
|
||||
2. execution_mode 分布对比
|
||||
3. H1/H2/H3 各自证实 / 证伪 / 部分证实
|
||||
4. d_to_p_sync 统计：调用数、成功数、失败原因 top
|
||||
5. 失败模式分析（如有）
|
||||
6. 与设计 `docs/D_TO_P_SYNC_DESIGN_ZH.md §3.2` 预测的对照
|
||||
|
||||
---
|
||||
|
||||
## 5. 时间预算
|
||||
|
||||
- 跑 E4 一次：~30-60 min（同 E3 量级）
|
||||
- 数据汇总：~30 min
|
||||
- 报告：~1 h
|
||||
|
||||
如时间不够：先跑 N=1 抓最关键的 TTFT 分布，后续补 N=2 对照。
|
||||
|
||||
---
|
||||
|
||||
## 6. 风险
|
||||
|
||||
| 风险 | 缓解 |
|
||||
|---|---|
|
||||
| `_attempt_d_to_p_sync` 在 reseed path 实际触发频率太低 | 调小 KV 池 + 调整 reject_threshold 让 reseed 多触发 |
|
||||
| RDMA dump 多次失败导致 D→P 链路变成 net negative | structural log 留好失败原因 → 抓 root cause |
|
||||
| SGLang scheduler 新引入的 RPC 干扰 PD pipeline | smoke test 已确认 RPC 互不影响 |
|
||||
| 量纲对错：D 推送的 KV bytes 在 P 端解码出错 | 完整 E4 跑完看下游 perplexity / TTFT 看异常 |
|
||||
|
||||
---
|
||||
|
||||
**核心句**：E4 是测试 D→P snapshot 在端到端工作负载中是否真能消除 reseed re-prefill 成本的核心实验。E4 胜过 E1 即证明 KVC + D→P 在保持设计独特性的前提下能跑赢 naive PD-disagg。
|
||||
82
scripts/sweep_e4_kvc_v2_d_to_p_sync.sh
Executable file
82
scripts/sweep_e4_kvc_v2_d_to_p_sync.sh
Executable file
@@ -0,0 +1,82 @@
|
||||
#!/usr/bin/env bash
|
||||
# E4 — KVC v2 + RDMA + load-floor bonus + D→P snapshot push
|
||||
#
|
||||
# Identical to scripts/sweep_e3_kvc_v2_loadfloor_rdma.sh except for the
|
||||
# additional --enable-d-to-p-sync flag (which causes agentic to orchestrate
|
||||
# the snapshot RPCs on the reseed slow path, and stack.py to set
|
||||
# SGLANG_SNAPSHOT_LINK_ENABLE=1 per worker).
|
||||
#
|
||||
# See docs/E4_PROTOCOL_ZH.md for hypothesis matrix.
|
||||
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
if [ -z "${CUDA_HOME:-}" ]; then
|
||||
echo "ERROR: CUDA_HOME not set. Source scripts/setup_env.sh first." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
MODEL=${MODEL:-/mnt/models/Qwen/Qwen3-30B-A3B-Instruct-2507}
|
||||
TRACE=${TRACE:-outputs/inferact_50sess.jsonl}
|
||||
OUTPUT=${OUTPUT:-outputs/e4_kvc_v2_d_to_p_sync_50sess}
|
||||
IB_DEVICE=${IB_DEVICE:-mlx5_60}
|
||||
LOAD_FLOOR_BONUS=${LOAD_FLOOR_BONUS:-200}
|
||||
|
||||
if [ ! -f "$TRACE" ]; then
|
||||
echo "ERROR: trace not found at $TRACE" >&2
|
||||
echo "Run: uv run --no-sync python scripts/sample_trace_subset.py --output $TRACE --sessions 50" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
mkdir -p "$OUTPUT"
|
||||
LOG="$OUTPUT/sweep.log"
|
||||
|
||||
log() { echo "[$(date '+%Y-%m-%d %H:%M:%S')] $*" | tee -a "$LOG"; }
|
||||
|
||||
log "=== E4: KVC v2 + RDMA + load-floor K=$LOAD_FLOOR_BONUS + D→P sync ==="
|
||||
log "MODEL=$MODEL"
|
||||
log "TRACE=$TRACE ($(wc -l < $TRACE) requests)"
|
||||
log "OUTPUT=$OUTPUT"
|
||||
log "IB_DEVICE=$IB_DEVICE"
|
||||
log "MC_TRANSFER_TIMEOUT=${MC_TRANSFER_TIMEOUT:-default-30s}"
|
||||
|
||||
label=e4_kvc_v2_d_to_p_sync_run1
|
||||
log ""
|
||||
log "=== [E4] $label starting ==="
|
||||
|
||||
uv run --no-sync python -m agentic_pd_hybrid.cli benchmark-live \
|
||||
--trace "$TRACE" \
|
||||
--output-root "$OUTPUT" \
|
||||
--mechanism kvcache-centric \
|
||||
--policy kv-aware \
|
||||
--model-path "$MODEL" \
|
||||
--prefill-workers 1 --decode-workers 3 \
|
||||
--prefill-tp-size 1 --decode-tp-size 1 \
|
||||
--prefill-gpu-ids 0 --decode-gpu-ids 1,2,3 \
|
||||
--transfer-backend mooncake \
|
||||
--force-rdma --ib-device "$IB_DEVICE" \
|
||||
--gpu-budget 4 \
|
||||
--time-scale 1 \
|
||||
--session-sample-rate 1.0 \
|
||||
--target-duration-s 100000 \
|
||||
--concurrency-limit 32 \
|
||||
--timeout-s 1800 \
|
||||
--request-timeout-s 300 \
|
||||
--kvcache-admission-mode worker \
|
||||
--kvcache-seed-min-turn-id 1 \
|
||||
--kvcache-seed-max-inflight-decode -1 \
|
||||
--kvcache-prefill-backup-policy release-after-transfer \
|
||||
--kvcache-prefill-priority-eviction \
|
||||
--kvcache-migration-reject-threshold 3 \
|
||||
--kvcache-direct-max-uncached-tokens 8192 \
|
||||
--kvcache-load-floor-bonus "$LOAD_FLOOR_BONUS" \
|
||||
--enable-d-to-p-sync 2>&1 | tee -a "$LOG"
|
||||
|
||||
run_dir=$(ls -td "$OUTPUT"/kvcache-centric-*/ 2>/dev/null | head -1)
|
||||
log "=== [E4] $label COMPLETED, artifacts at $run_dir ==="
|
||||
|
||||
if [ -f "$run_dir/request-metrics.jsonl.summary.json" ]; then
|
||||
cp "$run_dir/request-metrics.jsonl.summary.json" "$OUTPUT/${label}_summary.json"
|
||||
cp "$run_dir/request-metrics.jsonl" "$OUTPUT/${label}_metrics.jsonl"
|
||||
log "=== summary saved to $OUTPUT/${label}_summary.json ==="
|
||||
fi
|
||||
Reference in New Issue
Block a user