docs(kvc): correct reseed cost decomposition + flag D->P sync gap

After an independent Opus-agent forensic audit, the previous "(c) 增量
fetch (工程量较大,未实现)" line in V2_DEEP_ANALYSIS §4.2 was understating
the gap. The audit confirmed:

- No D->P KV transfer code exists in the framework at any layer
  (agentic_pd_hybrid orchestration, vendored SGLang disaggregation,
  or mooncake transport).
- Mooncake MooncakeKVManager has a hard role split: PREFILL = sender,
  DECODE = receiver-only loop. `add_transfer_request` asserts the
  disaggregation_mode is PREFILL.
- The BaseKVSender / BaseKVReceiver abstraction has no bidirectional slot.
- session_aware_cache.release_session only calls kv_pool_allocator.free()
  on eviction -- no serialization, no outbound network call.
- _commit_prefill_backup_residency is only called from the seed/reseed
  path (_invoke_kvcache_seeded_router). direct-to-D path never updates
  P-side backup state.
- "capacity-backup" policy semantics: it only skips the close on P after
  reseed -- the backup is the seed-time static snapshot, never refreshed
  by D-side append-prefill activity.

V2_DEEP_ANALYSIS §4.2:
- Decomposed the 3-7s reseed cost into the P-side re-prefill segment
  (1.5-3s, dominant) and the P->D mooncake transfer segment (1.5-4s).
- Quantified the realistic effect of enabling RDMA: only the transfer
  segment shrinks, reseed reduces to 1.7-3.2s, TTFT p99 ~0.7s, still
  loses to DP's 0.43s.
- Replaced the throwaway "(c) incremental fetch" line with a full
  paragraph explaining what D->P sync would require, why it's the
  largest engineering gap, and that the blocker is SGLang's radix-tree
  single-producer assumption, not the network layer.

KVC_ROUTER_ALGORITHM §9:
- Refined Open Question 3 (RDMA) to clarify it only helps the transfer
  segment, not the re-prefill segment.
- Added Open Question 4: D->P incremental KV sync as the central
  future-work contribution gap, with cited evidence for why it doesn't
  currently exist.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
kzlin
2026-05-11 22:07:14 +08:00
parent 517677d7f2
commit 9ccd853066
2 changed files with 20 additions and 7 deletions

View File

@@ -327,9 +327,11 @@ KVC 的关键架构权衡:**用 P 端 GPU 闲置换 D 端 TTFT 稳定性**。
2. **Algorithm 3 在更高压下行为如何**(例如 ts=10 加速、session 数 ≫ |D|·K_d/peak_input当前 ts=1 评测对应真实 agentic 区间,但算法在更高负载下的鲁棒性未经实验验证。
3. **真 RDMA 下的 reseed 代价**:本次评测的 37 s reseed 延迟主要来自 mooncake 在 TCP loopback 上的 transfer 开销。算法结构与传输无关,但 TTFT p99 长尾预期在 IB/RoCE 下缩短约 10×。待独立验证。
3. **真 RDMA 下的 reseed 代价**:本次评测的 37 s reseed 延迟由两段组成——P 端 re-prefill1.5-3s+ P→D mooncake transfer1.5-4s。当前 sweep 用的是 TCP loopback启用 IB/RoCE节点有 mlx5_0/_1 @ 200 Gb/s × 2 active需在 sweep 加 `--force-rdma --ib-device mlx5_0`)只能压缩 transfer 段到 ~200ms**不动 re-prefill 段**。预期 TTFT p99 从 1.28s 降到 ~0.7s(仍输 DP 0.43s。待独立验证。
4. **v2 代码路径下的确定性**v0 代码库的 ts=1 N=3 categorical 确定性已经证实;新增的 reset-on-success 分支与 threshold=8192 路径未被独立 re-validate。两个额外的 N=1 run 即可解决
4. **D→P 增量 KV 同步(核心 future-work 缺口)**reseed 长尾的真正消除需要让 P 端 backup 跟上 D 的 direct-to-D append 增长。经独立 forensic 审查,**当前代码、vendored SGLang、mooncake 三层均无 D→P KV transfer 实现**mooncake `MooncakeKVManager` 是 PREFILL=sender / DECODE=receiver 的硬角色分支(`add_transfer_request` 上有 `assert disaggregation_mode == PREFILL` 硬约束),`BaseKVSender` / `BaseKVReceiver` 抽象无 bidirectional slot`session_aware_cache.release_session` 在驱逐时只调 `kv_pool_allocator.free()` 无出站,`_commit_prefill_backup_residency` 唯一 caller 是 seed/reseed 路径;`capacity-backup` policy 的真实语义只是"reseed 完不关 P streaming session"——backup 是 seed-time 的静态快照,不随 direct-to-D append 同步。要实现 D→P 增量同步,工程量 ~1-2 周,最难的不是 mooncake 加 D-sender / P-receiver 角色(~400 LOC而是 **SGLang radix tree 改成允许从外部 worker 喂数据**——radix cache 当前假设单一生产者(本 worker model 输出)。这是论文里最值得做的 contribution 之一
5. **v2 代码路径下的确定性**v0 代码库的 ts=1 N=3 categorical 确定性已经证实;新增的 reset-on-success 分支与 threshold=8192 路径未被独立 re-validate。两个额外的 N=1 run 即可解决。
---