docs(kvc): correct reseed cost decomposition + flag D->P sync gap
After an independent Opus-agent forensic audit, the previous "(c) 增量 fetch (工程量较大,未实现)" line in V2_DEEP_ANALYSIS §4.2 was understating the gap. The audit confirmed: - No D->P KV transfer code exists in the framework at any layer (agentic_pd_hybrid orchestration, vendored SGLang disaggregation, or mooncake transport). - Mooncake MooncakeKVManager has a hard role split: PREFILL = sender, DECODE = receiver-only loop. `add_transfer_request` asserts the disaggregation_mode is PREFILL. - The BaseKVSender / BaseKVReceiver abstraction has no bidirectional slot. - session_aware_cache.release_session only calls kv_pool_allocator.free() on eviction -- no serialization, no outbound network call. - _commit_prefill_backup_residency is only called from the seed/reseed path (_invoke_kvcache_seeded_router). direct-to-D path never updates P-side backup state. - "capacity-backup" policy semantics: it only skips the close on P after reseed -- the backup is the seed-time static snapshot, never refreshed by D-side append-prefill activity. V2_DEEP_ANALYSIS §4.2: - Decomposed the 3-7s reseed cost into the P-side re-prefill segment (1.5-3s, dominant) and the P->D mooncake transfer segment (1.5-4s). - Quantified the realistic effect of enabling RDMA: only the transfer segment shrinks, reseed reduces to 1.7-3.2s, TTFT p99 ~0.7s, still loses to DP's 0.43s. - Replaced the throwaway "(c) incremental fetch" line with a full paragraph explaining what D->P sync would require, why it's the largest engineering gap, and that the blocker is SGLang's radix-tree single-producer assumption, not the network layer. KVC_ROUTER_ALGORITHM §9: - Refined Open Question 3 (RDMA) to clarify it only helps the transfer segment, not the re-prefill segment. - Added Open Question 4: D->P incremental KV sync as the central future-work contribution gap, with cited evidence for why it doesn't currently exist. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -327,9 +327,11 @@ KVC 的关键架构权衡:**用 P 端 GPU 闲置换 D 端 TTFT 稳定性**。
|
||||
|
||||
2. **Algorithm 3 在更高压下行为如何**(例如 ts=10 加速、session 数 ≫ |D|·K_d/peak_input)?当前 ts=1 评测对应真实 agentic 区间,但算法在更高负载下的鲁棒性未经实验验证。
|
||||
|
||||
3. **真 RDMA 下的 reseed 代价**:本次评测的 3–7 s reseed 延迟主要来自 mooncake 在 TCP loopback 上的 transfer 开销。算法结构与传输无关,但 TTFT p99 长尾预期在 IB/RoCE 下缩短约 10×。待独立验证。
|
||||
3. **真 RDMA 下的 reseed 代价**:本次评测的 3–7 s reseed 延迟由两段组成——P 端 re-prefill(1.5-3s)+ P→D mooncake transfer(1.5-4s)。当前 sweep 用的是 TCP loopback;启用 IB/RoCE(节点有 mlx5_0/_1 @ 200 Gb/s × 2 active,需在 sweep 加 `--force-rdma --ib-device mlx5_0`)只能压缩 transfer 段到 ~200ms,**不动 re-prefill 段**。预期 TTFT p99 从 1.28s 降到 ~0.7s(仍输 DP 0.43s)。待独立验证。
|
||||
|
||||
4. **v2 代码路径下的确定性**:v0 代码库的 ts=1 N=3 categorical 确定性已经证实;新增的 reset-on-success 分支与 threshold=8192 路径未被独立 re-validate。两个额外的 N=1 run 即可解决。
|
||||
4. **D→P 增量 KV 同步(核心 future-work 缺口)**:reseed 长尾的真正消除需要让 P 端 backup 跟上 D 的 direct-to-D append 增长。经独立 forensic 审查,**当前代码、vendored SGLang、mooncake 三层均无 D→P KV transfer 实现**:mooncake `MooncakeKVManager` 是 PREFILL=sender / DECODE=receiver 的硬角色分支(`add_transfer_request` 上有 `assert disaggregation_mode == PREFILL` 硬约束),`BaseKVSender` / `BaseKVReceiver` 抽象无 bidirectional slot,`session_aware_cache.release_session` 在驱逐时只调 `kv_pool_allocator.free()` 无出站,`_commit_prefill_backup_residency` 唯一 caller 是 seed/reseed 路径;`capacity-backup` policy 的真实语义只是"reseed 完不关 P streaming session"——backup 是 seed-time 的静态快照,不随 direct-to-D append 同步。要实现 D→P 增量同步,工程量 ~1-2 周,最难的不是 mooncake 加 D-sender / P-receiver 角色(~400 LOC),而是 **SGLang radix tree 改成允许从外部 worker 喂数据**——radix cache 当前假设单一生产者(本 worker model 输出)。这是论文里最值得做的 contribution 之一。
|
||||
|
||||
5. **v2 代码路径下的确定性**:v0 代码库的 ts=1 N=3 categorical 确定性已经证实;新增的 reset-on-success 分支与 threshold=8192 路径未被独立 re-validate。两个额外的 N=1 run 即可解决。
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user