feat(snapshot): D→P RDMA Phase 1b — GPU pointer path verified
Confirms snapshot_link works for cuda device pointers, not just host memory. Sender on cuda:0 pushes to receiver on cuda:1 via RDMA over mlx5_60. All 5 sizes (16K, 1M, 16M, 64M, 256M) pass SHA verification. 16 KB 8.3 ms 0.016 Gbps (cold openSegment) 1 MB 0.10 ms 87.6 Gbps 16 MB 0.84 ms 159 Gbps 64 MB 2.52 ms 213 Gbps 256 MB 8.54 ms 251 Gbps (~60% NDR400 line rate) For Inferact-scale sessions (~50K tokens × ~80 KB layer-per-token = ~4 GB), this projects D→P transfer time at ~130 ms — within the "reseed-savings" envelope sketched in design doc §3.2. Files: scripts/snapshot_link_receiver_gpu.py scripts/smoke_snapshot_link_gpu.py Next: SGLang scheduler integration for D-side dump + P-side ingest.
This commit is contained in:
@@ -92,13 +92,25 @@ peer.close()
|
||||
|
||||
## 4. 当前覆盖范围(清单)
|
||||
|
||||
- ✅ Host CPU 内存的 D→P RDMA byte transfer
|
||||
- ✅ Host CPU 内存的 D→P RDMA byte transfer (`scripts/smoke_snapshot_link.py`)
|
||||
- ✅ **GPU 内存** cuda:0 → cuda:1 的 D→P RDMA(`scripts/smoke_snapshot_link_gpu.py`,5/5 size 全 SHA 校验通过,256 MB 8.5 ms / 251 Gbps)
|
||||
- ✅ 单 IB device (mlx5_60)
|
||||
- ✅ 同节点 loopback(127.0.0.1)
|
||||
- ⏳ GPU 内存(设备指针 + `batch_transfer_write_on_cuda`)—— 现有 `push()` 走 `transfer_sync_write`,对 GPU 指针支持取决于 mooncake 的 protocol;下一步验证
|
||||
- ⏳ 跨节点(远端 IP)—— 设计上一致,未验证
|
||||
- ⏳ 多 D → 单 P(多 sender → 共享 recv buffer 的 offset 协调)—— 留给 Phase 3 整合时设计
|
||||
- ⏳ ZeroCopy 入 SGLang kv_pool slot —— 留给 Phase 2
|
||||
- ⏳ ZeroCopy 入 SGLang kv_pool slot —— 留给 Phase 2/3
|
||||
|
||||
### GPU smoke 性能
|
||||
|
||||
| Size | Push duration | Throughput |
|
||||
|---:|---:|---:|
|
||||
| 16 KB | 8.27 ms (cold) | 0.016 Gbps |
|
||||
| 1 MB | 0.096 ms | 87.6 Gbps |
|
||||
| 16 MB | 0.844 ms | 159 Gbps |
|
||||
| 64 MB | 2.52 ms | 213 Gbps |
|
||||
| **256 MB** | **8.54 ms** | **251 Gbps** |
|
||||
|
||||
GPU↔GPU 比 host↔host 慢一些(251 vs 316 Gbps for 64MB),但仍接近 mlx5_60 NDR 400Gb 的 60% 线率。对 KVC 单 session ~50K tokens × ~80 KB/token ≈ 4 GB 量级的 transfer,对应 D→P 时间约 130 ms。
|
||||
|
||||
## 5. 下一步(Phase 2 / Phase 3)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user