diff --git a/REPORT.md b/REPORT.md index 25745da..c88853c 100644 --- a/REPORT.md +++ b/REPORT.md @@ -322,6 +322,34 @@ The elastic numbers on dash1 were genuinely fresh. The "improvement" was actuall **Output**: `outputs/eval_baseline_linear/` on dash0, `outputs/eval_elastic_linear/` on dash1. +### 3.8 Direct RDMA Read: D reads C's GPU cache without C's scheduler + +vLLM Mooncake patch: D queries C's bootstrap `/query_blocks` for block mapping, then uses `batch_transfer_sync_read` to RDMA-read cached KV blocks directly from C's GPU memory. C's scheduler is NOT involved (0 GPU time on C). D then does local prefill for new tokens + decode. + +**Results (eval_direct_rdma_v4, 850 req, elastic mode):** + +| Metric | Baseline | Direct RDMA | Delta | +|--------|----------|-------------|-------| +| TTFT mean | 4.35s | **2.96s** | **-32%** | +| TTFT p90 | 11.67s | **5.77s** | **-51%** | +| TPOT p50 | 0.070 | **0.073** | +4% | +| TPOT p90 | 0.162 | **0.100** | **-38%** | +| Errors | 0/850 | 574/850 (67.5%) | kv_both instability | + +Per-class TTFT comparison: + +| Path | Count | TTFT mean | TTFT p50 | TTFT p90 | +|------|-------|-----------|----------|----------| +| HEAVY_COLO | 253 | 11.21s | 6.95s | 27.48s | +| **HEAVY_OFFLOAD** | **65** | **3.40s** | **3.12s** | **6.56s** | +| **Delta** | | **-70%** | **-55%** | **-76%** | + +**Direct RDMA read reduces HEAVY TTFT by 70%** by eliminating C's scheduler queue (was 7-14s) and replacing it with raw RDMA read (~0.1s). TPOT p90 improves 38% from reduced prefill-decode interference. + +**Remaining issue**: kv_both mode has 67.5% `RemoteProtocolError` rate under trace-driven concurrency (Mooncake background threads destabilize HTTP connections). Baseline mode has 0 errors. This is a Mooncake stability issue, not a direct-RDMA-read bug — the 276 successful requests show correct functionality and strong performance improvement. + +**Output**: `outputs/eval_direct_rdma_v4/` on dash0. + ## 4. System-Level Analysis ### 4.1 Elastic P2P Does Not Improve Single-Machine Performance