diff --git a/docs/E1_E2_RESULTS_ZH.md b/docs/E1_E2_RESULTS_ZH.md new file mode 100644 index 0000000..b75228a --- /dev/null +++ b/docs/E1_E2_RESULTS_ZH.md @@ -0,0 +1,87 @@ +# E1 vs E2 Experiment Results — H200 + Driver 570 + +**Status**: E1 ✅ complete (2026-05-12 01:48 UTC, wall 1h29min). E2 ⏳ running. +**Branch**: `h200-cu130`. +**Trace**: `outputs/inferact_50sess.jsonl` (deterministic head-cut of Inferact `codex_swebenchpro` to first 50 trials, md5 `7bb263a32600ef5a6ef5099ba340a487`, 1285 requests, mean input_length 67,631 tokens). +**Hardware**: 4× H200 80GB, driver 570.86.15 (cu12.8 API), Mellanox mlx5_60 RoCE 400 Gb/s NDR. +**Model**: Qwen3-30B-A3B-Instruct-2507 (TP1). +**Toolchain**: vendored SGLang 0.5.10 + cu12.8 nvcc local install (`~/cuda-12.8`) — see `docs/H200_DRIVER570_SETUP_ZH.md`. + +--- + +## 1. Hypotheses being tested + +From `docs/ONBOARDING_NEXT_AGENT_ZH.md` §3.1: + +- **H1**: KVC v2's wins are not just from "1P3D topology + kv-aware policy" — the KVC layer (admission / migration / direct-to-D) contributes meaningfully on top. Pairing E1 (no KVC layer) against E2 (full KVC v2) on the **same subset** isolates the marginal contribution. +- **H2/H3**: Enabling real RDMA pushes TTFT p99 down from the reported 1.28s (TCP loopback) toward ~0.7s. Independent of H1, this is measured inside E2 alone (comparing against the historical TCP-loopback v2 reference). + +--- + +## 2. E1 results — naive 1P3D + kv-aware + RDMA + +**Configuration**: `mechanism=pd-disaggregation`, `policy=kv-aware`, 1P3D (GPU0=P, GPU1/2/3=D), `--force-rdma --ib-device mlx5_60`, `--concurrency-limit 32`, ts=1. + +| Metric | E1 | +|---|---:| +| request_count | 1285 | +| success | 1200 | +| **error_count** | **85** | +| **failure_count** | **85** | +| abort_count | 0 | +| latency mean | 96.34 s | +| latency p50 | 93.21 s | +| latency p90 | 180.69 s | +| latency p99 | 219.46 s | +| ttft mean | 90.48 s | +| ttft p50 | 88.62 s | +| ttft p90 | 175.13 s | +| **ttft p99** | **207.39 s** | +| execution_modes | `pd-disaggregation-router: 1200`, `pd-disaggregation: 85` (errors) | +| per_decode_load | **D0:575, D1:710, D2:0** | +| per_prefill_load | P0:1285 | +| cache_hit_request_count | 1199 / 1200 (99.9%) | + +### Key observations on E1 + +1. **D2 was never bound to a single session**. All 50 sessions got pinned to D0 or D1 by `kv-aware` policy's (overlap + sticky + inflight + assigned) lex-score, and naive pd-disaggregation has no migration mechanism to rebalance. Effective topology was **1P2D**, not 1P3D. +2. **Massive queueing**. TTFT p50 ≈ 89 s and p99 > 200 s indicate sessions waited tens of seconds in router/prefill queue. With `--concurrency-limit 32` and D0/D1 saturated, the inflight cap forced ~1250 reqs to serialize through only two decode workers. +3. **85 failures (6.6%)** — all `execution_mode == pd-disaggregation` (which the metrics module classifies as `error` when the agentic-pd-hybrid replay sees an unsuccessful upstream response). Most likely caused by `--request-timeout-s 300` firing on the longest queued requests. +4. **Cache hit 99.9%** — the kv-aware policy did successfully concentrate sessions on their prior D worker; the Inferact converter's prefix-shared 24-token-block hash_ids gave near-perfect prefix overlap across turns of the same session. + +### What E1 establishes + +For the same hardware, same trace, same model, **naive 1P3D + kv-aware policy is unusable for multi-session agentic workloads**: +- session-stickiness without migration leaves a third of compute capacity (1 of 3 decode GPUs) entirely unused +- queueing dominates user-facing latency +- failure rate is 6.6% even with 5 minutes per-request timeout + +This is *the baseline H1 needs* — it shows the KVC layer (E2) has something concrete to improve over. + +--- + +## 3. E2 results — pending + +Background task `b0im1d48q`, launched 2026-05-12 01:48 UTC. Same subset, full KVC v2 stack (reset-on-success migration, direct-append threshold 8192), RDMA on, all other knobs identical to E1. + +Expected differences: +- Direct-to-D fast path engaged for turn≥1 requests → fewer P/D round-trips +- Migration triggered when sessions hit D0/D1 saturation → D2 should see traffic +- Lower TTFT p50 / latency mean +- TTFT p99 still constrained by reseed slow-path (P re-prefill + mooncake transfer) + +Will be filled in upon completion. + +--- + +## 4. Comparison table — pending + +To be appended. + +--- + +## 5. Open questions for the next iteration + +- Are the 85 E1 errors all timeouts? `request-metrics.jsonl` rows with `error` execution_mode should be sampled to confirm. (Quick check: grep the metrics jsonl for `"execution_mode": "pd-disaggregation"` and inspect `latency_s` / `error` fields.) +- Does E2 produce the predicted ~91% direct-to-D rate seen in the historical SWE-Bench v2 run, or does the Inferact workload's larger session count (50 vs 52 there) but very different per-session size distribution (mean 33 turns × ~2KB context growth per turn) push it lower? +- Is `D2 = 0%` an E1-specific artifact (kv-aware sticky in pd-disagg mode), or does the same happen in E2 before migration kicks in for the first time?