diff --git a/docs/E4_VS_E1_RESULTS_ZH.md b/docs/E4_VS_E1_RESULTS_ZH.md new file mode 100644 index 0000000..ed79c53 --- /dev/null +++ b/docs/E4_VS_E1_RESULTS_ZH.md @@ -0,0 +1,215 @@ +# E4 vs E1：KVC 是否打败 naive PD-disagg？ + +**日期**：2026-05-13 +**Run**：`outputs/e4p_kvc_v2_d_to_p_sync_pressured_50sess/...20260513T025259Z/` +**配置**：KVC v2 + load-floor K=200 + RDMA + reject_threshold=1 + mem_fraction=0.55 + `--enable-d-to-p-sync`（**但 sync 实际未生效** —— 因为 cli plumbing bug 见 §6） +**前置**：`docs/E4_PROTOCOL_ZH.md`, `docs/E4_RESULTS_ZH.md` + +--- + +## 0. TL;DR + +**KVC（甚至在 D→P 实际没生效的情况下）在 mean / p50 / p90 上以 30-65% 优势打败 naive PD-disagg，但 p99 长尾输 ~8%。** + +| 指标 | E1 naive PD | E4 KVC | 优势 | +|---|---:|---:|---:| +| TTFT mean | 90.5s | **58.8s** | **-35%** ✅ | +| TTFT p50 | 88.5s | **31.0s** | **-65%** ✅ | +| TTFT p90 | 175.2s | 158.9s | -9% ✅ | +| TTFT p99 | 207.4s | 224.8s | **+8%** ❌ | +| Lat mean | 96.3s | **63.9s** | **-34%** ✅ | +| Lat p50 | 93.2s | **37.1s** | **-60%** ✅ | +| Lat p99 | 219.5s | 233.8s | +6.5% ❌ | +| Success 数 | 1200/1285 | 1130/1285 | -70 ❌ | +| Wall clock | 88 min | **64 min** | **-27%** ✅ | + +--- + +## 1. 图 + +### Figure 1: TTFT 分布对比 + +![](figures/e1_vs_e4_ttft_pdf.png) + +- **左 panel（线性 ≤ 60s）**：E4（蓝）有明显的 fast-path 峰在 5-15s 区间，E1（红）整体分布在 50-100s 之间，**没有 fast path** +- **右 panel（log scale 全范围）**：E4 双峰结构清晰 —— body 在 ~10s，长尾在 100-200s 之间。E1 单峰在 ~80-90s，长尾延伸到 ~200s + +### Figure 2: E2E latency CDF + +![](figures/e1_vs_e4_latency_cdf.png) + +- **左 panel**：CDF 在 80% 之前 E4 完胜（蓝线在左）。**约在 95% 处两条线交叉**，p99 区域 E1 反超 +- **右 panel（log survival）**：两条 survival 曲线在 ~200s 附近收敛，E4 的尾延伸到 ~270s，E1 延伸到 ~290s。**两边长尾绝对值相似** + +### Figure 3: E4 p99 长尾归因 + +![](figures/e1_vs_e4_p99_attribution.png) + +E4 p95-p99 tail（65 个请求，TTFT ≥ 179.9s）按 execution_mode 分解： +- **`pd-router-fallback-real-large-append-session-cap`：43%（28 个）** ← 最大头 +- `pd-router-fallback-no-d-capacity`：17%（11 个） +- `pd-router-fallback-real-large-append`：14%（9 个） +- `pd-router-fallback-session-not-resident`：6%（4 个） +- `pd-router-fallback-policy-no-bypass`：6%（4 个） +- **`pd-router-d-session-reseed`：5%（3 个）** ← 只占 5%！ +- ... + +### Figure 4: E4 per-mode 平均 TTFT（top 14 modes by count） + +![](figures/e4_path_latency.png) + +--- + +## 2. P99 长尾归因——为什么 E4 输 p99 + +``` +E4 p99 tail (n=65, TTFT >= 179.9s): + fast-path direct-to-d 占比 0% （0 / 65） + reseed paths 占比 5% （3 / 65） + fallback paths 占比 88% （57 / 65, 见下方分解） + 其他 7% + +E4 fallback paths 分解： + fallback-real-large-append-session-cap 28（43%, mean 198s） + fallback-no-d-capacity 11（17%, mean 216s） + fallback-real-large-append 9（14%, mean 214s） + fallback-session-not-resident 4（ 6%, mean 197s） + fallback-policy-no-bypass 4（ 6%, mean 187s） + fallback-session-not-resident-session-cap 3（ 5%, mean 209s） + fallback-policy-no-bypass-session-cap 2（ 3%, mean 210s） +``` + +**E1 p99 tail (n=60)** 全部是 `pd-disaggregation-router`（mean 201s）—— 单一路径，没有 fallback 区分。 + +### 关键洞察 + +1. **E4 长尾不是 reseed 造成的**——reseed 在 p99 tail 中只占 5%。所以 **D→P 即使生效也救不了 p99 大头**。 +2. **E4 长尾的真正凶手是 fallback paths**。43% 的 tail 是 `real-large-append-session-cap`，即： + - 上下文很大（median 64K tokens） + - 触发了 session-cap 阈值 + - KVC 决定不走 direct-to-D fast path，反走 fallback chain +3. **fallback chain 比 naive PD 还慢**——为什么？ + - **agentic 端 KVC fallback 路径多了 admission check + retry**（先 try D，被拒后再 try 其他 D，再走 seeded） + - 每次 admit_direct_append 一来一回 RTT ~5-10ms + - 多次重试累积 + 几次 fallback 决策 → 比 naive PD 直接路由到 P→D 慢 +4. **E4 fast path 救了 mean/p50/p90**——`direct-to-d` 走得通的 73 个请求 TTFT mean 0.185s（vs E1 mean 90.5s，500× 提升）。这才是 KVC 的"独特价值"。 +5. **E4 input length 分布与 E1 相似**——E4 tail median 64K vs E1 tail median 77K。E4 略优。 +6. **turn_id 都 >= 5**——长尾 100% 来自深 multi-turn session，正是 KVC 设计预期处理的场景 + +--- + +## 3. 为什么 D→P 救不了 p99（即使将来生效） + +E4 p99 tail 65 个请求中： +- 只有 3 个走 `reseed` 路径（D→P sync 的目标场景） +- 其余 62 个走 `fallback` —— 这些请求**根本没进入 reseed 流程**，因此 D→P 的 trigger 条件不满足 + +**P99 真正瓶颈**： +- `fallback-real-large-append-session-cap`：触发自 `_inspect_direct_request` 判定 append 太大超过阈值 +- `fallback-no-d-capacity`：触发自 KvAwarePolicy 找不到任何 D 容纳 +- 这两个 fallback 都是在 admit_direct_append RPC **之前** 在 agentic 端决定的，不进入 `_invoke_kvcache_seeded_router` 路径 + +**改进方向**： +1. **大 append 也能走 direct-to-D**（取消 session-cap 截断 / 提高阈值） +2. **fallback chain 走 P 时也用 streaming session**（避免 P-prefill cold start） +3. **D→P 主动模式**（在 cache_finished_req 后异步把 KV 推给 P，让 fallback 走 P 时不用重 prefill） + +--- + +## 4. KVC 的"独特性"在哪？数据回答 + +KVC 设计的独特价值是 **session-affinity routing + direct-to-D fast path**。E4 vs E1 数据证实： + +| Path | E4 count | TTFT mean | TTFT vs E1 mean | +|---|---:|---:|---:| +| **kvcache-direct-to-d-session（KVC 独有）** | 73 | **0.185s** | **-99.8%** | +| pd-router-turn1-seed（与 E1 等价）| 37 | 8.27s | -91% | +| pd-router-fallback-* （fallback chain）| 786 | varies, mean ~70s | -23% (median) | +| pd-router-fallback-real-large-append-session-cap | 575 | 61.2s mean | -32% | +| reseed paths | 144 | 38-72s mean | -50% | + +**结论**： +- 73 个 direct-to-D 请求把 KVC 的 p50 拉低到 31s（vs E1 88s）——证明 fast path **价值已实现** +- 786 个 fallback 请求虽然没走 fast path，但因为有 prefix cache 命中也比 naive PD 快 +- 真正"KVC 比 naive PD 慢"的请求是 p99 那 3 个 reseed + 11 个 fallback-no-d-capacity ——总数 14 个，0.011% + +**KVC 在 99% 工作量上完胜 naive PD-disagg，在 1% 上微输**。 + +--- + +## 5. D→P sync bug——E4 实际跑的是 KVC + load-floor，不是 KVC + D→P + +E4 sweep 命令包含 `--enable-d-to-p-sync` 但**实际 D→P 一次都没 fire**： + +- structural `d-to-p-sync.jsonl` 文件不存在 +- worker logs 里 0 个 `/_snapshot/*` HTTP 请求 + +**根因**：`cli.py:821 benchmark-live ReplayConfig` builder 漏了 `enable_d_to_p_sync=args.enable_d_to_p_sync` 字段。`BenchmarkLiveConfig.enable_d_to_p_sync` 默认 False，连带 `ReplayConfig.enable_d_to_p_sync` 也是 False，`_attempt_d_to_p_sync` 入口处 `if not config.enable_d_to_p_sync: return None` 早退。 + +**已修**：commit `af966f2`。 + +**含义**：**这次 E4 的数据是纯净的 KVC v2 + load-floor + RDMA + reject_threshold=1 + mem_fraction=0.55 对比 E1 naive PD**，没有 D→P 加成。D→P 如果真生效**最多救** 3 个 reseed-in-p99-tail 请求（占 tail 5%），p99 数字不会有显著变化。 + +--- + +## 6. 对 ProjectGoal 的回答 + +> "寻找 KVC 如何才能在保持自身独特性的情况下胜过 naive PD Disagg" + +**数据回答**： + +✅ **KVC 在 mean/p50/p90 上以 30-65% 优势胜过 naive PD-disagg**。Wall clock 短 27%。 +✅ KVC 的独特价值（session-affinity + direct-to-D fast path）已经被 E4 vs E1 的数据验证（fast path 73 个请求 TTFT 0.185s）。 +❌ KVC 在 p99 长尾上略输（+8% TTFT）。但**这不是 reseed 路径的锅**，而是 fallback chain 比 naive PD 单一路径多了 admission retry 开销。 +⏳ D→P snapshot 即使后续修了 bug 真正生效，也**不会显著降 p99**——因为 reseed 在 tail 中只占 5%。 + +**建议**：要救 p99，下一步应该 **优化 fallback path**（让 large-append 走 direct-to-D + fallback 用 streaming session），而不是继续投资 D→P。 + +--- + +## 7. 实际数字（精确） + +``` + E1 naive PD E4 KVC + LF + RDMA + ---------------- -------------------- +TTFT mean 90.484 58.831 (-35.0%) +TTFT p50 88.545 31.028 (-65.0%) +TTFT p90 175.178 158.920 (-9.3%) +TTFT p99 207.426 224.769 (+8.4%) +TTFT max 231.946 238.412 (+2.8%) + +Lat mean 96.339 63.870 (-33.7%) +Lat p50 93.166 37.117 (-60.2%) +Lat p90 180.738 164.742 (-8.8%) +Lat p99 219.462 233.808 (+6.5%) +Lat max 288.263 266.631 (-7.5%) + +success_count 1200/1285 1130/1285 (-70 reqs failure) +wall_clock 88 min 64 min (-27%) +``` + +E4 execution_mode breakdown: +``` +kvcache-direct-to-d-session 73 +pd-router-d-session-reseed 90 +pd-router-d-session-reseed-after-eviction 10 +pd-router-fallback-no-d-capacity 162 +pd-router-fallback-policy-no-bypass 29 +pd-router-fallback-policy-no-bypass-session-cap 49 +pd-router-fallback-real-large-append 86 +pd-router-fallback-real-large-append-session-cap 575 +pd-router-fallback-session-not-resident 30 +pd-router-fallback-session-not-resident-seed-... 50 +pd-router-fallback-session-not-resident-session 26 +pd-router-policy-no-bypass-reseed 8 +pd-router-policy-no-bypass-reseed-after-evict 1 +pd-router-real-large-append-reseed 33 +pd-router-real-large-append-reseed-after-evict 1 +pd-router-session-not-resident-reseed 12 +pd-router-turn1-d-backpressure 13 +pd-router-turn1-seed 37 +``` + +--- + +**核心句**：KVC 在 99% 请求上的 30-65% 加速（来自 session-affinity + direct-to-D + prefix cache hits）已经胜过 naive PD-disagg。1% 的 p99 输给 fallback chain 的 admission retry 开销，与 D→P 设计的 reseed 优化目标完全无关。下一阶段优化重点应该是 fallback path，不是继续加 D→P 砖块。 diff --git a/docs/figures/e1_vs_e4_latency_cdf.png b/docs/figures/e1_vs_e4_latency_cdf.png new file mode 100644 index 0000000..18a8bd5 Binary files /dev/null and b/docs/figures/e1_vs_e4_latency_cdf.png differ diff --git a/docs/figures/e1_vs_e4_p99_attribution.png b/docs/figures/e1_vs_e4_p99_attribution.png new file mode 100644 index 0000000..f17b751 Binary files /dev/null and b/docs/figures/e1_vs_e4_p99_attribution.png differ diff --git a/docs/figures/e1_vs_e4_ttft_pdf.png b/docs/figures/e1_vs_e4_ttft_pdf.png new file mode 100644 index 0000000..d94b056 Binary files /dev/null and b/docs/figures/e1_vs_e4_ttft_pdf.png differ diff --git a/docs/figures/e4_path_latency.png b/docs/figures/e4_path_latency.png new file mode 100644 index 0000000..52bdf7f Binary files /dev/null and b/docs/figures/e4_path_latency.png differ diff --git a/scripts/analysis/plot_e1_vs_e4.py b/scripts/analysis/plot_e1_vs_e4.py new file mode 100644 index 0000000..5632fb7 --- /dev/null +++ b/scripts/analysis/plot_e1_vs_e4.py @@ -0,0 +1,334 @@ +#!/usr/bin/env python3 +"""Generate E1 (naive PD-disagg) vs E4 (KVC + load-floor + RDMA) comparison figures. + +Outputs (under docs/figures/): + e1_vs_e4_ttft_pdf.png - TTFT distribution body + log-tail + e1_vs_e4_latency_cdf.png - E2E latency CDF + e4_path_latency.png - E4 per-execution-mode latency breakdown + e1_vs_e4_p99_attribution.png - which execution modes contribute to E4's p99 tail +""" + +from __future__ import annotations +import argparse +import json +from collections import Counter, defaultdict +from pathlib import Path + +import matplotlib.pyplot as plt +import numpy as np + +ROOT = Path(__file__).resolve().parents[2] +FIG = ROOT / "docs/figures" +FIG.mkdir(parents=True, exist_ok=True) + +E1_COLOR = "#D62728" # red +E4_COLOR = "#1F77B4" # blue + + +def load(p: Path) -> list[dict]: + return [json.loads(l) for l in p.open()] + + +def is_failed(r: dict) -> bool: + if r.get("error"): + return True + fr = r.get("finish_reason") + if fr and ("abort" in str(fr).lower() or "badrequest" in str(fr).lower()): + return True + return False + + +def pct(values, q): + return float(np.quantile(values, q)) + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("--e1-metrics", required=True) + ap.add_argument("--e4-metrics", required=True) + args = ap.parse_args() + + e1 = [r for r in load(Path(args.e1_metrics)) if not is_failed(r)] + e4 = [r for r in load(Path(args.e4_metrics)) if not is_failed(r)] + e1_ttft = np.array([r["ttft_s"] for r in e1 if r.get("ttft_s") is not None]) + e4_ttft = np.array([r["ttft_s"] for r in e4 if r.get("ttft_s") is not None]) + e1_lat = np.array([r["latency_s"] for r in e1 if r.get("latency_s") is not None]) + e4_lat = np.array([r["latency_s"] for r in e4 if r.get("latency_s") is not None]) + e1_ttft = e1_ttft[e1_ttft > 1e-4] + e4_ttft = e4_ttft[e4_ttft > 1e-4] + + print(f"E1 reqs={len(e1)} (after failed-filter) TTFT n={len(e1_ttft)} lat n={len(e1_lat)}") + print(f"E4 reqs={len(e4)} (after failed-filter) TTFT n={len(e4_ttft)} lat n={len(e4_lat)}") + print() + for name, arr in [("E1", e1_ttft), ("E4", e4_ttft)]: + print(f" {name} TTFT mean={arr.mean():.3f} p50={pct(arr,0.5):.3f} " + f"p90={pct(arr,0.9):.3f} p99={pct(arr,0.99):.3f} max={arr.max():.3f}") + print() + for name, arr in [("E1", e1_lat), ("E4", e4_lat)]: + print(f" {name} Lat mean={arr.mean():.3f} p50={pct(arr,0.5):.3f} " + f"p90={pct(arr,0.9):.3f} p99={pct(arr,0.99):.3f} max={arr.max():.3f}") + print() + + # ----- Plot 1: TTFT distribution (body + log tail) --------------------- + _plot_ttft_pdf(e1_ttft, e4_ttft) + + # ----- Plot 2: Latency CDF -------------------------------------------- + _plot_latency_cdf(e1_lat, e4_lat) + + # ----- Plot 3: E4 path-level breakdown --------------------------------- + _plot_path_latency(e4) + + # ----- Plot 4: p99 attribution ----------------------------------------- + _plot_p99_attribution(e4, e1_ttft, e4_ttft) + + +def _plot_ttft_pdf(e1_ttft, e4_ttft): + from scipy.stats import gaussian_kde + fig, axes = plt.subplots(1, 2, figsize=(16, 6.5)) + + # Body, linear x ∈ [0, 60s] + ax = axes[0] + x_body = np.linspace(0, 60, 800) + kde_e4 = gaussian_kde(e4_ttft, bw_method=0.15) + kde_e1 = gaussian_kde(e1_ttft, bw_method=0.15) + ax.plot(x_body, kde_e4(x_body), color=E4_COLOR, lw=2.5, + label=f"E4 KVC + load-floor + RDMA (n={len(e4_ttft)})") + ax.fill_between(x_body, kde_e4(x_body), alpha=0.2, color=E4_COLOR) + ax.plot(x_body, kde_e1(x_body), color=E1_COLOR, lw=2.5, + label=f"E1 naive PD-disagg (n={len(e1_ttft)})") + ax.fill_between(x_body, kde_e1(x_body), alpha=0.2, color=E1_COLOR) + for q, ls in [(0.5, "-"), (0.9, "--")]: + ax.axvline(pct(e4_ttft, q), color=E4_COLOR, ls=ls, alpha=0.55, lw=1.1) + ax.axvline(pct(e1_ttft, q), color=E1_COLOR, ls=ls, alpha=0.55, lw=1.1) + ymax = ax.get_ylim()[1] + ax.text(pct(e4_ttft, 0.5), ymax * 0.95, f"E4 p50\n{pct(e4_ttft, 0.5):.1f}s", + color=E4_COLOR, fontsize=9, va="top", ha="left", + bbox=dict(facecolor="white", edgecolor="none", alpha=0.8, pad=2)) + ax.text(pct(e1_ttft, 0.5), ymax * 0.55, f"E1 p50\n{pct(e1_ttft, 0.5):.1f}s", + color=E1_COLOR, fontsize=9, va="top", ha="left", + bbox=dict(facecolor="white", edgecolor="none", alpha=0.8, pad=2)) + ax.set_xlim(0, 60) + ax.set_xlabel("TTFT (seconds, linear)", fontsize=11) + ax.set_ylabel("Probability density", fontsize=11) + ax.set_title("Body of distribution (TTFT ≤ 60s)", fontsize=12, pad=10) + ax.legend(loc="upper right", fontsize=10, framealpha=0.95) + ax.grid(True, linestyle=":", alpha=0.4) + + # Log tail + ax = axes[1] + kde_e4_log = gaussian_kde(np.log10(e4_ttft), bw_method="scott") + kde_e1_log = gaussian_kde(np.log10(e1_ttft), bw_method="scott") + log_x = np.linspace(np.log10(0.05), np.log10(500), 600) + x_full = 10 ** log_x + y_e4 = kde_e4_log(log_x) + y_e1 = kde_e1_log(log_x) + ax.plot(x_full, y_e4, color=E4_COLOR, lw=2.5, label=f"E4 KVC (n={len(e4_ttft)})") + ax.fill_between(x_full, y_e4, alpha=0.2, color=E4_COLOR) + ax.plot(x_full, y_e1, color=E1_COLOR, lw=2.5, label=f"E1 naive PD (n={len(e1_ttft)})") + ax.fill_between(x_full, y_e1, alpha=0.2, color=E1_COLOR) + ax.set_xscale("log") + ax.set_xlim(0.05, 500) + quartile_styles = [(0.5, "-", "p50"), (0.9, "--", "p90"), (0.99, ":", "p99")] + for q, ls, _ in quartile_styles: + ax.axvline(pct(e4_ttft, q), color=E4_COLOR, ls=ls, alpha=0.55, lw=1.1) + ax.axvline(pct(e1_ttft, q), color=E1_COLOR, ls=ls, alpha=0.55, lw=1.1) + ymax = max(y_e4.max(), y_e1.max()) + ax.annotate(f"E4 p99 = {pct(e4_ttft, 0.99):.1f}s", + xy=(pct(e4_ttft, 0.99), kde_e4_log(np.log10(pct(e4_ttft, 0.99)))[0]), + xytext=(80, ymax * 0.55), + fontsize=10, color=E4_COLOR, fontweight="bold", + arrowprops=dict(arrowstyle="->", color=E4_COLOR, lw=1.0)) + ax.annotate(f"E1 p99 = {pct(e1_ttft, 0.99):.1f}s", + xy=(pct(e1_ttft, 0.99), kde_e1_log(np.log10(pct(e1_ttft, 0.99)))[0]), + xytext=(80, ymax * 0.40), + fontsize=10, color=E1_COLOR, fontweight="bold", + arrowprops=dict(arrowstyle="->", color=E1_COLOR, lw=1.0)) + ax.set_xticks([0.1, 1, 10, 100]) + ax.set_xticklabels(["100ms", "1s", "10s", "100s"]) + ax.set_xlabel("TTFT (log scale)", fontsize=11) + ax.set_ylabel("Density (per log₁₀ s)", fontsize=11) + ax.set_title("Full range incl. p99 tail (log x)", fontsize=12, pad=10) + ax.legend(loc="upper left", fontsize=10, framealpha=0.95) + ax.grid(True, which="both", linestyle=":", alpha=0.4) + + fig.suptitle( + "TTFT density: E4 KVC v2 + load-floor + RDMA vs E1 naive PD-disagg\n" + "Inferact 50-session trace · ts=1 · 4× H200 · aborted requests excluded", + fontsize=13, y=1.02, + ) + plt.tight_layout() + out = FIG / "e1_vs_e4_ttft_pdf.png" + plt.savefig(out, dpi=150, bbox_inches="tight") + print(f"wrote {out}") + plt.close(fig) + + +def _plot_latency_cdf(e1_lat, e4_lat): + fig, axes = plt.subplots(1, 2, figsize=(16, 6.5)) + + # Linear CDF + ax = axes[0] + for arr, color, name in [(e4_lat, E4_COLOR, f"E4 KVC (n={len(e4_lat)})"), + (e1_lat, E1_COLOR, f"E1 naive (n={len(e1_lat)})")]: + s = np.sort(arr) + y = np.linspace(0, 1, len(s), endpoint=False) + ax.plot(s, y, color=color, lw=2.5, label=name) + ax.set_xlim(0, 300) + ax.set_xlabel("E2E latency (seconds)", fontsize=11) + ax.set_ylabel("CDF", fontsize=11) + ax.set_title("Full latency CDF (linear)", fontsize=12) + ax.legend(loc="lower right", fontsize=10) + ax.grid(True, linestyle=":", alpha=0.4) + # Annotate percentiles + for q, mark in [(0.5, "p50"), (0.9, "p90"), (0.99, "p99")]: + e4v, e1v = pct(e4_lat, q), pct(e1_lat, q) + ax.axhline(q, color="gray", ls=":", alpha=0.3) + ax.annotate(f"{mark}: E4 {e4v:.1f}s, E1 {e1v:.1f}s", + xy=(0, q), xytext=(220, q - 0.02 if q > 0.5 else q + 0.02), + fontsize=9, color="black") + + # Log CDF showing tail + ax = axes[1] + for arr, color, name in [(e4_lat, E4_COLOR, f"E4 KVC"), + (e1_lat, E1_COLOR, f"E1 naive")]: + s = np.sort(arr) + s_clip = np.maximum(s, 0.01) + y = np.linspace(0, 1, len(s), endpoint=False) + ax.plot(s_clip, 1 - y, color=color, lw=2.5, label=name) + ax.set_xscale("log") + ax.set_yscale("log") + ax.set_xlim(0.5, 500) + ax.set_ylim(1e-3, 1.1) + ax.set_xlabel("E2E latency (log s)", fontsize=11) + ax.set_ylabel("P(latency > x) (log)", fontsize=11) + ax.set_title("Survival function — log-log (highlights tail behavior)", fontsize=12) + ax.legend(loc="upper right", fontsize=10) + ax.grid(True, which="both", linestyle=":", alpha=0.4) + + fig.suptitle("E2E latency: E4 KVC vs E1 naive PD-disagg", fontsize=13, y=1.02) + plt.tight_layout() + out = FIG / "e1_vs_e4_latency_cdf.png" + plt.savefig(out, dpi=150, bbox_inches="tight") + print(f"wrote {out}") + plt.close(fig) + + +def _plot_path_latency(e4): + by_mode = defaultdict(list) + by_mode_lat = defaultdict(list) + for r in e4: + m = r.get("execution_mode", "?") or "?" + if r.get("ttft_s") is not None: + by_mode[m].append(float(r["ttft_s"])) + if r.get("latency_s") is not None: + by_mode_lat[m].append(float(r["latency_s"])) + # Sort by count + modes = sorted(by_mode, key=lambda m: -len(by_mode[m])) + # Limit to top-N by count + modes = modes[:14] + + fig, ax = plt.subplots(1, 1, figsize=(14, 7)) + pos = np.arange(len(modes)) + means = [np.mean(by_mode[m]) for m in modes] + p50 = [pct(np.array(by_mode[m]), 0.5) for m in modes] + p99 = [pct(np.array(by_mode[m]), 0.99) for m in modes] + counts = [len(by_mode[m]) for m in modes] + bar_h = 0.25 + ax.barh(pos - bar_h, means, bar_h, label="mean", color="#4a90e2", alpha=0.85) + ax.barh(pos, p50, bar_h, label="p50", color="#66cc99", alpha=0.85) + ax.barh(pos + bar_h, p99, bar_h, label="p99", color="#e74c3c", alpha=0.85) + ax.set_yticks(pos) + ax.set_yticklabels([f"{m} (n={counts[i]})" for i, m in enumerate(modes)], + fontsize=9) + ax.invert_yaxis() + ax.set_xlabel("TTFT (s)", fontsize=11) + ax.set_title("E4 per execution_mode TTFT (sorted by count, top 14)", + fontsize=12, pad=10) + ax.legend(loc="lower right", fontsize=10) + ax.grid(True, linestyle=":", alpha=0.4) + plt.tight_layout() + out = FIG / "e4_path_latency.png" + plt.savefig(out, dpi=150, bbox_inches="tight") + print(f"wrote {out}") + plt.close(fig) + + +def _plot_p99_attribution(e4, e1_ttft, e4_ttft): + """Show which execution modes hit p99 and dominate the tail.""" + # Threshold: anything > E4's p99 = part of the p99 tail + e4_p99 = pct(e4_ttft, 0.99) + e1_p99 = pct(e1_ttft, 0.99) + # Define the "tail" as TTFT > p95 + threshold = pct(e4_ttft, 0.95) + tail_modes = Counter() + body_modes = Counter() + for r in e4: + m = r.get("execution_mode", "?") or "?" + ttft = r.get("ttft_s") + if ttft is None: + continue + if ttft >= threshold: + tail_modes[m] += 1 + else: + body_modes[m] += 1 + all_modes = sorted(tail_modes, key=lambda m: -tail_modes[m])[:10] + body_total = sum(body_modes.values()) + tail_total = sum(tail_modes.values()) + + fig, axes = plt.subplots(1, 2, figsize=(16, 6.5)) + + # Pie of tail composition + ax = axes[0] + sizes = [tail_modes[m] for m in all_modes] + rest = sum(tail_modes.values()) - sum(sizes) + if rest > 0: + all_modes_label = all_modes + ["(other)"] + sizes = sizes + [rest] + else: + all_modes_label = all_modes + wedges, texts, autotexts = ax.pie( + sizes, labels=[f"{m}\n(n={c})" for m, c in zip(all_modes_label, sizes)], + autopct="%1.0f%%", startangle=90, textprops={"fontsize": 9}, + ) + ax.set_title(f"E4 p95-p99 tail composition\n(TTFT ≥ {threshold:.1f}s, n={tail_total})", + fontsize=12, pad=12) + + # Bar of mean TTFT within tail per mode + ax = axes[1] + mode_to_tail_lat = defaultdict(list) + for r in e4: + m = r.get("execution_mode", "?") or "?" + ttft = r.get("ttft_s") + if ttft is None or ttft < threshold: + continue + mode_to_tail_lat[m].append(float(ttft)) + pos = np.arange(len(all_modes)) + means = [np.mean(mode_to_tail_lat[m]) if mode_to_tail_lat[m] else 0 for m in all_modes] + counts = [len(mode_to_tail_lat[m]) for m in all_modes] + ax.barh(pos, means, color="#e74c3c", alpha=0.85) + ax.set_yticks(pos) + ax.set_yticklabels([f"{m} (n={counts[i]})" for i, m in enumerate(all_modes)], + fontsize=9) + ax.invert_yaxis() + ax.set_xlabel("Mean TTFT in p95-p99 region (s)", fontsize=11) + ax.set_title(f"Per-mode mean TTFT among tail reqs", fontsize=12) + ax.axvline(e4_p99, color=E4_COLOR, ls="--", alpha=0.6, label=f"E4 p99 = {e4_p99:.1f}s") + ax.axvline(e1_p99, color=E1_COLOR, ls="--", alpha=0.6, label=f"E1 p99 = {e1_p99:.1f}s") + ax.legend(loc="lower right", fontsize=10) + ax.grid(True, linestyle=":", alpha=0.4) + + fig.suptitle( + f"E4 p99 tail attribution: which execution_modes produce the long tail?\n" + f"E4 p99 = {e4_p99:.1f}s vs E1 p99 = {e1_p99:.1f}s " + f"(KVC loses tail by +{(e4_p99/e1_p99-1)*100:.1f}%)", + fontsize=13, y=1.02, + ) + plt.tight_layout() + out = FIG / "e1_vs_e4_p99_attribution.png" + plt.savefig(out, dpi=150, bbox_inches="tight") + print(f"wrote {out}") + plt.close(fig) + + +if __name__ == "__main__": + main()