f2c: switch to per-instance decode-concurrency view; correct KV pool ceiling
Old f2c plotted per-request KV footprint MiB against an "H20 ~95 GiB usable" reference line. That ceiling was wrong — a 30B-A3B bf16 deployment burns roughly: ~50% HBM for model params (~48 GiB on 96 GiB H20) ~10% for runtime activation buffers ~40% left for the KV cache pool (~38.4 GiB) so 95 GiB was overstating the available pool by 2.5×. New f2c reframes the same data into the answer that actually motivates the paper: how many concurrent decodes does a single instance hold, and how does PD-disagg change that? Grouped bars per percentile show system-wide concurrent decode capacity for three 8-GPU deployments: Combined 8C, PD-disagg 4P+4D (N_D=4), PD-disagg 6P+2D (N_D=2) Key reads off the figure: p50 (1.8 GiB/req): 20 fit/inst → 160 / 80 / 40 system-wide p90 (8.0 GiB/req): 4 fit/inst → 32 / 16 / 8 p95 (9.6 GiB/req): 4 fit/inst → 32 / 16 / 8 p99 (11.5 GiB/req): 3 fit/inst → 24 / 12 / 6 PD-disagg 4P+4D literally halves the decode population at the same per-request KV pressure — this is the concrete §3.2 "KV memory wall" penalty stated in terms users care about (concurrency). - analysis/characterization/render_window1_figures.py: fig_kv_footprint_cdf rewritten; reads same kv_footprint_summary.json but computes floor(KV_pool / req_size) × N_D and annotates the per-instance fit count below each percentile group. - figs/f2c_kv_footprint_cdf.png: regenerated. - MEETING.md / PAPER_OUTLINE.md §2.1, §2.4: prose updated with the new ceiling and the "3 p99 decodes per instance / halved by PD-disagg" framing. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -50,7 +50,7 @@ Agentic workload 与 chatbot 的三个本质差异:
|
||||
- **Prefill-dominated**:input/output token ratio **75x**,98% 计算在 prefill 阶段(chatbot 为 1-10x)
|
||||
- **Skewed sessions**(来自 Qwen3 production trace,n=1.3M session / 2.1M req / 7200s):top 1% 贡献 **46.5%** input token,top 5% **66.5%**,top 10% **74.6%**,top 25% **87.5%**,top 50% **96.0%** —— 半数 session 几乎占满全部 input mass
|
||||
|
||||
平均 session 长度 TBD turn、TBD 输入 token;p99 单请求 KV 占用 **11.49 GiB**(H20 96GB HBM 的 12%)。
|
||||
平均 session 长度 TBD turn、TBD 输入 token。Per-request KV footprint(Qwen3-Coder-30B-A3B, 98304 B/token):p50 **1.8 GiB**, p90 **8.0 GiB**, p95 **9.6 GiB**, p99 **11.5 GiB**. 单 instance KV pool ≈ 0.4 × 96 GiB = **38.4 GiB**(剩 50% model params bf16 + 10% runtime activation),所以 p99 请求一个 instance 只能装 **3 个 concurrent decode**;改 PD-disagg 4P+4D 让系统 decode 容量直接减半(系统并发 24 → 12)。
|
||||
|
||||
### §2.2 KV Cache Reuse Topology
|
||||
|
||||
@@ -70,7 +70,7 @@ Trace 上 KV reuse 的分解:
|
||||
|
||||

|
||||
|
||||

|
||||

|
||||
|
||||
> 📝 Layout TBD:三张拼成 1×3 还是分散到 §2.1/§2.2/§2.4 各一张。
|
||||
|
||||
|
||||
Reference in New Issue
Block a user