perf: KI-5 FIXED — single-GPU 40K->93K tok/s, DDP scaling 1.3x->5x@8

Device caching/pool allocator removes the per-op cudaMalloc serialization that
was the real DDP bottleneck (and a single-GPU cost). Measured on dash5 (8x RTX
5090, dim384/12L, per-rank batch 32, seq 256, steady-state tok/s):

  single-GPU: 40226 -> 92638 tok/s  (~2.3x)
  DDP scaling (global batch 32*world):
    world  before        after
      1    39801 1.00x    92385 1.00x
      2    47229 1.19x   146821 1.59x
      4    52854 1.33x   269867 2.92x
      8    48996 1.23x   461270 4.99x

8-GPU absolute throughput 49K -> 461K tok/s (9.4x); nvidia-smi shows all 8 GPUs
at 95-99% util during the run (KI-5 saw only 1-2/8 busy). Loss trajectories are
bit-identical before/after (10.9026->4.8453). xserv closed loop green: re-export
of the v3 ckpt is md5-identical to the registry safetensors and xserv serves it.

Mark KI-5 FIXED in docs/known-issues.md with before/after table; fill in the
design doc's measured numbers. Residual ~5x@8 (not perfectly linear) is the
~7% all-reduce + 8-GPU PCIe/launch overhead; process-per-GPU is the next lever
if v4 needs higher linearity.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

docs/10-caching-allocator.md

@@ -129,6 +129,32 @@ cuBLAS handle 仍串行），如实报告，并说明 **process-per-GPU**（每
   承重闸门是 loss-match（~5.7e-7）；本机 PCIe-only NCCL all-reduce run-to-run 跨 rank 非逐位可复现，
   diff ≤1.2e-7（几 ULP，数值无害）。`== 0.0` 过严 flaky。
 
-## Before → After
+## Before → After（dash5, 8× RTX 5090, sm_120）
 
-（dash5, 8× RTX 5090, 实测填入；见 known-issues.md KI-5 的 before/after 表与 commit。）
+实测（`train_ddp`, dim384/12L/12h·hd32 ffn1536 core 28.3M, per-rank batch 32, seq 256,
+steady-state tok/s；before = parent `d422c68`, after = pooled）：
+
+**单卡（KI-5 假设：per-op malloc 单卡也吃）**
+
+| | tok/s | GPU util |
+|---|---|---|
+| before | 40226 | 8 卡轮流忙，1–2/8 |
+| after | **92638** | — |
+
+→ 单卡 **~2.3×**，loss 轨迹逐位对住（10.9026→4.8453 before/after 一致）。
+
+**DDP 1/2/4/8 卡 scaling（global batch = 32×world）**
+
+| world | before tok/s | before speedup | after tok/s | after speedup |
+|---|---|---|---|---|
+| 1 | 39801 | 1.00× | 92385 | 1.00× |
+| 2 | 47229 | 1.19× | 146821 | 1.59× |
+| 4 | 52854 | 1.33× | 269867 | 2.92× |
+| 8 | 48996 | 1.23× | **461270** | **4.99×** |
+
+→ 8 卡绝对吞吐 **49K → 461K tok/s = 9.4×**；scaling 从「~1.3× 封顶」恢复到 **~5×@8**。
+8 卡运行 `nvidia-smi` 抽样 **8 卡全部 95–99% util**（KI-5 时只有 1–2/8 在忙）——
+per-op cudaMalloc 串行确是根因，pool 消掉后 GPU 变 compute-bound 喂满。
+
+**残留**：5×@8 非完美线性（grad all-reduce ~7% + 8 卡 PCIe / launch 余量），但弱扩展的悬崖已消。
+KI-5 标 **FIXED**。若 v4 要更高线性度，下一步才是 process-per-GPU（每 rank 独立 context）。