perf: KI-2 FIXED — dim768 bf16 fits batch 32, tok/s 31.5K→40.8K

bf16 mixed precision (fp32 master) solves the v4 dim768 fp32 batch-32
OOM and speeds up the now-compute-bound dim768 GEMMs (dash5 1× RTX
5090 32GB, dim768/18L/24h×32 ffn2048 seq256, steady-state):

  config          batch  peak mem   tok/s   fits 32GB
  fp32            16      27.2 GB    31.5K   yes
  bf16            16      19.3 GB    35.5K   yes   (-29% mem / +13% tok/s)
  fp32            32      —          —       OOM
  bf16            32      31.1 GB    40.8K   yes   (+29% vs fp32-b16)

Verified on dash5: fp32 suite green at tight tol + xserv export md5
bit-identical to registry; bf16 looser-tol (loss 1.2e-4, logits p99
6.8e-3, grad 1.0e-2) + 150-step convergence tracks fp32 (3.984 vs
3.988); 2-GPU bf16 DDP at per-rank batch 32 trains cleanly.

Mark KI-2 FIXED; fill docs/11 results.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-06-16 14:28:20 +08:00
parent 48922cb628
commit 320c1ae4fb
2 changed files with 39 additions and 8 deletions

View File

@@ -76,8 +76,23 @@
- **dim768 bf16 能跑 per-rank batch 32**v4 OOM 的触发点)。
- dim768 **bf16 vs fp32** 的峰值显存 + steady-state tok/s预期显存↓、tok/s↑)。
## 实测结果dash5, 8× RTX 5090, sm_120
## 实测结果dash5, 1× RTX 5090 32GB, sm_120
> 见 `docs/known-issues.md` KI-2 的 before→after 表fp32 batch32 OOM → bf16 batch32 fit显存 A→Btok/s A→B
**闸门 ① fp32 不回归**全套测试在原紧容差绿autograd 15 / structural 5 / GEMM 5 / batched==looped / overfit 27/27 / AdamW GPU bit-exact + host torch / checkpoint 逐位 / DDP 2)。**xserv 闭环**v3 ckpt T12 代码重导 `model.safetensors` registry **md5 逐位一致**`b04fc9f9a0c9af04c47d9ca649aea12e`)——export/fp32 数值零漂移
**闸门 ② bf16 正确性 + 收敛**
- **looser-toltests/bf16.rs** fp32 master fp32 vs bf16——loss rel `1.2e-4`logits mean rel `2.0e-3` / p99 `6.8e-3`grad worst scaled-mean `1.0e-2` NaNgrad fp32master 未动)。
- **收敛**dim768 短训 150 bf16-b16 loss 轨迹对住 fp32-b16step50 `4.40` vs `4.40`step149 `3.984` vs `3.988`单调降无发散
**闸门 ③ 显存 + 吞吐dim768/18L/24h×32 ffn2048 seq256, steady-state**
| config | per-rank batch | 峰值显存 | tok/s | fits 32GB? |
|---|---|---|---|---|
| fp32 | 16 (v4 fallback) | 27.2 GB | 31.5K | |
| **bf16** | 16 | **19.3 GB29%** | **35.5K+13%** | |
| fp32 | 32 | | | **OOM** |
| **bf16** | **32甜点区** | **31.1 GB** | **40.8K** | **解 OOM** |
batchbf16 显存 29% / tok/s +13%**bf16 fp32-batch32 OOM**batch32 40.8K tok/s+29% vs fp32-b16)。KI-2 **FIXED**
</content>
</invoke>