docs: Phase 14 design doc + benchmark, fix Phase 11/12 honesty

Phase 14 (Flash Attention):
- Design doc: FA2 algorithm, SM120 hardware constraints (FA4 incompatible),
  kernel config (BR=BC=64, 32KB smem), GQA mapping, causal tile-skip,
  known limitations and optimization roadmap
- Benchmark doc: correctness (9/10 top-1 match, identical to pre-FA baseline),
  performance tracking (6.9→10.3→12.9 tok/s across phases), memory savings
  analysis, remaining bottleneck breakdown

Phase 11 doc: title corrected from "Paged Attention" to "GPU-Resident KV Cache"
with explicit note that paged allocation was not implemented.

Phase 12 doc: "当前状态" updated from "未实现" to reflect actual state —
iteration-level scheduling implemented + verified (6.0x concurrent speedup),
batched GPU forward explicitly marked as not yet implemented.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-22 18:51:29 +08:00
parent d67dda404e
commit 6cc1c9332d
4 changed files with 285 additions and 3 deletions

View File

@@ -1,8 +1,10 @@
# Phase 11: Paged Attention + KV Cache Manager — Design Document
# Phase 11: GPU-Resident KV Cache — Design Document
> **注意**: 原计划为 "Paged Attention + KV Cache Manager",实际实现为 GPU 连续预分配 KV cache非 paged。Paged allocation 留待后续优化。
## Goal
将 KV cache 从 CPU Vec 迁移到 GPU使用 block-based paging 管理显存。消除每步 decode 的 CPU round-trip当前 KV cache 最大性能瓶颈之一)。
将 KV cache 从 CPU Vec 迁移到 GPU消除每步 decode 的 CPU round-trip当前 KV cache 最大性能瓶颈之一)。
## 当前问题