docs: Phase 14 design doc + benchmark, fix Phase 11/12 honesty
Phase 14 (Flash Attention): - Design doc: FA2 algorithm, SM120 hardware constraints (FA4 incompatible), kernel config (BR=BC=64, 32KB smem), GQA mapping, causal tile-skip, known limitations and optimization roadmap - Benchmark doc: correctness (9/10 top-1 match, identical to pre-FA baseline), performance tracking (6.9→10.3→12.9 tok/s across phases), memory savings analysis, remaining bottleneck breakdown Phase 11 doc: title corrected from "Paged Attention" to "GPU-Resident KV Cache" with explicit note that paged allocation was not implemented. Phase 12 doc: "当前状态" updated from "未实现" to reflect actual state — iteration-level scheduling implemented + verified (6.0x concurrent speedup), batched GPU forward explicitly marked as not yet implemented. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -1,8 +1,10 @@
|
||||
# Phase 11: Paged Attention + KV Cache Manager — Design Document
|
||||
# Phase 11: GPU-Resident KV Cache — Design Document
|
||||
|
||||
> **注意**: 原计划为 "Paged Attention + KV Cache Manager",实际实现为 GPU 连续预分配 KV cache(非 paged)。Paged allocation 留待后续优化。
|
||||
|
||||
## Goal
|
||||
|
||||
将 KV cache 从 CPU Vec 迁移到 GPU,使用 block-based paging 管理显存。消除每步 decode 的 CPU round-trip(当前 KV cache 最大性能瓶颈之一)。
|
||||
将 KV cache 从 CPU Vec 迁移到 GPU,消除每步 decode 的 CPU round-trip(当前 KV cache 最大性能瓶颈之一)。
|
||||
|
||||
## 当前问题
|
||||
|
||||
|
||||
Reference in New Issue
Block a user