docs: Phase 14 design doc + benchmark, fix Phase 11/12 honesty

Phase 14 (Flash Attention):
- Design doc: FA2 algorithm, SM120 hardware constraints (FA4 incompatible),
  kernel config (BR=BC=64, 32KB smem), GQA mapping, causal tile-skip,
  known limitations and optimization roadmap
- Benchmark doc: correctness (9/10 top-1 match, identical to pre-FA baseline),
  performance tracking (6.9→10.3→12.9 tok/s across phases), memory savings
  analysis, remaining bottleneck breakdown

Phase 11 doc: title corrected from "Paged Attention" to "GPU-Resident KV Cache"
with explicit note that paged allocation was not implemented.

Phase 12 doc: "当前状态" updated from "未实现" to reflect actual state —
iteration-level scheduling implemented + verified (6.0x concurrent speedup),
batched GPU forward explicitly marked as not yet implemented.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-22 18:51:29 +08:00
parent d67dda404e
commit 6cc1c9332d
4 changed files with 285 additions and 3 deletions

View File

@@ -150,4 +150,8 @@ HTTP Handler Engine Thread
## 当前状态
**实现**。当前是 FIFO 串行,一次只处理一个请求。本文档是实现的设计规格
**实现: iteration-level scheduling**。多请求可以并发进入 batch (max_batch_size),新请求在 mid-generation 动态加入。Prefill 和 decode 阶段在每轮迭代内分离处理
**未实现: batched GPU forward**。每个 seq 的 model forward 仍是串行调用 (per-seq forward_gpu_cache)。真正的 batched decode (多 seq 的 token 合并为一次 GPU forward) 需要 Flash Attention 的 variable-length attention 支持。Phase 14 实现了 FA2 kernel为后续 batched forward 提供了基础。
**验证**: 8 个并发请求 (max_batch=4) 总 wall clock 22.5s,各请求延迟之和 135.0s,调度加速 6.0x。Server log 确认 `decode batch_size=4`