docs: Phase 14 design doc + benchmark, fix Phase 11/12 honesty
Phase 14 (Flash Attention): - Design doc: FA2 algorithm, SM120 hardware constraints (FA4 incompatible), kernel config (BR=BC=64, 32KB smem), GQA mapping, causal tile-skip, known limitations and optimization roadmap - Benchmark doc: correctness (9/10 top-1 match, identical to pre-FA baseline), performance tracking (6.9→10.3→12.9 tok/s across phases), memory savings analysis, remaining bottleneck breakdown Phase 11 doc: title corrected from "Paged Attention" to "GPU-Resident KV Cache" with explicit note that paged allocation was not implemented. Phase 12 doc: "当前状态" updated from "未实现" to reflect actual state — iteration-level scheduling implemented + verified (6.0x concurrent speedup), batched GPU forward explicitly marked as not yet implemented. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -150,4 +150,8 @@ HTTP Handler Engine Thread
|
||||
|
||||
## 当前状态
|
||||
|
||||
**未实现**。当前是 FIFO 串行,一次只处理一个请求。本文档是实现的设计规格。
|
||||
**已实现: iteration-level scheduling**。多请求可以并发进入 batch (max_batch_size),新请求在 mid-generation 动态加入。Prefill 和 decode 阶段在每轮迭代内分离处理。
|
||||
|
||||
**未实现: batched GPU forward**。每个 seq 的 model forward 仍是串行调用 (per-seq forward_gpu_cache)。真正的 batched decode (多 seq 的 token 合并为一次 GPU forward) 需要 Flash Attention 的 variable-length attention 支持。Phase 14 实现了 FA2 kernel,为后续 batched forward 提供了基础。
|
||||
|
||||
**验证**: 8 个并发请求 (max_batch=4) 总 wall clock 22.5s,各请求延迟之和 135.0s,调度加速 6.0x。Server log 确认 `decode batch_size=4`。
|
||||
|
||||
Reference in New Issue
Block a user