docs: fill the Phase 19 gap, refresh README/roadmap to actual state

- docs/19-gpt-oss-moe.md: the numbered series jumped 18->20; write up
  gpt-oss arch deltas, harmony pitfalls, and the two CUDA debugging
  postmortems (fully-masked-tile NaN in flash-attention sinks;
  pre-__syncthreads early return reading uninitialized smem in the
  decode GEMV) — the highest-value learning content of that phase.
- README: models/perf/capabilities were frozen at the Qwen3-only era;
  now lists gpt-oss MoE, TP/PP, FP8/MXFP4, sparse MoE, and the
  llama.cpp standing.
- Roadmap: record where reality diverged from the plan at Phase 18+,
  add milestone entries and the ranked next-phase candidates
  (21 CUDA-graph MoE decode, 22 non-expert quant, 23 sparse prefill).
- sparse-moe benchmark doc: post-review-fix numbers.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-06-12 17:02:59 +08:00
parent 5343391dbd
commit 2a92f268a9
4 changed files with 168 additions and 13 deletions

View File

@@ -1748,6 +1748,27 @@ Text → Tokenizer → Text Tokens ────────────→
---
## 实际进展记录(与原计划的分叉,2026-06 更新)
Phase 017 按计划完成。Phase 18 起实际路线偏离了上面的原计划
(speculative decoding 与多模态推迟),实际走向是 MoE + 量化 + 稀疏化:
| 实际 Phase | 内容 | 文档 |
|---|---|---|
| 18 | Pipeline Parallelism(PP=2/4) | `18-pipeline-parallelism.md`、`benchmarks/pp-sweep.md` |
| 19 | **gpt-oss-20b MoE**:harmony 格式、attention sinks + sliding window、YaRN;两个 CUDA bug 实战(prefill sinks NaN、GEMV 未初始化 smem);GSM8K 94.5% 对齐 llama.cpp;FP8 W8A8 / MXFP4 W4A16 量化 | `19-gpt-oss-moe.md`、`benchmarks/{fp8-quantization,mxfp4-and-llama-decode}.md` |
| 20 | **稀疏 top-k MoE decode**:只算被路由的专家,decode 13.9→7.0ms,TP=2 下 decode/TTFT 全面快于 llama.cpp 同配置;gpt-oss 单卡 serving | `20-sparse-moe.md`、`benchmarks/sparse-moe.md` |
**下一步候选(按预期收益排序):**
| 候选 Phase | 内容 | 预期 |
|---|---|---|
| 21 | **gpt-oss decode CUDA Graph**:把 Phase 15 的 split-graph 方案(`decode_graph.rs`,目前只用于 Qwen3 单卡)推广到 MoE/TP 路径,消除 ~200 launch/token | TPOT 7.0 → ~4-5ms,逼近 llama 单卡 2.9ms |
| 22 | **非专家权重量化**:qkv/o + lm_head(1.16GB/token)仍是 BF16 | TPOT 再省 ~1-1.5ms |
| 23 | **稀疏 prefill**(按专家 permute + grouped GEMM) | 长 prompt TTFT 79 → ~40ms |
| 24 | server 侧 harmony channel 分离(`reasoning_content` 流式输出,对齐 llama-server 行为) | API 易用性 |
| — | Speculative Decoding、多模态(原 16/19) | 推迟 |
## 里程碑总结
| 里程碑 | Phase | 验收标准 |
@@ -1757,7 +1778,9 @@ Text → Tokenizer → Text Tokens ────────────→
| ③ E2E API | 13 | HTTP streaming API, Python OpenAI SDK 可调用, 10 并发正确 |
| ④ 性能达标 | 15 | throughput >= 50% vLLM, profiling 报告完成 |
| ⑤ 多卡推理 | 17 | TP=2/4 同组 GPU 推理正确, scaling benchmark 完成 |
| ⑥ 多模态 | 19 | 图片输入 → 文字回答, API 端到端 |
| ⑥ MoE 模型(实际) | 19 | gpt-oss-20b 端到端正确, GSM8K 与 llama.cpp 持平 ✅ |
| ⑦ 性能反超(实际) | 20 | 同配置 decode 快于 llama.cpp(TP=2 达成;单卡是 Phase 21+ 目标) ✅ |
| ⑧ 多模态 | 推迟 | 图片输入 → 文字回答, API 端到端 |
## 外部依赖清单