docs: fill the Phase 19 gap, refresh README/roadmap to actual state

- docs/19-gpt-oss-moe.md: the numbered series jumped 18->20; write up
  gpt-oss arch deltas, harmony pitfalls, and the two CUDA debugging
  postmortems (fully-masked-tile NaN in flash-attention sinks;
  pre-__syncthreads early return reading uninitialized smem in the
  decode GEMV) — the highest-value learning content of that phase.
- README: models/perf/capabilities were frozen at the Qwen3-only era;
  now lists gpt-oss MoE, TP/PP, FP8/MXFP4, sparse MoE, and the
  llama.cpp standing.
- Roadmap: record where reality diverged from the plan at Phase 18+,
  add milestone entries and the ranked next-phase candidates
  (21 CUDA-graph MoE decode, 22 non-expert quant, 23 sparse prefill).
- sparse-moe benchmark doc: post-review-fix numbers.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-06-12 17:02:59 +08:00
parent 5343391dbd
commit 2a92f268a9
4 changed files with 168 additions and 13 deletions

View File

@@ -40,6 +40,12 @@ MXFP4 runs W4A16. Dense path retained for prefill / `num_tokens > 8` and via
13.1 vs 6.6 ms before sparse). Remaining loss: long-prompt TTFT — prefill is
still the dense all-expert GEMM; sparse/grouped prefill is the next phase.
**Post-review fixes** (same harness, rerun): removing three leftover
`cudaDeviceSynchronize` from the decode hot path and replacing the CPU-tiled
prefill bias-add (96 D2H/H2D round-trips per prefill) with a GPU broadcast
kernel improved both axes — TPOT 7.19-7.32 → **6.99-7.21 ms**, TTFT
short/medium/long 35/49/94 → **29/42/79 ms**. GSM8K-50: 94% (unchanged).
## TP=1 head-to-head (single 5090; server now routes gpt-oss tp=1 to the TP engine)
| prompt | metric | xserv sparse FP8 | llama MXFP4 |