docs: fill the Phase 19 gap, refresh README/roadmap to actual state
- docs/19-gpt-oss-moe.md: the numbered series jumped 18->20; write up gpt-oss arch deltas, harmony pitfalls, and the two CUDA debugging postmortems (fully-masked-tile NaN in flash-attention sinks; pre-__syncthreads early return reading uninitialized smem in the decode GEMV) — the highest-value learning content of that phase. - README: models/perf/capabilities were frozen at the Qwen3-only era; now lists gpt-oss MoE, TP/PP, FP8/MXFP4, sparse MoE, and the llama.cpp standing. - Roadmap: record where reality diverged from the plan at Phase 18+, add milestone entries and the ranked next-phase candidates (21 CUDA-graph MoE decode, 22 non-expert quant, 23 sparse prefill). - sparse-moe benchmark doc: post-review-fix numbers. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -40,6 +40,12 @@ MXFP4 runs W4A16. Dense path retained for prefill / `num_tokens > 8` and via
|
||||
13.1 vs 6.6 ms before sparse). Remaining loss: long-prompt TTFT — prefill is
|
||||
still the dense all-expert GEMM; sparse/grouped prefill is the next phase.
|
||||
|
||||
**Post-review fixes** (same harness, rerun): removing three leftover
|
||||
`cudaDeviceSynchronize` from the decode hot path and replacing the CPU-tiled
|
||||
prefill bias-add (96 D2H/H2D round-trips per prefill) with a GPU broadcast
|
||||
kernel improved both axes — TPOT 7.19-7.32 → **6.99-7.21 ms**, TTFT
|
||||
short/medium/long 35/49/94 → **29/42/79 ms**. GSM8K-50: 94% (unchanged).
|
||||
|
||||
## TP=1 head-to-head (single 5090; server now routes gpt-oss tp=1 to the TP engine)
|
||||
|
||||
| prompt | metric | xserv sparse FP8 | llama MXFP4 |
|
||||
|
||||
Reference in New Issue
Block a user