Commit Graph

1 Commits

Author SHA1 Message Date
057a3c68a3 docs: Phase 19 MoE (gpt-oss-20b) design + progress snapshot
Architecture + exact HF reference math (router softmax-after-topk,
interleaved clamped (up+1)*glu experts, attention sinks, alternating
sliding window, head_dim 64, rope 150000), MXFP4 dequant plan, and the
correctness-first -> PP -> llama.cpp roadmap. MOE_PROGRESS.md captures
live state for resuming after a machine reboot (HF is firewalled here;
download via proxy + hf-mirror; gpt-oss-20b not yet downloaded).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-29 19:13:23 +08:00