YaRN RoPE was the missing piece — gpt-oss uses rope_type "yarn" (factor 32,
beta_fast 32, beta_slow 1, orig_max 4096); a plain theta RoPE garbled
attention. Added yarn_rope_cache (host-computed inv_freq + mscale, built
into a RopeCache directly). Experts kept CPU-resident and uploaded per-use
(the dequantized BF16 model is ~36GB, won't fit one 32GB card).
Verified: "The capital of France is" -> top-1 token 12366 = " Paris"
(logit 19.75), matching the llama.cpp oracle's behavior. This exercises the
full MoE path: top-4 router (softmax-after-topk), interleaved clamped
(up+1)*glu experts, attention sinks, sliding window, MXFP4->BF16 weights,
YaRN RoPE, head_dim 64, q/k/v/o biases.
Correctness-first (host attention + per-token MoE); GPU attention-with-sinks
kernel, KV cache, faster MoE, and PP-for-memory come next to run AIME/GSM8K
at speed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Gahow Wang
0dd8851e88