phase19: MoE support — gpt-oss-20b end-to-end inference with TP=2

Add Mixture-of-Experts support for the gpt-oss-20b model (20.9B params,
32 experts × top-4 routing). Key additions:

- ModelConfig: MoE fields (num_local_experts, layer_types, sliding_window,
  attention_bias, explicit head_dim, rope_scaling, swiglu_limit)
- YaRN RoPE: RopeCache::new_yarn() with correct frequency interpolation
  and attention_scaling = 0.1*ln(factor)+1
- Custom GLU kernel: gpt_oss_glu_bf16 (clamped sigmoid gate activation)
- Paged attention with sinks + sliding window kernel variant
- GptOss model struct with expert-parallel TP (split 32 experts across ranks)
- bench-gpt-oss binary for TP inference benchmarking

Verified on dash5 with 2x RTX 5090: 63.6 tok/s decode, ~160ms TTFT.
Model generates topically-coherent output (needs chat template for quality).

Known issues:
- Custom GEMV kernel produces NaN with small N (workaround: pad to M=2)
- Prefill doesn't use attention sinks (uses standard flash attention)
- Output quality requires chat template formatting

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

csrc/activation/activations.cu

@@ -58,6 +58,25 @@ __global__ void silu_mul_bf16_kernel(const __nv_bfloat16* gate, const __nv_bfloa
     }
 }
 
+// gpt-oss GLU: gate_up is [N, 2*D] with interleaved columns (gate=even, up=odd).
+// gate = gate_up[::2].clamp(max=limit)
+// up   = gate_up[1::2].clamp(-limit, limit)
+// glu  = gate * sigmoid(gate * alpha)
+// out  = (up + 1) * glu
+// Output: [N, D]
+__global__ void gpt_oss_glu_bf16_kernel(const __nv_bfloat16* gate_up, __nv_bfloat16* out,
+                                         int n_elements, float alpha, float limit) {
+    int idx = blockIdx.x * blockDim.x + threadIdx.x;
+    if (idx < n_elements) {
+        float g = __bfloat162float(gate_up[idx * 2]);
+        float u = __bfloat162float(gate_up[idx * 2 + 1]);
+        g = fminf(g, limit);
+        u = fmaxf(fminf(u, limit), -limit);
+        float glu = g / (1.0f + expf(-g * alpha));
+        out[idx] = __float2bfloat16((u + 1.0f) * glu);
+    }
+}
+
 // Element-wise add: out = a + b
 __global__ void add_f32_kernel(const float* a, const float* b, float* out, int n) {
     int idx = blockIdx.x * blockDim.x + threadIdx.x;
@@ -163,4 +182,13 @@ void launch_silu_mul_bf16(const void* gate, const void* up, void* out, int n, vo
     CUDA_CHECK_LAST_ERROR();
 }
 
+void launch_gpt_oss_glu_bf16(const void* gate_up, void* out, int n_elements,
+                              float alpha, float limit, void* stream) {
+    int block = 256;
+    int grid = (n_elements + block - 1) / block;
+    gpt_oss_glu_bf16_kernel<<<grid, block, 0, (cudaStream_t)stream>>>(
+        (const __nv_bfloat16*)gate_up, (__nv_bfloat16*)out, n_elements, alpha, limit);
+    CUDA_CHECK_LAST_ERROR();
+}
+
 }