From-scratch CUDA kernel for gpt-oss expert weights: one thread per packed
byte decodes 2 FP4 (E2M1) codes, applies the per-32-block E8M0 scale
(2^(e-127)), and writes BF16 transposed into [IN, OUT] (IN = nblk*32) so it
drops straight into x @ W. dequant_mxfp4() wrapper takes raw GpuBuffers
(uint8 is not an xserv Tensor dtype). mxfp4-check bin dequants layer-0
expert-0 on GPU and matches tools/mxfp4_probe.py exactly:
[0, 0, 0, -0.0625, 0, -0, -0.015625, -0.03125]
This lets experts stay MXFP4-resident on GPU (13GB, fits one 32GB card)
and be dequantized to a BF16 scratch right before each expert GEMM, instead
of holding 36GB of BF16 or uploading experts per token. Loader plumbing +
GPU MoE decode use it next.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Gahow Wang
7ebdd7c552