xserv

Files

Gahow Wang b8868d59ac moe: FIX decode — route all gpt-oss matmuls through dense cuBLAS

Root cause of the broken/non-deterministic KV-cache decode: matmul's m==1
fast path is a custom GEMV that reduces over K with a grid-split atomicAdd,
whose float accumulation order is non-deterministic. At decode (m==1) the
router logits jittered enough to flip the top-4 EXPERT SELECTION run-to-run,
so each step picked different experts -> wrong, non-deterministic output and
garbage generation. (Earlier partial fixes failed because only the expert
GEMMs were switched; router/qkv/o_proj still used the GEMV path.)

Fix: gptoss matmul2 -> gemm::matmul_dense (plain cublasGemmEx, no GEMV) for
every matmul. Forward (m>1) already used dense cuBLAS, unchanged + correct.

Verified THIS run (read full output file):
  gptoss-logits "The capital of France is":
    forward top5 == decode top5, byte for byte
      12650 " Paris" 15.50, 625, 25, 354, 261 ;  MATCH_TOP1: YES
    forward 3/3 identical, decode 3/3 identical (deterministic).
  gptoss-gen: first token 12650 " Paris", ~10.4 tok/s on one 5090.

KV-cache GPU decode (sink-attention + MXFP4 experts + YaRN RoPE) is now
correct and deterministic. Next: harmony chat template + server + AIME/GSM8K.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

2026-05-29 22:34:55 +08:00

xserv-cuda

kernels/cuda: paged-attention kernel, dispatch, pinned host memory

2026-05-28 19:58:36 +08:00

xserv-distributed

distributed: NCCL P2P primitives (PpContext + send/recv)

2026-05-29 18:45:42 +08:00

xserv-kernels

moe(wip): zero dequant output — decode STILL broken (not fixed)

2026-05-29 22:28:45 +08:00

xserv-model

moe: FIX decode — route all gpt-oss matmuls through dense cuBLAS