The KV-cache decode produced wrong, non-deterministic logits. Root cause:
dequant_mxfp4 allocated its output with Tensor::empty (uninitialized) and,
under MoE's heavy allocator churn, some output elements were read as
stale/garbage before use — deterministic in isolation (mxfp4-check) but not
in the full MoE path, where each run saw different dirty buffers. Allocating
the output with Tensor::zeros makes decode correct AND deterministic.
Verified (gptoss-logits, "The capital of France is"):
forward top-1 = 12650 " Paris" 15.5625
decode top-1 = 12650 " Paris" 15.5625 MATCH_TOP1: YES
3/3 identical decode runs.
gptoss-gen first generated token = 12650 " Paris" at ~9.5 tok/s, single 5090.
(Zeroing is a correct fix but also masks a possible coverage gap in the
dequant kernel; the kernel's per-thread index math looks complete, so this
is likely a stale-read hazard. Revisit if perf needs the memset gone.)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Gahow Wang
0b76207d21