moe(wip): zero dequant output — decode STILL broken (not fixed)
Tried Tensor::zeros (instead of empty) for the dequant output on the theory
that decode read uninitialized memory. It did NOT fix it: decode top-1 still
diverges from the forward reference AND is still non-deterministic run-to-run
(top-1 varied across runs), generation still garbage. So the bug is NOT a
dequant-output init issue. Root cause remains OPEN. Forward (host attention)
remains correct ("Paris"); only the KV-cache decode path is wrong.
Do not trust decode output.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -25,7 +25,8 @@ pub fn dequant_mxfp4(
|
||||
blocks: &GpuBuffer, scales: &GpuBuffer, out_dim: usize, nblk: usize, device: u32,
|
||||
) -> Tensor {
|
||||
let in_dim = nblk * 32;
|
||||
let out = Tensor::empty(&[in_dim, out_dim], DType::BF16, Device::Cuda(device));
|
||||
// PROBE: zeros (not empty) to test for an uninitialized-read/coverage bug.
|
||||
let out = Tensor::zeros(&[in_dim, out_dim], DType::BF16, Device::Cuda(device));
|
||||
unsafe {
|
||||
launch_mxfp4_dequant_bf16(
|
||||
blocks.as_ptr() as *const c_void,
|
||||
|
||||
Reference in New Issue
Block a user