Gahow Wang 7e7d077ff1 moe: KV-cached GPU decode for gpt-oss (O(1)/token) ...

decode_step(): single-token forward using GpuKVCache + the GPU
decode_attention_sink kernel (per-head sinks + sliding window) + per-token
MoE, so generation is O(1)/token instead of the host forward's O(n²)
recompute. generate(): prefill prompt token-by-token (causal attention =>
sequential == batched), then greedy decode to eos/max_new.

Verified against the reference host forward on the Paris prompt: both
predict top-1 token 12650 = " Paris" (MATCH_TOP1: YES; logits 15.375 vs
15.3125 — BF16 accumulation-order diff only). gptoss-logits now runs both
paths and asserts the match. Build green on dash5.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

2026-05-29 21:54:05 +08:00

..

bin

moe: KV-cached GPU decode for gpt-oss (O(1)/token)

2026-05-29 21:54:05 +08:00

config.rs

moe(wip): gpt-oss-20b groundwork — config fields, arch doc, MXFP4 tools

2026-05-29 21:01:53 +08:00

decode_graph.rs

model: paged KV cache with CPU swap pool, decode graph, qwen3 updates

2026-05-28 19:58:54 +08:00

gpt2.rs

model: paged KV cache with CPU swap pool, decode graph, qwen3 updates

2026-05-28 19:58:54 +08:00

gptoss.rs

moe: KV-cached GPU decode for gpt-oss (O(1)/token)

2026-05-29 21:54:05 +08:00

kv_cache.rs

model: paged KV cache with CPU swap pool, decode graph, qwen3 updates

2026-05-28 19:58:54 +08:00

lib.rs

moe: fix missing pub mod gptoss + record REAL verification

2026-05-29 21:17:31 +08:00

loader.rs

moe: MXFP4-resident experts on GPU (single-card gpt-oss)

2026-05-29 21:50:38 +08:00

paged_kv_cache.rs

model: tensor-parallel Qwen3 (sharded weights + AllReduce)

2026-05-29 11:10:24 +08:00

qwen3.rs

model: pipeline-parallel Qwen3 (from_weights_pp + stage forward)

2026-05-29 18:45:47 +08:00

sampling.rs

model: pipeline-parallel Qwen3 (from_weights_pp + stage forward)

2026-05-29 18:45:47 +08:00