decode_step(): single-token forward using GpuKVCache + the GPU
decode_attention_sink kernel (per-head sinks + sliding window) + per-token
MoE, so generation is O(1)/token instead of the host forward's O(n²)
recompute. generate(): prefill prompt token-by-token (causal attention =>
sequential == batched), then greedy decode to eos/max_new.
Verified against the reference host forward on the Paris prompt: both
predict top-1 token 12650 = " Paris" (MATCH_TOP1: YES; logits 15.375 vs
15.3125 — BF16 accumulation-order diff only). gptoss-logits now runs both
paths and asserts the match. Build green on dash5.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Gahow Wang
7e7d077ff1