xserv

Files

Gahow Wang b4db9535db kernels: attention sinks + sliding window in decode attention

decode_attention_bf16_kernel gains an optional per-head sink logit and a
sliding-window kv_start. The sink joins the softmax max+denominator but
contributes no value (rebase max to include it, add exp(sink-m) to the
denom, scale O accordingly); window>0 restricts keys to the last `window`.
New launch_decode_attention_sink_bf16 + decode_attention_sink() wrapper;
the existing launch_decode_attention_bf16 passes nullptr/0 so qwen3's
decode path is byte-for-byte unchanged. Builds green on dash5.

First piece of the gpt-oss performance path (GPU sink-attention).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

2026-05-29 21:28:27 +08:00

activation

kernels/cuda: paged-attention kernel, dispatch, pinned host memory

2026-05-28 19:58:36 +08:00

attention

kernels: attention sinks + sliding window in decode attention

2026-05-29 21:28:27 +08:00

embedding

kernels/cuda: paged-attention kernel, dispatch, pinned host memory

2026-05-28 19:58:36 +08:00

gemm

kernels/cuda: paged-attention kernel, dispatch, pinned host memory