model: pipeline-parallel Qwen3 (from_weights_pp + stage forward)

Layer-wise split: each stage loads only its contiguous layer range
[s*L, (s+1)*L); stage 0 keeps embed_tokens, the last stage keeps
norm/lm_head (others get a 1x1 placeholder). Heads are NOT split
(PP is orthogonal to TP). Adds embed/head and forward_layers_prefill/
forward_layers_decode that take and return the [tokens, hidden] hidden
state; per-stage PagedKVCache is indexed by local layer id.

sampling: derive Clone on SamplingParams (carried in the PP command enum).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-05-29 18:45:47 +08:00
parent 859c0cc0b6
commit da3aaa134a
2 changed files with 264 additions and 0 deletions

View File

@@ -2,6 +2,7 @@ use half::bf16;
use rand::Rng;
use xserv_tensor::{DType, Device, Tensor};
#[derive(Clone)]
pub struct SamplingParams {
pub temperature: f32,
pub top_k: usize,