model: tensor-parallel Qwen3 (sharded weights + AllReduce)

from_weights_tp shards each rank's weights (column-split q/k/v/gate/up,
row-split o/down; replicate norms/embed/lm_head) and the paged forward uses
local head counts + AllReduces after o_proj and down_proj. PagedKVCache::new_tp
sizes the pool for the rank's local KV heads (KV is sharded too). TP=1 is the
identity path. New bench-tp binary runs E2E multi-GPU generation per TP degree.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

crates/xserv-model/src/paged_kv_cache.rs

@@ -134,10 +134,29 @@ impl PagedKVCache {
         max_blocks_per_seq: usize,
         dtype: DType,
         device: u32,
+    ) -> Self {
+        Self::new_tp(
+            config, config.num_kv_heads(), total_blocks, cpu_total_blocks,
+            max_seqs, max_blocks_per_seq, dtype, device,
+        )
+    }
+
+    /// Like `new`, but with an explicit `num_kv_heads` — under tensor parallelism
+    /// each rank only stores its `num_kv_heads / world` heads, so the pool is
+    /// sized for the local head count, not the model's full count.
+    #[allow(clippy::too_many_arguments)]
+    pub fn new_tp(
+        config: &ModelConfig,
+        num_kv_heads: usize,
+        total_blocks: usize,
+        cpu_total_blocks: usize,
+        max_seqs: usize,
+        max_blocks_per_seq: usize,
+        dtype: DType,
+        device: u32,
     ) -> Self {
         assert!(total_blocks >= 2, "need at least 2 blocks (one is sentinel)");
         let num_layers = config.num_layers();
-        let num_kv_heads = config.num_kv_heads();
         let head_dim = config.head_dim();
         let elem_size = dtype.size_bytes();
         let block_bytes = num_kv_heads * BLOCK_SIZE * head_dim * elem_size;