server: tensor-parallel HTTP engine (--tp N)

tp_engine: rank-0 coordinator owns the scheduler and broadcasts per-token
commands (Register/Prefill/Decode/Free) to worker rank threads; the sampled
token always comes from rank 0, so it's correct for greedy and stochastic
sampling. Serial single-request path (sufficient for the quality benchmark).
--tp N selects it; TP=1 keeps the existing single-GPU Engine unchanged.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
2026-05-29 11:10:33 +08:00
parent f17011129e
commit 95eb61d639
3 changed files with 211 additions and 3 deletions

View File

@@ -13,6 +13,7 @@ xserv-tensor = { path = "../xserv-tensor" }
xserv-kernels = { path = "../xserv-kernels" }
xserv-model = { path = "../xserv-model" }
xserv-tokenizer = { path = "../xserv-tokenizer" }
xserv-distributed = { path = "../xserv-distributed" }
half.workspace = true
serde.workspace = true
serde_json.workspace = true