gahow/xserv
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Commit Graph

  • c7d0750c32 moe(wip): gpt-oss-20b groundwork — config fields, arch doc, MXFP4 tools Gahow Wang 2026-05-29 21:01:53 +08:00
  • 057a3c68a3 docs: Phase 19 MoE (gpt-oss-20b) design + progress snapshot Gahow Wang 2026-05-29 19:13:23 +08:00
  • 11e0154e4d docs: Phase 18 pipeline parallelism — design + benchmark results Gahow Wang 2026-05-29 18:46:06 +08:00
  • d5dcf1a5ab bench: PP harness (xserv --pp vs llama.cpp -sm layer) Gahow Wang 2026-05-29 18:45:59 +08:00
  • 824cc58daa server: pipeline-parallel HTTP engine (--pp N) Gahow Wang 2026-05-29 18:45:52 +08:00
  • da3aaa134a model: pipeline-parallel Qwen3 (from_weights_pp + stage forward) Gahow Wang 2026-05-29 18:45:47 +08:00
  • 859c0cc0b6 distributed: NCCL P2P primitives (PpContext + send/recv) Gahow Wang 2026-05-29 18:45:42 +08:00
  • c2362df1f1 fix(xserv-chat): UTF-8/CJK-aware line input Gahow Wang 2026-05-29 11:36:54 +08:00
  • 7b8b520cda docs: TP=1/2/4 xserv vs llama.cpp benchmark results Gahow Wang 2026-05-29 11:10:52 +08:00
  • a4a171d425 bench: TP sweep harness (xserv --tp, llama row-split, concurrent groups) Gahow Wang 2026-05-29 11:10:43 +08:00
  • 95eb61d639 server: tensor-parallel HTTP engine (--tp N) Gahow Wang 2026-05-29 11:10:33 +08:00
  • f17011129e model: tensor-parallel Qwen3 (sharded weights + AllReduce) Gahow Wang 2026-05-29 11:10:24 +08:00
  • 453520d622 distributed: NCCL tensor-parallel primitives (TpContext + AllReduce) Gahow Wang 2026-05-29 11:10:14 +08:00
  • 76fffb3b68 docs: Phase 17 tensor parallelism design Gahow Wang 2026-05-29 11:10:03 +08:00
  • 14a44b503e docs: add Chinese README (overview + usage) Gahow Wang 2026-05-28 21:38:20 +08:00
  • 80157e614a docs: update llama.cpp comparison with 8192 results (OOM fixed) Gahow Wang 2026-05-28 21:32:14 +08:00
  • fc1900a745 server: VRAM-sized KV pool + vLLM-style swap scheduler Gahow Wang 2026-05-28 19:59:06 +08:00
  • d52baa0006 model: paged KV cache with CPU swap pool, decode graph, qwen3 updates Gahow Wang 2026-05-28 19:58:54 +08:00
  • 4c3f914459 kernels/cuda: paged-attention kernel, dispatch, pinned host memory Gahow Wang 2026-05-28 19:58:36 +08:00
  • 3f1c3d429a docs: llama.cpp vs xserv benchmark results + summary Gahow Wang 2026-05-28 15:06:21 +08:00
  • 950ccf3822 bench: fix llama.cpp per-slot context (was 1/parallel of intended) Gahow Wang 2026-05-28 15:06:12 +08:00
  • 7cb9ee3870 bench: run one server at a time, match thinking mode, fix tools package Gahow Wang 2026-05-28 11:40:07 +08:00
  • 49c7653222 tools: add llama.cpp comparison baseline + standard benchmark suite Gahow Wang 2026-05-28 11:18:52 +08:00
  • 9bb5c5c328 tools: add correctness + performance test scripts for Qwen3-8B Gahow Wang 2026-05-23 14:13:49 +08:00
  • 986a289616 fix: 12 bug fixes from comprehensive review — 51 tok/s verified on RTX 5090 Gahow Wang 2026-05-23 14:13:43 +08:00
  • a67e724119 docs: Phase 15 design doc + benchmark report Gahow Wang 2026-05-23 00:39:27 +08:00
  • d5532ef209 phase 15: Tensor::empty + CUDA Graph infra — 50.3 tok/s (140% of HF, 45% roofline) Gahow Wang 2026-05-22 23:57:34 +08:00
  • e207523e21 phase 15: custom GEMV kernel — 46.6 tok/s serial (3.5x improvement, 130% of HF) Gahow Wang 2026-05-22 22:22:31 +08:00
  • 876d3f5d6a phase 15: batched decode forward — 35 tok/s (97% of HF transformers) Gahow Wang 2026-05-22 20:07:43 +08:00
  • 9783fcf410 phase 15: decode attention kernel + fused silu_mul + fused add_rmsnorm Gahow Wang 2026-05-22 19:40:56 +08:00
  • 6cc1c9332d docs: Phase 14 design doc + benchmark, fix Phase 11/12 honesty Gahow Wang 2026-05-22 18:51:29 +08:00
  • d67dda404e phase 14: Flash Attention 2 for SM120 (RTX 5090) Gahow Wang 2026-05-22 18:27:39 +08:00
  • ee68d3565d fix: comprehensive review + 14 bug fixes + Phase 12/14 overhaul Gahow Wang 2026-05-22 17:53:28 +08:00
  • d8493bd70f phase 12: implement real continuous batching scheduler phase13 Gahow Wang 2026-05-22 13:44:26 +08:00
  • 7d05ececa0 docs: split Phase 12 and Phase 13 into separate design documents Gahow Wang 2026-05-22 13:15:27 +08:00
  • da043554ba phase 12+13: HTTP API server with OpenAI-compatible endpoint (Milestone ③) Gahow Wang 2026-05-22 12:55:19 +08:00
  • 2be27d6d94 perf: GPU transpose/reshape/repeat_kv kernels (eliminate CPU round-trips) Gahow Wang 2026-05-22 12:01:07 +08:00
  • 2d48f25e66 phase 11: GPU-resident KV cache phase11 Gahow Wang 2026-05-22 11:50:12 +08:00
  • be5c64ea8a phase 10: GPU add/mul kernels + BF16 precision analysis phase10 Gahow Wang 2026-05-22 11:35:26 +08:00
  • 268e40d764 phase 10: add Qwen3-8B benchmark + performance fix Gahow Wang 2026-05-22 10:25:33 +08:00
  • 246ae1c590 phase 10: Qwen3-8B support (Milestone ②) Gahow Wang 2026-05-22 00:46:37 +08:00
  • 64084d3489 phase 9: KV cache + autoregressive generation phase9 Gahow Wang 2026-05-21 23:39:41 +08:00
  • cb12250ef0 phase 8: add benchmark framework + baseline results Gahow Wang 2026-05-21 23:29:41 +08:00
  • e1e75fc7f6 phase 6+7+8: model loading, BPE tokenizer, GPT-2 inference (Milestone ①) phase8 Gahow Wang 2026-05-21 22:04:00 +08:00
  • 6035ffdc0b phase 5: naive multi-head attention phase5 Gahow Wang 2026-05-21 21:17:23 +08:00
  • c8e8153702 phase 4: transformer core kernels phase4 Gahow Wang 2026-05-21 21:07:24 +08:00
  • 51a0f2eb14 docs: add design docs + takeaways for Phase 2 and Phase 3 Gahow Wang 2026-05-21 20:59:45 +08:00
  • d77f921a12 phase 3: GEMM kernels (naive, tiled, cuBLAS) phase3 Gahow Wang 2026-05-21 19:48:05 +08:00
  • a83971fa25 phase 2: tensor abstraction layer phase2 Gahow Wang 2026-05-21 19:45:22 +08:00
  • c8f7bc0c3c phase 0+1: fix Rust 2024 edition compat + memory query phase0-1 Gahow Wang 2026-05-21 19:40:49 +08:00
  • 9806b4db35 phase 0+1: project scaffold + xserv-cuda crate Gahow Wang 2026-05-21 18:40:22 +08:00