xserv

Go to file

Gahow Wang 7d05ececa0 docs: split Phase 12 and Phase 13 into separate design documents

- docs/12-continuous-batching.md: scheduler, sequence management,
  batching strategy (currently single-request, expandable)
- docs/13-http-api.md: HTTP server, OpenAI-compatible API,
  axum architecture, SSE streaming (TODO)

Phase 12 = WHAT to compute (scheduling decisions)
Phase 13 = HOW to expose it (HTTP protocol layer)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

2026-05-22 13:15:27 +08:00

crates

phase 12+13: HTTP API server with OpenAI-compatible endpoint (Milestone ③)

2026-05-22 12:55:19 +08:00

csrc

perf: GPU transpose/reshape/repeat_kv kernels (eliminate CPU round-trips)

2026-05-22 12:01:07 +08:00

docs

docs: split Phase 12 and Phase 13 into separate design documents

2026-05-22 13:15:27 +08:00

tools

phase 10: add Qwen3-8B benchmark + performance fix