model: paged KV cache with CPU swap pool, decode graph, qwen3 updates

- paged_kv_cache: new block-paged KV cache; adds a pinned-host swap pool with
  a second BlockAllocator, per-sequence Location {Gpu,Cpu}, and lossless
  swap_out/swap_in (block-granular D2H/H2D) for vLLM-style preemption.
  bytes_per_block helper exposes per-block cost for VRAM-based sizing.
- decode_graph: CUDA-graph decode path.
- qwen3/gpt2/kv_cache: paged prefill/decode forward + related updates.
- tokenizer/bins: BPE updates, new xserv-chat CLI, bench-qwen3 tweaks.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
2026-05-28 19:58:54 +08:00
parent 4c3f914459
commit d52baa0006
9 changed files with 1896 additions and 44 deletions

View File

@@ -1,13 +1,17 @@
pub mod config;
pub mod decode_graph;
pub mod gpt2;
pub mod kv_cache;
pub mod loader;
pub mod paged_kv_cache;
pub mod qwen3;
pub mod sampling;
pub use config::ModelConfig;
pub use decode_graph::{DecodeGraphState, LayerWeightPtrs};
pub use gpt2::{GPT2, KVCache};
pub use kv_cache::GpuKVCache;
pub use paged_kv_cache::{BlockAllocator, Location, PagedKVCache, BLOCK_SIZE};
pub use qwen3::Qwen3;
pub use sampling::{SamplingParams, sample};