gahow/xtrain
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Commit Graph

  • 25b032445d train: real batched step (drop loop+SUM) Gahow Wang 2026-06-16 00:44:33 +08:00
  • 5353b38402 model: batched forward [B,S] Gahow Wang 2026-06-16 00:44:25 +08:00
  • 7821bd9c34 autograd: batch dim for ops (flatten linears, batched attention) Gahow Wang 2026-06-16 00:44:15 +08:00
  • d2a585c5cb docs: KI-1 re-diagnosed in v3 — larger batch does NOT fix DDP weak scaling Gahow Wang 2026-06-15 23:20:26 +08:00
  • bf679f6f1f docs: run v2 — TinyStories, dim384/12L, DDP 4-card (val 1.71) Gahow Wang 2026-06-15 22:38:31 +08:00
  • c87a0bc44e docs: known-issues / perf backlog — KI-1 DDP weak scaling at small global batch Gahow Wang 2026-06-15 19:56:58 +08:00
  • 7090b475fb train: bring DDP trainer to parity with bin/train (val + checkpoint + cache + arch) Gahow Wang 2026-06-15 19:34:40 +08:00
  • 264660527f docs: run v1 — TinyStories full, dim256 Gahow Wang 2026-06-15 19:09:46 +08:00
  • ec8114ecbc train: --eval-ckpt eval-only mode (v0-vs-v1 same-set val loss) Gahow Wang 2026-06-15 18:44:40 +08:00
  • e44e50ef78 data: full TinyStories + tokenized-id cache, val loss, CLI arch Gahow Wang 2026-06-15 18:34:48 +08:00
  • 15f1e526c7 train: parameterize model size (scaling ladder) Gahow Wang 2026-06-15 18:34:39 +08:00
  • 8981cf7982 docs: T9 verification results (xserv == xtrain, dash5) Gahow Wang 2026-06-15 17:37:46 +08:00
  • e246c3bec2 export: dump_logits bin for xserv-vs-xtrain comparison Gahow Wang 2026-06-15 17:36:41 +08:00
  • 18c2229b4b docs: Phase T9 — export to xserv Gahow Wang 2026-06-15 17:33:32 +08:00
  • 1c76573cb4 export: safetensors + config.json for xserv qwen3 Gahow Wang 2026-06-15 17:33:26 +08:00
  • 7a4f69e430 model: add per-head QK-norm (Qwen3-compat) for xserv export Gahow Wang 2026-06-15 17:33:19 +08:00
  • ad82e8bf92 dist: lengthen scaling bench so NCCL init amortizes Gahow Wang 2026-06-15 17:18:23 +08:00
  • 818f76a18f dist: drop unused import; relax DDP-vs-single-GPU param tolerance Gahow Wang 2026-06-15 17:17:31 +08:00
  • 0131f05b26 docs: Phase T8 — distributed data parallel Gahow Wang 2026-06-15 17:15:49 +08:00
  • cf5e3987df dist: multi-rank launcher + ddp acceptance test Gahow Wang 2026-06-15 17:15:41 +08:00
  • 163f567c80 dist: ddp all-reduce + sharded batch Gahow Wang 2026-06-15 17:15:29 +08:00
  • e27df50ca9 dist: nccl ffi + comm bootstrap Gahow Wang 2026-06-15 17:14:56 +08:00
  • 5e8add2a41 docs: Phase T7 — performance Gahow Wang 2026-06-15 17:00:29 +08:00
  • a842e432b5 perf: streams / drop per-op sync Gahow Wang 2026-06-15 16:56:17 +08:00
  • 8070c1949a perf: make xtrain-cuda a regular dep of xtrain-optim (GPU AdamW) Gahow Wang 2026-06-15 16:53:52 +08:00
  • b0e397ca81 perf: GPU AdamW + grad-norm Gahow Wang 2026-06-15 16:53:09 +08:00
  • 0e5c7d22e2 perf: cuBLAS matmul fwd/bwd Gahow Wang 2026-06-15 16:48:35 +08:00
  • 5df1d4d57b test: resolve real_training corpus default via CARGO_MANIFEST_DIR Gahow Wang 2026-06-15 16:41:12 +08:00
  • 2f8118fda9 test: tighten AdamW parity (f32 reference, 10 steps, allclose tol) Gahow Wang 2026-06-15 16:34:18 +08:00
  • 29b4d30b6c docs: Phase T6 — training loop Gahow Wang 2026-06-15 16:30:14 +08:00
  • 22b7434b23 test: AdamW PyTorch parity + checkpoint round-trip + real training Gahow Wang 2026-06-15 16:30:06 +08:00
  • 77a82bfeee train: loop + checkpoint save/load + sampler + train binary Gahow Wang 2026-06-15 16:29:58 +08:00
  • 7d84a64f5c data: gpt2 bpe via xserv-tokenizer + TinyStories corpus + lr schedule + grad clip Gahow Wang 2026-06-15 16:29:32 +08:00
  • f22429f5b8 optim: hand-written AdamW (decoupled weight decay + bias correction) Gahow Wang 2026-06-15 16:28:23 +08:00
  • 8565565647 docs: Phase T5 — tiny transformer Gahow Wang 2026-06-15 16:09:30 +08:00
  • 603c85e1e0 model: silence torch parity warning (read loss before backward) Gahow Wang 2026-06-15 16:09:30 +08:00
  • 3366f30c4d model: PyTorch parity harness (weight dump + equivalent torch model) Gahow Wang 2026-06-15 16:07:30 +08:00
  • e3912c2380 model: tiny RoPE+RMSNorm+SwiGLU transformer + overfit test Gahow Wang 2026-06-15 16:05:20 +08:00
  • 0acfa5df11 ops: grad-check the T5 structural ops Gahow Wang 2026-06-15 16:05:20 +08:00
  • 7fb1a29057 ops: embedding/reshape/transpose/split-merge-heads fwd+bwd Gahow Wang 2026-06-15 16:05:09 +08:00
  • 777f3c7949 docs: Phase T4 — autograd engine Gahow Wang 2026-06-15 15:47:43 +08:00
  • e7ce504b1f ops: differentiable autograd nodes + per-op grad-check tests Gahow Wang 2026-06-15 15:47:43 +08:00
  • 224f750ee4 autograd: tape engine + grad accumulation Gahow Wang 2026-06-15 15:44:17 +08:00
  • 5aef3742d6 ops: transformer op fwd/bwd CUDA kernels + Tensor wrappers Gahow Wang 2026-06-15 15:44:09 +08:00
  • 88fbe0a85d gemm: realistic f32 tolerances in GEMM acceptance tests Gahow Wang 2026-06-15 15:28:57 +08:00
  • dde2fde297 docs: Phase T3 — GEMM fwd/bwd + finite-diff Gahow Wang 2026-06-15 15:27:03 +08:00
  • 1384044f27 gemm: GPU acceptance tests vs cuBLAS + finite-diff Gahow Wang 2026-06-15 15:26:58 +08:00
  • 08c88bf360 gemm: tiled F32 forward + transpose + backward (dA/dB) Gahow Wang 2026-06-15 15:26:51 +08:00
  • 9ca98efd98 autodiff: finite-diff gradient-check harness Gahow Wang 2026-06-15 15:26:42 +08:00
  • fbd07a578c tensor: minimal Tensor crate over xtrain-cuda Gahow Wang 2026-06-15 15:13:06 +08:00
  • 63dc05fd10 tensor: add scale elementwise CUDA kernel + FFI Gahow Wang 2026-06-15 15:13:06 +08:00
  • 8557a289a2 docs: Phase T2 — tensor abstraction Gahow Wang 2026-06-15 15:12:55 +08:00
  • c1b204296b docs: backfill T1 build-chain Gahow Wang 2026-06-15 15:12:55 +08:00
  • 92acf9f413 T1: scaffold repo + Rust/CUDA build chain (vecadd smoke test) Gahow Wang 2026-06-15 14:42:43 +08:00