Gahow Wang 64b2a8c09e run: v3 archive + export (dim512, single-GPU batched, val 1.30) ...

v3 trained (30000 steps × batch 32 × seq 256 = 245.8M tok, ~0.53 epoch),
single-GPU batched via T10 (~26K tok/s, ~2.65h). Archived to registry
~/projects/tiny-models/v3-tinystories-dim512/ (xtrain.ckpt + config.json +
model.safetensors BF16 179 tensors + tokenizer.json + RUN.md) and served in
xserv (loads 16L/dim512 qwen3, 2/3 prompts token-match xtrain greedy; 3rd
diverges on BF16 drift as in v1/v2).

best/final val 1.3027 (beats ~1.4 target). val ladder on the same held-out
1M-token set: v0 3.80 / v1 2.58 / v2 1.71 / v3 1.30. T10 (batched forward)
validated at scale (KI-1 root cause = launch-bound, not all-reduce); single-GPU
avoids KI-5. Update docs/runs/README.md comparison table.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

2026-06-16 03:37:36 +08:00

..

runs

run: v3 archive + export (dim512, single-GPU batched, val 1.30)

2026-06-16 03:37:36 +08:00

00-build-chain.md

docs: backfill T1 build-chain

2026-06-15 15:12:55 +08:00

01-tensor.md

docs: Phase T2 — tensor abstraction

2026-06-15 15:12:55 +08:00

02-gemm-autodiff.md

docs: Phase T3 — GEMM fwd/bwd + finite-diff

2026-06-15 15:27:03 +08:00

03-autograd-engine.md

docs: Phase T4 — autograd engine

2026-06-15 15:53:55 +08:00

04-tiny-transformer.md

docs: Phase T5 — tiny transformer

2026-06-15 16:09:30 +08:00

05-training-loop.md

docs: Phase T6 — training loop

2026-06-15 16:30:14 +08:00

06-performance.md

docs: Phase T7 — performance

2026-06-15 17:00:29 +08:00

07-distributed.md

docs: Phase T8 — distributed data parallel

2026-06-15 17:15:49 +08:00

08-export-xserv.md

docs: T9 verification results (xserv == xtrain, dash5)

2026-06-15 17:37:46 +08:00

09-batched-forward.md

docs: Phase T10 — batched forward

2026-06-16 00:44:50 +08:00

known-issues.md

perf: KI-1 fixed — GPU util 0-15%→37-54%, tok/s 1653→25627 (15.5x)

2026-06-16 00:44:43 +08:00