Gahow Wang gahow
0 Followers · 0 Following
  • Joined on 2026-04-03
Repositories
19
 Projects Packages Public Activity Starred Repositories
gahow pushed to main at gahow/aituner 2026-06-16 11:31:27 +00:00
a16016a876 Add harness vs naive ablation configs (27b, scale=0.2 substrate)
gahow pushed to main at gahow/xtrain 2026-06-16 11:30:54 +00:00
88bec270af docs: evolution overview — per-milestone changes across algorithm/arch/infra/dataset axes
gahow pushed to main at gahow/aituner 2026-06-16 11:16:43 +00:00
07f5d92e1d Add consolidated two-stop summary doc
f2ff0faebd Document Stop-B end-to-end on dense 27B: the improving climb + no-regression
4a64196a99 Add 27B Stop-B agentic-loop config (harness-driven, GPUs 2-7)
b17b213575 Tear down the engine on SIGTERM instead of orphaning it
93ce339d61 Document 27B TP sweep: per-GPU rises sharply with TP (dense), opposite of MoE
Compare 28 commits »
gahow pushed to main at gahow/xtrain 2026-06-16 11:04:47 +00:00
7e5ea9976b data: FineWeb-edu parquet->txt prep script (Scaling v6)
gahow pushed to feat/two-stop at gahow/aituner 2026-06-16 10:07:01 +00:00
f2ff0faebd Document Stop-B end-to-end on dense 27B: the improving climb + no-regression
gahow pushed to main at gahow/xtrain 2026-06-16 09:56:32 +00:00
579365f4a0 docs: run v5 — TinyStories saturation at dim768 (val 1.11)
8a1e29543b run: v5 archive + export (dim768, bf16, 5.33ep, val 1.11)
Compare 2 commits »
gahow pushed to main at gahow/xtrain 2026-06-16 06:29:26 +00:00
5b7dde1736 test: bf16 test reads f32-cast logits (forward now returns bf16)
gahow pushed to main at gahow/xtrain 2026-06-16 06:28:27 +00:00
320c1ae4fb perf: KI-2 FIXED — dim768 bf16 fits batch 32, tok/s 31.5K→40.8K
gahow pushed to main at gahow/xtrain 2026-06-16 06:20:50 +00:00
48922cb628 perf: keep bf16 logits (no persistent fp32 logits buffer)
gahow pushed to main at gahow/xtrain 2026-06-16 06:15:09 +00:00
30db62d8f2 docs: Phase T12 — bf16 mixed precision design
0a2a4dcaa8 train: --bf16 flag (fp32-master AMP) + bf16 correctness test
b0086b5214 autodiff: bf16 mixed-precision path (fp32 master via cast op)
d05115ddf3 cuda: bf16 cuBLAS GemmEx (16BF in/out, fp32 accum) + cast kernels
Compare 4 commits »
gahow pushed to main at gahow/xtrain 2026-06-16 05:14:52 +00:00
511ceebbb3 docs: KI-2 trigger — dim768 fp32 batch-32 OOM
ff79fee3c5 docs: run v4 — TinyStories, dim768, val 1.17
734e119db3 run: v4 archive + export (dim768, 8-GPU DDP, val 1.17)
Compare 3 commits »
gahow pushed to main at gahow/xtrain 2026-06-16 03:15:13 +00:00
f85bd4d276 perf: KI-5 FIXED — single-GPU 40K->93K tok/s, DDP scaling 1.3x->5x@8
gahow pushed to main at gahow/xtrain 2026-06-16 03:04:17 +00:00
4c3f332f64 docs: Phase T11 — caching allocator
b7104e2cb7 test: loosen flaky DDP cross-rank assertion to <1e-6; scale to world=8
28801fbfe5 cuda: device caching allocator (pool GpuBuffer alloc)
Compare 3 commits »
gahow pushed to main at gahow/xtrain 2026-06-16 01:42:23 +00:00
d422c68704 docs: KI-5 — correct cross-rank divergence attribution (pre-existing flaky)
gahow pushed to main at gahow/xtrain 2026-06-16 01:40:51 +00:00
84092fb28d docs: KI-5 re-diagnosis — all-reduce is NOT the DDP bottleneck (T11)
88c2c15768 Revert "dist: coalesce grads into buckets for all-reduce (KI-5)"
Compare 2 commits »
gahow pushed to main at gahow/xtrain 2026-06-16 01:09:54 +00:00
b8b58212dc dist: coalesce grads into buckets for all-reduce (KI-5)
gahow pushed to feat/two-stop at gahow/aituner 2026-06-16 01:08:48 +00:00
4a64196a99 Add 27B Stop-B agentic-loop config (harness-driven, GPUs 2-7)
gahow pushed to feat/two-stop at gahow/aituner 2026-06-16 01:08:11 +00:00
b17b213575 Tear down the engine on SIGTERM instead of orphaning it
gahow pushed to main at gahow/xtrain 2026-06-15 19:43:26 +00:00
a78502e0f0 docs: run v3 — TinyStories, dim512, val 1.30
64b2a8c09e run: v3 archive + export (dim512, single-GPU batched, val 1.30)
Compare 2 commits »
gahow pushed to feat/two-stop at gahow/aituner 2026-06-15 17:54:42 +00:00
93ce339d61 Document 27B TP sweep: per-GPU rises sharply with TP (dense), opposite of MoE