gemm: tiled F32 forward + transpose + backward (dA/dB)

Hand-written tiled GEMM (csrc/ops/gemm.cu, TILE_SIZE=32, FP32 accumulate,
boundary-masked) plus an out-of-place transpose kernel. Wire both through
xtrain-cuda FFI (no_cuda-gated) and expose at the tensor level:
Tensor::matmul, transpose_2d, and matmul_backward computing
dA = dC·Bᵀ and dB = Aᵀ·dC by materializing transposes and reusing the
forward. Also declare cuBLAS sgemm FFI + link cublas, used only as a
correctness reference in tests.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-06-15 15:26:51 +08:00
parent 9ca98efd98
commit 08c88bf360
4 changed files with 221 additions and 0 deletions

View File

@@ -22,6 +22,8 @@ fn main() {
println!("cargo:rustc-link-search=native={cuda_path}/lib64");
println!("cargo:rustc-link-lib=dylib=cudart");
println!("cargo:rustc-link-lib=dylib=cuda");
// cuBLAS is used only as a correctness reference for the hand-written GEMM.
println!("cargo:rustc-link-lib=dylib=cublas");
cc::Build::new()
.cuda(true)
@@ -29,6 +31,7 @@ fn main() {
.flag("-gencode=arch=compute_120,code=sm_120")
.file("../../csrc/test/vecadd.cu")
.file("../../csrc/ops/elementwise.cu")
.file("../../csrc/ops/gemm.cu")
.compile("xtrain_cuda_kernels");
}