autodiff: finite-diff gradient-check harness
New xtrain-autodiff crate with a reusable central finite-difference gradient check: grad_check(x, shape, f, analytic_grad, cfg) compares an analytic gradient against (f(x+ε)-f(x-ε))/2ε per element with a relative tolerance. Host-only (no CUDA): the loss closure owns any GPU work, so T4's per-op backward checks can reuse it directly. Includes host unit tests (sum(x²) grad 2x passes; a wrong grad is rejected). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
15
crates/xtrain-autodiff/src/lib.rs
Normal file
15
crates/xtrain-autodiff/src/lib.rs
Normal file
@@ -0,0 +1,15 @@
|
||||
//! Reusable numerical-gradient checking for xtrain (Phase T3+).
|
||||
//!
|
||||
//! Given a scalar loss `f(x)` and an analytic gradient `g`, verify that `g`
|
||||
//! matches the central finite-difference estimate
|
||||
//! `(f(x+ε·eᵢ) - f(x-ε·eᵢ)) / 2ε` for every element `i`, within a relative
|
||||
//! tolerance. Later phases (T4 autograd) reuse this per-op: wrap each op's
|
||||
//! forward as the loss, run its backward to get `g`, and `grad_check`.
|
||||
//!
|
||||
//! The harness is host-only and dtype-agnostic at this layer: it works on a
|
||||
//! flat `&[f32]` parameter vector + shape and a closure. The closure is free to
|
||||
//! push the data to the GPU and run kernels — that detail stays out of here.
|
||||
|
||||
pub mod finite_diff;
|
||||
|
||||
pub use finite_diff::{GradCheckConfig, GradCheckResult, ParamFn, grad_check};
|
||||
Reference in New Issue
Block a user