Gahow Wang
224f750ee4
Var = Rc<RefCell<VarNode>> on a define-by-run tape: value + optional grad + parents + backward closure. backward() seeds a scalar loss, walks reverse topo order, and pushes grads to parents. push_grad always SUMs into the grad slot — the fan-out accumulation path T3 lacked. Per-crate build.rs emits the no_cuda cfg (does not propagate); engine gated, grad_check stays host-only. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
27 lines
1.1 KiB
Rust
27 lines
1.1 KiB
Rust
//! Reusable numerical-gradient checking for xtrain (Phase T3+).
|
|
//!
|
|
//! Given a scalar loss `f(x)` and an analytic gradient `g`, verify that `g`
|
|
//! matches the central finite-difference estimate
|
|
//! `(f(x+ε·eᵢ) - f(x-ε·eᵢ)) / 2ε` for every element `i`, within a relative
|
|
//! tolerance. Later phases (T4 autograd) reuse this per-op: wrap each op's
|
|
//! forward as the loss, run its backward to get `g`, and `grad_check`.
|
|
//!
|
|
//! The harness is host-only and dtype-agnostic at this layer: it works on a
|
|
//! flat `&[f32]` parameter vector + shape and a closure. The closure is free to
|
|
//! push the data to the GPU and run kernels — that detail stays out of here.
|
|
|
|
pub mod finite_diff;
|
|
|
|
pub use finite_diff::{GradCheckConfig, GradCheckResult, ParamFn, grad_check};
|
|
|
|
// Tape-based autograd engine + differentiable ops (Phase T4). These call GPU
|
|
// kernels via xtrain-tensor, so they are gated behind `not(no_cuda)` (the
|
|
// per-crate convention); the grad_check harness above stays host-only.
|
|
#[cfg(not(no_cuda))]
|
|
pub mod ops;
|
|
#[cfg(not(no_cuda))]
|
|
pub mod tape;
|
|
|
|
#[cfg(not(no_cuda))]
|
|
pub use tape::Var;
|