distributed: process-per-GPU launcher + worker (proc.rs)
torchrun-style process-per-GPU: launch_processes spawns one worker process per
GPU (re-exec current_exe with XTRAIN_{RANK,WORLD,LOCAL_RANK,NCCL_ID} env),
mints the ncclUniqueId once in the launcher and hex-injects it via env (no
shared FS/TCP, race-free). worker_env/run_worker read the env, bind the device
(own CUDA context), DdpContext::init + build_model + train_rank reused from T8
UNCHANGED. hex_encode/decode_unique_id are host-testable pure fns.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -18,8 +18,13 @@
|
||||
|
||||
pub mod ddp;
|
||||
pub mod ffi;
|
||||
pub mod proc;
|
||||
|
||||
pub use ddp::{DdpConfig, DdpResult, build_model, launch, train_rank};
|
||||
pub use proc::{
|
||||
ModelOpts, WorkerEnv, build_worker_model, hex_decode_unique_id, hex_encode_unique_id,
|
||||
launch_processes, run_worker, worker_env,
|
||||
};
|
||||
|
||||
use std::ffi::c_void;
|
||||
|
||||
|
||||
Reference in New Issue
Block a user