experiment: pin cu129 real pilot runtime

.research/experiments/frontier-workload-regime-taxonomy-20260720.md

@@ -173,7 +173,7 @@ The accompanying mock figure is schematic, not data. Panel A shows the state kne
 - Active v4 allocation: dash1=TP1 prefix off/on, dash2=TP2 prefix off/on, dash3=TP4 prefix off, dash4=TP4 prefix on. The four fleet jobs are running from fresh `sim-v4` output roots. First-process audit found the explicit isolated `--metrics_config_cache_dir` on all hosts and zero cross-version warnings.
 - First valid v4 tranche: 16/16 observed cells completed, each with 129 requests, request metrics, and a stage-batch ledger; no traceback, fallback, or version warning was found. The tranche covers all five W0 load points at TP1/TP2/TP4-MNS8 plus the first W5 prefix points at TP4-MNS8.
 - Early load-boundary observation: W0 at `rho=0.05` is low-latency for TP4-MNS8 (simulator TTFT mean 109.25 ms) but already queues for TP1-MNS8 (25.70 s); at `rho=0.25`, even TP4-MNS8 reaches 27.13 s mean TTFT. Because `rho` normalizes decode tokens only, high-rate short-output W0 also raises prefill and active-sequence pressure. These points map the overload boundary and are not eligible as reasonable-latency real pilots.
-- Real-runtime gate: a stock vLLM 0.20.0 environment passed import/H20 checks but used CUDA 13.0, so it is excluded from comparison with the historical CUDA 12.9 baseline. A separate official vLLM `0.20.0+cu129` environment is being materialized from the shared package cache before any real pilot is launched.
+- Real-runtime gate: a stock vLLM 0.20.0 environment passed import/H20 checks but used CUDA 13.0, so it is excluded from comparison with the historical CUDA 12.9 baseline. The replacement environment `vllm-0.20.0-cu129-workload-regime-v2` passes `vllm CLI=0.20.0+cu129`, torch `2.11.0+cu129`, CUDA runtime 12.9, H20 visibility, and all 179 package dependency checks. The first CPFS install used file copies and was stopped after download because it was still copying roughly 7 GB after 12 minutes; its incomplete directory is retained with an `invalid-copy-incomplete` suffix, while v2 uses same-filesystem hardlinks from the validated cache.
 - Load-contract correction: the original Fixed-PD surface held request rate per GPU constant, so global arrival rate scaled with TP. The v4 sweep holds global arrival rate constant and is retained as the control that isolates service-topology changes. A matched per-GPU sweep is now required to reproduce the original closed-loop intervention: TP1/TP2/TP4 receive `1x/2x/4x` global arrival rate at the same per-GPU `rho`.
 - Per-GPU low-load materialization: 105 cases = W0--W6 × `rho {0.0025,0.005,0.01,0.02,0.05}` × TP `{1,2,4}` were generated under `traces-per-gpu-low`. Audit passed 105 unique paths, 129 public/private rows per case, digests, arrival alignment, and exact `global_rate / TP = per_gpu_rate`. W0 `rho=0.01` is 0.239375 req/s/GPU, bracketing the original 0.215 req/s/GPU Fixed-PD point with `rho=0.005`.
 - The per-GPU sweep writes to a separate `sim-per-gpu-v1` result root but reuses the completed v4 predictor cache for the same TP/prefix/config. Predictor cache provenance is explicit in every surface manifest; workload results and state ledgers are never shared.