P2: all routing policies read real state via eff_ accessors + ablation harness
InstanceState.eff_{num_requests,pending_prefill,ongoing_decode,ongoing_tokens}
= max(shadow, real) when feed fresh (fixes 30s-stale under-count, keeps
in-flight RaceFix), plus real-only r_max_prefill_remaining / r_kv_used_frac.
Wired into load_only, lmetric, sticky, unified(_kv_both), unified_v3, and
snapshot logging. Feed off => identical to before. run_v3_trace.sh gains ES=1
toggle (always deploys enhanced proxy); run_ablation_es.sh runs each config
ES0-vs-ES1 to test whether real state changes policy performance/ranking.
All unit-tested without GPU.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -52,10 +52,31 @@ dash0, so file backend is the working default).
|
||||
3. Proxy: `EXTRA_PROXY_ARGS="--engine-state-uri file:///dev/shm/agentic_engine_state ..."`.
|
||||
4. Revert the patch + `rm -rf /dev/shm/agentic_engine_state` after.
|
||||
|
||||
## ALL policies now read the real state (update)
|
||||
`InstanceState` exposes effective accessors used by **every** picker:
|
||||
`eff_num_requests / eff_pending_prefill / eff_ongoing_decode /
|
||||
eff_ongoing_tokens` = `max(shadow, real)` when the feed is fresh (real fixes
|
||||
the 30s-stale under-count; shadow's atomic pre-await reservation still covers
|
||||
the in-flight window, preserving the RaceFix), plus real-only
|
||||
`r_max_prefill_remaining / r_kv_used_frac`. Wired into: `load_only`, `lmetric`,
|
||||
`sticky`, `pick_instance` (legacy), `pick_instance_unified_hybrid`
|
||||
(unified / unified_kv_both), `pick_instance_unified_v3` (gate + Mechanism B),
|
||||
and `snapshot_workers` (logged scores now match the decision + real fields).
|
||||
Feed off ⇒ `real_state is None` ⇒ accessors return shadow ⇒ byte-identical to
|
||||
before. (legacy `unified_v2` left on shadow — retired, not in the ablation.)
|
||||
|
||||
## Ablation (when GPU free)
|
||||
`run_v3_trace.sh` gains `ES=1` (apply engine-state patch + feed + proxy flag)
|
||||
and always deploys the enhanced proxy (dormant when feed/write-mode off).
|
||||
`run_ablation_es.sh` runs each config twice (ES=0 vs ES=1) so the only
|
||||
difference is the state source. Default decisive set (4 runs): champion
|
||||
`unified+A+B` and `unified_v3+A+B+layerwise`, each ES0/ES1. Extend CONFIGS for
|
||||
`lmetric` / `unified_kv_both` / `load_only`. Compares per-policy TTFT
|
||||
(overall + migrated) and whether the **ranking** changes with ground-truth
|
||||
state.
|
||||
|
||||
## Status / scope
|
||||
- Built + unit-tested; NOT yet run against live engines (GPU busy).
|
||||
- Scoped to **migration target selection** (the P2 ask). The same real-load
|
||||
signal could also de-stale the base `pick_instance_unified_hybrid` LMetric
|
||||
fallback (the 8007-hotspot class from UNIFIED_ABLATION) — follow-up.
|
||||
- Built + unit-tested (snapshot, round-trip, target scorer, eff_ accessors,
|
||||
end-to-end publish→read→select); NOT yet run against live engines (GPU busy).
|
||||
- TP=1 only (one EngineCore/instance → one publisher/engine_id). TP>1 needs
|
||||
per-rank ids.
|
||||
|
||||
Reference in New Issue
Block a user