P2: real engine-state feed replaces stale shadow counters for migration targeting

vLLM scheduler publishes real state (running/waiting, KV free, and the
max-in-progress-prefill signal /metrics lacks) to a tmpfs/redis store ~20Hz;
router reads it and avoids GIL-stall (mid-large-prefill) + KV-capacity-wall
targets, using real load over 30s-stale shadow counters. Components:
engine_state.py (canonical+reader), instrument_engine_state.py (scheduler
patch, file/redis writer), migration_target.py (scorer), proxy wiring
(--engine-state-uri, off=unchanged). All unit-tested without GPU; not yet
run live. See P2_ENGINE_STATE.md.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

microbench/connector_tax/layerwise/P2_ENGINE_STATE.md

@@ -0,0 +1,61 @@
+# P2: real engine-state feed for migration target selection
+
+Problem: the router (`cache_aware_proxy.py`) decides migration targets from
+**shadow counters** it maintains itself (incremented at dispatch, decremented
+at completion) and reconciles to vLLM `/metrics` only every **30 s**
+(`_reconcile_loop`). So every routing/migration decision is on stale state.
+Worse, the signal that predicts the ~45% control-plane stall — *is the target
+mid-large-prefill?* (a big prefill holds the GIL and starves the mooncake
+receiver_loop) — isn't visible at all, and `/metrics` doesn't expose it either.
+
+Fix: vLLM publishes **real** per-engine state to a shared store ~20 Hz; the
+router reads ground truth and avoids GIL-stall / capacity-wall targets.
+
+## Components (all unit-tested without GPUs)
+
+- `engine_state.py` — canonical `compute_snapshot(scheduler, id)`, `StateWriter`,
+  `StateReader`. Schema per engine: `ts, num_running, num_waiting,
+  gpu_blocks_total/free, gpu_kv_used_frac, pending_prefill_tokens,
+  ongoing_decode_tokens, num_prefilling, max_prefill_remaining`.
+- `instrument_engine_state.py` — vLLM `Scheduler` patch (apply/revert markers
+  `ES_INSTRUMENT_*`): a daemon thread publishes the snapshot every
+  `AGENTIC_ENGINE_STATE_PERIOD_MS` (50 ms) off the forward hot path. Inlined
+  writer (engine process needs no repo import). Coexists with MB5.
+- `migration_target.py` — pure target scorer: avoid `max_prefill_remaining ≥
+  es_big_prefill_threshold` (GIL stall) and `gpu_kv_used_frac ≥ es_kv_wall_frac`
+  (capacity wall), then rank by cache-richness and **real** load.
+- `cache_aware_proxy.WRITEMODE.py` — wired: `InstanceState.real_state`,
+  `_engine_state_poll_loop` (instance i ← `engine_{i}`), `_real_load`/Gate-3 and
+  Mechanism-B now real-state-aware. `--engine-state-uri` flag; off ⇒ identical
+  to before (shadow only).
+
+Transport (`AGENTIC_ENGINE_STATE_URI` / `--engine-state-uri`):
+`file:///dev/shm/agentic_engine_state` (default, zero-dep, single-node) or
+`redis://host:port/0` (multi-node; needs redis-py + server — not installed on
+dash0, so file backend is the working default).
+
+## Tests (no GPU)
+- `compute_snapshot` field math (mock scheduler): running/waiting,
+  max_prefill_remaining, pending, decode, kv_used_frac.
+- writer→reader round-trip + staleness drop (file backend).
+- target scorer: 5 cases incl. *avoid GIL-stall target even when its shadow
+  load is lower*, *real load beats stale shadow*, *cache-rich wins*,
+  *avoid KV wall*, *graceful fallback when feed missing*.
+- end-to-end: publish 8 engines (one mid-130k-prefill) → proxy inlined reader →
+  target selection avoids it.
+
+## Enabling in a GPU run (when free)
+1. `instrument_engine_state.py --apply` on the dash0 venv.
+2. `export AGENTIC_ENGINE_STATE_URI=file:///dev/shm/agentic_engine_state`
+   before the launcher (vLLM instances inherit it; `AGENTIC_WORKER_ID=engine_{i}`
+   already set by `b3_isolated_policy.sh` → publishes as `engine_{i}`).
+3. Proxy: `EXTRA_PROXY_ARGS="--engine-state-uri file:///dev/shm/agentic_engine_state ..."`.
+4. Revert the patch + `rm -rf /dev/shm/agentic_engine_state` after.
+
+## Status / scope
+- Built + unit-tested; NOT yet run against live engines (GPU busy).
+- Scoped to **migration target selection** (the P2 ask). The same real-load
+  signal could also de-stale the base `pick_instance_unified_hybrid` LMetric
+  fallback (the 8007-hotspot class from UNIFIED_ABLATION) — follow-up.
+- TP=1 only (one EngineCore/instance → one publisher/engine_id). TP>1 needs
+  per-rank ids.