Docs: reconcile routing docs with current hybrid direction

Per analysis/unified_routing_fix_review.md #2, several docs still
presented the retired single-argmin + PUSH-migration design as the
final algorithm. Mark them superseded and document the current hybrid
direction (commit 255c8e6).

- REPORT.md §1.1 / §3.9: add errata callout and section header noting
  the "Final Design" framing was retired after cc6e562 / 4c583f2;
  point readers to docs/migration-policy-design.md.

- docs/migration-policy-design.md: rewrite. Opens with the current
  hybrid algorithm (LMetric base + cache_ratio>0.5 affinity gate +
  tie-breaker), then a "What Was Retired" commit table, then the old
  Approach A numbers preserved as "Historical Baseline-Mode Comparison".

- analysis/research_findings.md §2.2 / §5: correct the LMetric framing.
  LMetric isn't "neutralized by affinity constraints" (pure --policy
  lmetric has no affinity at all); it converges to similar placements
  because P_tokens includes new_uncached_tokens, giving it implicit
  soft affinity.

- analysis/elastic_hypotheses.md: same LMetric correction in the
  "DOESN'T work" summary, plus a footer cross-referencing the current
  routing direction.

- analysis/unified_routing_fix_review.md: track this file (was
  untracked); it is the review handoff cited from the updated docs.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
2026-05-25 10:47:14 +08:00
parent ac6534c3ff
commit 6a27f75337
5 changed files with 591 additions and 53 deletions

View File

@@ -27,6 +27,12 @@ For agentic LLM workloads (long input, short output, high KV cache reuse), is pr
> was removed when replay moved to trace-driven dispatch. The next-step
> experiment requires restoring the flag first (see `FIXES.md` §B2
> route A) before any production-concurrency numbers can be produced.
> - **§3.9 "Final Design" framing**: the single-argmin + PUSH-migration
> design was retired after `cc6e562` / `4c583f2` showed forced and
> relaxed-gate migration variants both regressed E2E tail. Current
> policy is the hybrid LMetric + high-cache affinity landed in
> `255c8e6`. See the per-section note in §3.9 and the active algorithm
> in `docs/migration-policy-design.md`.
>
> The authoritative results are in **§3.6 and §3.7**.
@@ -356,7 +362,23 @@ The elastic numbers on dash1 were genuinely fresh. The "improvement" was actuall
**Output**: `outputs/eval_direct_rdma_v*/` on dash0.
### 3.9 Unified Routing (Final Design)
### 3.9 Unified Routing (Historical — superseded)
> **Superseded by git history.** The "single argmin + PUSH migration" design
> described here was implemented in `6b255fa`, refined through
> `5892739` (soft affinity), `2b9eae0` (numbers below), and `4b50c5a`
> (queue/overload-gate fixes). Follow-on attempts to scale migration —
> `e991960`/`5772149` (forced session migration) and `bf4469a` (relaxed
> push gate) — were both reverted (`cc6e562`, `4c583f2`) after they
> regressed E2E tail (57 migrations → HEAVY TTFT p90 15.9s → 59.1s;
> 134 offloads → E2E p90 37s → 82s).
>
> Current implementation is the **hybrid LMetric + high-cache affinity**
> direction landed in `255c8e6`. See `docs/migration-policy-design.md`
> for the active algorithm and `analysis/unified_routing_fix_review.md`
> for the reasoning. The numbers below remain valid for the
> `eval_unified_v3` artifact; do not treat them as the current
> production policy.
Replaced two-phase routing (pick_instance offload gate) with single `argmin(expected_latency)` per instance: