Docs: reconcile routing docs with current hybrid direction
Per analysis/unified_routing_fix_review.md #2, several docs still presented the retired single-argmin + PUSH-migration design as the final algorithm. Mark them superseded and document the current hybrid direction (commit255c8e6). - REPORT.md §1.1 / §3.9: add errata callout and section header noting the "Final Design" framing was retired aftercc6e562/ 4c583f2; point readers to docs/migration-policy-design.md. - docs/migration-policy-design.md: rewrite. Opens with the current hybrid algorithm (LMetric base + cache_ratio>0.5 affinity gate + tie-breaker), then a "What Was Retired" commit table, then the old Approach A numbers preserved as "Historical Baseline-Mode Comparison". - analysis/research_findings.md §2.2 / §5: correct the LMetric framing. LMetric isn't "neutralized by affinity constraints" (pure --policy lmetric has no affinity at all); it converges to similar placements because P_tokens includes new_uncached_tokens, giving it implicit soft affinity. - analysis/elastic_hypotheses.md: same LMetric correction in the "DOESN'T work" summary, plus a footer cross-referencing the current routing direction. - analysis/unified_routing_fix_review.md: track this file (was untracked); it is the review handoff cited from the updated docs. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
24
REPORT.md
24
REPORT.md
@@ -27,6 +27,12 @@ For agentic LLM workloads (long input, short output, high KV cache reuse), is pr
|
||||
> was removed when replay moved to trace-driven dispatch. The next-step
|
||||
> experiment requires restoring the flag first (see `FIXES.md` §B2
|
||||
> route A) before any production-concurrency numbers can be produced.
|
||||
> - **§3.9 "Final Design" framing**: the single-argmin + PUSH-migration
|
||||
> design was retired after `cc6e562` / `4c583f2` showed forced and
|
||||
> relaxed-gate migration variants both regressed E2E tail. Current
|
||||
> policy is the hybrid LMetric + high-cache affinity landed in
|
||||
> `255c8e6`. See the per-section note in §3.9 and the active algorithm
|
||||
> in `docs/migration-policy-design.md`.
|
||||
>
|
||||
> The authoritative results are in **§3.6 and §3.7**.
|
||||
|
||||
@@ -356,7 +362,23 @@ The elastic numbers on dash1 were genuinely fresh. The "improvement" was actuall
|
||||
|
||||
**Output**: `outputs/eval_direct_rdma_v*/` on dash0.
|
||||
|
||||
### 3.9 Unified Routing (Final Design)
|
||||
### 3.9 Unified Routing (Historical — superseded)
|
||||
|
||||
> **Superseded by git history.** The "single argmin + PUSH migration" design
|
||||
> described here was implemented in `6b255fa`, refined through
|
||||
> `5892739` (soft affinity), `2b9eae0` (numbers below), and `4b50c5a`
|
||||
> (queue/overload-gate fixes). Follow-on attempts to scale migration —
|
||||
> `e991960`/`5772149` (forced session migration) and `bf4469a` (relaxed
|
||||
> push gate) — were both reverted (`cc6e562`, `4c583f2`) after they
|
||||
> regressed E2E tail (57 migrations → HEAVY TTFT p90 15.9s → 59.1s;
|
||||
> 134 offloads → E2E p90 37s → 82s).
|
||||
>
|
||||
> Current implementation is the **hybrid LMetric + high-cache affinity**
|
||||
> direction landed in `255c8e6`. See `docs/migration-policy-design.md`
|
||||
> for the active algorithm and `analysis/unified_routing_fix_review.md`
|
||||
> for the reasoning. The numbers below remain valid for the
|
||||
> `eval_unified_v3` artifact; do not treat them as the current
|
||||
> production policy.
|
||||
|
||||
Replaced two-phase routing (pick_instance → offload gate) with single `argmin(expected_latency)` per instance:
|
||||
|
||||
|
||||
Reference in New Issue
Block a user