H1 (load balance) confirmed at the 15-min checkpoint: D2 received
22.5% of bindings (225 out of 1001) covering 30 unique sessions,
versus 0 in both E1 and E2. The graduated load-floor formula with
K=200 produces the intended distribution: fresh sessions on
under-loaded D, sticky sessions stay put.
But decode-1 crashed at 11:51:21 (~5 min into benchmark) with an
SGLang AssertionError in schedule_batch.py:1646. Root cause: the
streaming-session correction at line 1572-1585 patches
req.extend_input_len to 0 when len(fill_ids) < len(prefix_indices),
but the downstream invariant uses raw fill_ids/prefix_indices
lengths, so the arithmetic check fails. This is a pre-existing
landmine in the b8e6f13 SGLang vendor patch, not caused by the
load-floor bonus. It just happened to be masked in E2 by the
failure cascade preventing sessions from accumulating deep enough
prefix to trigger the correction.
Crash session 1000195 stayed on decode-1 the whole time (not a
migration race). E3 exposes this faster because sessions actually
run further with rebalanced load.
5 fix options evaluated. Recommended: Fix A — local patch at
schedule_batch.py:1646 to skip zero-extend-len reqs before
asserting. Less invasive than C (recomputing seq/prefix arrays);
addresses the actual case (D and E are workarounds, not fixes).
4 decision points for review; no code changes in this commit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
