H7 OVERLOAD_FACTOR sweep: negative result + H4 GPU profiling

H7: Sweeping OVERLOAD_FACTOR (2.0/1.5/1.3/1.0) has no effect on GPU
imbalance (~3.5-4x across all settings). Root cause: imbalance is from
workload skew at session placement (turn 1), not from routing at turn 2+.

H4 GPU profiling confirms: GPU balance improvement IS real (4.0x→2.0x),
and it directly improves HEAVY_COLO TTFT by 10.5%. But RDMA-offloaded
requests have bimodal transfer times (0.6s or 18-31s) that negate the
routing benefit.

Updated elastic_hypotheses.md with H7 results and next directions:
higher load experiments where contention amplifies routing differences.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

scripts/bench.sh

@@ -35,6 +35,7 @@ TIME_SCALE=20
 MAX_SESSIONS=8
 HEAVY_THRESHOLD=20000
 NO_OFFLOAD=false
+OVERLOAD_FACTOR_ARG=""
 
 # Parse args
 while [[ $# -gt 0 ]]; do
@@ -48,6 +49,7 @@ while [[ $# -gt 0 ]]; do
         --sessions) MAX_SESSIONS="$2"; shift 2 ;;
         --heavy-threshold) HEAVY_THRESHOLD="$2"; shift 2 ;;
         --no-offload) NO_OFFLOAD=true; shift ;;
+        --overload-factor) OVERLOAD_FACTOR_ARG="$2"; shift 2 ;;
         *) echo "Unknown: $1"; exit 1 ;;
     esac
 done
@@ -77,6 +79,7 @@ cat > "$OUTDIR/config.json" << CONF
     "max_sessions": $MAX_SESSIONS,
     "heavy_threshold": $HEAVY_THRESHOLD,
     "no_offload": "$NO_OFFLOAD",
+    "overload_factor": "${OVERLOAD_FACTOR_ARG:-2.0}",
     "timestamp": "$(date -Iseconds)",
     "hostname": "$(hostname)"
 }
@@ -193,6 +196,9 @@ launch_proxy() {
     done
 
     local extra_args="--policy $POLICY"
+    if [ -n "$OVERLOAD_FACTOR_ARG" ]; then
+        extra_args="$extra_args --overload-factor $OVERLOAD_FACTOR_ARG"
+    fi
     if [ "$MODE" = "elastic" ]; then
         local bp_list=""
         for i in $(seq 0 $((N_INSTANCES - 1))); do
@@ -307,7 +313,7 @@ print('=' * 70)
 
 echo "================================================================"
 echo "  bench.sh: $TAG"
-echo "  mode=$MODE  policy=$POLICY  requests=$REQUESTS"
+echo "  mode=$MODE  policy=$POLICY  requests=$REQUESTS  overload_factor=${OVERLOAD_FACTOR_ARG:-2.0}"
 echo "  $(date)"
 echo "================================================================"
 

scripts/cache_aware_proxy.py

@@ -28,7 +28,7 @@ from fastapi.responses import StreamingResponse
 BLOCK_SIZE = 512
 CACHE_HIT_ALPHA = 1.0
 HEAVY_THRESHOLD = 20000  # default; overridden by --heavy-threshold
-OVERLOAD_FACTOR = 2.0
+OVERLOAD_FACTOR = 2.0    # default; overridden by --overload-factor
 
 
 class InstanceState:
@@ -627,6 +627,8 @@ def parse_args():
                    help="Comma-separated bootstrap ports for combined instances (for offload mode)")
     p.add_argument("--policy", type=str, default="linear", choices=["linear", "lmetric"],
                    help="Routing policy: linear (default) or lmetric (P_tokens × BS, OSDI'26)")
+    p.add_argument("--overload-factor", type=float, default=2.0,
+                   help="Break session affinity when instance load > factor * avg")
     args = p.parse_args()
 
     args.prefill = []
@@ -645,4 +647,5 @@ def parse_args():
 if __name__ == "__main__":
     global_args = parse_args()
     HEAVY_THRESHOLD = global_args.heavy_threshold
+    OVERLOAD_FACTOR = global_args.overload_factor
     uvicorn.run(app, host=global_args.host, port=global_args.port)