H7 OVERLOAD_FACTOR sweep: negative result + H4 GPU profiling
H7: Sweeping OVERLOAD_FACTOR (2.0/1.5/1.3/1.0) has no effect on GPU imbalance (~3.5-4x across all settings). Root cause: imbalance is from workload skew at session placement (turn 1), not from routing at turn 2+. H4 GPU profiling confirms: GPU balance improvement IS real (4.0x→2.0x), and it directly improves HEAVY_COLO TTFT by 10.5%. But RDMA-offloaded requests have bimodal transfer times (0.6s or 18-31s) that negate the routing benefit. Updated elastic_hypotheses.md with H7 results and next directions: higher load experiments where contention amplifies routing differences. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -35,6 +35,7 @@ TIME_SCALE=20
|
||||
MAX_SESSIONS=8
|
||||
HEAVY_THRESHOLD=20000
|
||||
NO_OFFLOAD=false
|
||||
OVERLOAD_FACTOR_ARG=""
|
||||
|
||||
# Parse args
|
||||
while [[ $# -gt 0 ]]; do
|
||||
@@ -48,6 +49,7 @@ while [[ $# -gt 0 ]]; do
|
||||
--sessions) MAX_SESSIONS="$2"; shift 2 ;;
|
||||
--heavy-threshold) HEAVY_THRESHOLD="$2"; shift 2 ;;
|
||||
--no-offload) NO_OFFLOAD=true; shift ;;
|
||||
--overload-factor) OVERLOAD_FACTOR_ARG="$2"; shift 2 ;;
|
||||
*) echo "Unknown: $1"; exit 1 ;;
|
||||
esac
|
||||
done
|
||||
@@ -77,6 +79,7 @@ cat > "$OUTDIR/config.json" << CONF
|
||||
"max_sessions": $MAX_SESSIONS,
|
||||
"heavy_threshold": $HEAVY_THRESHOLD,
|
||||
"no_offload": "$NO_OFFLOAD",
|
||||
"overload_factor": "${OVERLOAD_FACTOR_ARG:-2.0}",
|
||||
"timestamp": "$(date -Iseconds)",
|
||||
"hostname": "$(hostname)"
|
||||
}
|
||||
@@ -193,6 +196,9 @@ launch_proxy() {
|
||||
done
|
||||
|
||||
local extra_args="--policy $POLICY"
|
||||
if [ -n "$OVERLOAD_FACTOR_ARG" ]; then
|
||||
extra_args="$extra_args --overload-factor $OVERLOAD_FACTOR_ARG"
|
||||
fi
|
||||
if [ "$MODE" = "elastic" ]; then
|
||||
local bp_list=""
|
||||
for i in $(seq 0 $((N_INSTANCES - 1))); do
|
||||
@@ -307,7 +313,7 @@ print('=' * 70)
|
||||
|
||||
echo "================================================================"
|
||||
echo " bench.sh: $TAG"
|
||||
echo " mode=$MODE policy=$POLICY requests=$REQUESTS"
|
||||
echo " mode=$MODE policy=$POLICY requests=$REQUESTS overload_factor=${OVERLOAD_FACTOR_ARG:-2.0}"
|
||||
echo " $(date)"
|
||||
echo "================================================================"
|
||||
|
||||
|
||||
Reference in New Issue
Block a user