Gahow Wang
3594f7dce0
LMetric was incorrectly sharing session-sticky logic with Linear policy. Fixed to pure per-request routing: score = P_tokens × BS where P = pending_prefill + (input - cache_hit), BS = num_requests. Experiment result (200 req, fresh restart): Linear vs corrected LMetric show <2% difference on all metrics — LMetric's cache-hit estimation provides implicit soft affinity that preserves locality without explicit session stickiness. Also fix bench.sh missing cd (replayer module not found from non-project cwd) and rewrite run_lmetric_ab.sh as thin wrapper around bench.sh to eliminate duplicated launch/cleanup logic that broke under set -euo. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
57 lines
2.1 KiB
Bash
Executable File
57 lines
2.1 KiB
Bash
Executable File
#!/bin/bash
|
|
# A/B comparison: linear (session-sticky) vs lmetric (OSDI'26, no affinity).
|
|
# Wraps bench.sh for guaranteed fresh state between experiments.
|
|
#
|
|
# Usage:
|
|
# bash scripts/run_lmetric_ab.sh # defaults: 200 req
|
|
# bash scripts/run_lmetric_ab.sh --requests 1000 # override request count
|
|
|
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
PROJECT_DIR="$(dirname "$SCRIPT_DIR")"
|
|
VENV="${VENV_PATH:-$PROJECT_DIR/.venv/bin}"
|
|
TS=$(date +%Y%m%d_%H%M%S)
|
|
|
|
echo "================================================================"
|
|
echo " A/B: Linear vs LMetric routing policy"
|
|
echo " $(date)"
|
|
echo "================================================================"
|
|
|
|
echo ""
|
|
echo "=== Experiment 1/2: Linear (session-sticky) ==="
|
|
bash "$SCRIPT_DIR/bench.sh" --tag "ab_linear_${TS}" --mode baseline --policy linear "$@"
|
|
|
|
echo ""
|
|
echo "=== Experiment 2/2: LMetric (no affinity) ==="
|
|
bash "$SCRIPT_DIR/bench.sh" --tag "ab_lmetric_${TS}" --mode baseline --policy lmetric "$@"
|
|
|
|
echo ""
|
|
echo "================================================================"
|
|
echo " Results comparison"
|
|
echo "================================================================"
|
|
"$VENV/python" -c "
|
|
import json
|
|
|
|
def summarize(path):
|
|
rows = [json.loads(l) for l in open(path)]
|
|
ok = [r for r in rows if not r.get('error')]
|
|
p = lambda v,q: sorted(v)[min(int(q*len(v)),len(v)-1)] if v else 0
|
|
ttfts = [r['ttft_s'] for r in ok if r.get('ttft_s')]
|
|
tpots = [r['tpot_s'] for r in ok if r.get('tpot_s') and r['tpot_s']>0]
|
|
e2es = [r['latency_s'] for r in ok]
|
|
return len(ok), len(rows), p(ttfts,.5), p(ttfts,.9), p(tpots,.9), p(e2es,.5)
|
|
|
|
lin = summarize('$PROJECT_DIR/outputs/ab_linear_${TS}/metrics.jsonl')
|
|
lm = summarize('$PROJECT_DIR/outputs/ab_lmetric_${TS}/metrics.jsonl')
|
|
|
|
fmt = ' %-20s OK=%3d/%3d TTFT50=%7.3f TTFT90=%7.3f TPOT90=%6.4f E2E50=%7.3f'
|
|
print(fmt % ('Linear', *lin))
|
|
print(fmt % ('LMetric', *lm))
|
|
print()
|
|
for name, i in [('TTFT50',2),('TTFT90',3),('TPOT90',4),('E2E50',5)]:
|
|
d = (lm[i] - lin[i]) / lin[i] * 100 if lin[i] else 0
|
|
print(' %s delta: %+.1f%%' % (name, d))
|
|
"
|
|
|
|
echo ""
|
|
echo "Done at $(date)"
|