Gahow Wang
d376d91fe1
Real-time engine state is NOT the routing lever. Across 6 policies × es0/es1, real state reshuffles 44-76% of decisions but never beats the champion (unified+A+B, p90 7.62s). The effect's SIGN is set by reactivity: one-shot placement (sticky) HELPS -26%; per-request affinity-dominated is a wash; per-request pure-load (lmetric +17%, load_only +27%) HURTS via herding (stale shadow was a dampener). Feed verified fresh (median 25ms, <=92ms during prefills). Prior shadow-state results stand. ES_ABLATION_RESULTS.md has the table + mechanism; run_full_ablation.sh / fresh_sampler.py / cmp_es.py are the harness. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
31 lines
1.3 KiB
Python
31 lines
1.3 KiB
Python
#!/usr/bin/env python3
|
|
"""Sample engine-state feed freshness during the es1 run.
|
|
Writes one jsonl record per engine per tick: age_s = now - state.ts.
|
|
Stops when the DONE marker appears (run finished + /dev/shm wiped) or after 90min.
|
|
"""
|
|
import json, os, time, sys, glob
|
|
|
|
esdir, outpath, donemarker = sys.argv[1], sys.argv[2], sys.argv[3]
|
|
deadline = time.time() + 90 * 60
|
|
with open(outpath, "a") as out:
|
|
while not os.path.exists(donemarker) and time.time() < deadline:
|
|
now = time.time()
|
|
if os.path.isdir(esdir):
|
|
for f in sorted(glob.glob(os.path.join(esdir, "engine_*.json"))):
|
|
try:
|
|
s = json.load(open(f))
|
|
rec = {
|
|
"now": round(now, 3),
|
|
"engine": os.path.basename(f)[:-5],
|
|
"age_s": round(now - s.get("ts", 0), 4),
|
|
"num_running": s.get("num_running"),
|
|
"num_prefilling": s.get("num_prefilling"),
|
|
"max_prefill_remaining": s.get("max_prefill_remaining"),
|
|
"kv_used": s.get("gpu_kv_used_frac"),
|
|
}
|
|
out.write(json.dumps(rec) + "\n")
|
|
except Exception:
|
|
pass
|
|
out.flush()
|
|
time.sleep(5)
|