Fig18 substrate: real output_length + criterion-A time_scale + Stop-A drain deadline
Replace the out=128 / scale=0.5 ablation substrate with a paper-faithful one: - Use the trace's real output_length (drop completion_tokens_override=128). The 0-8k chat window has p50=531 / p99=2436 / max=35168 output tokens, so decode (TPOT) becomes the dominant bottleneck instead of an artificial 128-token cap. - replay_time_scale=0.8775, chosen by criterion-A: binary-search the smallest scale whose A-family L-C-A similarity to the real (scale=1.0) arrivals stays >= tau (0.90). The old scale=0.5 had sim_A=0.56, distorting the arrival axis far below the tau bar used everywhere else. New calibrator: scripts/calibrate_time_scale.py. - Per-probe Stop-A-consistent drain deadline (worker._probe_drain_deadline): the wall-clock a *feasible* config needs to drain the LCA-admitted set (last_arrival + worst-case TTFT + p99_out * TPOT budget + margin). With real outputs decode dominates wall-clock, so the old fixed 320s cap would truncate the Stop-A offered window mid-decode. early_stop_max_elapsed_s (1000s) is now a hard ceiling; the per-probe deadline governs. The lag cap still cuts overload. 12-iter paired driver (both arms on dash1, removes the dash0/dash1 host confound): scripts/run_ablation_pair_d1.sh. 115 tests pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -18,7 +18,7 @@ from .engine import build_launch_recipe
|
||||
from .http_client import HttpClientError, stream_chat_completion, wait_for_server
|
||||
from .lca import find_convergence_prefix, resolve_length_mode
|
||||
from .search import ThresholdProbe, binary_search_max_feasible
|
||||
from .slo import RequestOutcome, evaluate_request, summarize_evaluations
|
||||
from .slo import RequestOutcome, _rule_threshold_ms, evaluate_request, summarize_evaluations
|
||||
from .spec import ConfigPatch, SamplingSearchSpec, TrialSpec, load_study_spec, to_jsonable
|
||||
from .trace import TraceRequest, load_trace_requests, select_requests_for_threshold
|
||||
|
||||
@@ -254,6 +254,34 @@ def _ignore_sigterm_if_main() -> None:
|
||||
pass
|
||||
|
||||
|
||||
def _probe_drain_deadline(
|
||||
reqs: list[TraceRequest], slo: Any, *, ceiling: float | None
|
||||
) -> float | None:
|
||||
"""Stop-A-consistent per-probe drain deadline (wall-clock seconds).
|
||||
|
||||
The deadline is the time a *feasible* config needs to drain the admitted set:
|
||||
the last admitted arrival plus the worst-case TTFT budget plus the p99 output
|
||||
length times the TPOT budget. A config that cannot finish by this deadline is
|
||||
genuinely SLO-infeasible, so the clock never pre-empts the LCA-matched offered
|
||||
window (Stop-A) -- it only fails the unfit. ``ceiling`` is a hard safety cap.
|
||||
"""
|
||||
if not reqs or slo.tpot_rule is None:
|
||||
return ceiling
|
||||
last_arrival = max(float(r.arrival_s or 0.0) for r in reqs)
|
||||
inputs = sorted(int(r.prompt_tokens_hint or 0) for r in reqs)
|
||||
outputs = sorted(int(r.completion_tokens_hint or 0) for r in reqs)
|
||||
|
||||
def _p99(xs: list[int]) -> int:
|
||||
return xs[min(len(xs) - 1, int(0.99 * len(xs)))] if xs else 0
|
||||
|
||||
p99_in, p99_out = _p99(inputs), _p99(outputs)
|
||||
tpot_ms = _rule_threshold_ms(slo.tpot_rule, p99_in)
|
||||
ttft_ms = _rule_threshold_ms(slo.ttft_rule, p99_in) if slo.ttft_rule is not None else 0.0
|
||||
margin_s = 30.0
|
||||
deadline = last_arrival + (ttft_ms + p99_out * tpot_ms) / 1000.0 + margin_s
|
||||
return min(float(ceiling), deadline) if ceiling else deadline
|
||||
|
||||
|
||||
def _adaptive_replay_set(
|
||||
selected: list[TraceRequest],
|
||||
*,
|
||||
@@ -640,7 +668,9 @@ def run_trial(trial_spec_path: Path) -> dict[str, Any]:
|
||||
max_concurrency=study.trace.max_concurrency,
|
||||
target_pass_rate=study.slo.target_pass_rate,
|
||||
max_lag_s=study.trace.early_stop_max_lag_s,
|
||||
max_elapsed_s=study.trace.early_stop_max_elapsed_s,
|
||||
max_elapsed_s=_probe_drain_deadline(
|
||||
reqs, study.slo, ceiling=study.trace.early_stop_max_elapsed_s
|
||||
),
|
||||
evaluate_outcome=lambda outcome: evaluate_request(outcome, study.slo),
|
||||
drain_inflight_on_early_stop=not restart_after_early_stop,
|
||||
)
|
||||
|
||||
Reference in New Issue
Block a user