Gahow Wang
0c23285f39
Replace the out=128 / scale=0.5 ablation substrate with a paper-faithful one: - Use the trace's real output_length (drop completion_tokens_override=128). The 0-8k chat window has p50=531 / p99=2436 / max=35168 output tokens, so decode (TPOT) becomes the dominant bottleneck instead of an artificial 128-token cap. - replay_time_scale=0.8775, chosen by criterion-A: binary-search the smallest scale whose A-family L-C-A similarity to the real (scale=1.0) arrivals stays >= tau (0.90). The old scale=0.5 had sim_A=0.56, distorting the arrival axis far below the tau bar used everywhere else. New calibrator: scripts/calibrate_time_scale.py. - Per-probe Stop-A-consistent drain deadline (worker._probe_drain_deadline): the wall-clock a *feasible* config needs to drain the LCA-admitted set (last_arrival + worst-case TTFT + p99_out * TPOT budget + margin). With real outputs decode dominates wall-clock, so the old fixed 320s cap would truncate the Stop-A offered window mid-decode. early_stop_max_elapsed_s (1000s) is now a hard ceiling; the per-probe deadline governs. The lag cap still cuts overload. 12-iter paired driver (both arms on dash1, removes the dash0/dash1 host confound): scripts/run_ablation_pair_d1.sh. 115 tests pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
AITuner
AITuner is a small study orchestrator for OpenAI-compatible serving engines. It replays trace windows, searches for the highest feasible offered load under configured SLOs, and records enough trial context for LLM- or harness-guided configuration proposals.
Status
This repository is research tooling. Treat reported experiment numbers as valid
only when the matching study spec, trial artifacts, probe history, and
probe_details.jsonl files are available for audit.
Install
python3 -m pip install -e .
Test
The test suite uses the Python standard library unittest runner:
PYTHONPATH=src python3 -m unittest discover -s tests -v
If the package is installed in editable mode, PYTHONPATH=src is optional.
Basic Workflow
Initialize a study:
aituner study init --spec configs/examples/study.example.json
Run a local tuning loop:
aituner study tune --spec configs/examples/study.example.json --max-trials 2
Run a compare:
aituner compare run --spec configs/examples/compare.example.json
Remote experiment notes for this checkout live in AGENTS.md. The default
remote host is dash0, and code should be synchronized through Git before
remote runs.
Experiment Integrity
- Fixed-length replay requests are scored only when completion token usage is verifiable and matches the trace expectation.
- Each trial writes aggregate probe history and per-request probe details.
request_rate_per_gpuis the primary cross-topology metric:best_feasible_request_rate / (tensor_parallel_size * data_parallel_size).- Compare reports include failed and no-feasible window counts; do not interpret mean request rates without those counts.
- Bounded replays using
max_requests_per_probe,completion_tokens_override, orreplay_time_scaleare convergence tests for that bounded workload, not production benchmarks.
Configuration Notes
Example specs that use llm.endpoint.provider=codex resolve the endpoint from
the local Codex configuration unless llm.endpoint.base_url or
AITUNER_CODEX_BASE_URL is set. Public, reproducible examples should prefer an
explicit endpoint or omit the LLM endpoint and use proposal files.