Document Stop-B end-to-end on dense 27B: the improving climb + no-regression

Real gpt-5.4 agentic loop raised per-GPU TP1 0.123 -> TP2 0.2925 -> TP4 1.0012 (8.1x),
each a correctly-diagnosed real gain; then a TP4 runtime tweak measured 0.942 < 1.00
and was correctly rejected (no regression). With the 30B run (validator stop + LLM-stop
veto), all Stop-B behaviors are now validated end-to-end. The SIGTERM-teardown fix was
validated in practice (clean engine teardown, no GPU leak on stop). Efficiency finding:
at scale=1.0, infeasible high-theta probes burn the 900s elapsed cap, so a practical
loop needs a lower cap; this is why the run was stopped after iter-4 rather than driven
to an explicit Stop-B firing.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

.gitignore

@@ -2,6 +2,8 @@
 .aituner-smoke/
 .aituner-decode/
 .aituner-tight/
+.aituner-prefill/
+.aituner-compare/
 .env
 __pycache__/
 *.pyc

AGENTS.md

@@ -0,0 +1,15 @@
+# Project Operating Notes
+
+## Remote experiment host
+
+- Default experiment machine: `dash0`.
+- Hardware expectation: 8 NVIDIA H20 GPUs.
+- SSH check: use `ssh dash0` before scheduling or debugging remote runs.
+- Remote project path: `/home/admin/cpfs/wjh/aituner/aituner`.
+
+## Local/remote sync workflow
+
+- Treat this local repository and the `dash0` repository as the same project checkout.
+- Synchronize code through Git using `commit`, `push`, and `pull`.
+- For remote experiments, commit local changes, push to `origin`, then pull on `dash0` in `/home/admin/cpfs/wjh/aituner/aituner` before running.
+- Do not ask for the remote host or project path again unless the user explicitly changes them.

CONTRIBUTING.md

@@ -0,0 +1,23 @@
+# Contributing
+
+## Development Setup
+
+```bash
+python3 -m pip install -e .
+PYTHONPATH=src python3 -m unittest discover -s tests -v
+```
+
+## Change Requirements
+
+- Add or update tests for behavior changes.
+- Keep experiment claims tied to reproducible artifacts: study spec, trial spec,
+  result JSON, probe history, and per-request probe details.
+- Do not publish benchmark conclusions from bounded or time-compressed replays
+  without clearly labeling the replay controls.
+- Keep example configs free of private credentials and prefer explicit,
+  reproducible endpoint settings.
+
+## Commit Hygiene
+
+Use small commits grouped by behavior: measurement integrity, orchestration
+logic, documentation, or infrastructure.

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 AITuner contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -0,0 +1,72 @@
+# AITuner
+
+AITuner is a small study orchestrator for OpenAI-compatible serving engines. It
+replays trace windows, searches for the highest feasible offered load under
+configured SLOs, and records enough trial context for LLM- or harness-guided
+configuration proposals.
+
+## Status
+
+This repository is research tooling. Treat reported experiment numbers as valid
+only when the matching study spec, trial artifacts, probe history, and
+`probe_details.jsonl` files are available for audit.
+
+## Install
+
+```bash
+python3 -m pip install -e .
+```
+
+## Test
+
+The test suite uses the Python standard library `unittest` runner:
+
+```bash
+PYTHONPATH=src python3 -m unittest discover -s tests -v
+```
+
+If the package is installed in editable mode, `PYTHONPATH=src` is optional.
+
+## Basic Workflow
+
+Initialize a study:
+
+```bash
+aituner study init --spec configs/examples/study.example.json
+```
+
+Run a local tuning loop:
+
+```bash
+aituner study tune --spec configs/examples/study.example.json --max-trials 2
+```
+
+Run a compare:
+
+```bash
+aituner compare run --spec configs/examples/compare.example.json
+```
+
+Remote experiment notes for this checkout live in `AGENTS.md`. The default
+remote host is `dash0`, and code should be synchronized through Git before
+remote runs.
+
+## Experiment Integrity
+
+- Fixed-length replay requests are scored only when completion token usage is
+  verifiable and matches the trace expectation.
+- Each trial writes aggregate probe history and per-request probe details.
+- `request_rate_per_gpu` is the primary cross-topology metric:
+  `best_feasible_request_rate / (tensor_parallel_size * data_parallel_size)`.
+- Compare reports include failed and no-feasible window counts; do not interpret
+  mean request rates without those counts.
+- Bounded replays using `max_requests_per_probe`, `completion_tokens_override`,
+  or `replay_time_scale` are convergence tests for that bounded workload, not
+  production benchmarks.
+
+## Configuration Notes
+
+Example specs that use `llm.endpoint.provider=codex` resolve the endpoint from
+the local Codex configuration unless `llm.endpoint.base_url` or
+`AITUNER_CODEX_BASE_URL` is set. Public, reproducible examples should prefer an
+explicit endpoint or omit the LLM endpoint and use proposal files.

SECURITY.md

@@ -0,0 +1,19 @@
+# Security
+
+AITuner launches local or remote serving engines and may replay trace payloads.
+Do not commit secrets, API keys, private trace content, or private model access
+tokens.
+
+## Reporting
+
+Report security issues privately to the project maintainers. If this repository
+is mirrored to a public forge, use that forge's private vulnerability reporting
+flow when available.
+
+## Operational Guidance
+
+- Keep `.env` files local; `.env.example` documents expected variable names.
+- Review generated trial artifacts before publishing them, because request
+  payloads may contain trace text.
+- Treat remote execution configs as sensitive when they include internal host
+  names, paths, or scheduler details.

configs/examples/dash0_qwen235b_decode_thinking_run1.json

@@ -157,7 +157,8 @@
     "max_concurrency": 128,
     "replay_time_scale": 1.0,
     "early_stop_max_lag_s": 180.0,
-    "early_stop_max_elapsed_s": 1200.0
+    "early_stop_max_elapsed_s": 1200.0,
+    "restart_engine_after_early_stop": true
   },
   "slo": {
     "target_pass_rate": 0.95,

configs/examples/dash0_qwen235b_decode_thinking_run2_tpot40.json

@@ -167,7 +167,8 @@
     "max_concurrency": 128,
     "replay_time_scale": 1.0,
     "early_stop_max_lag_s": 180.0,
-    "early_stop_max_elapsed_s": 1200.0
+    "early_stop_max_elapsed_s": 1200.0,
+    "restart_engine_after_early_stop": true
   },
   "slo": {
     "target_pass_rate": 0.95,

configs/examples/dash0_qwen235b_prefill_thinking_run3_ttft_tight_0323.json

@@ -0,0 +1,208 @@
+{
+  "study_id": "dash0-qwen235b-prefill-thinking-run3-ttft-tight-0323-topology",
+  "hardware": {
+    "gpu_count": 8,
+    "gpu_model": "H20",
+    "host_candidates": [
+      "dash0"
+    ]
+  },
+  "model": {
+    "model_id": "qwen3-235b-a22b-256k-0717-internal",
+    "served_model_name": "qwen3-235b-prefill"
+  },
+  "engine": {
+    "engine_name": "vllm",
+    "engine_version": "internal-on-dash0",
+    "exec_path": "/usr/local/bin/vllm",
+    "cwd": "/home/admin/cpfs/wjh/aituner/aituner",
+    "host": "127.0.0.1",
+    "port": 18127,
+    "healthcheck_path": "/v1/models",
+    "ready_timeout_s": 1800,
+    "request_timeout_s": 1800,
+    "launch_args": [
+      "serve",
+      "/home/admin/resource/model/464482ce.qwen3-235b-a22b/256k-0717"
+    ],
+    "base_envs": {
+      "CUDA_VISIBLE_DEVICES": "0,1,2,3,4,5,6,7",
+      "VLLM_USE_V1": "1",
+      "VLLM_ATTENTION_BACKEND": "FLASH_ATTN",
+      "VLLM_QUANTIZATION_LAYER_WISE": "1",
+      "VLLM_MOE_USE_DEEPEP": "0",
+      "VLLM_MOE_BALANCED_GATING": "0",
+      "VLLM_MOE_RANDOM_GATING": "0",
+      "VLLM_FUSED_MOE_CHUNK_SIZE": "4096",
+      "VLLM_DP_META_USE_CPU_GROUP": "0",
+      "VLLM_MLA_FP8_ATTENTION": "0",
+      "VLLM_MOE_EXPERTS_OVERLAP": "0",
+      "VLLM_USE_FLASHINFER_SAMPLER": "0",
+      "VLLM_RESPONSE_TIMEOUT": "290",
+      "VLLM_FP8_USE_BLADNN": "1",
+      "VLLM_MOE_USE_BLADNN": "1",
+      "VLLM_USE_DEEP_GEMM": "0",
+      "VLLM_PD_TRY_CONNECT_TIMEOUT_SECONDS": "120",
+      "VLLM_DEEP_GEMM_WARMUP": "skip",
+      "DEEPEP_LL_COMBINE_USE_FP8": "1",
+      "DEEPEP_LL_BUFFER_FP8_OPT": "1",
+      "DEEPEP_LL_DISPATCH_USE_NVL": "1",
+      "DEEPEP_LL_COMBINE_USE_NVL": "1",
+      "ACCL_LOW_LATENCY_OPTIMIZE": "2",
+      "ACCL_WRITEBATCH_OPT": "2",
+      "ACCL_IBV_MTU": "9000",
+      "ACCL_TX_DEPTH": "1024",
+      "ACCL_RETRANSMIT_TIMEOUT": "17",
+      "NVSHMEM_IBGDA_NUM_RC_PER_PE": "4",
+      "BLLM_KVTRANS_RDMA_SP": "2",
+      "NCCL_SOCKET_IFNAME": "eth1",
+      "NVSHMEM_BOOTSTRAP_UID_SOCK_IFNAME": "eth1",
+      "GLOO_SOCKET_IFNAME": "eth1"
+    },
+    "base_flags": {
+      "host": "127.0.0.1",
+      "port": 18127,
+      "served-model-name": "qwen3-235b-prefill",
+      "tensor-parallel-size": 4,
+      "gpu-memory-utilization": 0.85,
+      "enable-prefix-caching": true,
+      "enable-chunked-prefill": true,
+      "max-num-batched-tokens": 8192,
+      "disable-hybrid-kv-cache-manager": true,
+      "max-model-len": 262144,
+      "block-size": 64,
+      "max-num-seqs": 64,
+      "quantization": "fp8",
+      "cuda-graph-sizes": [
+        16,
+        32,
+        64,
+        96,
+        128,
+        160,
+        192,
+        224,
+        256,
+        288,
+        320,
+        352,
+        384,
+        416,
+        448,
+        480,
+        512,
+        544,
+        576,
+        608,
+        640,
+        672,
+        704,
+        736,
+        768,
+        800,
+        832,
+        864,
+        896,
+        928,
+        960,
+        992,
+        1024
+      ],
+      "compilation-config": "{\"cudagraph_mode\":\"PIECEWISE\",\"use_inductor\":false,\"custom_ops\":[\"all\"],\"max_cudagraph_capture_size\":2048}",
+      "speculative-config": "{\"method\":\"eagle3\",\"num_speculative_tokens\":1,\"hf_overrides\":{\"rope_scaling\":{\"type\":\"yarn\",\"factor\":128,\"original_max_position_embeddings\":2048,\"semi_dynamic\":false,\"dynamic\":true},\"num_experts\":0},\"model\":\"/home/admin/resource/model/464482ce.qwen3-235b-a22b/0717-eagle-0820\"}",
+      "hf-overrides": "{\"architectures\":[\"Qwen3MoeForCausalLM\"],\"model_type\":\"qwen3_moe\"}",
+      "kv-cache-dtype": "fp8",
+      "disable-log-requests": true
+    },
+    "tunable_envs": [
+      "VLLM_ENABLE_TORCH_COMPILE"
+    ],
+    "tunable_flags": [
+      "tensor-parallel-size",
+      "data-parallel-size",
+      "enable-expert-parallel",
+      "expert-parallel-size",
+      "gpu-memory-utilization",
+      "max-num-batched-tokens",
+      "max-num-seqs",
+      "block-size",
+      "enable-prefix-caching",
+      "enable-chunked-prefill"
+    ],
+    "topology_constraints": {
+      "require_tp_dp_product_equals_gpu_count": false,
+      "require_ep_size_leq_tp_dp_product": true,
+      "require_ep_size_divides_tp_dp_product": true,
+      "require_enable_expert_parallel_when_ep_gt_one": true,
+      "validate_cuda_graph_sizes_divisible_by_tp_when_tp_ep_reduce_scatter": true,
+      "allowed_tp_dp_products": [
+        4,
+        8
+      ],
+      "allowed_tensor_parallel_sizes": [
+        4,
+        8
+      ],
+      "allowed_data_parallel_sizes": [
+        1,
+        2
+      ],
+      "allowed_expert_parallel_sizes": [
+        1,
+        2,
+        4,
+        8
+      ]
+    },
+    "python_executable": "python3"
+  },
+  "trace": {
+    "windows_path": "/home/admin/cpfs/wjh/aituner/aituner/trace_windows/windows.json",
+    "window_id": "thinking_w20260323_1000",
+    "request_mode": "chat",
+    "completion_tokens_override": 1,
+    "u_field": "sampling_u",
+    "timestamp_field": "timestamp",
+    "max_concurrency": 64,
+    "replay_time_scale": 1.0,
+    "early_stop_max_lag_s": 180.0,
+    "early_stop_max_elapsed_s": 1200.0
+  },
+  "slo": {
+    "target_pass_rate": 0.95,
+    "ttft_rule": {
+      "kind": "step_ms",
+      "buckets": [
+        {
+          "max_input_tokens": 8191,
+          "threshold_ms": 2000
+        },
+        {
+          "max_input_tokens": 32767,
+          "threshold_ms": 4000
+        },
+        {
+          "threshold_ms": 6000
+        }
+      ]
+    }
+  },
+  "search": {
+    "low": 0.0,
+    "high": 0.125,
+    "tolerance": 0.001,
+    "max_probes": 6,
+    "sample_seed": 20260325
+  },
+  "llm": {
+    "system_prompt": "You are tuning a prefill-dominated vLLM serving stack. The trace replay forces completion length to exactly 1 token, so optimize for TTFT under the configured stepped SLO. Propose one launch-safe config patch that increases the maximum feasible sampling_u while respecting the topology constraints and avoiding known launch failures.",
+    "max_history_trials": 8,
+    "endpoint": {
+      "provider": "codex",
+      "model": "gpt-5.4",
+      "stream": true,
+      "api_key_env": "OPENAI_API_KEY",
+      "timeout_s": 240
+    }
+  }
+}

configs/examples/dash0_qwen27b_stopB_loop.json

@@ -0,0 +1,177 @@
+{
+  "study_id": "dash0-qwen27b-stopB-loop-chat-0-8k",
+  "hardware": {
+    "gpu_count": 8,
+    "gpu_model": "H20",
+    "host_candidates": [
+      "dash0"
+    ]
+  },
+  "model": {
+    "model_id": "qwen3.5-27b-256k-0223-internal",
+    "served_model_name": "qwen35-27b-aituner"
+  },
+  "engine": {
+    "engine_name": "vllm",
+    "engine_version": "latest-release-on-dash0",
+    "exec_path": "/usr/local/bin/vllm",
+    "cwd": "/home/admin/cpfs/wjh/aituner/aituner",
+    "host": "127.0.0.1",
+    "port": 18082,
+    "healthcheck_path": "/v1/models",
+    "ready_timeout_s": 900,
+    "request_timeout_s": 180,
+    "launch_args": [
+      "serve",
+      "/home/admin/resource/model/464482ce/qwen3.5-27b/256k-0223-internal"
+    ],
+    "base_envs": {
+      "VLLM_DISABLE_COMPILE_CACHE": "1",
+      "DS_LLM_IGNORE_WARMUP": "1",
+      "DS_LLM_IGNORE_CHECK_WARMUP": "1",
+      "VLLM_ENABLE_MODEL_RUNNER_WARMUP": "1",
+      "VLLM_GDN_USE_FUSED_QKVZBA_KERNEL": "0",
+      "PARAM_TOTAL_MAX": "262144",
+      "PARAM_IN_LENGTH_MAX": "262144",
+      "PARAM_MAX_LENGTH_MAX": "131072",
+      "DS_LLM_MAX_THINK_TOKENS": "81920",
+      "DS_LLM_GRACEFUL_SHUTDOWN_WAIT_SECONDS": "600",
+      "VLLM_FP8_USE_BLADNN": "1",
+      "VLLM_MOE_USE_BLADNN": "1",
+      "VLLM_GDN_USE_BLADNN": "0",
+      "VLLM_USE_V1": "1",
+      "VLLM_IS_HYBRID_MODEL": "1",
+      "VLLM_ENABLE_TORCH_COMPILE": "1",
+      "VLLM_ATTENTION_BACKEND": "FLASH_ATTN",
+      "VLLM_QUANTIZE_ROUTED_EXPERTS_ONLY": "1",
+      "VLLM_USE_FLASHINFER_SAMPLER": "0",
+      "VLLM_DP_MASTER_PORT": "9528",
+      "VLLM_RESPONSE_TIMEOUT": "300",
+      "VLLM_LOG_REQ_KV_LENS": "1",
+      "DS_LLM_GRACEFUL_SHUTDOWN_KEEP_SECONDS": "600",
+      "CUDA_VISIBLE_DEVICES": "2,3,4,5,6,7"
+    },
+    "base_flags": {
+      "host": "127.0.0.1",
+      "port": 18082,
+      "served-model-name": "qwen35-27b-aituner",
+      "trust-remote-code": true,
+      "dtype": "bfloat16",
+      "gpu-memory-utilization": 0.9,
+      "enable-prefix-caching": true,
+      "mamba-cache-mode": "light",
+      "distributed-executor-backend": "mp",
+      "block-size": 64,
+      "enable-chunked-prefill": true,
+      "max-num-batched-tokens": 8192,
+      "disable-cascade-attn": true,
+      "max-model-len": 262144,
+      "speculative-config": "{\"method\":\"qwen3_next_vl_mtp\",\"num_speculative_tokens\":3}",
+      "mm-processor-cache-gb": 0,
+      "limit-mm-per-prompt": "{\"image\":256,\"video\":64}",
+      "compilation-config": "{\"cudagraph_mode\":\"FULL_AND_PIECEWISE\",\"use_inductor\":false,\"pass_config\":{\"fuse_norm_quant\":false,\"fuse_act_quant\":false,\"fuse_attn_quant\":false}}",
+      "mamba-cache-dtype": "float32",
+      "skip-mm-profiling": true,
+      "quantization": "fp8",
+      "tensor-parallel-size": 1,
+      "disable-log-requests": true
+    },
+    "tunable_envs": [
+      "VLLM_ENABLE_TORCH_COMPILE"
+    ],
+    "tunable_flags": [
+      "tensor-parallel-size",
+      "data-parallel-size",
+      "expert-parallel-size",
+      "gpu-memory-utilization",
+      "block-size",
+      "max-num-batched-tokens",
+      "max-num-seqs",
+      "enable-prefix-caching",
+      "enable-chunked-prefill"
+    ],
+    "topology_constraints": {
+      "require_tp_dp_product_equals_gpu_count": false,
+      "require_ep_size_leq_tp_dp_product": true,
+      "require_ep_size_divides_tp_dp_product": true,
+      "require_enable_expert_parallel_when_ep_gt_one": true,
+      "validate_cuda_graph_sizes_divisible_by_tp_when_tp_ep_reduce_scatter": true,
+      "allowed_tp_dp_products": [
+        1,
+        2,
+        4,
+        8
+      ],
+      "allowed_tensor_parallel_sizes": [
+        1,
+        2,
+        4,
+        8
+      ],
+      "allowed_data_parallel_sizes": [
+        1,
+        2,
+        4,
+        8
+      ],
+      "allowed_expert_parallel_sizes": [
+        1
+      ]
+    },
+    "python_executable": "python3"
+  },
+  "trace": {
+    "windows_path": "/home/admin/cpfs/wjh/aituner/aituner/trace_windows/windows.json",
+    "window_id": "chat_w20260311_1000",
+    "u_field": "sampling_u",
+    "timestamp_field": "timestamp",
+    "max_concurrency": 32,
+    "input_length_filter": {
+      "min_input_tokens": 0,
+      "max_input_tokens": 8192
+    },
+    "replay_time_scale": 1.0,
+    "early_stop_max_lag_s": 120.0,
+    "early_stop_max_elapsed_s": 900.0,
+    "adaptive_stop": {
+      "enabled": true,
+      "tau": 0.9,
+      "tau_c": 0.9,
+      "stable_checks": 3,
+      "max_checks": 20,
+      "min_fraction": 0.1,
+      "boundary_delta": 0.02
+    }
+  },
+  "slo": {
+    "target_pass_rate": 0.95,
+    "ttft_rule": {
+      "kind": "linear_ms",
+      "intercept_ms": 4000,
+      "per_token_ms": 0.125
+    },
+    "tpot_rule": {
+      "kind": "fixed_ms",
+      "threshold_ms": 50
+    }
+  },
+  "search": {
+    "low": 0.0,
+    "high": 0.25,
+    "tolerance": 0.001,
+    "max_probes": 6,
+    "sample_seed": 20260325,
+    "inherit_incumbent_floor": true
+  },
+  "llm": {
+    "system_prompt": "Propose a single engine config patch that increases the maximum feasible sampling_u under the SLO target. Favor launch-safe changes grounded in the incumbent result and only propose knobs that plausibly improve throughput above the incumbent request rate.",
+    "max_history_trials": 8,
+    "endpoint": {
+      "provider": "codex",
+      "model": "gpt-5.4",
+      "stream": true,
+      "api_key_env": "OPENAI_API_KEY",
+      "timeout_s": 180
+    }
+  }
+}

configs/examples/dash0_qwen27b_tp_ab.json

@@ -0,0 +1,176 @@
+{
+  "study_id": "dash0-qwen27b-tp-ab-chat-0-8k",
+  "hardware": {
+    "gpu_count": 8,
+    "gpu_model": "H20",
+    "host_candidates": [
+      "dash0"
+    ]
+  },
+  "model": {
+    "model_id": "qwen3.5-27b-256k-0223-internal",
+    "served_model_name": "qwen35-27b-aituner"
+  },
+  "engine": {
+    "engine_name": "vllm",
+    "engine_version": "latest-release-on-dash0",
+    "exec_path": "/usr/local/bin/vllm",
+    "cwd": "/home/admin/cpfs/wjh/aituner/aituner",
+    "host": "127.0.0.1",
+    "port": 18082,
+    "healthcheck_path": "/v1/models",
+    "ready_timeout_s": 900,
+    "request_timeout_s": 180,
+    "launch_args": [
+      "serve",
+      "/home/admin/resource/model/464482ce/qwen3.5-27b/256k-0223-internal"
+    ],
+    "base_envs": {
+      "VLLM_DISABLE_COMPILE_CACHE": "1",
+      "DS_LLM_IGNORE_WARMUP": "1",
+      "DS_LLM_IGNORE_CHECK_WARMUP": "1",
+      "VLLM_ENABLE_MODEL_RUNNER_WARMUP": "1",
+      "VLLM_GDN_USE_FUSED_QKVZBA_KERNEL": "0",
+      "PARAM_TOTAL_MAX": "262144",
+      "PARAM_IN_LENGTH_MAX": "262144",
+      "PARAM_MAX_LENGTH_MAX": "131072",
+      "DS_LLM_MAX_THINK_TOKENS": "81920",
+      "DS_LLM_GRACEFUL_SHUTDOWN_WAIT_SECONDS": "600",
+      "VLLM_FP8_USE_BLADNN": "1",
+      "VLLM_MOE_USE_BLADNN": "1",
+      "VLLM_GDN_USE_BLADNN": "0",
+      "VLLM_USE_V1": "1",
+      "VLLM_IS_HYBRID_MODEL": "1",
+      "VLLM_ENABLE_TORCH_COMPILE": "1",
+      "VLLM_ATTENTION_BACKEND": "FLASH_ATTN",
+      "VLLM_QUANTIZE_ROUTED_EXPERTS_ONLY": "1",
+      "VLLM_USE_FLASHINFER_SAMPLER": "0",
+      "VLLM_DP_MASTER_PORT": "9528",
+      "VLLM_RESPONSE_TIMEOUT": "300",
+      "VLLM_LOG_REQ_KV_LENS": "1",
+      "DS_LLM_GRACEFUL_SHUTDOWN_KEEP_SECONDS": "600",
+      "CUDA_VISIBLE_DEVICES": "2,3,4,5,6,7"
+    },
+    "base_flags": {
+      "host": "127.0.0.1",
+      "port": 18082,
+      "served-model-name": "qwen35-27b-aituner",
+      "trust-remote-code": true,
+      "dtype": "bfloat16",
+      "gpu-memory-utilization": 0.9,
+      "enable-prefix-caching": true,
+      "mamba-cache-mode": "light",
+      "distributed-executor-backend": "mp",
+      "block-size": 64,
+      "enable-chunked-prefill": true,
+      "max-num-batched-tokens": 8192,
+      "disable-cascade-attn": true,
+      "max-model-len": 262144,
+      "speculative-config": "{\"method\":\"qwen3_next_vl_mtp\",\"num_speculative_tokens\":3}",
+      "mm-processor-cache-gb": 0,
+      "limit-mm-per-prompt": "{\"image\":256,\"video\":64}",
+      "compilation-config": "{\"cudagraph_mode\":\"FULL_AND_PIECEWISE\",\"use_inductor\":false,\"pass_config\":{\"fuse_norm_quant\":false,\"fuse_act_quant\":false,\"fuse_attn_quant\":false}}",
+      "mamba-cache-dtype": "float32",
+      "skip-mm-profiling": true,
+      "quantization": "fp8",
+      "tensor-parallel-size": 1,
+      "disable-log-requests": true
+    },
+    "tunable_envs": [
+      "VLLM_ENABLE_TORCH_COMPILE"
+    ],
+    "tunable_flags": [
+      "tensor-parallel-size",
+      "data-parallel-size",
+      "expert-parallel-size",
+      "gpu-memory-utilization",
+      "block-size",
+      "max-num-batched-tokens",
+      "max-num-seqs",
+      "enable-prefix-caching",
+      "enable-chunked-prefill"
+    ],
+    "topology_constraints": {
+      "require_tp_dp_product_equals_gpu_count": false,
+      "require_ep_size_leq_tp_dp_product": true,
+      "require_ep_size_divides_tp_dp_product": true,
+      "require_enable_expert_parallel_when_ep_gt_one": true,
+      "validate_cuda_graph_sizes_divisible_by_tp_when_tp_ep_reduce_scatter": true,
+      "allowed_tp_dp_products": [
+        1,
+        2,
+        4,
+        8
+      ],
+      "allowed_tensor_parallel_sizes": [
+        1,
+        2,
+        4,
+        8
+      ],
+      "allowed_data_parallel_sizes": [
+        1,
+        2,
+        4,
+        8
+      ],
+      "allowed_expert_parallel_sizes": [
+        1
+      ]
+    },
+    "python_executable": "python3"
+  },
+  "trace": {
+    "windows_path": "/home/admin/cpfs/wjh/aituner/aituner/trace_windows/windows.json",
+    "window_id": "chat_w20260311_1000",
+    "u_field": "sampling_u",
+    "timestamp_field": "timestamp",
+    "max_concurrency": 32,
+    "input_length_filter": {
+      "min_input_tokens": 0,
+      "max_input_tokens": 8192
+    },
+    "replay_time_scale": 1.0,
+    "early_stop_max_lag_s": 120.0,
+    "early_stop_max_elapsed_s": 900.0,
+    "adaptive_stop": {
+      "enabled": true,
+      "tau": 0.9,
+      "tau_c": 0.9,
+      "stable_checks": 3,
+      "max_checks": 20,
+      "min_fraction": 0.1,
+      "boundary_delta": 0.02
+    }
+  },
+  "slo": {
+    "target_pass_rate": 0.95,
+    "ttft_rule": {
+      "kind": "linear_ms",
+      "intercept_ms": 4000,
+      "per_token_ms": 0.125
+    },
+    "tpot_rule": {
+      "kind": "fixed_ms",
+      "threshold_ms": 50
+    }
+  },
+  "search": {
+    "low": 0.0,
+    "high": 0.125,
+    "tolerance": 0.001,
+    "max_probes": 6,
+    "sample_seed": 20260325
+  },
+  "llm": {
+    "system_prompt": "Propose a single engine config patch that increases the maximum feasible sampling_u under the SLO target. Favor launch-safe changes grounded in the incumbent result and only propose knobs that plausibly improve throughput above the incumbent request rate.",
+    "max_history_trials": 8,
+    "endpoint": {
+      "provider": "codex",
+      "model": "gpt-5.4",
+      "stream": true,
+      "api_key_env": "OPENAI_API_KEY",
+      "timeout_s": 180
+    }
+  }
+}

configs/examples/dash0_qwen30b_a3b_community_vllm020_harness.json

@@ -0,0 +1,122 @@
+{
+  "study_id": "dash0-qwen30b-a3b-community-vllm020-chat-0-8k-out128-scale01-harness",
+  "hardware": {
+    "gpu_count": 8,
+    "gpu_model": "H20",
+    "host_candidates": [
+      "dash0"
+    ]
+  },
+  "model": {
+    "model_id": "Qwen/Qwen3-30B-A3B",
+    "served_model_name": "qwen3-30b-a3b-community"
+  },
+  "engine": {
+    "engine_name": "vllm",
+    "engine_version": "0.20.0",
+    "exec_path": "/tmp/wjh/venvs/vllm-0.20.0-cu129/bin/vllm",
+    "cwd": "/home/admin/cpfs/wjh/aituner/aituner",
+    "host": "127.0.0.1",
+    "port": 18230,
+    "healthcheck_path": "/v1/models",
+    "ready_timeout_s": 900,
+    "request_timeout_s": 900,
+    "launch_args": [
+      "serve",
+      "/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B"
+    ],
+    "base_envs": {
+      "CUDA_VISIBLE_DEVICES": "0,1,2,3,4,5,6,7",
+      "HOME": "/tmp/wjh",
+      "XDG_CACHE_HOME": "/tmp/wjh/.cache"
+    },
+    "base_flags": {
+      "host": "127.0.0.1",
+      "port": 18230,
+      "served-model-name": "qwen3-30b-a3b-community"
+    },
+    "tunable_envs": [],
+    "tunable_flags": [
+      "tensor-parallel-size",
+      "data-parallel-size",
+      "enable-expert-parallel",
+      "expert-parallel-size",
+      "gpu-memory-utilization",
+      "max-num-batched-tokens",
+      "max-num-seqs",
+      "block-size",
+      "enable-prefix-caching",
+      "enable-chunked-prefill"
+    ],
+    "topology_constraints": {
+      "require_tp_dp_product_equals_gpu_count": false,
+      "require_ep_size_leq_tp_dp_product": true,
+      "require_ep_size_divides_tp_dp_product": true,
+      "require_enable_expert_parallel_when_ep_gt_one": true,
+      "validate_cuda_graph_sizes_divisible_by_tp_when_tp_ep_reduce_scatter": true,
+      "allowed_tp_dp_products": [1, 2, 4, 8],
+      "allowed_tensor_parallel_sizes": [1, 2, 4, 8],
+      "allowed_data_parallel_sizes": [1, 2, 4, 8],
+      "allowed_expert_parallel_sizes": [1, 2, 4, 8]
+    },
+    "python_executable": "/tmp/wjh/venvs/vllm-0.20.0-cu129/bin/python"
+  },
+  "trace": {
+    "windows_path": "/home/admin/cpfs/wjh/aituner/aituner/trace_windows/windows.json",
+    "window_id": "chat_w20260311_1000",
+    "completion_tokens_override": 128,
+    "u_field": "sampling_u",
+    "timestamp_field": "timestamp",
+    "max_concurrency": 64,
+    "input_length_filter": {
+      "min_input_tokens": 0,
+      "max_input_tokens": 8192
+    },
+    "max_requests_per_probe": 512,
+    "replay_time_scale": 0.1,
+    "early_stop_max_lag_s": 120.0,
+    "early_stop_max_elapsed_s": 900.0
+  },
+  "slo": {
+    "target_pass_rate": 0.95,
+    "ttft_rule": {
+      "kind": "step_ms",
+      "buckets": [
+        {
+          "max_input_tokens": 4096,
+          "threshold_ms": 2000
+        },
+        {
+          "max_input_tokens": 32768,
+          "threshold_ms": 4000
+        },
+        {
+          "threshold_ms": 6000
+        }
+      ]
+    },
+    "tpot_rule": {
+      "kind": "fixed_ms",
+      "threshold_ms": 50
+    }
+  },
+  "search": {
+    "low": 0.0,
+    "high": 0.125,
+    "tolerance": 0.001,
+    "max_probes": 4,
+    "sample_seed": 20260325
+  },
+  "llm": {
+    "system_prompt": "Tune community vLLM 0.20.0 serving for Qwen3-30B-A3B. Start from the default vLLM engine configuration, use only launch-safe patches, and optimize request_rate_per_gpu under the configured SLO.",
+    "max_history_trials": 8,
+    "use_harness": true,
+    "endpoint": {
+      "provider": "codex",
+      "model": "gpt-5.4",
+      "stream": true,
+      "api_key_env": "OPENAI_API_KEY",
+      "timeout_s": 240
+    }
+  }
+}

configs/examples/dash0_qwen30b_a3b_community_vllm020_noharness.json

@@ -0,0 +1,122 @@
+{
+  "study_id": "dash0-qwen30b-a3b-community-vllm020-chat-0-8k-out128-scale01-noharness",
+  "hardware": {
+    "gpu_count": 8,
+    "gpu_model": "H20",
+    "host_candidates": [
+      "dash0"
+    ]
+  },
+  "model": {
+    "model_id": "Qwen/Qwen3-30B-A3B",
+    "served_model_name": "qwen3-30b-a3b-community"
+  },
+  "engine": {
+    "engine_name": "vllm",
+    "engine_version": "0.20.0",
+    "exec_path": "/tmp/wjh/venvs/vllm-0.20.0-cu129/bin/vllm",
+    "cwd": "/home/admin/cpfs/wjh/aituner/aituner",
+    "host": "127.0.0.1",
+    "port": 18231,
+    "healthcheck_path": "/v1/models",
+    "ready_timeout_s": 900,
+    "request_timeout_s": 900,
+    "launch_args": [
+      "serve",
+      "/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B"
+    ],
+    "base_envs": {
+      "CUDA_VISIBLE_DEVICES": "0,1,2,3,4,5,6,7",
+      "HOME": "/tmp/wjh",
+      "XDG_CACHE_HOME": "/tmp/wjh/.cache"
+    },
+    "base_flags": {
+      "host": "127.0.0.1",
+      "port": 18231,
+      "served-model-name": "qwen3-30b-a3b-community"
+    },
+    "tunable_envs": [],
+    "tunable_flags": [
+      "tensor-parallel-size",
+      "data-parallel-size",
+      "enable-expert-parallel",
+      "expert-parallel-size",
+      "gpu-memory-utilization",
+      "max-num-batched-tokens",
+      "max-num-seqs",
+      "block-size",
+      "enable-prefix-caching",
+      "enable-chunked-prefill"
+    ],
+    "topology_constraints": {
+      "require_tp_dp_product_equals_gpu_count": false,
+      "require_ep_size_leq_tp_dp_product": true,
+      "require_ep_size_divides_tp_dp_product": true,
+      "require_enable_expert_parallel_when_ep_gt_one": true,
+      "validate_cuda_graph_sizes_divisible_by_tp_when_tp_ep_reduce_scatter": true,
+      "allowed_tp_dp_products": [1, 2, 4, 8],
+      "allowed_tensor_parallel_sizes": [1, 2, 4, 8],
+      "allowed_data_parallel_sizes": [1, 2, 4, 8],
+      "allowed_expert_parallel_sizes": [1, 2, 4, 8]
+    },
+    "python_executable": "/tmp/wjh/venvs/vllm-0.20.0-cu129/bin/python"
+  },
+  "trace": {
+    "windows_path": "/home/admin/cpfs/wjh/aituner/aituner/trace_windows/windows.json",
+    "window_id": "chat_w20260311_1000",
+    "completion_tokens_override": 128,
+    "u_field": "sampling_u",
+    "timestamp_field": "timestamp",
+    "max_concurrency": 64,
+    "input_length_filter": {
+      "min_input_tokens": 0,
+      "max_input_tokens": 8192
+    },
+    "max_requests_per_probe": 512,
+    "replay_time_scale": 0.1,
+    "early_stop_max_lag_s": 120.0,
+    "early_stop_max_elapsed_s": 900.0
+  },
+  "slo": {
+    "target_pass_rate": 0.95,
+    "ttft_rule": {
+      "kind": "step_ms",
+      "buckets": [
+        {
+          "max_input_tokens": 4096,
+          "threshold_ms": 2000
+        },
+        {
+          "max_input_tokens": 32768,
+          "threshold_ms": 4000
+        },
+        {
+          "threshold_ms": 6000
+        }
+      ]
+    },
+    "tpot_rule": {
+      "kind": "fixed_ms",
+      "threshold_ms": 50
+    }
+  },
+  "search": {
+    "low": 0.0,
+    "high": 0.125,
+    "tolerance": 0.001,
+    "max_probes": 4,
+    "sample_seed": 20260325
+  },
+  "llm": {
+    "system_prompt": "Tune community vLLM 0.20.0 serving for Qwen3-30B-A3B. Start from the default vLLM engine configuration, use only launch-safe patches, and optimize request_rate_per_gpu under the configured SLO.",
+    "max_history_trials": 8,
+    "use_harness": false,
+    "endpoint": {
+      "provider": "codex",
+      "model": "gpt-5.4",
+      "stream": true,
+      "api_key_env": "OPENAI_API_KEY",
+      "timeout_s": 240
+    }
+  }
+}

configs/examples/dash0_qwen30b_a3b_stopA_fulldata.json

@@ -0,0 +1,138 @@
+{
+  "study_id": "dash0-qwen30b-a3b-stopA-fulldata-chat-0-8k",
+  "hardware": {
+    "gpu_count": 8,
+    "gpu_model": "H20",
+    "host_candidates": [
+      "dash0"
+    ]
+  },
+  "model": {
+    "model_id": "Qwen/Qwen3-30B-A3B",
+    "served_model_name": "qwen3-30b-a3b-community"
+  },
+  "engine": {
+    "engine_name": "vllm",
+    "engine_version": "0.20.0",
+    "exec_path": "/usr/local/bin/vllm",
+    "cwd": "/home/admin/cpfs/wjh/aituner/aituner",
+    "host": "127.0.0.1",
+    "port": 18230,
+    "healthcheck_path": "/v1/models",
+    "ready_timeout_s": 900,
+    "request_timeout_s": 900,
+    "launch_args": [
+      "serve",
+      "/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B"
+    ],
+    "base_envs": {
+      "CUDA_VISIBLE_DEVICES": "0,1,2,3,4,5,6,7",
+      "HOME": "/tmp/wjh",
+      "XDG_CACHE_HOME": "/tmp/wjh/.cache"
+    },
+    "base_flags": {
+      "host": "127.0.0.1",
+      "port": 18230,
+      "served-model-name": "qwen3-30b-a3b-community",
+      "gpu-memory-utilization": 0.9,
+      "max-model-len": 16384,
+      "trust-remote-code": true,
+      "enable-prefix-caching": true
+    },
+    "tunable_envs": [],
+    "tunable_flags": [
+      "tensor-parallel-size",
+      "data-parallel-size",
+      "enable-expert-parallel",
+      "expert-parallel-size",
+      "gpu-memory-utilization",
+      "max-num-batched-tokens",
+      "max-num-seqs",
+      "block-size",
+      "enable-prefix-caching",
+      "enable-chunked-prefill"
+    ],
+    "topology_constraints": {
+      "require_tp_dp_product_equals_gpu_count": false,
+      "require_ep_size_leq_tp_dp_product": true,
+      "require_ep_size_divides_tp_dp_product": true,
+      "require_enable_expert_parallel_when_ep_gt_one": true,
+      "validate_cuda_graph_sizes_divisible_by_tp_when_tp_ep_reduce_scatter": true,
+      "allowed_tp_dp_products": [
+        1,
+        2,
+        4,
+        8
+      ],
+      "allowed_tensor_parallel_sizes": [
+        1,
+        2,
+        4,
+        8
+      ],
+      "allowed_data_parallel_sizes": [
+        1,
+        2,
+        4,
+        8
+      ],
+      "allowed_expert_parallel_sizes": [
+        1,
+        2,
+        4,
+        8
+      ]
+    },
+    "python_executable": "/tmp/wjh/venvs/vllm-0.20.0-cu129/bin/python"
+  },
+  "trace": {
+    "windows_path": "/home/admin/cpfs/wjh/aituner/aituner/trace_windows/windows.json",
+    "window_id": "chat_w20260311_1000",
+    "completion_tokens_override": 128,
+    "u_field": "sampling_u",
+    "timestamp_field": "timestamp",
+    "max_concurrency": 64,
+    "input_length_filter": {
+      "min_input_tokens": 0,
+      "max_input_tokens": 8192
+    },
+    "replay_time_scale": 1.0,
+    "early_stop_max_lag_s": 120.0,
+    "early_stop_max_elapsed_s": 900.0
+  },
+  "slo": {
+    "target_pass_rate": 0.95,
+    "ttft_rule": {
+      "kind": "step_ms",
+      "buckets": [
+        {
+          "max_input_tokens": 4096,
+          "threshold_ms": 2000
+        },
+        {
+          "max_input_tokens": 32768,
+          "threshold_ms": 4000
+        },
+        {
+          "threshold_ms": 6000
+        }
+      ]
+    },
+    "tpot_rule": {
+      "kind": "fixed_ms",
+      "threshold_ms": 50
+    }
+  },
+  "search": {
+    "low": 0.0,
+    "high": 0.125,
+    "tolerance": 0.001,
+    "max_probes": 4,
+    "sample_seed": 20260325
+  },
+  "llm": {
+    "system_prompt": "Tune community vLLM 0.20.0 serving for Qwen3-30B-A3B. Start from the default vLLM engine configuration, use only launch-safe patches, and optimize request_rate_per_gpu under the configured SLO.",
+    "max_history_trials": 8,
+    "use_harness": false
+  }
+}

configs/examples/dash0_qwen30b_a3b_stopA_on.json

@@ -0,0 +1,147 @@
+{
+  "study_id": "dash0-qwen30b-a3b-stopA-on-chat-0-8k",
+  "hardware": {
+    "gpu_count": 8,
+    "gpu_model": "H20",
+    "host_candidates": [
+      "dash0"
+    ]
+  },
+  "model": {
+    "model_id": "Qwen/Qwen3-30B-A3B",
+    "served_model_name": "qwen3-30b-a3b-community"
+  },
+  "engine": {
+    "engine_name": "vllm",
+    "engine_version": "0.20.0",
+    "exec_path": "/usr/local/bin/vllm",
+    "cwd": "/home/admin/cpfs/wjh/aituner/aituner",
+    "host": "127.0.0.1",
+    "port": 18230,
+    "healthcheck_path": "/v1/models",
+    "ready_timeout_s": 900,
+    "request_timeout_s": 900,
+    "launch_args": [
+      "serve",
+      "/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B"
+    ],
+    "base_envs": {
+      "CUDA_VISIBLE_DEVICES": "0,1,2,3,4,5,6,7",
+      "HOME": "/tmp/wjh",
+      "XDG_CACHE_HOME": "/tmp/wjh/.cache"
+    },
+    "base_flags": {
+      "host": "127.0.0.1",
+      "port": 18230,
+      "served-model-name": "qwen3-30b-a3b-community",
+      "gpu-memory-utilization": 0.9,
+      "max-model-len": 16384,
+      "trust-remote-code": true,
+      "enable-prefix-caching": true
+    },
+    "tunable_envs": [],
+    "tunable_flags": [
+      "tensor-parallel-size",
+      "data-parallel-size",
+      "enable-expert-parallel",
+      "expert-parallel-size",
+      "gpu-memory-utilization",
+      "max-num-batched-tokens",
+      "max-num-seqs",
+      "block-size",
+      "enable-prefix-caching",
+      "enable-chunked-prefill"
+    ],
+    "topology_constraints": {
+      "require_tp_dp_product_equals_gpu_count": false,
+      "require_ep_size_leq_tp_dp_product": true,
+      "require_ep_size_divides_tp_dp_product": true,
+      "require_enable_expert_parallel_when_ep_gt_one": true,
+      "validate_cuda_graph_sizes_divisible_by_tp_when_tp_ep_reduce_scatter": true,
+      "allowed_tp_dp_products": [
+        1,
+        2,
+        4,
+        8
+      ],
+      "allowed_tensor_parallel_sizes": [
+        1,
+        2,
+        4,
+        8
+      ],
+      "allowed_data_parallel_sizes": [
+        1,
+        2,
+        4,
+        8
+      ],
+      "allowed_expert_parallel_sizes": [
+        1,
+        2,
+        4,
+        8
+      ]
+    },
+    "python_executable": "/tmp/wjh/venvs/vllm-0.20.0-cu129/bin/python"
+  },
+  "trace": {
+    "windows_path": "/home/admin/cpfs/wjh/aituner/aituner/trace_windows/windows.json",
+    "window_id": "chat_w20260311_1000",
+    "completion_tokens_override": 128,
+    "u_field": "sampling_u",
+    "timestamp_field": "timestamp",
+    "max_concurrency": 64,
+    "input_length_filter": {
+      "min_input_tokens": 0,
+      "max_input_tokens": 8192
+    },
+    "replay_time_scale": 1.0,
+    "early_stop_max_lag_s": 120.0,
+    "early_stop_max_elapsed_s": 900.0,
+    "adaptive_stop": {
+      "enabled": true,
+      "tau": 0.9,
+      "tau_c": 0.9,
+      "stable_checks": 3,
+      "max_checks": 20,
+      "min_fraction": 0.1,
+      "boundary_delta": 0.02
+    }
+  },
+  "slo": {
+    "target_pass_rate": 0.95,
+    "ttft_rule": {
+      "kind": "step_ms",
+      "buckets": [
+        {
+          "max_input_tokens": 4096,
+          "threshold_ms": 2000
+        },
+        {
+          "max_input_tokens": 32768,
+          "threshold_ms": 4000
+        },
+        {
+          "threshold_ms": 6000
+        }
+      ]
+    },
+    "tpot_rule": {
+      "kind": "fixed_ms",
+      "threshold_ms": 50
+    }
+  },
+  "search": {
+    "low": 0.0,
+    "high": 0.125,
+    "tolerance": 0.001,
+    "max_probes": 4,
+    "sample_seed": 20260325
+  },
+  "llm": {
+    "system_prompt": "Tune community vLLM 0.20.0 serving for Qwen3-30B-A3B. Start from the default vLLM engine configuration, use only launch-safe patches, and optimize request_rate_per_gpu under the configured SLO.",
+    "max_history_trials": 8,
+    "use_harness": false
+  }
+}

configs/examples/dash0_qwen30b_a3b_stopB_e2e.json

@@ -0,0 +1,155 @@
+{
+  "study_id": "dash0-qwen30b-a3b-stopB-e2e-hi-chat-0-8k",
+  "hardware": {
+    "gpu_count": 8,
+    "gpu_model": "H20",
+    "host_candidates": [
+      "dash0"
+    ]
+  },
+  "model": {
+    "model_id": "Qwen/Qwen3-30B-A3B",
+    "served_model_name": "qwen3-30b-a3b-community"
+  },
+  "engine": {
+    "engine_name": "vllm",
+    "engine_version": "0.20.0",
+    "exec_path": "/usr/local/bin/vllm",
+    "cwd": "/home/admin/cpfs/wjh/aituner/aituner",
+    "host": "127.0.0.1",
+    "port": 18230,
+    "healthcheck_path": "/v1/models",
+    "ready_timeout_s": 900,
+    "request_timeout_s": 900,
+    "launch_args": [
+      "serve",
+      "/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B"
+    ],
+    "base_envs": {
+      "CUDA_VISIBLE_DEVICES": "0,1,2,3,4,5,6,7",
+      "HOME": "/tmp/wjh",
+      "XDG_CACHE_HOME": "/tmp/wjh/.cache"
+    },
+    "base_flags": {
+      "host": "127.0.0.1",
+      "port": 18230,
+      "served-model-name": "qwen3-30b-a3b-community",
+      "gpu-memory-utilization": 0.9,
+      "max-model-len": 16384,
+      "trust-remote-code": true,
+      "enable-prefix-caching": true
+    },
+    "tunable_envs": [],
+    "tunable_flags": [
+      "tensor-parallel-size",
+      "data-parallel-size",
+      "enable-expert-parallel",
+      "expert-parallel-size",
+      "gpu-memory-utilization",
+      "max-num-batched-tokens",
+      "max-num-seqs",
+      "block-size",
+      "enable-prefix-caching",
+      "enable-chunked-prefill"
+    ],
+    "topology_constraints": {
+      "require_tp_dp_product_equals_gpu_count": false,
+      "require_ep_size_leq_tp_dp_product": true,
+      "require_ep_size_divides_tp_dp_product": true,
+      "require_enable_expert_parallel_when_ep_gt_one": true,
+      "validate_cuda_graph_sizes_divisible_by_tp_when_tp_ep_reduce_scatter": true,
+      "allowed_tp_dp_products": [
+        1,
+        2,
+        4,
+        8
+      ],
+      "allowed_tensor_parallel_sizes": [
+        1,
+        2,
+        4,
+        8
+      ],
+      "allowed_data_parallel_sizes": [
+        1,
+        2,
+        4,
+        8
+      ],
+      "allowed_expert_parallel_sizes": [
+        1,
+        2,
+        4,
+        8
+      ]
+    },
+    "python_executable": "/tmp/wjh/venvs/vllm-0.20.0-cu129/bin/python"
+  },
+  "trace": {
+    "windows_path": "/home/admin/cpfs/wjh/aituner/aituner/trace_windows/windows.json",
+    "window_id": "chat_w20260311_1000",
+    "completion_tokens_override": 128,
+    "u_field": "sampling_u",
+    "timestamp_field": "timestamp",
+    "max_concurrency": 64,
+    "input_length_filter": {
+      "min_input_tokens": 0,
+      "max_input_tokens": 8192
+    },
+    "max_requests_per_probe": 512,
+    "replay_time_scale": 0.1,
+    "early_stop_max_lag_s": 120.0,
+    "early_stop_max_elapsed_s": 900.0,
+    "adaptive_stop": {
+      "enabled": true,
+      "tau": 0.9,
+      "tau_c": 0.9,
+      "stable_checks": 3,
+      "max_checks": 20,
+      "min_fraction": 0.1,
+      "boundary_delta": 0.02
+    }
+  },
+  "slo": {
+    "target_pass_rate": 0.95,
+    "ttft_rule": {
+      "kind": "step_ms",
+      "buckets": [
+        {
+          "max_input_tokens": 4096,
+          "threshold_ms": 2000
+        },
+        {
+          "max_input_tokens": 32768,
+          "threshold_ms": 4000
+        },
+        {
+          "threshold_ms": 6000
+        }
+      ]
+    },
+    "tpot_rule": {
+      "kind": "fixed_ms",
+      "threshold_ms": 50
+    }
+  },
+  "search": {
+    "low": 0.0,
+    "high": 1.0,
+    "tolerance": 0.001,
+    "max_probes": 6,
+    "sample_seed": 20260325
+  },
+  "llm": {
+    "system_prompt": "Tune community vLLM 0.20.0 serving for Qwen3-30B-A3B. Start from the default vLLM engine configuration, use only launch-safe patches, and optimize request_rate_per_gpu under the configured SLO.",
+    "max_history_trials": 8,
+    "use_harness": true,
+    "endpoint": {
+      "provider": "codex",
+      "model": "gpt-5.4",
+      "stream": true,
+      "api_key_env": "OPENAI_API_KEY",
+      "timeout_s": 240
+    }
+  }
+}

configs/examples/dash1_qwen235b_prefill_thinking_7day_compare.json

@@ -0,0 +1,52 @@
+{
+  "compare_id": "dash1-qwen235b-prefill-thinking-7day-baseline-vs-0323-vs-0327",
+  "study_spec_path": "dash0_qwen235b_prefill_thinking_run3_ttft_tight_0323.json",
+  "output_root": "../../.aituner-compare/dash1-qwen235b-prefill-thinking-7day-baseline-vs-0323-vs-0327",
+  "window_ids": [
+    "thinking_w20260321_1000",
+    "thinking_w20260322_1000",
+    "thinking_w20260323_1000",
+    "thinking_w20260324_1000",
+    "thinking_w20260325_1000",
+    "thinking_w20260326_1000",
+    "thinking_w20260327_1000"
+  ],
+  "candidates": [
+    {
+      "name": "baseline",
+      "phase": 1,
+      "config_patch": {
+        "env_patch": {},
+        "flag_patch": {}
+      },
+      "runtime": {
+        "cuda_visible_devices": "0,1,2,3",
+        "port": 18141
+      }
+    },
+    {
+      "name": "tuned_0323",
+      "phase": 1,
+      "trial_ref": {
+        "study_root": "../../.aituner-prefill/dash0-qwen235b-prefill-thinking-run3-ttft-tight-0323-topology",
+        "trial_id": "trial-0006"
+      },
+      "runtime": {
+        "cuda_visible_devices": "4,5,6,7",
+        "port": 18142
+      }
+    },
+    {
+      "name": "tuned_0327",
+      "phase": 2,
+      "trial_ref": {
+        "study_root": "../../.aituner-prefill/dash0-qwen235b-prefill-thinking-run2-ttft-tight-topology",
+        "trial_id": "trial-0012"
+      },
+      "runtime": {
+        "cuda_visible_devices": "0,1,2,3,4,5,6,7",
+        "port": 18143
+      }
+    }
+  ]
+}

configs/examples/dash1_qwen235b_prefill_thinking_run4_ttft_tight_0_32k.json

@@ -0,0 +1,212 @@
+{
+  "study_id": "dash1-qwen235b-prefill-thinking-run4-ttft-tight-0-32k-topology",
+  "hardware": {
+    "gpu_count": 8,
+    "gpu_model": "H20",
+    "host_candidates": [
+      "dash1"
+    ]
+  },
+  "model": {
+    "model_id": "qwen3-235b-a22b-256k-0717-internal",
+    "served_model_name": "qwen3-235b-prefill"
+  },
+  "engine": {
+    "engine_name": "vllm",
+    "engine_version": "internal-on-dash1",
+    "exec_path": "/usr/local/bin/vllm",
+    "cwd": "/home/admin/cpfs/wjh/aituner/aituner",
+    "host": "127.0.0.1",
+    "port": 18145,
+    "healthcheck_path": "/v1/models",
+    "ready_timeout_s": 1800,
+    "request_timeout_s": 1800,
+    "launch_args": [
+      "serve",
+      "/home/admin/resource/model/464482ce.qwen3-235b-a22b/256k-0717"
+    ],
+    "base_envs": {
+      "CUDA_VISIBLE_DEVICES": "0,1,2,3,4,5,6,7",
+      "VLLM_USE_V1": "1",
+      "VLLM_ATTENTION_BACKEND": "FLASH_ATTN",
+      "VLLM_QUANTIZATION_LAYER_WISE": "1",
+      "VLLM_MOE_USE_DEEPEP": "0",
+      "VLLM_MOE_BALANCED_GATING": "0",
+      "VLLM_MOE_RANDOM_GATING": "0",
+      "VLLM_FUSED_MOE_CHUNK_SIZE": "4096",
+      "VLLM_DP_META_USE_CPU_GROUP": "0",
+      "VLLM_MLA_FP8_ATTENTION": "0",
+      "VLLM_MOE_EXPERTS_OVERLAP": "0",
+      "VLLM_USE_FLASHINFER_SAMPLER": "0",
+      "VLLM_RESPONSE_TIMEOUT": "290",
+      "VLLM_FP8_USE_BLADNN": "1",
+      "VLLM_MOE_USE_BLADNN": "1",
+      "VLLM_USE_DEEP_GEMM": "0",
+      "VLLM_PD_TRY_CONNECT_TIMEOUT_SECONDS": "120",
+      "VLLM_DEEP_GEMM_WARMUP": "skip",
+      "DEEPEP_LL_COMBINE_USE_FP8": "1",
+      "DEEPEP_LL_BUFFER_FP8_OPT": "1",
+      "DEEPEP_LL_DISPATCH_USE_NVL": "1",
+      "DEEPEP_LL_COMBINE_USE_NVL": "1",
+      "ACCL_LOW_LATENCY_OPTIMIZE": "2",
+      "ACCL_WRITEBATCH_OPT": "2",
+      "ACCL_IBV_MTU": "9000",
+      "ACCL_TX_DEPTH": "1024",
+      "ACCL_RETRANSMIT_TIMEOUT": "17",
+      "NVSHMEM_IBGDA_NUM_RC_PER_PE": "4",
+      "BLLM_KVTRANS_RDMA_SP": "2",
+      "NCCL_SOCKET_IFNAME": "eth1",
+      "NVSHMEM_BOOTSTRAP_UID_SOCK_IFNAME": "eth1",
+      "GLOO_SOCKET_IFNAME": "eth1"
+    },
+    "base_flags": {
+      "host": "127.0.0.1",
+      "port": 18145,
+      "served-model-name": "qwen3-235b-prefill",
+      "tensor-parallel-size": 4,
+      "gpu-memory-utilization": 0.85,
+      "enable-prefix-caching": true,
+      "enable-chunked-prefill": true,
+      "max-num-batched-tokens": 8192,
+      "disable-hybrid-kv-cache-manager": true,
+      "max-model-len": 262144,
+      "block-size": 64,
+      "max-num-seqs": 64,
+      "quantization": "fp8",
+      "cuda-graph-sizes": [
+        16,
+        32,
+        64,
+        96,
+        128,
+        160,
+        192,
+        224,
+        256,
+        288,
+        320,
+        352,
+        384,
+        416,
+        448,
+        480,
+        512,
+        544,
+        576,
+        608,
+        640,
+        672,
+        704,
+        736,
+        768,
+        800,
+        832,
+        864,
+        896,
+        928,
+        960,
+        992,
+        1024
+      ],
+      "compilation-config": "{\"cudagraph_mode\":\"PIECEWISE\",\"use_inductor\":false,\"custom_ops\":[\"all\"],\"max_cudagraph_capture_size\":2048}",
+      "speculative-config": "{\"method\":\"eagle3\",\"num_speculative_tokens\":1,\"hf_overrides\":{\"rope_scaling\":{\"type\":\"yarn\",\"factor\":128,\"original_max_position_embeddings\":2048,\"semi_dynamic\":false,\"dynamic\":true},\"num_experts\":0},\"model\":\"/home/admin/resource/model/464482ce.qwen3-235b-a22b/0717-eagle-0820\"}",
+      "hf-overrides": "{\"architectures\":[\"Qwen3MoeForCausalLM\"],\"model_type\":\"qwen3_moe\"}",
+      "kv-cache-dtype": "fp8",
+      "disable-log-requests": true
+    },
+    "tunable_envs": [
+      "VLLM_ENABLE_TORCH_COMPILE"
+    ],
+    "tunable_flags": [
+      "tensor-parallel-size",
+      "data-parallel-size",
+      "enable-expert-parallel",
+      "expert-parallel-size",
+      "gpu-memory-utilization",
+      "max-num-batched-tokens",
+      "max-num-seqs",
+      "block-size",
+      "enable-prefix-caching",
+      "enable-chunked-prefill"
+    ],
+    "topology_constraints": {
+      "require_tp_dp_product_equals_gpu_count": false,
+      "require_ep_size_leq_tp_dp_product": true,
+      "require_ep_size_divides_tp_dp_product": true,
+      "require_enable_expert_parallel_when_ep_gt_one": true,
+      "validate_cuda_graph_sizes_divisible_by_tp_when_tp_ep_reduce_scatter": true,
+      "allowed_tp_dp_products": [
+        4,
+        8
+      ],
+      "allowed_tensor_parallel_sizes": [
+        4,
+        8
+      ],
+      "allowed_data_parallel_sizes": [
+        1,
+        2
+      ],
+      "allowed_expert_parallel_sizes": [
+        1,
+        2,
+        4,
+        8
+      ]
+    },
+    "python_executable": "python3"
+  },
+  "trace": {
+    "windows_path": "/home/admin/cpfs/wjh/aituner/aituner/trace_windows/windows.json",
+    "window_id": "thinking_w20260327_1000",
+    "request_mode": "chat",
+    "completion_tokens_override": 1,
+    "u_field": "sampling_u",
+    "timestamp_field": "timestamp",
+    "max_concurrency": 64,
+    "input_length_filter": {
+      "min_input_tokens": 0,
+      "max_input_tokens": 32768
+    },
+    "replay_time_scale": 1.0,
+    "early_stop_max_lag_s": 180.0,
+    "early_stop_max_elapsed_s": 1200.0
+  },
+  "slo": {
+    "target_pass_rate": 0.95,
+    "ttft_rule": {
+      "kind": "step_ms",
+      "buckets": [
+        {
+          "max_input_tokens": 8191,
+          "threshold_ms": 2000
+        },
+        {
+          "max_input_tokens": 32767,
+          "threshold_ms": 4000
+        },
+        {
+          "threshold_ms": 6000
+        }
+      ]
+    }
+  },
+  "search": {
+    "low": 0.0,
+    "high": 0.125,
+    "tolerance": 0.001,
+    "max_probes": 6,
+    "sample_seed": 20260325
+  },
+  "llm": {
+    "system_prompt": "You are tuning a prefill-dominated vLLM serving stack on the 0~32k input-length bucket. The trace replay forces completion length to exactly 1 token, so optimize for TTFT under the configured stepped SLO. Propose one launch-safe config patch that increases the maximum feasible sampling_u while respecting the topology constraints and avoiding known launch failures.",
+    "max_history_trials": 8,
+    "endpoint": {
+      "provider": "codex",
+      "model": "gpt-5.4",
+      "stream": true,
+      "api_key_env": "OPENAI_API_KEY",
+      "timeout_s": 240
+    }
+  }
+}

configs/examples/dash1_qwen235b_prefill_thinking_run5_ttft_1s_2s_0_32k.json

@@ -0,0 +1,212 @@
+{
+  "study_id": "dash1-qwen235b-prefill-thinking-run5-ttft-1s-2s-0-32k-topology",
+  "hardware": {
+    "gpu_count": 8,
+    "gpu_model": "H20",
+    "host_candidates": [
+      "dash1"
+    ]
+  },
+  "model": {
+    "model_id": "qwen3-235b-a22b-256k-0717-internal",
+    "served_model_name": "qwen3-235b-prefill"
+  },
+  "engine": {
+    "engine_name": "vllm",
+    "engine_version": "internal-on-dash1",
+    "exec_path": "/usr/local/bin/vllm",
+    "cwd": "/home/admin/cpfs/wjh/aituner/aituner",
+    "host": "127.0.0.1",
+    "port": 18146,
+    "healthcheck_path": "/v1/models",
+    "ready_timeout_s": 1800,
+    "request_timeout_s": 1800,
+    "launch_args": [
+      "serve",
+      "/home/admin/resource/model/464482ce.qwen3-235b-a22b/256k-0717"
+    ],
+    "base_envs": {
+      "CUDA_VISIBLE_DEVICES": "0,1,2,3,4,5,6,7",
+      "VLLM_USE_V1": "1",
+      "VLLM_ATTENTION_BACKEND": "FLASH_ATTN",
+      "VLLM_QUANTIZATION_LAYER_WISE": "1",
+      "VLLM_MOE_USE_DEEPEP": "0",
+      "VLLM_MOE_BALANCED_GATING": "0",
+      "VLLM_MOE_RANDOM_GATING": "0",
+      "VLLM_FUSED_MOE_CHUNK_SIZE": "4096",
+      "VLLM_DP_META_USE_CPU_GROUP": "0",
+      "VLLM_MLA_FP8_ATTENTION": "0",
+      "VLLM_MOE_EXPERTS_OVERLAP": "0",
+      "VLLM_USE_FLASHINFER_SAMPLER": "0",
+      "VLLM_RESPONSE_TIMEOUT": "290",
+      "VLLM_FP8_USE_BLADNN": "1",
+      "VLLM_MOE_USE_BLADNN": "1",
+      "VLLM_USE_DEEP_GEMM": "0",
+      "VLLM_PD_TRY_CONNECT_TIMEOUT_SECONDS": "120",
+      "VLLM_DEEP_GEMM_WARMUP": "skip",
+      "DEEPEP_LL_COMBINE_USE_FP8": "1",
+      "DEEPEP_LL_BUFFER_FP8_OPT": "1",
+      "DEEPEP_LL_DISPATCH_USE_NVL": "1",
+      "DEEPEP_LL_COMBINE_USE_NVL": "1",
+      "ACCL_LOW_LATENCY_OPTIMIZE": "2",
+      "ACCL_WRITEBATCH_OPT": "2",
+      "ACCL_IBV_MTU": "9000",
+      "ACCL_TX_DEPTH": "1024",
+      "ACCL_RETRANSMIT_TIMEOUT": "17",
+      "NVSHMEM_IBGDA_NUM_RC_PER_PE": "4",
+      "BLLM_KVTRANS_RDMA_SP": "2",
+      "NCCL_SOCKET_IFNAME": "eth1",
+      "NVSHMEM_BOOTSTRAP_UID_SOCK_IFNAME": "eth1",
+      "GLOO_SOCKET_IFNAME": "eth1"
+    },
+    "base_flags": {
+      "host": "127.0.0.1",
+      "port": 18146,
+      "served-model-name": "qwen3-235b-prefill",
+      "tensor-parallel-size": 4,
+      "gpu-memory-utilization": 0.85,
+      "enable-prefix-caching": true,
+      "enable-chunked-prefill": true,
+      "max-num-batched-tokens": 8192,
+      "disable-hybrid-kv-cache-manager": true,
+      "max-model-len": 262144,
+      "block-size": 64,
+      "max-num-seqs": 64,
+      "quantization": "fp8",
+      "cuda-graph-sizes": [
+        16,
+        32,
+        64,
+        96,
+        128,
+        160,
+        192,
+        224,
+        256,
+        288,
+        320,
+        352,
+        384,
+        416,
+        448,
+        480,
+        512,
+        544,
+        576,
+        608,
+        640,
+        672,
+        704,
+        736,
+        768,
+        800,
+        832,
+        864,
+        896,
+        928,
+        960,
+        992,
+        1024
+      ],
+      "compilation-config": "{\"cudagraph_mode\":\"PIECEWISE\",\"use_inductor\":false,\"custom_ops\":[\"all\"],\"max_cudagraph_capture_size\":2048}",
+      "speculative-config": "{\"method\":\"eagle3\",\"num_speculative_tokens\":1,\"hf_overrides\":{\"rope_scaling\":{\"type\":\"yarn\",\"factor\":128,\"original_max_position_embeddings\":2048,\"semi_dynamic\":false,\"dynamic\":true},\"num_experts\":0},\"model\":\"/home/admin/resource/model/464482ce.qwen3-235b-a22b/0717-eagle-0820\"}",
+      "hf-overrides": "{\"architectures\":[\"Qwen3MoeForCausalLM\"],\"model_type\":\"qwen3_moe\"}",
+      "kv-cache-dtype": "fp8",
+      "disable-log-requests": true
+    },
+    "tunable_envs": [
+      "VLLM_ENABLE_TORCH_COMPILE"
+    ],
+    "tunable_flags": [
+      "tensor-parallel-size",
+      "data-parallel-size",
+      "enable-expert-parallel",
+      "expert-parallel-size",
+      "gpu-memory-utilization",
+      "max-num-batched-tokens",
+      "max-num-seqs",
+      "block-size",
+      "enable-prefix-caching",
+      "enable-chunked-prefill"
+    ],
+    "topology_constraints": {
+      "require_tp_dp_product_equals_gpu_count": false,
+      "require_ep_size_leq_tp_dp_product": true,
+      "require_ep_size_divides_tp_dp_product": true,
+      "require_enable_expert_parallel_when_ep_gt_one": true,
+      "validate_cuda_graph_sizes_divisible_by_tp_when_tp_ep_reduce_scatter": true,
+      "allowed_tp_dp_products": [
+        4,
+        8
+      ],
+      "allowed_tensor_parallel_sizes": [
+        4,
+        8
+      ],
+      "allowed_data_parallel_sizes": [
+        1,
+        2
+      ],
+      "allowed_expert_parallel_sizes": [
+        1,
+        2,
+        4,
+        8
+      ]
+    },
+    "python_executable": "python3"
+  },
+  "trace": {
+    "windows_path": "/home/admin/cpfs/wjh/aituner/aituner/trace_windows/windows.json",
+    "window_id": "thinking_w20260327_1000",
+    "request_mode": "chat",
+    "completion_tokens_override": 1,
+    "u_field": "sampling_u",
+    "timestamp_field": "timestamp",
+    "max_concurrency": 64,
+    "input_length_filter": {
+      "min_input_tokens": 0,
+      "max_input_tokens": 32768
+    },
+    "replay_time_scale": 1.0,
+    "early_stop_max_lag_s": 180.0,
+    "early_stop_max_elapsed_s": 1200.0
+  },
+  "slo": {
+    "target_pass_rate": 0.95,
+    "ttft_rule": {
+      "kind": "step_ms",
+      "buckets": [
+        {
+          "max_input_tokens": 8191,
+          "threshold_ms": 1000
+        },
+        {
+          "max_input_tokens": 32767,
+          "threshold_ms": 2000
+        },
+        {
+          "threshold_ms": 2000
+        }
+      ]
+    }
+  },
+  "search": {
+    "low": 0.0,
+    "high": 0.125,
+    "tolerance": 0.001,
+    "max_probes": 6,
+    "sample_seed": 20260325
+  },
+  "llm": {
+    "system_prompt": "You are tuning a prefill-dominated vLLM serving stack on the 0~32k input-length bucket. The trace replay forces completion length to exactly 1 token, so optimize for TTFT under the configured stepped SLO. The SLO is stricter than the previous 2s/4s regime: <=8k prompts must hit 1s TTFT and <=32k prompts must hit 2s TTFT. Propose one launch-safe config patch that increases the maximum feasible sampling_u while respecting the topology constraints and avoiding known launch failures.",
+    "max_history_trials": 8,
+    "endpoint": {
+      "provider": "codex",
+      "model": "gpt-5.4",
+      "stream": true,
+      "api_key_env": "OPENAI_API_KEY",
+      "timeout_s": 240
+    }
+  }
+}

configs/examples/stopb_27b_ab/p1_tp1.json

@@ -0,0 +1,13 @@
+{
+  "observation": "baseline TP1 (deployed flags)",
+  "diagnosis": "deterministic TP A/B point",
+  "config_patch": {
+    "env_patch": {},
+    "flag_patch": {}
+  },
+  "expected_effects": [
+    "measure peak request_rate_per_gpu at this topology"
+  ],
+  "why_not_previous_failures": "n/a",
+  "should_stop": false
+}

configs/examples/stopb_27b_ab/p2_tp2.json

@@ -0,0 +1,15 @@
+{
+  "observation": "TP2",
+  "diagnosis": "deterministic TP A/B point",
+  "config_patch": {
+    "env_patch": {},
+    "flag_patch": {
+      "tensor-parallel-size": 2
+    }
+  },
+  "expected_effects": [
+    "measure peak request_rate_per_gpu at this topology"
+  ],
+  "why_not_previous_failures": "n/a",
+  "should_stop": false
+}

configs/examples/stopb_27b_ab/p3_tp4.json

@@ -0,0 +1,15 @@
+{
+  "observation": "TP4",
+  "diagnosis": "deterministic TP A/B point",
+  "config_patch": {
+    "env_patch": {},
+    "flag_patch": {
+      "tensor-parallel-size": 4
+    }
+  },
+  "expected_effects": [
+    "measure peak request_rate_per_gpu at this topology"
+  ],
+  "why_not_previous_failures": "n/a",
+  "should_stop": false
+}

docs/aituner-harness-summary.md

@@ -0,0 +1,125 @@
+# AITuner Harness Summary
+
+## What The Harness Adds
+
+The harness turns each LLM proposal from open-ended config search into a bottleneck-directed decision.
+
+1. Workload profile
+   - Extracts L-C-A features from the trace window:
+     - L: prompt length percentiles and tail ratio.
+     - C: prefix/cache reuse estimates from `hash_ids` when available.
+     - A: request rate, burst ratio, and interarrival variation.
+   - These features are injected into the prompt as a structured `Harnesses` section.
+
+2. Trial diagnostics
+   - Reads recent trial result JSON.
+   - Summarizes feasible probes, all-infeasible probes, pass rates, request rates, latency percentiles, and failed SLO reason counts.
+   - Classifies the active bottleneck as `ttft_prefill`, `decode_tpot`, `admission_or_queueing`, `launch_or_memory`, or unknown.
+
+3. Knob-family harnesses
+   - Maps bottlenecks to a small number of plausible knob families.
+   - Current harness families:
+     - `tensor-parallel-size`: long-prompt TTFT/prefill bottlenecks.
+     - `max-num-batched-tokens`: prefill batching or fragmentation, with trust-region guards.
+     - `max-num-seqs`: cache-heavy or admission-limited workloads.
+     - `enable-chunked-prefill`: long-tail prompt blocking.
+     - `gpu-memory-utilization`: memory headroom after topology and batching are stable.
+   - Each family has `use_when`, `procedure`, `guards`, and `active_now` fields.
+
+4. Proposal discipline and early stop
+   - The prompt requires the LLM to choose at most one primary knob family unless history proves a coupled change is needed.
+   - It must use adjacent legal topology choices and stay inside topology constraints.
+   - It receives tested config signatures, so it should not repeat already-tried configs.
+   - A deterministic harness stop can now emit `should_stop=true` before calling the LLM when completed validation evidence says another trial is not justified.
+
+5. Baseline-first loop
+   - LLM-driven `study tune` now evaluates the initial engine config first unless `--skip-baseline` is passed.
+   - This aligns the loop with evaluate-then-search: the first LLM proposal sees measured bottleneck evidence rather than guessing from static config.
+
+## What Accelerates Convergence
+
+The speedup comes from reducing wasted proposal families, not from changing the benchmark metric.
+
+1. Topology-before-runtime on prefill bottlenecks
+   - For long-prompt, low-cache-reuse windows, the harness activates the TP harness before speculative runtime knobs.
+   - Example: qwen27b 0-8k chat reached `TP=2, DP=1` at iter 2 under harness replay, while the original run spent iter 2 on `DP=2` and iter 3 on `DP=4`.
+
+2. Guarded stop after validation, not immediately after a strong incumbent
+   - If the newest trial is the incumbent and improves per-GPU throughput by at least `1.8x` over baseline, the harness requires direct evidence before trying runtime-only tweaks.
+   - It does not stop at the first large gain. It requires post-incumbent validation trials across nearby topology/runtime families, and stops only if those trials fail to produce a feasible per-GPU improvement.
+   - With the guard, `study tune` can write a `harness-stop-XXXX` proposal and exit without spending another GPU trial.
+
+3. All-infeasible plateau detection
+   - When recent all-infeasible trials at the same sampling threshold stop improving pass rate and p95 TTFT, the harness blocks repeating the same primary knob family.
+   - This prevents continuing a direction such as DP-only scale-out after DP4 and DP8 plateau.
+   - Plateau alone does not trigger deterministic early stop; it forces either a different justified family or a later validation/convergence stop.
+
+4. Cleaner early-stop handling
+   - Early-stopped probes no longer leave in-flight requests polluting the next probe.
+   - Default behavior drains in-flight requests for comparable production runs.
+   - Engine relaunch after early stop is available as opt-in for faster smoke studies, but it is not the default because it can change warm-state comparability.
+
+5. Search-high saturation stop
+   - If the incumbent's highest measured probe is feasible and is within the configured binary-search resolution of `search.high`, the harness stops before asking the LLM for another proposal. Individual request failures can be present when the aggregate probe still meets the configured pass-rate SLO.
+   - This is not a model-specific threshold. It means the workload search range, not the engine config, is currently the limiting measurement bound.
+
+6. Deterministic first probes
+   - After a baseline latency bottleneck, the harness can propose the adjacent legal TP increase before asking the LLM.
+   - After a TP incumbent improves per-GPU throughput, the harness keeps that topology and applies a same-topology runtime seed before trying DP/EP or broad runtime changes.
+
+## qwen27b 0-8k Evidence
+
+Source: `docs/qwen27b-chat-0-8k-harness-fig18.md`.
+
+Metric: best-so-far feasible `request_rate_per_gpu`.
+
+| Variant | Iter 1 | Iter 2 | Iter 3 | Iter 4 | Iter 5-12 |
+| --- | ---: | ---: | ---: | ---: | ---: |
+| Before harness | 0.0350 | 0.0617 | 0.0617 | 0.2025 | 0.2025 |
+| After harness strict replay | 0.0350 | 0.2025 | 0.2025 stop | 0.2025 | 0.2025 |
+
+Result:
+
+- Before harness reached the best value at iter 4.
+- After harness reached the same value at iter 2 and stopped at iter 3.
+- Iterations-to-best improved from `4` to `2`, a `2x` convergence speedup on this case.
+- The harness also avoided eight post-best infeasible runtime-only probes.
+
+## Current Risks
+
+- The harness is still prompt-guided for choosing the next non-stop proposal. The deterministic stop path is hard-coded in `study tune`, but proposal-family blocking is not yet enforced by a separate validator.
+- Strong-incumbent stopping is intentionally biased toward fewer GPU trials after validation evidence accumulates. Workloads with very narrow runtime sweet spots may still need a "continue local refinement" exception when the user wants absolute best throughput rather than fastest convergence to a good config.
+- Full fresh reruns on large models are expensive. Strict replay is useful for measuring proposal-path improvements when the proposed configs already exist in prior measured runs, but publication-quality claims still need fresh no-relaunch runs when time allows.
+
+## Qwen3-30B-A3B Community vLLM Evidence
+
+Source: `docs/qwen30b-community-vllm020/harness-early-stop-ablation-20260502.md`.
+
+Metric: best-so-far feasible `request_rate_per_gpu` on the bounded 0-8k chat replay with 128 output tokens and `replay_time_scale=0.1`.
+
+Initial `search.high=0.125` result:
+
+| Variant | Iter 1 | Iter 2 | Iter 3-12 |
+| --- | ---: | ---: | ---: |
+| no-harness | 1.0333 | 1.0333 | 1.0333 |
+| harness | 1.0333 | 1.0333 stop | 1.0333 |
+
+Result:
+
+- Both variants found the same best measured config: the default community vLLM launch.
+- Harness stopped at iter 2 because the incumbent saturated `search.high`; no LLM proposal or GPU trial was needed after baseline.
+- No-harness spent the full 12-iteration budget: iter 2 was worse per GPU, and iter 3-12 were launch failures.
+- This was a measurement-ceiling result, not proof of global optimality.
+
+High `search.high=1.0` rerun:
+
+| Variant | Iter 1 | Iter 2 | Iter 3 | Iter 4 | Iter 5 | Iter 6 | Iter 7 | Iter 8-12 |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
+| no-harness | 2.2000 | 3.2583 | 3.2583 | 3.2583 | 3.2583 | 3.3000 | 3.3500 | 3.3500 |
+| harness-guided-v2 | 2.3833 | 3.2583 | 3.2833 | 3.3000 | 3.3000 stop | 3.3000 | 3.3000 | 3.3000 |
+
+Result:
+
+- Raising `search.high` showed that default vLLM was not actually optimal; the prior run was capped by workload range.
+- Harness reached the same TP2/runtime config family in 4 iterations instead of 7 by making deterministic first TP and same-topology runtime proposals.
+- The single-run best value differs by about 1.5% (`3.3000` vs `3.3500`) for the same config family, so this should be interpreted as faster convergence to the same region, not an exact single-run throughput win.

docs/harness-ablation/profile-driven-harness-design.md

@@ -0,0 +1,250 @@
+# Profile-Driven Harness Design
+
+## Problem
+
+The current harness is too rule-like. It can help in a narrow setup, but it can also overfit to a previous observation such as "TP=2 is enough" and then fail when the SLO or workload changes. Adding one more rule for each failure mode is not acceptable.
+
+The harness should make AITuner behave more like a performance engineer:
+
+1. Profile the workload and engine behavior.
+2. Diagnose the active bottleneck from measurements.
+3. Form hypotheses about which knob families can relieve that bottleneck.
+4. Choose the next experiment that best disambiguates or improves the system.
+5. Interpret the result and update the diagnosis.
+6. Stop only when the measured search space is exhausted or the remaining hypotheses have low value.
+
+The harness should not encode a fixed answer for qwen27b, TP=4, TPOT25, or any testcase-specific threshold.
+
+## Design Principles
+
+### 1. Harness is an evidence system, not a prompt trick
+
+The harness should provide structured evidence and a decision protocol. It should not simply add more natural-language hints to the prompt.
+
+The output should include:
+
+- `profile`: measured workload and engine traits.
+- `bottleneck_diagnosis`: ranked bottleneck hypotheses with evidence.
+- `candidate_actions`: candidate config changes with predicted effect and risk.
+- `experiment_plan`: the next config to test, why it is informative, and what outcome would confirm/refute the hypothesis.
+- `stop_decision`: only if no useful experiment remains under the current measurement budget.
+
+### 2. Separate profiling from tuning
+
+AITuner currently learns mostly from full trial results. That is too coarse. The harness needs a profile layer that extracts bottleneck signals from every probe:
+
+- workload profile: input/output token distributions, cache reuse, burstiness, selected-request count per sampling threshold;
+- latency profile: TTFT and TPOT mean/p50/p95/p99 by threshold and optionally by input/output token bucket;
+- engine profile: prefill/decode throughput, waiting/running request counts, KV cache usage, preemptions, GPU utilization if available;
+- topology profile: TP/DP/EP product, per-GPU request rate, launch stability, memory pressure;
+- measurement profile: search high/low/tolerance, which thresholds were measured, whether failures were early-stop artifacts or true low-load infeasibility.
+
+This profile should be persisted per trial so later decisions do not depend on prompt memory.
+
+### 3. Diagnose bottlenecks from counters and slopes
+
+A professional tuning loop should not infer bottlenecks from a single failed reason. It should compare how metrics change across thresholds and configs.
+
+Useful diagnostics:
+
+- TTFT-bound: TTFT p95/p99 rises sharply with request rate, long prompts dominate failures, prefill throughput is saturated, decode TPOT is still healthy.
+- TPOT-bound: TPOT p95/p99 fails while TTFT is acceptable, decode throughput or per-token latency dominates, high sequence concurrency or batching creates token latency tails.
+- Admission/queueing-bound: waiting requests and arrival lag grow, both TTFT and TPOT may degrade, utilization may be high but service time per request is not the only issue.
+- Memory/KV-bound: KV cache usage, preemptions, OOM/launch failures, or max model length/cache pressure limit throughput.
+- Topology/communication-bound: increasing TP lowers per-request latency but may reduce per-GPU throughput; DP improves admission but may not reduce per-request latency.
+
+The diagnosis should be a ranked list with confidence, not a single label:
+
+```json
+{
+  "bottlenecks": [
+    {
+      "name": "decode_tpot",
+      "confidence": 0.62,
+      "evidence": ["tpot_p95 fails at last infeasible probe", "ttft_p95 remains below SLO"]
+    },
+    {
+      "name": "prefill_ttft",
+      "confidence": 0.31,
+      "evidence": ["long prompt tail", "ttft_p99 grows near knee"]
+    }
+  ]
+}
+```
+
+### 4. Candidate actions come from a knob-effect model
+
+The harness should maintain a generic model of knob effects, not case-specific rules.
+
+Examples:
+
+| Knob family | Expected effect | Main risks |
+|---|---|---|
+| Increase TP | lower per-request compute latency, may help TTFT/TPOT tails | communication overhead, lower per-GPU efficiency, fewer independent replicas |
+| Increase DP | more replicas, better admission/queueing | does not reduce per-request compute latency, may waste memory |
+| Increase EP | MoE expert distribution, possible decode/prefill balance | launch complexity, communication, only relevant with model/engine evidence |
+| Lower max-num-seqs | reduce decode contention and tail TPOT | underutilization, lower throughput |
+| Raise max-num-seqs | improve admission/concurrency | TPOT/TTFT tails, memory pressure |
+| Raise max-num-batched-tokens | improve prefill batching and GPU occupancy | long-prefill HoL blocking, memory pressure |
+| Lower max-num-batched-tokens | reduce long-prefill interference | underfilled prefill batches |
+| Change block-size | cache fragmentation/allocator effects | launch/runtime instability, workload-specific |
+
+The harness should score candidates with:
+
+```text
+score = expected_bottleneck_relief
+      + information_gain
+      + launch_safety
+      - regression_risk
+      - measurement_cost
+```
+
+This lets it choose between "exploit current incumbent" and "explore unmeasured topology" from evidence, not hardcoded order.
+
+### 5. Use hypothesis-driven experiments
+
+Each trial should answer a specific question.
+
+Bad:
+
+- "Try TP=4 because previous run needed TP=4."
+
+Good:
+
+- "At the current best TP=2, the last infeasible probe is TPOT-bound. Increasing TP to 4 should reduce per-token compute latency but may reduce per-GPU efficiency. This trial tests whether the SLO is compute-latency limited or topology-overhead limited."
+
+Every candidate should have:
+
+- hypothesis;
+- expected metric movement;
+- risk;
+- confirm condition;
+- reject condition;
+- fallback next family.
+
+### 6. Stop is a decision under measured search limits
+
+Early stop should not mean "we found something good." It should mean:
+
+- the configured `search.high` is saturated by a feasible incumbent; or
+- the remaining high-value hypotheses are already measured or invalidated; or
+- all remaining candidates have low expected information gain or high launch risk relative to the current best; or
+- the baseline is infeasible even at the lowest load and the SLO is too tight.
+
+For Fig18 raw mode, stop can be disabled or recorded as `stop` rather than filling remaining columns with best-so-far.
+
+## Proposed Architecture
+
+### Components
+
+1. `Profiler`
+   - Input: trial result, probe history, engine logs/metrics if available, study spec.
+   - Output: `TrialProfile`.
+
+2. `BottleneckAnalyzer`
+   - Input: recent `TrialProfile`s, workload profile, SLO.
+   - Output: ranked bottleneck hypotheses.
+
+3. `KnobEffectModel`
+   - Generic mapping from bottleneck hypotheses to possible knob families.
+   - Contains no model-specific or SLO-specific constants.
+
+4. `ExperimentPlanner`
+   - Generates candidate config patches.
+   - Scores candidates by expected relief, information gain, risk, and cost.
+   - Emits the next experiment and rationale.
+
+5. `StopPolicy`
+   - Uses measured search coverage and remaining candidate scores.
+   - Does not stop just because a strong incumbent appears.
+
+6. `PromptRenderer`
+   - Renders the structured plan for the LLM when LLM involvement is needed.
+   - The LLM can choose among candidates or refine rationale, but should not invent arbitrary directions without evidence.
+
+### Data Flow
+
+```text
+study spec + workload trace
+        |
+        v
+workload profile
+        |
+trial result/probes/logs --> trial profile
+        |
+        v
+bottleneck analyzer --> ranked hypotheses
+        |
+        v
+knob effect model --> candidate action set
+        |
+        v
+experiment planner --> next config / stop
+        |
+        v
+AITuner evaluates config over configured search range
+```
+
+## How This Handles the Current Failure
+
+Current fixed TTFT4s + TPOT25 data:
+
+- TP=2 reached `0.1992 req/s/GPU`.
+- Later TP=2 local runtime probes did not improve.
+- Min-prompt no-harness found TP=4 with `0.2696-0.2758 req/s/GPU`.
+
+A profile-driven harness should have reasoned:
+
+1. The current incumbent is good but not proof of optimum.
+2. The failed probes still show SLO pressure, especially token-latency/tail behavior.
+3. Higher TP is a candidate action because it may reduce per-request compute latency.
+4. TP=4 is an unmeasured topology hypothesis within allowed constraints.
+5. Before spending many local TP=2 runtime trials or stopping, test TP=4 as a high-information experiment.
+
+This is not because TP=4 is hardcoded. If TP=4 had already been measured and underperformed, the planner would prefer local runtime refinement or stop. If the bottleneck were admission/queueing with TTFT/TPOT healthy, the planner might prefer DP or max-num-seqs instead.
+
+## Implementation Plan
+
+### Phase 1: Make the current harness evidence-first
+
+- Persist a compact `TrialProfile` for every trial.
+- Build ranked bottleneck hypotheses from probe sequences.
+- Replace single `active_bottleneck` with ranked hypotheses and evidence.
+- Replace deterministic local refinement with candidate scoring.
+- Keep existing rules only as entries in the generic `KnobEffectModel`.
+
+### Phase 2: Candidate scoring
+
+- Generate candidates from:
+  - adjacent topology moves;
+  - same-topology runtime moves;
+  - rollback/avoidance after failures;
+  - no-op stop.
+- Score candidates by:
+  - whether they directly target top bottleneck;
+  - whether they test an unmeasured high-information direction;
+  - launch risk from prior failures;
+  - expected impact on request_rate_per_gpu;
+  - measurement cost.
+
+### Phase 3: Controlled experiments
+
+- Add ablations:
+  - old strong-prompt no-harness;
+  - min-prompt no-harness;
+  - current rule harness;
+  - profile-driven harness.
+- Run each under the same SLO/workload/search.
+- Report raw `perf[i]`, best-so-far, failed configs, and stop reason.
+
+## Acceptance Criteria
+
+The profile-driven harness is acceptable only if:
+
+1. It does not encode model names, fixed SLO values, or known winning configs.
+2. It can explain every proposal as a hypothesis tied to measured bottleneck evidence.
+3. It does not early-stop while an unmeasured high-score candidate remains.
+4. It improves or matches min-prompt no-harness convergence on at least the qwen27b TTFT4s/TPOT25 setup.
+5. It does not regress the previous stepped TTFT/TPOT50 setup.
+6. It records enough evidence for a human engineer to audit why each trial was chosen.
+

docs/harness-ablation/profile-driven-harness-implementation-20260512.md

@@ -0,0 +1,155 @@
+# Profile-Driven Harness Implementation Log
+
+Date: 2026-05-12
+
+## Goal
+
+The harness should accelerate AITuner as a general tuning system, not as a collection of case-specific rules. The current implementation moves the harness toward a performance-engineering loop:
+
+1. extract a compact profile from each measured trial;
+2. rank bottleneck hypotheses from workload and probe evidence;
+3. generate generic candidate actions from a knob-effect model;
+4. score candidates by expected bottleneck relief, information gain, launch safety, and regression risk;
+5. block early stop while a high-value untested candidate remains.
+
+This is intended to apply across qwen3.5-27b chat, qwen3-235b prefill-only, qwen3-235b decode-only, and different SLOs without encoding model names, SLO constants, or known winning configs.
+
+## Code Changes
+
+- `src/aituner/harness.py`
+  - Added `trial_profiles` to normalize trial topology, performance, probe failures, latency quantiles, and launch failure evidence.
+  - Added `bottleneck_hypotheses`, a ranked list instead of a single active bottleneck label.
+  - Added `candidate_actions`, generated from topology and runtime knob families.
+  - Added `experiment_plan`, which selects the next high-score candidate or declares stop readiness.
+  - Updated harness proposal generation to prefer the profile-driven next action before falling back to legacy deterministic proposal code.
+  - Updated harness stop logic so convergence/validation stop is blocked when the planner still has a high-value untested candidate.
+
+- `tests/test_core_flow.py`
+  - Added coverage that a strong TP=2 incumbent with TTFT pressure still selects an unmeasured TP=4 topology candidate.
+  - Added coverage that decode-only TPOT pressure at max TP can prefer lowering `max-num-seqs` instead of blindly lowering TP.
+
+## Current Scoring Model
+
+The candidate score is intentionally generic:
+
+```text
+score = expected_bottleneck_relief * bottleneck_confidence
+      + information_gain
+      + launch_safety
+      - regression_risk
+```
+
+Examples:
+
+- TTFT/prefill bottleneck: increasing TP and prefill batching candidates receive relief score.
+- Decode TPOT bottleneck: increasing TP is useful if a higher legal TP exists; if already at high TP, lowering decode concurrency can become the higher-value candidate.
+- Admission/queueing bottleneck: more DP or higher safe concurrency receives relief score.
+
+The scores are not tied to qwen27b/qwen235b or a fixed TPOT/TTFT threshold. They are tied to the measured bottleneck class and legal tunable space.
+
+## Verification
+
+Local:
+
+```bash
+python3 -m compileall -q src tests
+PYTHONPATH=src python3 -m unittest tests.test_core_flow
+```
+
+Result: `93` tests passed.
+
+## Next Experiment
+
+Run the same qwen3.5-27b chat 0-8k setup as the current ablation baseline:
+
+- workload: chat, input length 0-8k
+- SLO: TTFT p95 <= 4000ms, TPOT p95 <= 25ms, target pass rate 0.95
+- search: full range, `inherit_incumbent_floor=false`
+- budget: 12 total tuning iterations
+- LLM model: `gpt-5.4`
+- variant: harness enabled with profile-driven planner
+
+The no-harness min-prompt baseline is already available and only needs to be reused for comparison unless the setup changes.
+
+## Experiment Started
+
+Started on `dash0` (`11.73.2.172`) at commit `17e9681`.
+
+- tmux session: `qwen27b-profileplanner-harness-20260512`
+- spec: `.aituner-tight/specs/dash0-qwen27b-chat-0-8k-ttft4s-tpot25-gpu3skip-12iter-harness-profileplanner-20260512.json`
+- study id: `dash0-qwen27b-chat-0-8k-ttft4s-tpot25-gpu3skip-12iter-harness-profileplanner-20260512`
+- log: `.aituner-tight/logs/qwen27b-profileplanner-harness-20260512.log`
+- status at launch check: `trial-0001` baseline is running under AITuner; no manual intervention in the tuning loop.
+
+## V1 Early Stop
+
+The first profile-planner run was stopped before accepting it as evidence. A read-only replay of its completed baseline probe history showed that the planner would choose `max-num-seqs=64` for iter2. That was a diagnosis bug:
+
+- `slo_pass_rate_unrecoverable` is a binary-search early-stop summary, not a bottleneck class.
+- The harness was counting that summary as an admission/queueing failure.
+- Because this count dominated the real TTFT/TPOT failure counts, the planner selected a concurrency action instead of testing TP.
+
+Fix commit: `e3ed775`.
+
+The fix excludes `slo_pass_rate_unrecoverable` from the admission/queueing failure bucket. With the same baseline probe history, the planner now ranks `ttft_prefill` first and proposes `tensor-parallel-size=2` for iter2.
+
+## V2 Experiment Started
+
+Started on `dash0` (`11.73.2.172`) at commit `e3ed775`.
+
+- tmux session: `qwen27b-profileplanner-v2-harness-20260512`
+- spec: `.aituner-tight/specs/dash0-qwen27b-chat-0-8k-ttft4s-tpot25-gpu3skip-12iter-harness-profileplanner-v2-20260512.json`
+- study id: `dash0-qwen27b-chat-0-8k-ttft4s-tpot25-gpu3skip-12iter-harness-profileplanner-v2-20260512`
+- log: `.aituner-tight/logs/qwen27b-profileplanner-v2-harness-20260512.log`
+- monitor: read-only subagent `Wegener`
+
+Acceptance for this run is based on end-to-end trial results, not unit tests. If the first four trials lag the min-prompt no-harness baseline (`0.0650`, `0.1992`, `0.2696`, then failed/NA), the run should be treated as a failed harness iteration and the harness should be optimized again.
+
+## V2 Result And Failure
+
+V2 was stopped early after four trials because it did not improve the no-harness baseline and made a preventable launch-risk proposal.
+
+Raw `request_rate/GPU`:
+
+| Variant | iter1 | iter2 | iter3 | iter4 |
+| --- | ---: | ---: | ---: | --- |
+| no-harness min-prompt | 0.0650 | 0.1992 | 0.2696 | 0.2696 |
+| harness v2 | 0.0650 | 0.1992 | 0.2696 | failed |
+
+Harness v2 did correctly diagnose the first bottleneck and proposed:
+
+- iter2: `tensor-parallel-size=2`, raw `0.1992 req/s/GPU`;
+- iter3: `tensor-parallel-size=4`, raw `0.2696 req/s/GPU`.
+
+However, iter4 proposed `tensor-parallel-size=8` and failed at engine launch. The study's `hardware.gpu_count` is 8, but the launch environment sets `CUDA_VISIBLE_DEVICES=0,1,2,4,5,6,7`, which exposes only 7 GPUs. Therefore TP=8 should not have been considered launch-safe.
+
+This is a general harness bug: topology planning must use the effective visible GPU count from the execution profile, not just the nominal hardware count.
+
+Fix:
+
+- parse `engine.base_envs.CUDA_VISIBLE_DEVICES`;
+- compute effective GPU count as `min(hardware.gpu_count, visible_device_count)`;
+- filter topology candidates and adjacent TP frontier candidates by the effective GPU count.
+
+## GPU Visibility Correction
+
+On 2026-05-13 we corrected the intended experiment setup: `CUDA_VISIBLE_DEVICES` should be `0,1,2,3,4,5,6,7`, not the previous `0,1,2,4,5,6,7`.
+
+This invalidates direct comparison between the old `gpu3skip` runs and new 8-GPU runs. The old v2 failure was real under the old visible-device profile, but it was not the intended 8-card H20 setup.
+
+New comparable studies:
+
+| Variant | Study ID | Status |
+| --- | --- | --- |
+| no-harness baseline | `dash0-qwen27b-chat-0-8k-ttft4s-tpot25-gpu8-12iter-noharness-minprompt-gpt54-20260513` | completed |
+| harness | `dash0-qwen27b-chat-0-8k-ttft4s-tpot25-gpu8-12iter-harness-profileplanner-20260513` | completed |
+
+Both specs set:
+
+- `CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7`
+- model endpoint: `gpt-5.4`
+- workload: qwen3.5-27b chat 0-8k
+- SLO: TTFT p95 <= 4000ms, TPOT p95 <= 25ms, target pass rate 0.95
+- search: full range, `inherit_incumbent_floor=false`
+
+Results: harness best `0.2696 req/s/GPU` (TP=4, MBT=7680) vs no-harness best `0.1233 req/s/GPU` (prefix-caching=false), a **+118.6%** improvement. Full analysis in `qwen27b-chat-0-8k-ttft4s-tpot25-gpu8-20260513.md`.

docs/harness-ablation/qwen235b-thinking-prefill-ttft-20260510.md

@@ -0,0 +1,89 @@
+# qwen235b Thinking Prefill Harness Ablation, 2026-05-10
+
+**Superseded** by `qwen235b-thinking-prefill-ttft-3s6s9s-20260514.md` (updated SLO thresholds, 8-GPU setup). This document is retained for reference only.
+
+## Setup
+
+- Host: `dash0`
+- Engine: internal vLLM at `/usr/local/bin/vllm`
+- Model: `/home/admin/resource/model/464482ce.qwen3-235b-a22b/256k-0717`
+- Trace window: `thinking_w20260327_1000`
+- Request mode: chat, with `completion_tokens_override=1` for prefill-only measurement
+- SLO: TTFT-only stepped p95 pass target, target pass rate `0.95`
+  - input tokens `<=4096`: `3000 ms`
+  - input tokens `<=32768`: `6000 ms`
+  - otherwise: `9000 ms`
+- Search: `sampling_u` in `[0, 0.125]`, tolerance `0.001`, max probes `6`
+- Trial budget: no-harness allowed 12 GPU trials; harness allowed 12 but could stop early
+- Store root: `.aituner-prefill`
+
+The two fresh specs were identical except `study_id` and `llm.use_harness`:
+
+- no-harness: `.aituner-prefill/specs/dash0-qwen235b-prefill-thinking-run1-ttft-harness-ablation-12iter-noharness-rerun2-20260510.json`
+- harness: `.aituner-prefill/specs/dash0-qwen235b-prefill-thinking-run1-ttft-harness-ablation-12iter-harness-rerun2-20260510.json`
+
+Both runs were launched through `python3 -m aituner.cli study tune`; no proposal or study state was edited manually during tuning.
+
+## Result
+
+The table below is the raw per-iteration performance for a Fig18-style plot. Use this table as `perf[i]`; do not replace missing points with `max(perf[:i+1])`.
+
+Metric: `best_request_rate_per_gpu` from that trial's own `result.json`. `NA` means the proposed config did not produce a feasible point in the measured search range, either because the engine/probe failed or because every sampled probe was infeasible.
+
+Important caveat: these runs were produced before the lower-range fallback fix. For same-parallel-size runtime patches, AITuner inherited the incumbent `sampling_u` as the new search floor. If the config was infeasible above that floor, the old worker wrote `NA` without searching below the floor. Therefore the `NA` entries below are not complete Fig18-quality raw performance points; they are "no feasible point above inherited floor." A rerun with the fixed worker is required to fill their true lower-load performance.
+
+| Variant | iter1 | iter2 | iter3 | iter4 | iter5 | iter6 | iter7 | iter8 | iter9 | iter10 | iter11 | iter12 |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
+| no-harness raw `perf[i]` | 0.2029 | NA | NA | 0.3863 | NA | NA | NA | 0.3879 | 0.3892 | 0.3896 | 0.3900 | 0.3900 |
+| harness raw `perf[i]` | 0.2029 | NA | 0.3863 | stop | stop | stop | stop | stop | stop | stop | stop | stop |
+
+The raw no-harness curve is therefore not monotonic. The apparent monotonic 12-iter sequence comes only from plotting best-so-far rather than the measured performance of each proposal.
+
+Per-trial details:
+
+| Variant | iter1 | iter2 | iter3 | iter4 | iter5 | iter6 | iter7 | iter8 | iter9 | iter10 | iter11 | iter12 |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
+| no-harness, per-trial | 0.2029 | - | - | 0.3863 | - | - | - | 0.3879 | 0.3892 | 0.3896 | 0.3900 | 0.3900 |
+| harness, per-trial | 0.2029 | - | 0.3863 | stop | stop | stop | stop | stop | stop | stop | stop | stop |
+
+Best-so-far curve, shown only to explain final incumbent selection:
+
+| Variant | iter1 | iter2 | iter3 | iter4 | iter5 | iter6 | iter7 | iter8 | iter9 | iter10 | iter11 | iter12 |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
+| no-harness | 0.2029 | 0.2029 | 0.2029 | 0.3863 | 0.3863 | 0.3863 | 0.3863 | 0.3879 | 0.3892 | 0.3896 | 0.3900 | 0.3900 |
+| harness | 0.2029 | 0.2029 | 0.3863 | 0.3863 | 0.3863 | 0.3863 | 0.3863 | 0.3863 | 0.3863 | 0.3863 | 0.3863 | 0.3863 |
+
+For plotting raw `perf[i]`, the failed/infeasible points should stay missing or be rendered as invalid trials. If a plotting script requires numeric values, use `0` only with an explicit label that this means "no feasible configuration under the configured SLO"; do not forward-fill from the incumbent.
+
+Final best:
+
+| Variant | GPU trials spent | Best trial | Best config summary | Best req/s | Best req/s/GPU | Final vs no-harness |
+| --- | ---: | --- | --- | ---: | ---: | ---: |
+| no-harness | 12 | `trial-0011`/`trial-0012` | TP8, DP1, EP off, `max-num-batched-tokens` 7936/8064 | 3.1200 | 0.3900 | baseline |
+| harness | 3 | `trial-0003` | TP8, DP1, EP off | 3.0900 | 0.3863 | -0.96% |
+
+Harness reached `0.38625 req/s/GPU` at iter3. No-harness first reached the same TP8 family at iter4, then spent eight more GPU trials to move from `0.38625` to `0.39000 req/s/GPU`, an absolute gain of `0.00375 req/s/GPU` or `0.97%`.
+
+## What the Harness Did
+
+The harness did not use a testcase-specific throughput threshold. The stop decision came from the generic search-high saturation rule:
+
+- incumbent highest feasible probe: `sampling_u=0.123046875`
+- configured `search.high`: `0.125`
+- binary-search resolution: `(0.125 - 0.0) / 2^6 = 0.001953125`
+- gap to search high: `0.001953125`
+
+Because the incumbent was feasible and within one configured search resolution of `search.high`, the harness emitted `harness-stop-0004` before launching another GPU trial. This means the current study could no longer measure a materially higher workload without increasing `search.high`; it is not a claim of global engine optimality.
+
+The harness context also made the LLM response more directed after failure:
+
+- After baseline, it exposed the TTFT-only prefill bottleneck and the sharp queueing knee around `sampling_u=0.03515625`.
+- The LLM first chose TP4/DP2 to use the idle 4 GPUs while preserving the validated TP4 shard shape. This failed with `connection refused`, matching the no-harness failure family.
+- The next harness prompt included that failure, and the LLM switched to TP8/DP1 with EP off, explicitly avoiding the failed DP2 family.
+- No-harness inserted an extra EP4 launch-failure trial before reaching TP8/DP1.
+
+## Conclusion
+
+Harness accelerated convergence mainly through early stopping, not by finding a much better final config on this setup. It reduced GPU trials from 12 to 3 while preserving 99.0% of the no-harness final throughput. It also reached the first strong TP8 point one trial earlier than no-harness.
+
+The limitation is that the generic search-high stop guard stopped before local runtime tuning of `max-num-batched-tokens`, which no-harness used to recover a small additional `0.97%`. For this setup, that tradeoff is acceptable if the goal is fast convergence under a fixed measurement ceiling; if the goal is exact final throughput, the next study should raise `search.high` or disable search-high early stop for a local-polish phase.

docs/harness-ablation/qwen235b-thinking-prefill-ttft-3s6s9s-20260514.md

@@ -0,0 +1,117 @@
+# qwen235b Thinking Prefill Harness Ablation (TTFT 3s/6s/9s)
+
+Date: 2026-05-14 / 2026-05-15
+
+Supersedes: `qwen235b-thinking-prefill-ttft-20260510.md` (different SLO thresholds).
+
+## Setup
+
+- Host: `dash0`
+- Engine: internal vLLM at `/usr/local/bin/vllm`
+- Model: `/home/admin/resource/model/464482ce.qwen3-235b-a22b/256k-0717`
+- Trace window: `thinking_w20260327_1000`
+- Request mode: chat, with `completion_tokens_override=1` for prefill-only measurement
+- SLO: TTFT-only stepped p95 pass target, target pass rate `0.95`
+  - input tokens `<=4096`: `3000 ms`
+  - input tokens `<=32768`: `6000 ms`
+  - otherwise: `9000 ms`
+- GPU env: `CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7` (8x H20)
+- Baseline topology: `TP=4`
+- LLM: `gpt-5.4`
+- Code: profile-driven harness planner, post GPU-visibility fix (`5c2958e`+)
+
+## Studies
+
+| Variant | Study ID | search.high |
+| --- | --- | ---: |
+| no-harness | `dash0-qwen235b-prefill-thinking-ttft-3s6s9s-12iter-noharness-minprompt-gpt54-20260514` | 0.125 |
+| harness | `dash0-qwen235b-prefill-thinking-ttft-3s6s9s-12iter-harness-profileplanner-gpt54-20260514` | 0.125 |
+| harness (high=0.25) | `dash0-qwen235b-prefill-thinking-ttft-3s6s9s-high025-12iter-harness-profileplanner-gpt54-20260515` | 0.25 |
+
+The `harness (high=0.25)` run was added to test whether raising `search.high` lets the harness find a better runtime config after reaching the search ceiling at `0.125`.
+
+## Result
+
+Raw per-iteration performance for Fig18-style plot. Metric: `best_request_rate_per_gpu`. `NA` means the proposed config did not produce a feasible point. `fail` means engine launch failure. `stop` means harness stopped before launching another trial.
+
+| Variant | iter1 | iter2 | iter3 | iter4 | iter5 | iter6 | iter7 | iter8 | iter9 | iter10 | iter11 | iter12 |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
+| no-harness raw `perf[i]` | 0.1804 | fail | 0.1892 | fail | 0.1892 | 0.1804 | 0.2217 | 0.2029 | 0.2029 | 0.2029 | 0.1892 | 0.1804 |
+| harness raw `perf[i]` | 0.2029 | 0.3863 | stop | stop | stop | stop | stop | stop | stop | stop | stop | stop |
+| harness (high=0.25) raw `perf[i]` | 0.2029 | 0.3921 | 0.3442 | 0.3921 | 0.3821 | 0.3821 | 0.3821 | 0.3688 | 0.3821 | 0.3821 | 0.3821 | 0.3821 |
+
+| Variant | GPU trials | Best iter | Best req/s | Best req/s/GPU | Best config summary |
+| --- | ---: | ---: | ---: | ---: | --- |
+| no-harness | 12 | 7 | 0.8867 | 0.2217 | TP=4, MNS=112, MBT=7168 |
+| harness | 2 (stop) | 2 | 3.0900 | 0.3863 | TP=8 |
+| harness (high=0.25) | 12 | 2 | 3.1367 | **0.3921** | TP=8 |
+
+Harness reached **+74.2%** over no-harness at iter 2. With `search.high=0.25`, the harness found `0.3921 req/s/GPU` (+76.8%).
+
+## Incumbent Curve
+
+Best-so-far request rate per GPU after each iteration.
+
+| Variant | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
+| no-harness | 0.1804 | 0.1804 | 0.1892 | 0.1892 | 0.1892 | 0.1892 | 0.2217 | 0.2217 | 0.2217 | 0.2217 | 0.2217 | 0.2217 |
+| harness | 0.2029 | 0.3863 | stop | stop | stop | stop | stop | stop | stop | stop | stop | stop |
+| harness (high=0.25) | 0.2029 | 0.3921 | 0.3921 | 0.3921 | 0.3921 | 0.3921 | 0.3921 | 0.3921 | 0.3921 | 0.3921 | 0.3921 | 0.3921 |
+
+## Trial Details
+
+No-harness:
+
+| Iter | Result / GPU | Incumbent / GPU | Status | Config summary |
+| ---: | ---: | ---: | --- | --- |
+| 1 | 0.1804 | 0.1804 | completed | baseline (TP=4) |
+| 2 | - | 0.1804 | launch fail | TP=4, EP=4, MNS=128 |
+| 3 | 0.1892 | 0.1892 | completed | MNS=96 |
+| 4 | - | 0.1892 | launch fail | TP=4, DP=2, EP off, MNS=96 |
+| 5 | 0.1892 | 0.1892 | completed | MNS=112 |
+| 6 | 0.1804 | 0.1892 | completed | MNS=112, MBT=9216 |
+| 7 | 0.2217 | 0.2217 | completed | MNS=112, MBT=7168 |
+| 8 | 0.2029 | 0.2217 | completed | MNS=112, MBT=6144 |
+| 9 | 0.2029 | 0.2217 | completed | MNS=120, MBT=7168 |
+| 10 | 0.2029 | 0.2217 | completed | TP=4, DP=1, EP off, MNS=108, MBT=7168 |
+| 11 | 0.1892 | 0.2217 | completed | MNS=112, MBT=7680 |
+| 12 | 0.1804 | 0.2217 | completed | MNS=112, MBT=6912 |
+
+Harness (`search.high=0.125`):
+
+| Iter | Result / GPU | Incumbent / GPU | Status | Config summary |
+| ---: | ---: | ---: | --- | --- |
+| 1 | 0.2029 | 0.2029 | completed | baseline (TP=4) |
+| 2 | 0.3863 | 0.3863 | completed | TP=8 |
+| 3 | - | - | harness stop | search-high saturation (`sampling_u=0.123` vs `search.high=0.125`) |
+
+Harness (`search.high=0.25`):
+
+| Iter | Result / GPU | Incumbent / GPU | Status | Config summary |
+| ---: | ---: | ---: | --- | --- |
+| 1 | 0.2029 | 0.2029 | completed | baseline (TP=4) |
+| 2 | 0.3921 | 0.3921 | completed | TP=8 |
+| 3 | 0.3442 | 0.3921 | completed | TP=8, chunked-prefill, MBT=32768 |
+| 4 | 0.3921 | 0.3921 | completed | TP=8, MBT=12288 |
+| 5 | 0.3821 | 0.3921 | completed | TP=8, EP off, MBT=16384 |
+| 6 | 0.3821 | 0.3921 | completed | TP=8, EP off, MBT=14336 |
+| 7 | 0.3821 | 0.3921 | completed | TP=8, EP off, MBT=10240 |
+| 8 | 0.3688 | 0.3921 | completed | TP=8, EP off, MBT=11776 |
+| 9 | 0.3821 | 0.3921 | completed | TP=8, EP off, MBT=13312 |
+| 10 | 0.3821 | 0.3921 | completed | TP=8, EP off, MBT=7168 |
+| 11 | 0.3821 | 0.3921 | completed | TP=8, EP off, MBT=12032 |
+| 12 | 0.3821 | 0.3921 | completed | TP=8, EP off, MBT=12800 |
+
+## Interpretation
+
+No-harness never attempted TP=8. It stayed on the TP=4 baseline, encountered two launch failures (EP=4 and DP=2), and spent all remaining trials on runtime knob tuning within the TP=4 family. Its best finding was `MNS=112, MBT=7168` at iter 7 (`0.2217 req/s/GPU`).
+
+Harness identified `ttft_prefill` as the dominant bottleneck from the baseline trial and immediately proposed TP=8 as the first topology move. This is the correct direction for a prefill-only workload with heavy-tail prompts (p95 ~19.7k tokens, p99 ~30k tokens).
+
+With `search.high=0.125`, the harness stopped at iter 2 because the incumbent's best feasible `sampling_u=0.123` was within one search resolution of `search.high`. With `search.high=0.25`, the harness continued for 12 trials but the best remained iter 2 (`TP=8, default MBT`). The additional 10 trials explored MBT variations on TP=8 but none improved per-GPU throughput. This confirms the 2-trial harness result was already at or near the local optimum.
+
+The gap between harness and no-harness (`+76.8%`) comes entirely from topology: TP=8 doubles the per-GPU prefill compute bandwidth compared to TP=4, which directly reduces TTFT and allows higher admitted request rates under the stepped TTFT SLO.
+
+## Comparison with Previous Run (2026-05-10)
+
+The 2026-05-10 run used different SLO thresholds and is documented in `qwen235b-thinking-prefill-ttft-20260510.md`. The core finding is consistent: harness finds TP=8 at iter 2-3 while no-harness gets stuck on TP=4 runtime tuning.

docs/harness-ablation/qwen27b-chat-0-8k-ttft4s-tpot25-20260510.md

@@ -0,0 +1,103 @@
+# Qwen27B Chat 0-8k Harness Ablation
+
+Date: 2026-05-10
+
+**Superseded** by `qwen27b-chat-0-8k-ttft4s-tpot25-gpu8-20260513.md` (corrected 8-GPU setup). This document used `CUDA_VISIBLE_DEVICES=0,1,2,4,5,6,7` (7 GPUs) and is retained for reference only.
+
+## Setup
+
+- Host: `dash0` (`172.27.114.84`)
+- Model: `/home/admin/resource/model/464482ce/qwen3.5-27b/256k-0223-internal`
+- Workload: chat, 0-8k input window
+- SLO: TTFT <= 4000ms and TPOT <= 25ms, target pass rate = 0.95
+- Trial budget: 12 total tuning iterations per study
+- Execution: direct `python3 -m aituner.cli study tune ... --max-trials 12`
+- GPU env: `CUDA_VISIBLE_DEVICES=0,1,2,4,5,6,7`
+- Code commit: `adc4351`
+
+The previous no-harness run was affected by the `dash0` migration and had many engine launch failures. This document uses the clean no-harness rerun from 2026-05-09.
+
+## Studies
+
+| Variant | Study ID |
+| --- | --- |
+| no-harness rerun | `dash0-qwen27b-chat-0-8k-ttft4s-tpot25-gpu3skip-12iter-noharness-rerun-20260509` |
+| harness | `dash0-qwen27b-chat-0-8k-ttft4s-tpot25-gpu3skip-12iter-harness-20260508` |
+
+## Result
+
+The table below is the raw per-iteration performance for a Fig18-style plot. Use this table as `perf[i]`; do not replace missing points with `max(perf[:i+1])`.
+
+Metric: `best_request_rate_per_gpu` from that trial's own `result.json`. `NA` means the proposed config did not produce a feasible point in the measured search range. `stop` means the harness stopped before launching another GPU trial.
+
+Important caveat: these runs were produced before the lower-range fallback fix. For same-parallel-size runtime patches, AITuner inherited the incumbent `sampling_u` as the new search floor. If the config was infeasible above that floor, the old worker wrote `NA` without searching below the floor. Therefore the `NA` entries below are not complete Fig18-quality raw performance points; they are "no feasible point above inherited floor." A rerun with the fixed worker is required to fill their true lower-load performance.
+
+| Variant | iter1 | iter2 | iter3 | iter4 | iter5 | iter6 | iter7 | iter8 | iter9 | iter10 | iter11 | iter12 |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
+| no-harness raw `perf[i]` | 0.0650 | 0.0617 | 0.0308 | NA | NA | NA | NA | NA | NA | 0.2025 | NA | NA |
+| harness raw `perf[i]` | 0.0650 | 0.0617 | 0.2025 | NA | 0.1283 | NA | 0.2696 | 0.2742 | NA | NA | NA | stop |
+
+The raw no-harness curve is not monotonic: iter2 and iter3 are worse than the baseline, and iter4-9 do not produce feasible configs. The monotonic curve below is best-so-far/incumbent tracking, not the measured performance of each proposal.
+
+| Variant | Best iter | Best request rate | Best request rate / GPU | Best config summary |
+| --- | ---: | ---: | ---: | --- |
+| no-harness rerun | 10 | 0.4050 | 0.2025 | `tensor-parallel-size=2`, `data-parallel-size=1`, `max-num-batched-tokens=12288` |
+| harness | 8 | 1.0967 | 0.2742 | `tensor-parallel-size=4`, `enable-chunked-prefill=true`, `max-num-batched-tokens=16384` |
+
+Harness reached a higher incumbent and did so earlier. Final best request rate per GPU improved by about `35.4%` over the clean no-harness rerun.
+
+## Incumbent Curve
+
+Values are incumbent best request rate per GPU after each tuning iteration. This table is useful for explaining final best selection, but it should not be used as Fig18 raw `perf[i]`.
+
+| Variant | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
+| no-harness rerun | 0.0650 | 0.0650 | 0.0650 | 0.0650 | 0.0650 | 0.0650 | 0.0650 | 0.0650 | 0.0650 | 0.2025 | 0.2025 | 0.2025 |
+| harness | 0.0650 | 0.0650 | 0.2025 | 0.2025 | 0.2025 | 0.2025 | 0.2696 | 0.2742 | 0.2742 | 0.2742 | 0.2742 | stop |
+
+For plotting raw `perf[i]`, keep `NA` points missing or render them as invalid trials. If a plotting script requires numeric values, use `0` only with an explicit label that this means "no feasible configuration under the configured SLO"; do not forward-fill from the incumbent.
+
+## Trial Details
+
+No-harness rerun:
+
+| Iter | Trial result / GPU | Incumbent / GPU | Status | Config summary |
+| ---: | ---: | ---: | --- | --- |
+| 1 | 0.0650 | 0.0650 | completed | baseline |
+| 2 | 0.0617 | 0.0650 | completed | `tp=1`, `dp=2`, `max-num-batched-tokens=12288` |
+| 3 | 0.0308 | 0.0650 | completed | `tp=1`, `dp=4` |
+| 4 | - | 0.0650 | completed, infeasible | `max-num-batched-tokens=12288` |
+| 5 | - | 0.0650 | completed, infeasible | `tp=1`, `dp=2`, `max-num-batched-tokens=16384` |
+| 6 | - | 0.0650 | completed, infeasible | `tp=1`, `dp=2`, `max-num-batched-tokens=12288`, `block-size=32` |
+| 7 | - | 0.0650 | completed, infeasible | `max-num-batched-tokens=10240` |
+| 8 | - | 0.0650 | completed, infeasible | `max-num-batched-tokens=7168` |
+| 9 | - | 0.0650 | completed, infeasible | `tp=1`, `dp=2` |
+| 10 | 0.2025 | 0.2025 | completed | `tp=2`, `dp=1`, `max-num-batched-tokens=12288` |
+| 11 | - | 0.2025 | completed, infeasible | `tp=2`, `dp=1`, `max-num-batched-tokens=10240` |
+| 12 | - | 0.2025 | completed, infeasible | `tp=2`, `dp=1`, `max-num-batched-tokens=13312` |
+
+Harness:
+
+| Iter | Trial result / GPU | Incumbent / GPU | Status | Config summary |
+| ---: | ---: | ---: | --- | --- |
+| 1 | 0.0650 | 0.0650 | completed | baseline |
+| 2 | 0.0617 | 0.0650 | completed | `tp=1`, `dp=2` |
+| 3 | 0.2025 | 0.2025 | completed | `tp=2`, `dp=1` |
+| 4 | - | 0.2025 | completed, infeasible | `tp=2`, chunked prefill, `max-num-batched-tokens=16384` |
+| 5 | 0.1283 | 0.2025 | completed | `tp=2`, `dp=2` |
+| 6 | - | 0.2025 | completed, infeasible | `tp=2`, `dp=1`, `max-num-seqs=4` |
+| 7 | 0.2696 | 0.2696 | completed | `tp=4`, `dp=1` |
+| 8 | 0.2742 | 0.2742 | completed | `tp=4`, chunked prefill, `max-num-batched-tokens=16384` |
+| 9 | - | 0.2742 | completed, infeasible | `tp=4`, chunked prefill, `max-num-batched-tokens=24576` |
+| 10 | - | 0.2742 | completed, infeasible | `tp=4`, chunked prefill, `max-num-batched-tokens=16384`, `max-num-seqs=8` |
+| 11 | - | 0.2742 | completed, infeasible | `tp=4`, chunked prefill, `max-num-batched-tokens=16384`, `max-num-seqs=16` |
+| 12 | - | 0.2742 | harness stop | validation exhausted after strong incumbent |
+
+## Interpretation
+
+The clean no-harness rerun eventually found the `tp=2` topology at iter 10, so the old migration-tainted no-harness result was indeed too pessimistic. Harness still improves the process in two ways:
+
+- It reaches the `tp=2` topology by iter 3 instead of iter 10.
+- It then escalates to `tp=4` and a nearby batching refinement, reaching `0.2742 req/s/GPU`.
+
+The harness effect is not "one iter to best"; it is directional search. It turns bottleneck evidence into topology validation probes, then validates runtime refinements around the stronger incumbent and stops when further nearby probes do not improve.

docs/harness-ablation/qwen27b-chat-0-8k-ttft4s-tpot25-gpu8-20260513.md

@@ -0,0 +1,99 @@
+# Qwen27B Chat 0-8k Harness Ablation (8-GPU)
+
+Date: 2026-05-13
+
+Supersedes: `qwen27b-chat-0-8k-ttft4s-tpot25-20260510.md` (7-GPU / gpu3skip setup).
+
+## Setup
+
+- Host: `dash0`
+- Model: `/home/admin/resource/model/464482ce/qwen3.5-27b/256k-0223-internal`
+- Workload: `chat_w20260311_1000`, chat, 0-8k input window
+- SLO: TTFT <= 4000ms and TPOT <= 25ms, target pass rate = 0.95
+- Trial budget: 12 total tuning iterations per study
+- Search: `sampling_u` in `[0, 0.0625]`, tolerance `0.001`, max probes `6`
+- Execution: `python3 -m aituner.cli study tune ... --max-trials 12`
+- GPU env: `CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7` (8x H20)
+- Baseline topology: `TP=1`
+- LLM: `gpt-5.4`
+- Code: profile-driven harness planner, post GPU-visibility fix (`5c2958e`+)
+
+## Studies
+
+| Variant | Study ID |
+| --- | --- |
+| no-harness | `dash0-qwen27b-chat-0-8k-ttft4s-tpot25-gpu8-12iter-noharness-minprompt-gpt54-20260513` |
+| harness | `dash0-qwen27b-chat-0-8k-ttft4s-tpot25-gpu8-12iter-harness-profileplanner-20260513` |
+
+## Result
+
+Raw per-iteration performance for Fig18-style plot. Metric: `best_request_rate_per_gpu` from that trial's own `result.json`. `NA` means the proposed config did not produce a feasible point. `fail` means engine launch failure.
+
+| Variant | iter1 | iter2 | iter3 | iter4 | iter5 | iter6 | iter7 | iter8 | iter9 | iter10 | iter11 | iter12 |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
+| no-harness raw `perf[i]` | 0.0650 | fail | fail | 0.0617 | 0.0650 | 0.1233 | 0.1050 | 0.1233 | 0.0650 | 0.0650 | 0.0617 | 0.1233 |
+| harness raw `perf[i]` | 0.0650 | 0.1992 | 0.2621 | 0.2056 | 0.1544 | 0.2696 | 0.2621 | 0.2621 | 0.2696 | 0.2621 | 0.2621 | 0.2621 |
+
+| Variant | Best iter | Best request rate | Best request rate / GPU | Best config summary |
+| --- | ---: | ---: | ---: | --- |
+| no-harness | 6 | 0.1233 | 0.1233 | `enable-prefix-caching=false` |
+| harness | 6 | 1.0783 | **0.2696** | `tensor-parallel-size=4`, `max-num-batched-tokens=7680` |
+
+Harness final best is **+118.6%** over no-harness.
+
+## Incumbent Curve
+
+Best-so-far request rate per GPU after each iteration.
+
+| Variant | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
+| no-harness | 0.0650 | 0.0650 | 0.0650 | 0.0650 | 0.0650 | 0.1233 | 0.1233 | 0.1233 | 0.1233 | 0.1233 | 0.1233 | 0.1233 |
+| harness | 0.0650 | 0.1992 | 0.2621 | 0.2621 | 0.2621 | 0.2696 | 0.2696 | 0.2696 | 0.2696 | 0.2696 | 0.2696 | 0.2696 |
+
+## Trial Details
+
+No-harness:
+
+| Iter | Result / GPU | Incumbent / GPU | Status | Config summary |
+| ---: | ---: | ---: | --- | --- |
+| 1 | 0.0650 | 0.0650 | completed | baseline |
+| 2 | - | 0.0650 | launch fail | `gpu-memory-utilization=0.94`, `max-num-batched-tokens=16384` |
+| 3 | - | 0.0650 | launch fail | `enable-chunked-prefill=false` |
+| 4 | 0.0617 | 0.0650 | completed | `data-parallel-size=2` |
+| 5 | 0.0650 | 0.0650 | completed | `block-size=32` |
+| 6 | 0.1233 | 0.1233 | completed | `enable-prefix-caching=false` |
+| 7 | 0.1050 | 0.1233 | completed | `enable-prefix-caching=false`, `block-size=32` |
+| 8 | 0.1233 | 0.1233 | completed | `enable-prefix-caching=false`, `max-num-seqs=32` |
+| 9 | 0.0650 | 0.1233 | completed | `enable-prefix-caching=false`, `max-num-batched-tokens=4096` |
+| 10 | 0.0650 | 0.1233 | completed | `enable-prefix-caching=false`, `max-num-seqs=16` |
+| 11 | 0.0617 | 0.1233 | completed | `data-parallel-size=2`, `enable-prefix-caching=false` |
+| 12 | 0.1233 | 0.1233 | completed | `enable-prefix-caching=false` (+ torch compile off) |
+
+Harness:
+
+| Iter | Result / GPU | Incumbent / GPU | Status | Config summary |
+| ---: | ---: | ---: | --- | --- |
+| 1 | 0.0650 | 0.0650 | completed | baseline |
+| 2 | 0.1992 | 0.1992 | completed | `tensor-parallel-size=2` |
+| 3 | 0.2621 | 0.2621 | completed | `tensor-parallel-size=4` |
+| 4 | 0.2056 | 0.2621 | completed | `tensor-parallel-size=8` |
+| 5 | 0.1544 | 0.2621 | completed | `tensor-parallel-size=4`, `data-parallel-size=2` |
+| 6 | 0.2696 | 0.2696 | completed | `tensor-parallel-size=4`, `max-num-batched-tokens=7680` |
+| 7 | 0.2621 | 0.2696 | completed | `tensor-parallel-size=4`, `enable-chunked-prefill=true`, `max-num-batched-tokens=12288` |
+| 8 | 0.2621 | 0.2696 | completed | `tensor-parallel-size=4`, `max-num-batched-tokens=7424` |
+| 9 | 0.2696 | 0.2696 | completed | `tensor-parallel-size=4`, `max-num-batched-tokens=7680`, `max-num-seqs=64` |
+| 10 | 0.2621 | 0.2696 | completed | `tensor-parallel-size=4`, `max-num-batched-tokens=7680`, `max-num-seqs=56` |
+| 11 | 0.2621 | 0.2696 | completed | `tensor-parallel-size=4`, `max-num-batched-tokens=7680`, `max-num-seqs=60` |
+| 12 | 0.2621 | 0.2696 | completed | `tensor-parallel-size=4`, `max-num-batched-tokens=7680`, `max-num-seqs=63` |
+
+## Interpretation
+
+No-harness never tested any TP change in 12 trials. It started from TP=1, encountered two early launch failures, then spent all remaining budget on runtime knobs (`enable-prefix-caching`, `block-size`, `max-num-seqs`, `max-num-batched-tokens`). Its best discovery was disabling prefix caching at iter 6, reaching only `0.1233 req/s/GPU`.
+
+Harness systematically explored the TP frontier: iter 2 tested TP=2, iter 3 tested TP=4, iter 4 tested TP=8. The profile-driven planner identified `ttft_prefill` as the ranked bottleneck and proposed increasing TP as the primary relief action. After TP=4 proved best per-GPU, the harness tested TP=4/DP=2 (worse) then shifted to runtime refinement within the TP=4 family, settling on `max-num-batched-tokens=7680` as the marginal improvement.
+
+The result demonstrates that topology exploration is critical for this workload: the no-harness LLM failed to discover TP>1 configurations entirely, while the harness reached the optimal TP=4 topology by iter 3 and refined it by iter 6.
+
+## Comparison with Previous 7-GPU Run
+
+The 7-GPU (`gpu3skip`) run from 2026-05-10 used `CUDA_VISIBLE_DEVICES=0,1,2,4,5,6,7` and is not directly comparable. The harness result on 7-GPU was `0.2742 req/s/GPU` (TP=4, chunked-prefill, MBT=16384). On 8-GPU, the harness found a similar TP=4 optimum at `0.2696 req/s/GPU` with slightly different runtime tuning. The core finding is consistent: harness accelerates topology discovery and significantly outperforms no-harness.

docs/harness-ablation/qwen27b-tp-sweep-20260616.md

@@ -0,0 +1,51 @@
+# Qwen3.5-27B TP sweep under length-aware TTFT SLO — 2026-06-16
+
+Branch `feat/two-stop`. Deterministic ground-truth A/B (proposal files, no LLM):
+TP1 vs TP2 vs TP4 on the dense Qwen3.5-27B (internal 256k, fp8, spec-decode) at
+0–8k chat, vLLM 0.11.1, H20, `replay_time_scale=1.0` (no smoke), Stop-A enabled,
+pinned to GPUs 2–7.
+
+**SLO**: TTFT ≤ `4000 + 0.125·L_in` ms (= 4s + L_in/8k), TPOT ≤ 50 ms, pass ≥ 95%.
+
+## Result
+
+| config | best_u | raw req/s | req/s/GPU | pass | saturated |
+| --- | --- | --- | --- | --- | --- |
+| TP1 | 0.00195 | 0.065 | **0.065** | 1.00 | no |
+| TP2 | 0.0195 | 0.585 | **0.2925** | 0.96 | no |
+| TP4 | 0.123 | 3.63 | **≥0.908** | 0.98 | **yes (best_u≈high=0.125)** |
+
+- **Per-GPU throughput rises sharply with TP for the dense 27B**: TP2 = 4.5× TP1,
+  TP4 ≥ 14× TP1. Opposite of the MoE Qwen3-30B-A3B (TP1 best per-GPU) — confirms the
+  dense-vs-MoE distinction.
+- **Mechanism**: TP1 is TPOT-bound — one H20 cannot decode a 27B under 50 ms/token
+  once the batch grows, so it saturates at ~0.065 req/s/GPU. Loosening TTFT (2s→4-5s)
+  did *not* change TP1 (still 0.065), confirming TPOT — not TTFT — is TP1's binding
+  constraint. Each TP doubling speeds decode+prefill enough to more than recover the
+  added GPUs.
+- **TP4 saturated** the offered-load ceiling (`best_u=0.123 ≈ 0.125`): still feasible
+  after ~the whole trace, so 0.908 is a lower bound. True peak (and TP8) need a
+  raised `search.high` to measure.
+
+## Process findings (fed back into the harness)
+
+- **Bug fixed**: a request exceeding `request_timeout_s` raised a raw `TimeoutError`
+  mid-stream that escaped `_run_one_request` and crashed the whole trial; now wrapped
+  as `HttpClientError` (failed request, not failed trial). Commit `2fcaf80`.
+- **Open gap**: killing a `study tune` run orphans the `VLLM::EngineCore` workers
+  (SIGTERM/SIGKILL of the loop doesn't tear down the engine), which twice left leaked
+  GPU memory on GPUs 0/1 (dead PIDs still pinning KV, only clearable via root
+  `nvidia-smi --gpu-reset`). Fix: SIGTERM handler in the CLI loop + make
+  `_terminate_process_tree` match `EngineCore` workers, not just `vllm serve`.
+- Experiment hygiene: scale=1.0 makes each probe take real arrival time; `search.high`
+  must bracket the config's boundary (too wide wastes probes on a low-capacity config;
+  too low saturates a high-capacity one), and `request_timeout_s` must be modest so
+  overloaded probes drain fast.
+
+## Next
+
+- Re-measure TP4 (and TP8) with `search.high` raised (e.g. 0.5) to find the true peak
+  per-GPU and the TP knee.
+- Run the Stop-B agentic loop on this 27B stack: unlike the 30B (baseline already
+  optimal), here the loop should climb TP1→TP2→TP4 and stop — a real improving
+  trajectory (the original Phase-5 "A" goal).

docs/harness-ablation/stop-a-validation-20260615.md

@@ -0,0 +1,121 @@
+# Stop-A validation (Phase 3) — 2026-06-15
+
+Branch `feat/two-stop`. Stop-A = truncate each probe's replay once the offered
+L-C-A of the replayed prefix converges to the full set (pure L-C-A criterion +
+C-gate). This note records the CPU calibration and the GPU fidelity check.
+
+## 1. Calibration (CPU, no serving)
+
+`scripts/stop_a_calibration.py` on the dash0 0321 10:00–10:10 windows:
+
+| dim | chat (19239 req, hit≈7%) | coder (2451 req, structured reuse) |
+| --- | --- | --- |
+| A | ≥0.95 by frac 0.10 | fast |
+| L | ≥0.96 from frac 0.05 | 0.05=0.75 (heavy tail) → ≥0.94 by 0.20 |
+| **C (slowest)** | noisy, dips (0.50→0.885, 0.55→0.835), stable ≥0.92 only ~0.85 | smooth, stable ≥0.92 by ~0.70 |
+
+Stop fraction (τ_L=τ_A=0.90, W=3):
+
+| τ_c | chat | coder |
+| --- | --- | --- |
+| 0.85 | 0.45 (273s) | 0.45 (255s) |
+| 0.90 | 0.70 (423s) | 0.55 (318s) |
+| 0.92 | 0.85 (513s) | 0.70 (411s) |
+
+Findings:
+- **C is the slowest dimension in both workloads** — reproduces paper §5.2 / Fig 9.
+- **What makes C hard to call converged is signal *noise*, not reuse magnitude.**
+  Low-reuse chat has a sparse/spiky ideal-hit-length series, so its C similarity
+  oscillates and is *harder* to stabilize than the structured, higher-reuse coder.
+  Consequence: a strict τ_c (0.92) gives chat only ~15% saving. A more robust C
+  feature for the low-reuse regime is future work.
+
+## 2. GPU fidelity check (Qwen3-30B-A3B, vLLM 0.11.1, H20)
+
+One full-window run (`adaptive_stop` disabled, `replay_time_scale=1.0`, window
+`chat_w20260311_1000`, 0–8k, out=128), then `scripts/stop_a_validate.py`
+recomputes each probe's convergence prefix and compares the truncated verdict to
+the full verdict — so a single GPU run validates truncation fidelity (no second run).
+
+Trial result: best feasible `sampling_u=0.078125`, request_rate **2.30 req/s**,
+pass_rate 0.973.
+
+Per-probe verdict (τ=0.9):
+
+| τ_c | verdict matches | mean replay saved |
+| --- | --- | --- |
+| 0.85 | 3/4 | 54% |
+| 0.90 | 3/4 | 52% |
+| 0.92 | 3/4 | 38% |
+
+The mismatch is the same probe at every τ_c — the feasibility knee `0.08594`:
+
+```
+thresh    full_pass  prefix_pass  full_feas  prefix_feas
+0.08594   0.946      0.956–0.961  False      True   <- mismatch
+0.07812   0.973      0.987–0.990  True       True
+0.06250   0.986      1.000        True       True
+0.09375   0.268      0.49–0.54    False      False
+```
+
+## 3. Interpretation
+
+- **Stop-A works and saves ~50% of replay** (vs the full 600 s window) while
+  preserving 3/4 probe verdicts. (The paper's ~70% is vs a 30-min fixed baseline;
+  our baseline is the 600 s window, so the percentages are not directly comparable.)
+- **The one failure is a boundary false-positive at the feasibility knee.** At
+  `0.08594` the full window is 0.946 (just below the 0.95 SLO) but the prefix is
+  0.956–0.961 (just above): the *second half* of the window degraded — engine-state
+  drift (KV fill / fragmentation / later-arriving harder requests) that the
+  *offered* L-C-A cannot see. The C-gate did not help because offered-C had
+  converged; the divergence is in the measured pass-rate, not in C.
+- If Stop-A were enabled, the binary search would accept `0.08594`, overestimating
+  the peak sustainable rate by one binary step (~10%).
+
+**This is the boundary jitter we accepted when choosing the pure-L-C-A criterion.**
+The data now argues for revisiting the previously-declined **SLO-boundary guard**:
+keep replaying while the measured pass-rate is within ±δ of the target, even after
+L-C-A converges. It targets exactly this knee case at low extra cost (it only
+extends replay on probes sitting on the feasibility boundary). Recommend adding it
+as a small Stop-A enhancement before enabling Stop-A in production studies.
+
+## 4. SLO-boundary guard (implemented + validated)
+
+Added `trace.adaptive_stop.boundary_delta` (default 0.02): when a truncated probe's
+measured pass-rate lands within ±δ of the SLO target, re-measure on the full window
+and use that verdict. Re-ran the same config with `adaptive_stop` enabled
+(τ=0.9, τ_c=0.90, δ=0.02):
+
+| threshold | feasible | pass | selected | replayed | boundary_extended |
+| --- | --- | --- | --- | --- | --- |
+| 0.06250 | True | 1.000 | 1086 | 487 (45%) | — |
+| 0.09375 | False | 0.444 | 1656 | 822 (50%) | — |
+| 0.07812 | True | 0.994 | 1378 | 682 (49%) | — |
+| 0.08594 | **False** | 0.947 | 1523 | **1523 (100%)** | **True** |
+
+Result: best feasible `sampling_u=0.078125` (rate 2.30 req/s) — **identical to the
+full-replay baseline**. The guard fired on exactly the one knee probe and
+re-measured it to the correct infeasible verdict; the other three probes truncated
+to ~45–50%. Net replayed 3514/5643 requests ≈ **38% replay saved on this trial
+while recovering the correct peak rate** (no one-step overestimate).
+
+**Conclusion: Stop-A with the boundary guard is correct (verdict matches full
+replay) and still saves replay time. Safe to enable.** Configs:
+`dash0_qwen30b_a3b_stopA_fulldata.json` (OFF baseline) and
+`dash0_qwen30b_a3b_stopA_on.json` (ON).
+
+## Repro
+
+```
+# calibration
+PYTHONPATH=src python3 scripts/stop_a_calibration.py \
+  --jsonl <DIR>/qwen_chat_blksz_64_032109-032111.jsonl --block-size 64 \
+  --window-start 3600 --window-end 4200 --gpu-count 8 --label chat
+# GPU run + fidelity
+PYTHONPATH=src python3 -m aituner.cli study tune \
+  --spec configs/examples/dash0_qwen30b_a3b_stopA_fulldata.json \
+  --store-root .aituner/stopA-fulldata --max-trials 1
+PYTHONPATH=src python3 scripts/stop_a_validate.py \
+  --spec configs/examples/dash0_qwen30b_a3b_stopA_fulldata.json \
+  --store-root .aituner/stopA-fulldata --tau 0.9 --tau-c 0.90
+```

docs/harness-ablation/stop-b-e2e-20260615.md

@@ -0,0 +1,86 @@
+# Stop-B end-to-end validation (Phase 5) — 2026-06-15
+
+Branch `feat/two-stop`. Real agentic loop on dash0: Qwen3-30B-A3B / vLLM 0.11.1 /
+8×H20, `gpt-5.4` (via codex/prism) proposing configs, Stop-A enabled to accelerate
+each evaluation, `use_harness=True` so the Stop-B deterministic validator + LLM-stop
+veto are active. Config `dash0_qwen30b_a3b_stopB_e2e.json`, `search.high=1.0`,
+`max_probes=6`, `--max-trials 8`.
+
+## Two stop paths exercised
+
+**Run A (`search.high=0.125`)** — the default config already saturates the offered-load
+search range, so Stop-B fired immediately via the **search-high-saturation** path:
+`stop_authorized_by: validator`, reason *"the incumbent's highest measured probe is
+feasible and within the binary-search resolution of search.high."* Correct
+measurement-ceiling stop (no point proposing configs when the load range, not the
+config, is the bound).
+
+**Run B (`search.high=1.0`)** — forces a real multi-iteration search:
+
+| trial | TP | DP | EP | feasible | raw req/s | **req/s/GPU** | source |
+| --- | --- | --- | --- | --- | --- | --- | --- |
+| 0001 | 1 | 1 | 1 | ✓ | 2.90 | **2.900** | baseline |
+| 0002 | 2 | 1 | 1 | ✓ | 4.42 | 2.208 | harness TP-seed |
+| 0003 | 2 | 1 | 2 | ✗ launch-fail | — | — | harness (EP) |
+| 0004 | 1 | 2 | 1 | ✓ | 4.42 | 2.208 | LLM (after veto) |
+| 0005 | 2 | 2 | 1 | ✓ | 8.37 | 2.092 | harness |
+| 0006 | 2 | 2 | 2 | ✗ launch-fail | — | — | harness (EP) |
+| 0007 | 4 | 1 | 1 | ✓ | 8.37 | 2.092 | LLM |
+| (0008) | — | — | — | **STOP** | — | — | LLM stop, honored after veto budget |
+
+Incumbent: **trial-0001 (TP1), 2.90 req/s/GPU — never beaten.**
+
+> **⚠️ The per-GPU trajectory above is NOT a valid benchmark — it validates only
+> the Stop-B *mechanics*.** Two confounds:
+> 1. **Trace-ceiling saturation.** TP2·DP2 and TP4 reached `best_sampling_u≈0.98`
+>    (still feasible after consuming ~the whole window), so their *true* peak
+>    per-GPU is higher than the 2.09 shown — we ran out of offered load to push
+>    them to their boundary. Only TP1 (u=0.31), TP2 (u=0.48) and DP2 (u=0.48)
+>    found real boundaries. The `sampling_u` axis maxes at the full trace, so any
+>    config that sustains more than the window's offered rate cannot be measured.
+> 2. **Smoke regime.** This run inherited `replay_time_scale=0.1` +
+>    `max_requests_per_probe=512` (README: convergence test, *not* a benchmark) —
+>    compressed arrivals distort A and the 512 cap imposes a ~8.4 req/s ceiling.
+>
+> The below-ceiling TP1 (2.90) > TP2 (2.21) ordering *may* be real for this model
+> (Qwen3-30B-A3B is an MoE with ~3B active params → little compute per token → TP
+> adds all-reduce overhead with little benefit), which differs from the dense
+> Qwen3.5-27B where TP2 wins. But this run cannot establish it. A valid benchmark
+> needs `scale=1.0`, no cap, and enough offered-load headroom that strong configs
+> are not trace-saturated — see the 27B TP A/B follow-up.
+
+## Phase-5 acceptance
+
+- **No regression.** The primary metric `request_rate_per_gpu` stayed 2.90 the whole
+  run. Scaling TP/DP raised *raw* throughput (4.42, 8.37) but lowered per-GPU
+  efficiency (2.21, 2.09); the loop correctly kept the TP1 baseline as incumbent and
+  never adopted a worse-per-GPU config. (Matches the paper: long-prompt, low-reuse
+  chat prefers small TP for per-GPU efficiency.)
+- **Stop-B authority validated live.** At trial 4 the LLM tried to stop
+  (`should_stop=true`); the deterministic validator **vetoed** it
+  (`validator_did_not_authorize_stop`, `continue_harness_guided_search`), forcing one
+  more confirmation (DP2, which also failed to beat baseline). After the budget, the
+  LLM's later, well-justified stop was honored (`stop_authorized_by:
+  llm_after_veto_budget`). The bounded veto behaved exactly as designed.
+- **Auditable reason chain.** Every stop/veto carries a diagnosis grounded in the
+  measured evidence (e.g. *"increasing TP 1→2 lowers per-GPU efficiency even though
+  token latency improves … EP is explicitly blocked by launch-failure evidence"*).
+- **Launch-failure robustness.** Two EP configs (trial-0003, 0006) failed to launch
+  under vLLM 0.11.1; the harness recorded them as hard-negative evidence and the LLM
+  explicitly stopped proposing EP.
+
+## Notes / limitations
+
+- For this workload the baseline (TP1) is already per-GPU optimal, so iterations-to-
+  *best* = 1; the remaining trials are the loop *confirming* no config beats baseline
+  before stopping. A workload with an under-tuned default would show an improving
+  trajectory; this run validates the stop/no-regression machinery, not a tuning win.
+- The final stop came via `llm_after_veto_budget` (validator vetoed once, then
+  deferred), not a pure deterministic validator stop — because the deterministic
+  conditions (3-within-2%, saturation, validation-exhausted) did not cleanly fire
+  when every trial was a distinct config with a distinct per-GPU rate. The validator
+  acted as the *guard* (preventing premature stop), which is its designed role.
+- 7 trials > the paper's 3–6 average, inflated by the wider search range, 2 EP
+  launch-failures, and the veto. Acceptable for a validation run.
+- LLM token: the non-interactive shell lacks `OPENAI_API_KEY`; export it from the
+  codex `auth.json` (`~/.codex/auth.json`) before the run.

docs/harness-ablation/stop-b-e2e-27b-20260616.md

@@ -0,0 +1,64 @@
+# Stop-B end-to-end on dense Qwen3.5-27B (the improving trajectory) — 2026-06-16
+
+Branch `feat/two-stop`. Real `gpt-5.4` agentic loop (codex/prism), Stop-A enabled,
+length-aware TTFT SLO (4s + L_in/8k, TPOT ≤ 50 ms), vLLM 0.11.1, H20, GPUs 2–7,
+`replay_time_scale=1.0`, `search.high=0.25`, `inherit_incumbent_floor=true`.
+Config `dash0_qwen27b_stopB_loop.json`. Companion to the 30B run
+(`stop-b-e2e-20260616.md`); together they cover all Stop-B behaviors.
+
+## Trajectory (incumbent = TP4 @ 1.00 req/s/GPU)
+
+| iter | proposed by | config | per_gpu | adopted? |
+| --- | --- | --- | --- | --- |
+| 1 | baseline | TP1 | 0.123 | incumbent |
+| 2 | gpt-5.4 | TP2 | 0.2925 (2.4×) | ✅ new incumbent |
+| 3 | gpt-5.4 | TP4 | **1.0012 (8.1×)** | ✅ new incumbent |
+| 4 | gpt-5.4 | TP4 + chunked-prefill + mbt=16384 | 0.942 | ❌ **worse → rejected** |
+| 5 | gpt-5.4 | TP2 + DP2 | (loop stopped before completing) | — |
+
+(Run stopped manually after iter-4 — see "Why stopped" below. Incumbent preserved
+at TP4.)
+
+## What this demonstrates (the piece the 30B run could not)
+
+- **A genuine improving climb.** `gpt-5.4` + the harness raised per-GPU throughput
+  TP1 → TP2 → TP4 (0.123 → 0.29 → 1.00, 8.1×), each step a correctly-diagnosed real
+  gain: TP1 is TPOT-bound, so the agent scaled tensor-parallelism, then — once
+  topology was won — pivoted to **runtime tuning on the winning family** (chunked
+  prefill + larger batched tokens).
+- **No regression.** The runtime tweak (iter-4) measured *below* plain TP4
+  (0.942 < 1.00), and the harness correctly **kept TP4 as the incumbent** rather than
+  adopting the worse config — the core Stop-B guarantee, shown live.
+- Combined with the 30B run (search-high-saturation `validator`-authorized stop +
+  premature-LLM-stop veto), every Stop-B behavior is now validated end-to-end:
+  improving climb, correct bottleneck-driven proposals, no regression, deterministic
+  stop authority, and the LLM-stop veto.
+
+## Process wins / findings
+
+- **SIGTERM teardown fix validated in practice.** This loop was stopped with a plain
+  SIGTERM and the engine + EngineCore workers torn down cleanly — GPUs 2–7 freed, no
+  orphan, no leaked memory (contrast: the pre-fix runs twice leaked GPU0/1). Commit
+  `b17b213`.
+- **Timeout-as-failed-request fix** (`2fcaf80`) held — no trial crashed on
+  request timeouts this run.
+
+## Why stopped (efficiency finding — feeds next round)
+
+The loop was stopped after iter-4 rather than run to an explicit Stop-B firing,
+because each TP4-family trial took ~3 h: at `scale=1.0`, infeasible high-θ probes
+each run to the **`early_stop_max_elapsed_s=900` cap** (`probe_elapsed_s>900`), and
+the primary+fallback binary search doubles the probe count. Stop-A truncates a
+*converged* replay but does not shortcut an *overloaded* probe that simply runs out
+the clock. **For a practical agentic loop at scale=1.0, lower `early_stop_max_elapsed_s`
+(≈300 s)** so overloaded probes die fast; consider also having an infeasible-and-
+overloaded probe early-stop on a fast lag/throughput signal rather than the elapsed
+cap. The convergence itself was already evident (iter-4's runtime tweak and the
+queued TP2+DP2 were not beating TP4).
+
+## Next
+
+- Lower the elapsed cap and (optionally) re-run to capture the explicit Stop-B stop
+  on this 27B stack.
+- Land the deferred items: more robust C feature for the low-reuse regime; Stop-C
+  cross-day retune trigger; §7 baselines (SCOOT/naive/community).

docs/harness-tuning-progress.md

@@ -0,0 +1,129 @@
+# Harness-Guided AITuner Progress
+
+## Goal
+
+Improve AITuner convergence for the `dash0` internal vLLM + Qwen3.5-27B 0-8k chat study. The prior 12-iteration run can still propose worse configs after finding good ones. The new harness should make config proposals bottleneck-directed and stop spending GPU trials once no adjacent harness-guided probe is justified.
+
+## Paper Alignment
+
+- Prompt structure now includes an explicit `[Harnesses]` section aligned with paper Figure 12.
+- The harness uses the paper's L-C-A workload model:
+  - L: prompt length percentiles and tail ratio.
+  - C: prefix/KV-cache reuse estimated from repeated `hash_ids` blocks when available.
+  - A: request rate, 1-second QPS burst ratio, and interarrival CV.
+- Knob rules follow the paper's Figure 13 style:
+  - map active bottleneck to a knob family;
+  - probe adjacent legal choices;
+  - enforce guard conditions to avoid harmful side effects;
+  - prefer stopping over weak exploratory proposals after convergence.
+
+## Local Implementation Log
+
+- Added `src/aituner/harness.py`.
+  - Builds structured harness context for prompt injection.
+  - Adds TP, max-num-seqs, max-num-batched-tokens, chunked-prefill, and memory-utilization harnesses when those knobs are tunable.
+  - Extracts compact recent trial diagnostics from result JSON files.
+  - Adds a convergence guard based on recent completed trial performance.
+  - Adds an infeasible-progress guard: when recent all-infeasible trials at the same sampling threshold stop improving pass rate and p95 TTFT after changing one knob family, the next proposal must switch primary family or stop.
+  - Classifies `slo_pass_rate_unrecoverable` by latency failure counts first, and ignores probe-budget markers such as `probe_elapsed_s>` for bottleneck voting, so TTFT-heavy failures stay aligned to prefill/TP or batching harnesses instead of being treated as generic queueing.
+- Extended `src/aituner/trace.py`.
+  - `summarize_window` now reports L-C-A features.
+  - `TraceRequest` now carries optional metadata for `hash_ids`, turn, parent chat id, and trace type.
+- Extended `src/aituner/llm.py`.
+  - Prompt now includes tested config signatures and the structured harness section.
+  - Prompt schema now asks for `should_stop`.
+- Extended `src/aituner/spec.py`.
+  - `Proposal` accepts optional `should_stop`.
+- Extended `src/aituner/cli.py`.
+  - `study tune` honors `should_stop=true` by recording the proposal and not launching another GPU trial.
+- Extended `tests/test_core_flow.py`.
+  - Prompt includes harness context.
+  - Trace summary includes new L-C-A fields.
+  - Proposal parsing accepts `should_stop`.
+  - CLI does not launch a trial for a stop proposal.
+
+## Local Verification
+
+- `python3 -m compileall -q src tests`: passed.
+- `PYTHONPATH=src python3 -m unittest tests.test_core_flow`: passed, 62 tests.
+- `pytest -q` and `python3 -m pytest -q`: not runnable locally because `pytest` is not installed.
+
+## Remote Experiment Log
+
+### 2026-04-25 16:30-16:45 CST
+
+- Pushed commit `2c5e9af` to `origin/main` and pulled it on `dash0`.
+- Remote prompt check command:
+  - `PYTHONPATH=src python3 -m aituner.cli study prompt --study-root /tmp/aituner-harness-prompt-check/dash0-qwen27b-tight-slo-10min-run4-chat-0-8k --store-root /tmp/aituner-harness-prompt-check --prompt-name harness-check`
+- Harness profile for `chat_w20260311_1000`, after applying the 0-8k filter:
+  - L: p50 1992, p95 7628, p99 8102, tail ratio 3.83, regime `moderate_tail_prefill_sensitive`.
+  - C: repeated token ratio estimate 0.191, repeated block ratio 0.189, multi-turn ratio 0.160, regime `low_prefix_reuse`.
+  - A: request rate 29.52 req/s, p95 1s QPS 40, burst ratio 1.36, regime `smooth`.
+  - Active harnesses: `tensor-parallel-size` and `max-num-batched-tokens`, which matches a TTFT/prefill-sensitive 0-8k chat workload.
+- Remote `compileall` passed.
+- Remote `unittest discover` initially exposed two pre-existing path-sensitive tests that hardcoded `/home/gahow/phd/aituner`; fixed them to derive `REPO_ROOT` from the test file path.
+
+### 2026-04-25 16:38-16:58 CST
+
+- Started real run in tmux session `aituner_harness_qwen27b_0_8k_20260425`.
+- Store root: `.aituner/harness-studies-20260425`.
+- First proposal followed the harness:
+  - proposal: `tensor-parallel-size: 2`;
+  - rationale: L profile is prefill-sensitive, prefix reuse is low, arrivals are smooth, so probe adjacent TP before runtime batching knobs.
+- First high-load probe at `sampling_u=0.03125` was infeasible:
+  - request rate 0.895 req/s;
+  - pass rate 0.145;
+  - p95 TTFT 4063 ms and p95 TPOT 113 ms;
+  - failed reasons included `tpot_ms>50.0` and `slo_pass_rate_unrecoverable`.
+- Important implementation issue found: after an early-stopped probe, the worker returned while in-flight HTTP requests could continue occupying the engine, stalling/polluting the next binary-search probe.
+- Action: stopped the run and freed GPUs. Updating `worker._replay_requests` to drain in-flight requests after early stop before the next probe starts.
+
+### 2026-04-25 17:00-17:12 CST
+
+- r2 confirmed that draining avoids immediate cross-probe pollution, but the first LLM trial still started from a speculative TP=2 edit without a measured incumbent.
+- This is not aligned with the paper's agentic loop, which evaluates the initial configuration first and then searches from measured feedback.
+- Action: update `study tune` so LLM-driven studies automatically materialize a baseline empty-patch trial first, unless `--skip-baseline` is passed. This should reduce early bad proposals because the first LLM edit will see real baseline bottleneck diagnostics and an incumbent request_rate_per_gpu.
+
+### 2026-04-25 17:20-18:30 CST
+
+- r3 started with baseline-first enabled, but the full 0-8k run was too slow for fast iteration with raw chat completions. Stopped it before using it as a convergence signal.
+- A fast validation using `max_requests_per_probe=160` was invalid: the trace is downsampled before threshold selection, so lower thresholds can end up with `request_count=0`. Do not use that result for performance claims.
+- Prefill smoke v1 used `completion_tokens_override=1` but kept the TPOT SLO. That made TPOT missing failures dominate, so it was useful only for checking control flow, not for performance.
+
+### 2026-04-25 18:30-20:10 CST
+
+- Prefill smoke v2 used real dash0 internal vLLM, Qwen3.5-27B, the real 0-8k prompt distribution and arrivals, `completion_tokens_override=1`, and `tpot_rule=null`.
+- Trial 0001 baseline TP1/DP1:
+  - sampling `0.0078125`: pass rate 0.270, mean TTFT 2033.9 ms, p95 TTFT 5656.7 ms, p99 TTFT 6832.8 ms.
+- Trial 0002 TP1/DP2:
+  - sampling `0.0078125`: pass rate 0.277, mean TTFT 1766.9 ms, p95 TTFT 4215.3 ms, p99 TTFT 5801.7 ms.
+- Trial 0003 TP1/DP4:
+  - sampling `0.0078125`: pass rate 0.345, mean TTFT 1668.9 ms, p95 TTFT 3818.4 ms, p99 TTFT 5804.9 ms.
+- Trial 0004 TP1/DP8:
+  - sampling `0.0078125`: pass rate 0.345, mean TTFT 1675.7 ms, p95 TTFT 3823.4 ms.
+- Interpretation:
+  - The harness improved directionality: after the measured baseline, proposals followed a consistent scale-out path and avoided random runtime-knob churn.
+  - The smoke result improved p95 TTFT by about 32% versus baseline at the low sampling threshold and improved pass rate from 0.270 to 0.345 within 3-4 trials.
+  - It did not reach the 95% pass-rate SLO in this smoke setting, so this is not a full proof of convergence to a good production config.
+  - DP8 did not improve over DP4, which exposed a gap: when every trial is infeasible, the prior convergence guard had no feasible incumbent and could not detect plateau.
+
+### 2026-04-25 20:10 CST
+
+- Added the all-infeasible plateau guard described above.
+- Added unit coverage for:
+  - TTFT failure classification under `slo_pass_rate_unrecoverable`;
+  - blocking a repeat of the DP family after DP4 and DP8 show no material improvement at the same sampling threshold.
+- Pulled the commit on `dash0` and reran remote verification:
+  - `python3 -m compileall -q src tests`: passed.
+  - `PYTHONPATH=src python3 -m unittest discover -s tests -p "test_*.py"`: passed, 62 tests.
+- Regenerated a prompt against the real smoke v2 history:
+  - `convergence_guard.reason`: `data-parallel-size_plateau_on_infeasible_trials`.
+  - `should_stop_if_no_harness_can_justify_a_new_adjacent_probe`: `true`.
+  - blocked primary family: `data-parallel-size`.
+  - latest two active bottlenecks after ignoring `probe_elapsed_s>` for voting: `ttft_prefill`, `ttft_prefill`.
+- Current status: the harness now has the mechanism needed to avoid continuing the exact DP-only direction seen in the smoke v2 plateau. The next real experiment should either switch to a bottleneck-justified mixed TP/DP candidate or return `should_stop=true`.
+
+Remaining next steps:
+
+1. Use the Fig18-style qwen27b 0-8k comparison in `docs/qwen27b-chat-0-8k-harness-fig18.md` as the current convergence evidence.
+2. If a future full no-relaunch rerun is required for publication-quality reproduction, reserve a multi-hour dash0 window; the comparable full-chat evaluator keeps drain-based probe isolation and is much slower than prefill smoke.

docs/qwen235b-thinking-decode/0323.md

@@ -0,0 +1,85 @@
+# qwen235b-thinking-decode-0323
+
+qwen3-235b-a22b `thinking` trace, `decode_only` mode, internal vLLM (`/usr/local/bin/vllm`), tuned on `thinking_w20260323_1000` with `TPOT <= 40ms`.
+
+## Setup
+
+- Hardware: `dash2`, `8x H20`
+- Model: `/home/admin/resource/model/464482ce.qwen3-235b-a22b/256k-0717`
+- Engine: internal vLLM, decode-only mode with `--kv-transfer-config {"kv_connector":"DecodeBenchConnector","kv_role":"kv_both"}`
+- Baseline topology: `TP=4, DP=2, EP=8`
+- Trace: `thinking_w20260323_1000`
+- Trace source: `trace_windows/traces/thinking_w20260323_1000.jsonl`
+- Window duration: `600s` (`10:00-10:10`, `2026-03-23`)
+- Request mode: `decode_only`
+- SLO:
+  - pass target: `95%`
+  - `TPOT <= 40ms`
+  - `TTFT` not enforced
+- Search:
+  - `sampling_u in [0, 0.125]`
+  - `max_probes = 6`
+  - `12` trials total
+- Proposal model: `codex / gpt-5.4`
+
+## Run assets
+
+- Study root: `/home/admin/cpfs/wjh/aituner/aituner/.aituner-decode/dash2-qwen235b-decode-thinking-run1-0323-tpot40-topology`
+- State: `/home/admin/cpfs/wjh/aituner/aituner/.aituner-decode/dash2-qwen235b-decode-thinking-run1-0323-tpot40-topology/state.json`
+- Spec: `/home/admin/cpfs/wjh/aituner/aituner/.aituner-decode/specs/dash2_qwen235b_decode_thinking_run1_0323_tpot40_topology.json`
+
+## Best result
+
+- Best trial: `trial-0007`
+- Best config delta:
+  - `gpu-memory-utilization=0.86`
+- Best topology stayed unchanged:
+  - `tensor-parallel-size=4`
+  - `data-parallel-size=2`
+  - `expert-parallel-size=8`
+- Best `sampling_u`: `0.033736228943`
+- Best request rate: `0.48333333333333334 req/s`
+- Best request rate per GPU: `0.06041666666666667 req/s/gpu`
+- Best pass rate: `0.9551724137931035`
+
+Compared with baseline:
+
+- `trial-0001`: `0.43333333333333335 req/s`, `0.05416666666666667 req/s/gpu`
+- `trial-0007`: `0.48333333333333334 req/s`, `0.06041666666666667 req/s/gpu`
+- Throughput gain: `1.12x`
+
+Best-point latency:
+
+- `TPOT mean/p50/p90/p95/p99 = 26.18 / 24.04 / 38.46 / 39.55 / 40.76 ms`
+
+## 12-trial summary
+
+| Trial | Proposed config delta | Result |
+| --- | --- | --- |
+| `trial-0001` | baseline `TP4/DP2/EP8` | `0.4333 req/s`, feasible |
+| `trial-0002` | `EP=4` | launch fail |
+| `trial-0003` | `gpu-memory-utilization=0.8`, `max-num-seqs=224` | infeasible |
+| `trial-0004` | `gpu-memory-utilization=0.8` | infeasible |
+| `trial-0005` | `gpu-memory-utilization=0.82` | `0.4650 req/s`, feasible |
+| `trial-0006` | `gpu-memory-utilization=0.84` | infeasible |
+| `trial-0007` | `gpu-memory-utilization=0.86` | `0.4833 req/s`, feasible, best |
+| `trial-0008` | `gpu-memory-utilization=0.87` | infeasible |
+| `trial-0009` | `gpu-memory-utilization=0.86`, `block-size=32` | launch fail |
+| `trial-0010` | `gpu-memory-utilization=0.86`, `max-num-seqs=208` | infeasible |
+| `trial-0011` | `gpu-memory-utilization=0.86`, `max-num-seqs=176` | infeasible |
+| `trial-0012` | `gpu-memory-utilization=0.86`, `max-num-batched-tokens=896` | infeasible |
+
+## Key insights
+
+- This `0323` window did not produce a better topology than the baseline. The winning move was a small memory-headroom increase, not a TP/DP/EP change.
+- `EP=4` was not viable under the current deployment shape and failed at launch, so the run quickly converged away from topology changes.
+- The best point is very close to the SLO edge: `TPOT p95 ~= 39.55ms`, so the remaining headroom is small.
+- Compared with the heavier `0327` decode-only tuning, `0323` is a milder window: baseline is already strong, and tuning only adds about `11.5%`.
+
+## Recommendation
+
+For a `0323`-like decode-only `thinking` window, keep the baseline `TP4/DP2/EP8` topology and use:
+
+- `gpu-memory-utilization=0.86`
+
+Do not treat this run as evidence that `0323` prefers the same topology changes as `0327`; this study mainly supports a small residency-headroom refinement.

docs/qwen235b-thinking-decode/README.md

@@ -77,6 +77,7 @@ Best-point latency:
 - `TP1/DP8/EP8` launched, but did not beat `TP2/DP4/EP8`.
 - `EP4` under `TP2/DP4` failed at launch and should be treated as negative evidence for this stack.
 - After topology settled at `TP2/DP4/EP8`, the useful runtime refinement was tighter decode batching: `max-num-seqs=128`, `max-num-batched-tokens=256`.
+- Harness mechanism and ablation notes are in `one-shot-mechanism-ablation-20260502.md`.
 
 ## Current recommendation
 

docs/qwen235b-thinking-decode/harness-20260428.md

@@ -0,0 +1,129 @@
+# Qwen235B Thinking Decode-Only Harness Run, 2026-04-28
+
+## Goal
+
+Run the qwen235b thinking decode-only tuning with the same harness-guided workflow used for the prefill-only test, while keeping the harness generic. The harness must use workload mode, configured SLOs, legal topology constraints, and measured trial history rather than testcase-specific throughput thresholds.
+
+## Baseline Reference
+
+The before-harness comparison run is `dash0-qwen235b-decode-thinking-run5-tpot40-topology`:
+
+| Iter | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 |
+| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
+| before harness request/s | 0.1267 | 0.2450 | infeasible | launch fail | infeasible | infeasible | infeasible | infeasible | 0.2817 | infeasible | infeasible | infeasible |
+
+Before harness, the best feasible config appeared at iter 9 with 0.2817 request/s.
+
+## Harness Change
+
+The decode-only harness now defaults to `decode_tpot` when `trace.request_mode=decode_only` and a TPOT SLO is configured. This avoids treating long decode-only prompt hints as a TTFT-prefill workload.
+
+Active decode harness families are generic:
+
+- `tensor-parallel-size`: legal TP/DP redistribution, judged by configured SLO pass rate and request_rate_per_gpu.
+- `data-parallel-size`: legal replica topology changes for decode/admission bottlenecks.
+- `max-num-seqs`: concurrency adjustment from observed TPOT failures or SLO headroom.
+- `max-num-batched-tokens`: decode batching adjustment after topology is stable.
+- `expert-parallel`: preserve known-valid EP topology, but change EP size only with EP-specific evidence.
+
+No qwen235b-specific threshold or testcase-specific rule was added.
+
+## Current Run
+
+Started on dash0, 8x H20.
+
+- Remote spec: `.aituner/harness-qwen235b-decode-20260428/dash0_qwen235b_decode_thinking_harness_20260428.json`
+- Remote store: `.aituner/harness-qwen235b-decode-20260428/dash0-qwen235b-decode-thinking-harness-20260428`
+- Remote tmux: `aituner_qwen235b_decode_harness_20260428`
+- Remote log: `logs/qwen235b_decode_harness_20260428.log`
+- Code commit: `39aa47f`
+- Verification: local and dash0 both passed `PYTHONPATH=src python3 -m unittest discover -s tests`.
+
+The first attempt started a duplicate `trial-0001` baseline. Because the identical baseline was already measured in run5 and the decode probe can run for many minutes, that duplicate run was stopped and GPUs were freed.
+
+The active run is now seeded from the real run5 baseline and continues from `trial-0002`:
+
+- Remote spec: `.aituner/harness-qwen235b-decode-20260428-seeded/dash0_qwen235b_decode_thinking_harness_seeded_20260428.json`
+- Remote store: `.aituner/harness-qwen235b-decode-20260428-seeded/dash0-qwen235b-decode-thinking-harness-seeded-20260428`
+- Seeded `trial-0001`: 0.1267 request/s, 0.0158 request/s/GPU, pass rate 0.9868.
+- `proposal-0002`: legal adjacent decode topology move from `TP4/DP2/EP8` to `TP2/DP4/EP8`; no EP-size search and no testcase threshold.
+- `trial-0002`: completed, 0.3767 request/s, 0.0471 request/s/GPU, pass rate 0.9779.
+- `trial-0003`: completed with no feasible point for `TP1/DP8/EP8`.
+- `trial-0004`: completed with no feasible point for `max-num-seqs=160`.
+- Important caveat: `trial-0004` did not actually validate `TP2/DP4/EP8 + max-num-seqs=160`. AITuner applies `config_patch` relative to the study base config, and the proposal only patched `max-num-seqs`. The actual launch therefore used the base topology `TP4/DP2/EP8 + max-num-seqs=160`, so this is not evidence that same-topology refinement around `trial-0002` is exhausted.
+- `trial-0005`: corrected same-topology validation, `TP2/DP4/EP8 + max-num-seqs=160`; completed with no feasible point.
+
+The `trial-0002` proposal matches the first useful topology direction from the earlier before-harness run, but the new harness-controlled run measured substantially better throughput for that topology.
+
+## Result Judgment
+
+Fig-18-style raw throughput table:
+
+| Run | Iter 1 | Iter 2 | Iter 3 | Iter 4 | Iter 5 | Iter 6 | Iter 7 | Iter 8 | Iter 9 | Iter 10 | Iter 11 | Iter 12 |
+| --- | ---: | ---: | --- | --- | --- | --- | --- | --- | ---: | --- | --- | --- |
+| before harness request/s | 0.1267 | 0.2450 | infeasible | launch fail | infeasible | infeasible | infeasible | infeasible | 0.2817 | infeasible | infeasible | infeasible |
+| harness request/s | 0.1267 | 0.3767 | infeasible | infeasible | infeasible | not run | not run | not run | not run | not run | not run | not run |
+
+Per-GPU throughput table:
+
+| Run | Iter 1 | Iter 2 | Iter 3 | Iter 4 | Iter 5 | Iter 6 | Iter 7 | Iter 8 | Iter 9 | Iter 10 | Iter 11 | Iter 12 |
+| --- | ---: | ---: | --- | --- | --- | --- | --- | --- | ---: | --- | --- | --- |
+| before harness req/s/GPU | 0.0158 | 0.0306 | infeasible | launch fail | infeasible | infeasible | infeasible | infeasible | 0.0352 | infeasible | infeasible | infeasible |
+| harness req/s/GPU | 0.0158 | 0.0471 | infeasible | infeasible | infeasible | not run | not run | not run | not run | not run | not run | not run |
+
+Decision: the harness accelerated convergence on qwen235b decode-only, but this is not a proof of global optimality after one proposal. The before-harness run first reached its best observed throughput at iter 9 with 0.2817 request/s. The harness run exceeded that value at iter 2 with 0.3767 request/s, a 1.34x improvement over the before-harness 12-iter best and a 2.97x improvement over the baseline config.
+
+The harness did not stop cleanly after finding the strong incumbent. It spent one additional trial on `TP1/DP8/EP8`, which found no feasible point. The next proposal intended same-topology runtime validation, but omitted the incumbent topology fields, so the materialized trial validated the base topology instead. This issue was corrected with `trial-0005`.
+
+Important interpretation: `trial-0002` should be called the current best observed config, not a global optimum proof. The harness got there quickly because the decode-only harness biases the first proposal toward the most relevant adjacent topology redistribution, `TP4/DP2/EP8 -> TP2/DP4/EP8`, instead of spending trials on prefill-oriented runtime knobs. Later validation now supports local optimality against the tested adjacent topology and the tested same-topology `max-num-seqs=160` runtime refinement.
+
+Follow-up implementation after this result:
+
+- `strong_incumbent.guard_active` no longer directly contributes to `should_stop_if_no_harness_can_justify_a_new_adjacent_probe`.
+- A strong incumbent now means "enter validation phase": run adjacent topology or same-topology runtime probes that could falsify the incumbent.
+- The proposal rules now explicitly say not to stop solely because a strong incumbent appeared.
+- Proposal parsing now accepts structured `observation`/`diagnosis` by converting them to text, so a usable validation proposal is not dropped only because the LLM used an object instead of a string.
+
+After the implementation fix, the previously rejected `proposal-0004` was resumed as a validation trial:
+
+- `trial-0004`: intended same-topology validation with `max-num-seqs=160`, but actually ran on base topology because the proposal omitted `TP2/DP4/EP8`.
+- Remote tmux: `aituner_qwen235b_decode_harness_validate_20260428`.
+- Result: completed with no feasible point. This is useful negative evidence for the base topology plus `max-num-seqs=160`, but not for the `trial-0002` incumbent topology.
+
+A second validation trial was then launched with the full incumbent topology in the patch:
+
+- `trial-0005` config: `TP2/DP4/EP8 + max-num-seqs=160`.
+- Search range: low `0.017028808593`, high `0.125`, tolerance `0.001`, max probes `6`.
+- Result: completed with no feasible point; `trial-0002` remained the best trial.
+- Probe outcomes:
+
+| Probe sampling_u | Request/s | Pass rate | Feasible | Early-stop reason |
+| ---: | ---: | ---: | --- | --- |
+| 0.0710144 | 1.7800 | 0.2818 | no | `slo_pass_rate_unrecoverable` |
+| 0.0440216 | 1.0900 | 0.1789 | no | `slo_pass_rate_unrecoverable` |
+| 0.0305252 | 0.7050 | 0.3002 | no | `slo_pass_rate_unrecoverable` |
+| 0.0237770 | 0.5417 | 0.4092 | no | `slo_pass_rate_unrecoverable` |
+| 0.0204029 | 0.4533 | 0.4890 | no | `slo_pass_rate_unrecoverable` |
+| 0.0187159 | 0.4117 | 0.5466 | no | `slo_pass_rate_unrecoverable` |
+
+This directly answers the one-iter-to-best concern for this refinement: the harness did not stop after `trial-0002`; it ran a corrected same-topology validation, and every tested point above the incumbent search floor failed the 95% TPOT SLO. Therefore `max-num-seqs=160` does not falsify `trial-0002` as the current best.
+
+## Follow-up Fix
+
+The seeded prompt exposed a generic diagnosis issue: if the best feasible probe had no latency failures, the harness could miss the prior infeasible probe that showed the real bottleneck at higher load. The harness now scans the probe sequence backward and uses the nearest non-trivial bottleneck before falling back to the best feasible probe. This keeps decode-only runs focused on `decode_tpot` after a feasible low-load point, without adding testcase thresholds.
+
+A second generic diagnosis bug was fixed: non-SLO bookkeeping counts such as `probe_elapsed_s>...` no longer collapse to `ttft_prefill` when TTFT/TPOT/request failure counts are all zero.
+
+## Follow-up Fix, 2026-04-30
+
+The base-relative patch issue is now guarded in code, not only in the LLM prompt. When `StudyStore.materialize_trial` sees a runtime/env-only proposal after a non-base incumbent has been found, it inherits the incumbent topology patch into the trial spec unless the proposal explicitly provides a topology. This keeps same-topology runtime validation on the actual incumbent while preserving the ability to test the base topology by stating it explicitly.
+
+Local verification at that commit: `PYTHONPATH=src python3 -m unittest discover -s tests` passed. The current repository suite has since grown; rerun the command rather than relying on this historical test count.
+
+## Current Harness Judgment
+
+For qwen235b decode-only, the harness still accelerates convergence: before harness, the best observed 12-iter result appeared at iter 9 with 0.2817 request/s; with harness, iter 2 reached 0.3767 request/s and later validation did not find a better adjacent or same-topology runtime point.
+
+The remaining optimization is validation cost, not convergence quality. `trial-0005` took a long time because early-stopped decode-only probes still had to wait for in-flight long-output requests unless the engine is restarted after early stop. As of 2026-05-02, decode-only studies default to `trace.restart_engine_after_early_stop=true` when the field is not explicitly set, and the qwen235b decode examples set it explicitly.
+
+See `docs/qwen235b-thinking-decode/one-shot-mechanism-ablation-20260502.md` for the detailed mechanism explanation and harness ablation.

docs/qwen235b-thinking-decode/one-shot-mechanism-ablation-20260502.md

@@ -0,0 +1,150 @@
+# qwen235b Decode Harness One-Shot Mechanism and Ablation, 2026-05-02
+
+## Question
+
+The harness run reached its best observed qwen235b decode-only config at iter 2:
+
+`TP4/DP2/EP8 -> TP2/DP4/EP8`
+
+This document explains why that happened, what information the harness added to the LLM prompt, what the LLM did with that information, and what the non-harness ablation shows.
+
+## Short Answer
+
+The iter-2 result is not magic and should not be described as a global optimum proof. It is a local topology sweet spot for this decode-only workload:
+
+- Baseline `TP4/DP2/EP8` has only 2 data-parallel replicas and pays tensor-parallel communication on every decode step.
+- `TP2/DP4/EP8` halves tensor-parallel width and doubles independent decode replicas while preserving the known-good EP8 MoE sharding.
+- `TP1/DP8/EP8` goes too far: it was tested next and produced no feasible point.
+- Same-topology `TP2/DP4/EP8 + max-num-seqs=160` was also tested later and produced no feasible point.
+
+So the harness run's iter 2 is best observed because it hit the nearby topology balance point early, and the follow-up validation probes did not falsify it.
+
+## Mechanism
+
+The workload is `decode_only` with `TPOT <= 40ms` and no TTFT objective. In this regime, the critical cost is steady-state token generation rather than prompt prefill latency.
+
+For a large MoE decode stack:
+
+- Higher TP can reduce per-GPU model shard size, but it also adds tensor-parallel collectives to every decode step.
+- Higher DP gives more independent serving replicas and absorbs bursty arrivals better, but each replica has less tensor parallelism.
+- EP should not be changed without EP-specific evidence, because MoE expert sharding affects launch safety, memory layout, and expert dispatch.
+
+The baseline shape `TP4/DP2/EP8` is therefore not obviously optimal for decode. The adjacent legal move `TP2/DP4/EP8` is the natural first test: reduce repeated per-token TP communication and increase replica count while keeping all 8 GPUs and EP8 fixed.
+
+The measured data supports this:
+
+| Run | Config | Best sampling_u | Request/s | Pass rate |
+| --- | --- | ---: | ---: | ---: |
+| harness trial-0001 | `TP4/DP2/EP8` | 0.0058594 | 0.1267 | 0.9868 |
+| harness trial-0002 | `TP2/DP4/EP8` | 0.0170288 | 0.3767 | 0.9779 |
+| harness trial-0003 | `TP1/DP8/EP8` | none | infeasible | none |
+| harness trial-0005 | `TP2/DP4/EP8 + max-num-seqs=160` | none | infeasible | none |
+
+## Harness Information Added
+
+The harness prompt added structured context that the non-harness prompt did not have:
+
+| Harness field | Concrete value in this run | How it affected the proposal |
+| --- | --- | --- |
+| workload mode | `request_mode=decode_only`; TTFT not an objective | Avoid prefill-first reasoning; optimize TPOT and decode throughput. |
+| active bottleneck | `decode_tpot` | Make TP/DP redistribution and decode batching relevant, not TTFT knobs. |
+| L-C-A profile | prompt p50 1491, p95 19670, p99 29961; prefix reuse about 0.41; burst ratio about 1.40 | Treat the workload as long-tail, moderately cache-reused, moderately bursty decode. |
+| current best | baseline request/s/GPU 0.0158, pass rate 0.9868 | Require proposals to improve per-GPU throughput under SLO. |
+| legal topology candidates | TP/DP products constrained to 8 GPUs; candidate includes `TP2/DP4/EP8` and `TP1/DP8/EP8` | Restrict search to launch-plausible adjacent topologies. |
+| knob harness rules | topology-first for `decode_tpot`; keep EP fixed without EP-specific evidence | Pick `TP2/DP4/EP8`, not EP changes or runtime-only knobs first. |
+| tested signatures | only baseline tested at iter 2 | Avoid repeating baseline; choose first adjacent topology. |
+
+The relevant LLM response for harness `proposal-0002` followed this structure:
+
+```json
+{
+  "diagnosis": "Follow the topology-first harness for decode_tpot. Because the incumbent already satisfies the TPOT SLO, the next justified adjacent probe is to trade some tensor parallelism for more data-parallel replicas, while keeping expert parallel fixed to avoid introducing an EP-specific variable without evidence. The adjacent legal move from TP4/DP2 is TP2/DP4 with EP8 preserved.",
+  "config_patch": {
+    "flag_patch": {
+      "tensor-parallel-size": 2,
+      "data-parallel-size": 4,
+      "expert-parallel-size": 8
+    }
+  }
+}
+```
+
+The important behavior is not just "choose TP2/DP4"; it is "choose the adjacent topology, keep EP fixed, judge by request_rate_per_gpu and TPOT SLO, then validate nearby alternatives."
+
+## Non-Harness Ablation
+
+The before-harness run is `dash0-qwen235b-decode-thinking-run5-tpot40-topology`.
+
+It is a useful ablation because it used the same trace, same model family, same baseline topology, same TPOT SLO, and same 12-trial budget, but did not include the structured harness context.
+
+| Iter | Non-harness proposal | Result |
+| ---: | --- | --- |
+| 1 | baseline `TP4/DP2/EP8` | 0.1267 request/s |
+| 2 | `TP2/DP4` | 0.2450 request/s |
+| 3 | `TP1/DP8/EP8` | infeasible |
+| 4 | `TP2/DP4/EP4` | launch fail |
+| 5 | `gpu-memory-utilization=0.8`, `max-num-seqs=256` | infeasible |
+| 6 | `max-num-seqs=128` | infeasible |
+| 7 | `block-size=128` | infeasible |
+| 8 | `max-num-batched-tokens=384` | infeasible |
+| 9 | `TP2/DP4/EP8 + max-num-seqs=128 + max-num-batched-tokens=256` | 0.2817 request/s |
+| 10 | trial 9 + `block-size=128` | infeasible |
+| 11 | `TP1/DP8/EP8 + max-num-seqs=128 + max-num-batched-tokens=256` | infeasible |
+| 12 | `TP2/DP4/EP8 + max-num-seqs=96 + max-num-batched-tokens=192` | infeasible |
+
+The non-harness LLM also found `TP2/DP4` at iter 2, so we should not claim that the harness uniquely discovered the direction. The difference is that the non-harness prompt left the model to reason from raw history and raw topology candidates. After iter 2 it spent trials on EP changes, memory/concurrency changes, block size, and batch-token variants before finding its best observed point at iter 9.
+
+## Full Harness Ablation
+
+| Run | Iter 1 | Iter 2 | Iter 3 | Iter 4 | Iter 5 | Best observed |
+| --- | ---: | ---: | --- | --- | --- | ---: |
+| non-harness | 0.1267 | 0.2450 | infeasible | launch fail | infeasible | 0.2817 at iter 9 |
+| harness | 0.1267 | 0.3767 | infeasible | infeasible | infeasible | 0.3767 at iter 2 |
+
+The harness accelerated convergence in two ways:
+
+- It made the first post-baseline trial a structured adjacent topology test with EP fixed.
+- It converted later iterations into validation of local alternatives rather than broad, weakly justified search.
+
+The measured validation points after iter 2:
+
+| Trial | Config | Outcome |
+| --- | --- | --- |
+| trial-0003 | `TP1/DP8/EP8` | no feasible point |
+| trial-0004 | intended `max-num-seqs=160`, but actually base topology due old base-relative patch issue | no feasible point, not valid incumbent validation |
+| trial-0005 | `TP2/DP4/EP8 + max-num-seqs=160` | no feasible point |
+
+For `trial-0005`, every probe above the incumbent floor failed:
+
+| Probe sampling_u | Request/s | Pass rate | Feasible |
+| ---: | ---: | ---: | --- |
+| 0.0710144 | 1.7800 | 0.2818 | no |
+| 0.0440216 | 1.0900 | 0.1789 | no |
+| 0.0305252 | 0.7050 | 0.3002 | no |
+| 0.0237770 | 0.5417 | 0.4092 | no |
+| 0.0204029 | 0.4533 | 0.4890 | no |
+| 0.0187159 | 0.4117 | 0.5466 | no |
+
+This is the evidence that iter 2 was not just a premature stop. The harness continued probing nearby alternatives, and those alternatives did not beat the incumbent.
+
+## Implementation Update
+
+Long decode-only validation exposed a cost issue: once a probe became SLO-unrecoverable, the worker still waited for in-flight long-output requests unless the study explicitly enabled engine restart after early stop.
+
+The implementation now makes this the default for decode-only studies:
+
+- `trace.request_mode=decode_only` and no explicit `restart_engine_after_early_stop` means `restart_engine_after_early_stop=true`.
+- An explicit `restart_engine_after_early_stop=false` is still honored.
+- Chat/prefill studies keep the old default `false`.
+- The LLM prompt now includes `early_stop_max_lag_s`, `early_stop_max_elapsed_s`, and `restart_engine_after_early_stop` in the trace block.
+- qwen235b decode example specs now explicitly set `restart_engine_after_early_stop=true`.
+
+This change does not alter the SLO decision for a probe. It changes the cost model after an already-unrecoverable probe: cancel in-flight requests, restart the engine cleanly, and move to the next probe instead of waiting for long decode tails.
+
+## Interpretation
+
+The correct claim is:
+
+The harness did not prove global optimality in one iteration. It made the first post-baseline proposal land on the correct local topology neighborhood, and follow-up harness validation failed to find a better adjacent topology or tested same-topology runtime refinement. On this workload, that was enough for iter 2 to remain the best observed configuration.
+
+The non-harness ablation shows that the model could guess the same topology direction, but without harness structure it spent the remaining budget exploring less controlled directions and only reached its best observed result at iter 9.

docs/qwen235b-thinking-prefill-harness-20260427.md

@@ -0,0 +1,58 @@
+# qwen235b Thinking Prefill Harness Test
+
+## Setup
+
+- Workload: `qwen3-235b-a22b` thinking trace, prefill-only replay with `min_tokens=max_tokens=1`.
+- Window: `thinking_w20260327_1000`.
+- SLO: 95% pass rate, stepped TTFT `3s/6s/9s`.
+- Metric: best-so-far feasible `request_rate_per_gpu`.
+- Before-harness source: actual 12-trial run
+  `.aituner-prefill/dash0-qwen235b-prefill-thinking-run1-ttft-topology`.
+- Harness test source:
+  `.aituner/harness-qwen235b-prefill-20260427/dash0-qwen235b-prefill-thinking-harness-run1-20260427`.
+
+## Result So Far
+
+The harness run was stopped after establishing the convergence result and observing the next weak proposal. The useful comparison is already visible by iter 2.
+
+| Variant | Iter 1 | Iter 2 | Iter 3 | Iter 4 | Iter 5 | Iter 6 | Iter 7 | Iter 8 | Iter 9 | Iter 10 | Iter 11 | Iter 12 |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
+| Before harness, actual run1 | 0.2029 | 0.2029 | 0.2029 | 0.2029 | 0.2029 | 0.3575 | 0.3575 | 0.3708 | 0.3708 | 0.3794 | 0.3794 | 0.3794 |
+| Harness, actual 2026-04-27 run | 0.1892 | 0.3863 | 0.3863 | 0.3863 | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a |
+
+## Trial Details
+
+| Variant | Iter | Config | Result |
+| --- | ---: | --- | --- |
+| Before harness | 1 | baseline `TP4/DP1/EP-off`, `MBT=8192` | `0.2029 req/s/gpu` |
+| Before harness | 2 | `DP=2`, `MBT=4096` | runtime failure |
+| Before harness | 3 | `DP=2`, `MBT=8192` | runtime failure |
+| Before harness | 4 | `EP=4` | launch failure |
+| Before harness | 6 | `TP8/DP1/EP-off`, `MBT=4096` | `0.3575 req/s/gpu` |
+| Before harness | 10 | `TP8/DP1/EP-off`, `MBT=3712` | `0.3794 req/s/gpu`, best |
+| Harness | 1 | baseline `TP4/DP1/EP-off`, `MBT=8192` | `0.1892 req/s/gpu` |
+| Harness | 2 | `TP8/DP1/EP-off`, `MBT=8192` | `0.3863 req/s/gpu`, best so far |
+| Harness | 3 | `TP8/DP1/EP=2` | launch failure |
+
+The harness baseline was slightly lower than the original baseline (`0.1892` vs `0.2029 req/s/gpu`), but iter 2 still exceeded the original 12-trial best (`0.3863` vs `0.3794 req/s/gpu`).
+
+## Convergence Judgment
+
+- Before harness reached its best at iter 10.
+- Harness reached a better result at iter 2.
+- Iterations-to-best improved from `10` to `2`, a `5x` improvement on this run.
+- The important behavior change is that the harness skipped the original failed DP2 and EP4 exploration and moved directly from baseline to `TP8/DP1`.
+
+## Follow-Up Optimization
+
+The run also exposed a remaining weakness: after reaching the strong `TP8/DP1` incumbent, the LLM proposed `EP=2`, which failed at launch. To address that, the harness was tightened after this test:
+
+- strong-incumbent stop threshold changed from `3x` to `1.8x` over baseline;
+- expert parallel is now explicitly guarded and should not be introduced for TTFT-prefill bottlenecks without direct positive EP evidence.
+
+With the new guard, the intended behavior after this iter-2 result is `should_stop=true` unless a same-topology runtime harness has strong direct evidence.
+
+## Run Status
+
+- The 2026-04-27 harness run was stopped after collecting the iter-2 convergence result and the iter-3 EP failure.
+- GPUs were freed after stopping the run.

docs/qwen235b-thinking-prefill/7day-compare.md

@@ -0,0 +1,79 @@
+# qwen235b-thinking-prefill-7day-compare
+
+qwen3-235b-a22b `thinking` trace, prefill-only replay with `output_length=1`, comparing 3 configs across 7 daily `10:00-10:10` windows by `request_rate_per_gpu`.
+
+## Setup
+
+- Hardware: `dash1`, `8x H20`
+- Model: `/home/admin/resource/model/464482ce.qwen3-235b-a22b/256k-0717`
+- Engine: internal vLLM, baseline aligned to `~/run_qwen235b.sh`
+- Trace set: `thinking_w20260321_1000` to `thinking_w20260327_1000`
+- Window duration: `600s` each
+- Request mode: `chat`
+- Replay override: `min_tokens=max_tokens=1`
+- SLO:
+  - pass target: `95%`
+  - `TTFT <= 2000ms` for `<=8191` input tokens
+  - `TTFT <= 4000ms` for `<=32767` input tokens
+  - `TTFT <= 6000ms` for `>32767` input tokens
+- Search:
+  - each candidate independently binary-searches its own `sampling_u`
+  - `sampling_u in [0, 0.125]`
+  - `max_probes = 6`
+
+## Candidates
+
+- `baseline`
+  - `TP=4, DP=1, EP=off`
+  - baseline `run_qwen235b.sh` shape
+- `tuned_0323`
+  - tuned on `thinking_w20260323_1000`
+  - `TP=4, DP=1, EP=off`
+  - `max-num-batched-tokens=3072`
+  - `max-num-seqs=32`
+- `tuned_0327`
+  - tuned on `thinking_w20260327_1000`
+  - `TP=8, DP=1, EP=off`
+  - `max-num-batched-tokens=6144`
+  - `max-num-seqs=48`
+  - `block-size=32`
+
+## Run assets
+
+- Compare root: `/home/admin/cpfs/wjh/aituner/aituner/.aituner-compare/dash1-qwen235b-prefill-thinking-7day-baseline-vs-0323-vs-0327`
+- Summary: `/home/admin/cpfs/wjh/aituner/aituner/.aituner-compare/dash1-qwen235b-prefill-thinking-7day-baseline-vs-0323-vs-0327/summary.json`
+- Report: `/home/admin/cpfs/wjh/aituner/aituner/.aituner-compare/dash1-qwen235b-prefill-thinking-7day-baseline-vs-0323-vs-0327/report.md`
+- Compare spec: `/home/admin/cpfs/wjh/aituner/aituner/configs/examples/dash1_qwen235b_prefill_thinking_7day_compare.json`
+
+## Aggregate result
+
+- Wins by `request_rate_per_gpu`:
+  - `tuned_0327`: `5 / 7`
+  - `baseline`: `2 / 7`
+  - `tuned_0323`: `0 / 7`
+- Mean `request_rate_per_gpu`:
+  - `baseline`: `0.13845`
+  - `tuned_0323`: `0.12756`
+  - `tuned_0327`: `0.17232`
+- Relative to baseline:
+  - `tuned_0323`: `0.92x` mean per-GPU throughput
+  - `tuned_0327`: `1.24x` mean per-GPU throughput
+
+## Per-day result
+
+| Date | baseline req/s/gpu | tuned_0323 req/s/gpu | tuned_0327 req/s/gpu | Winner |
+| --- | ---: | ---: | ---: | --- |
+| `2026-03-21` | `0.08500` | `0.03917` | `0.14375` | `tuned_0327` |
+| `2026-03-22` | `0.10125` | `0.12083` | `0.15313` | `tuned_0327` |
+| `2026-03-23` | `0.12792` | `0.12792` | `0.19167` | `tuned_0327` |
+| `2026-03-24` | `0.09000` | `0.09583` | `0.11250` | `tuned_0327` |
+| `2026-03-25` | `0.13792` | `0.13208` | `0.13146` | `baseline` |
+| `2026-03-26` | `0.32000` | `0.25917` | `0.23375` | `baseline` |
+| `2026-03-27` | `0.10708` | `0.11792` | `0.24000` | `tuned_0327` |
+
+## Key takeaways
+
+- `tuned_0327` is the only candidate with clear cross-day value. It wins `5/7` windows and improves mean per-GPU throughput by about `24%`.
+- `tuned_0323` does not generalize. It is slightly more conservative and keeps high pass rate, but mean per-GPU throughput is below baseline.
+- The `0327` winner is not universal. On `2026-03-25` and especially `2026-03-26`, the 4-GPU baseline is more efficient per GPU than the `TP8` tuned shape.
+- The practical reading is that prefill-only tuning has workload-regime sensitivity. `TP8 + 6144 + 48 + block-size=32` is a strong default candidate, but not a global static optimum across all days.

docs/qwen235b-thinking-prefill/ttft-1s-2s-0-32k.md

@@ -0,0 +1,98 @@
+# qwen235b-thinking-prefill-ttft-1s-2s-0-32k
+
+qwen3-235b-a22b `thinking` trace, prefill-only replay with `output_length=1`, internal vLLM (`/usr/local/bin/vllm`), tuned on the `0~32k` input bucket under a stricter stepped TTFT SLO.
+
+## Setup
+
+- Hardware: `dash1`, `8x H20`
+- Model: `/home/admin/resource/model/464482ce.qwen3-235b-a22b/256k-0717`
+- Engine: internal vLLM, baseline aligned to `~/run_qwen235b.sh`
+- Baseline topology: `TP=4, DP=1, EP=1`
+- Trace: `thinking_w20260327_1000`
+- Trace source: `trace_windows/traces/thinking_w20260327_1000.jsonl`
+- Window duration: `600s` (`10:00-10:10`, `2026-03-27`)
+- Request mode: `chat`
+- Replay override: `min_tokens=max_tokens=1`
+- Input bucket: `0 <= input_length <= 32768`
+- SLO:
+  - pass target: `95%`
+  - `TTFT <= 1000ms` for `<=8191` input tokens
+  - `TTFT <= 2000ms` for `<=32767` input tokens
+  - `TTFT <= 2000ms` fallback bucket
+- Search:
+  - `sampling_u in [0, 0.125]`
+  - `max_probes = 6`
+  - `12` trials total
+- Proposal model: `codex / gpt-5.4`
+
+## Run assets
+
+- Study root: `/home/admin/cpfs/wjh/aituner/aituner/.aituner-prefill/dash1-qwen235b-prefill-thinking-run5-ttft-1s-2s-0-32k-topology`
+- State: `/home/admin/cpfs/wjh/aituner/aituner/.aituner-prefill/dash1-qwen235b-prefill-thinking-run5-ttft-1s-2s-0-32k-topology/state.json`
+- Log: `/home/admin/cpfs/wjh/aituner/aituner/logs/q235b_prefill_1s2s_0_32k.log`
+- Spec: `/home/admin/cpfs/wjh/aituner/aituner/configs/examples/dash1_qwen235b_prefill_thinking_run5_ttft_1s_2s_0_32k.json`
+
+## Best result
+
+- Best trial: `trial-0011`
+- Best config:
+  - `tensor-parallel-size=8`
+  - `data-parallel-size=1`
+  - `enable-expert-parallel=false`
+  - `max-num-batched-tokens=4096`
+  - `max-num-seqs=16`
+  - `VLLM_ENABLE_TORCH_COMPILE=0`
+- Best `sampling_u`: `0.073767628521`
+- Best request rate: `1.8516666666666666 req/s`
+- Best request rate per GPU: `0.23145833333333332 req/s/gpu`
+- Best pass rate: `0.9558955895589559`
+
+Compared with baseline:
+
+- `trial-0001`: `0.47 req/s`, `0.1175 req/s/gpu`
+- `trial-0011`: `1.8516666666666666 req/s`, `0.23145833333333332 req/s/gpu`
+- Raw throughput gain: `3.94x`
+- Per-GPU throughput gain: `1.97x`
+
+Best-point latency:
+
+- baseline `trial-0001` TTFT mean/p50/p90/p95/p99 = `236.68 / 75.19 / 294.39 / 1378.79 / 3118.86 ms`
+- best `trial-0011` TTFT mean/p50/p90/p95/p99 = `223.70 / 65.67 / 261.69 / 1065.31 / 3648.34 ms`
+
+## 12-trial summary
+
+| Trial | Proposed config delta | Result |
+| --- | --- | --- |
+| `trial-0001` | baseline `TP4/DP1/EP-off`, compile on | `0.4700 req/s`, feasible |
+| `trial-0002` | `TP4/DP2`, `EP-off` | probe-search failure |
+| `trial-0003` | `TP4/DP1/EP4`, `max-num-batched-tokens=4096` | launch fail |
+| `trial-0004` | `VLLM_ENABLE_TORCH_COMPILE=0`, `max-num-batched-tokens=6144` | infeasible |
+| `trial-0005` | compile off, `max-num-batched-tokens=4096` | infeasible |
+| `trial-0006` | compile off, `max-num-seqs=32` | infeasible |
+| `trial-0007` | compile off, `TP8/DP1/EP-off` | `1.3817 req/s`, feasible |
+| `trial-0008` | `trial-0007 + max-num-seqs=32` | `1.5983 req/s`, feasible |
+| `trial-0009` | `trial-0008 + max-num-batched-tokens=6144` | `1.8017 req/s`, feasible |
+| `trial-0010` | `trial-0008 + max-num-batched-tokens=4096` | `1.8300 req/s`, feasible |
+| `trial-0011` | `trial-0010 + max-num-seqs=16` | `1.8517 req/s`, feasible, best |
+| `trial-0012` | `trial-0011 + max-num-batched-tokens=3072` | infeasible |
+
+## Key insights
+
+- Under the stricter `1s/2s` TTFT SLO, the main win still came from topology first: `TP4 -> TP8`.
+- `TP4/DP2` and `EP4` remain negative evidence in this stack. The former failed in probe search; the latter failed at engine launch.
+- Runtime-only tuning inside the 4-GPU topology did not beat baseline at all. The useful search space opened only after moving to `TP8/DP1`.
+- After the `TP8` switch, the winning runtime shape became more conservative than the looser prefill studies: `max-num-batched-tokens=4096` and `max-num-seqs=16`.
+- This run shows that even under a much tighter TTFT target, the `TP8` shape still improves both raw throughput and per-GPU throughput materially over baseline.
+
+## Recommendation
+
+For `qwen235b thinking prefill-only` on the `0~32k` bucket under the `1s/2s` stepped TTFT SLO, use:
+
+- `tensor-parallel-size=8`
+- `data-parallel-size=1`
+- `enable-expert-parallel=false`
+- `max-num-batched-tokens=4096`
+- `max-num-seqs=16`
+- `VLLM_ENABLE_TORCH_COMPILE=0`
+
+Keep the rest of the `run_qwen235b.sh` baseline unchanged.

docs/qwen235b-thinking-prefill/ttft-tight-0327.md

@@ -0,0 +1,102 @@
+# qwen235b-thinking-prefill-ttft-tight-0327
+
+qwen3-235b-a22b `thinking` trace, prefill-only replay with `output_length=1`, internal vLLM (`/usr/local/bin/vllm`), tuned on `thinking_w20260327_1000` under tighter stepped TTFT SLO.
+
+## Setup
+
+- Hardware: `dash0`, `8x H20`
+- Model: `/home/admin/resource/model/464482ce.qwen3-235b-a22b/256k-0717`
+- Engine: internal vLLM, baseline aligned to `~/run_qwen235b.sh`
+- Baseline topology: `TP=4, DP=1, EP=1`
+- Trace: `thinking_w20260327_1000`
+- Trace source: `trace_windows/traces/thinking_w20260327_1000.jsonl`
+- Window duration: `600s` (`10:00-10:10`, `2026-03-27`)
+- Request mode: `chat`
+- Replay override: `min_tokens=max_tokens=1`
+- SLO:
+  - pass target: `95%`
+  - `TTFT <= 2000ms` for `<=8191` input tokens
+  - `TTFT <= 4000ms` for `<=32767` input tokens
+  - `TTFT <= 6000ms` for `>32767` input tokens
+- Search:
+  - `sampling_u in [0, 0.125]`
+  - `max_probes = 6`
+  - `12` trials total
+- Proposal model: `codex / gpt-5.4`
+
+## Run assets
+
+- Study root: `/home/admin/cpfs/wjh/aituner/aituner/.aituner-prefill/dash0-qwen235b-prefill-thinking-run2-ttft-tight-topology`
+- State: `/home/admin/cpfs/wjh/aituner/aituner/.aituner-prefill/dash0-qwen235b-prefill-thinking-run2-ttft-tight-topology/state.json`
+- Spec: `/home/admin/cpfs/wjh/aituner/aituner/configs/examples/dash0_qwen235b_prefill_thinking_run2_ttft_tight.json`
+
+## Best result
+
+- Best trial: `trial-0012`
+- Best config:
+  - `tensor-parallel-size=8`
+  - `data-parallel-size=1`
+  - `enable-expert-parallel=false`
+  - `max-num-batched-tokens=6144`
+  - `max-num-seqs=48`
+  - `block-size=32`
+- Best `sampling_u`: `0.098106384277`
+- Best request rate: `2.4966666666666666 req/s`
+- Best request rate per GPU: `0.3120833333333333 req/s/gpu`
+- Best pass rate: `0.9506008010680908`
+
+Compared with baseline:
+
+- `trial-0001`: `0.4716666666666667 req/s`, `0.11791666666666667 req/s/gpu`
+- `trial-0012`: `2.4966666666666666 req/s`, `0.3120833333333333 req/s/gpu`
+- Raw throughput gain: `5.29x`
+- Per-GPU throughput gain: `2.65x`
+
+Compared with the looser TTFT study on the same `2026-03-27` window:
+
+- looser-SLO best: `3.035 req/s`, `0.379375 req/s/gpu`
+- tighter-SLO best: `2.4966666666666666 req/s`, `0.3120833333333333 req/s/gpu`
+- throughput retained: `82.26%`
+- throughput drop: `17.74%`
+
+Best-point latency:
+
+- `TTFT mean/p50/p90/p95/p99 = 413.92 / 67.86 / 1456.32 / 2286.90 / 5326.23 ms`
+
+## 12-trial summary
+
+| Trial | Proposed config delta | Result |
+| --- | --- | --- |
+| `trial-0001` | baseline `TP4/DP1/EP-off`, `max-num-batched-tokens=8192` | `0.4717 req/s`, feasible |
+| `trial-0002` | `TP4/DP2` | probe-search failure |
+| `trial-0003` | `TP8/DP1/EP-off` | `1.9200 req/s`, feasible |
+| `trial-0004` | `TP8/DP1/EP8` | launch fail |
+| `trial-0005` | `trial-0003 + max-num-batched-tokens=6144` | `2.2517 req/s`, feasible |
+| `trial-0006` | `trial-0003 + max-num-batched-tokens=4096` | infeasible |
+| `trial-0007` | `trial-0003 + max-num-batched-tokens=5120` | infeasible |
+| `trial-0008` | `trial-0003 + max-num-batched-tokens=5632` | infeasible |
+| `trial-0009` | `trial-0005 + max-num-seqs=32` | infeasible |
+| `trial-0010` | `trial-0005 + max-num-seqs=48` | infeasible |
+| `trial-0011` | `trial-0005 + block-size=32` | infeasible |
+| `trial-0012` | `trial-0005 + max-num-seqs=48, block-size=32` | `2.4967 req/s`, feasible, best |
+
+## Key insights
+
+- This tuning is also on `2026-03-27`, not a different day. The change is the tighter TTFT step SLO.
+- The best topology still moved to `TP8/DP1/no-EP`; tighter TTFT did not change the topology conclusion.
+- Tighter TTFT did change the runtime sweet spot. The best runtime shape is not the looser-study `3712` token batch, but `6144 + max-num-seqs=48 + block-size=32`.
+- `DP2` and `EP` remained negative evidence in this stack: `TP4/DP2` failed during probing, and `TP8 + EP8` failed at launch.
+- Relative to the looser TTFT study on the same day, stricter TTFT costs about `17.7%` throughput, but the tuned result still keeps a large margin over baseline.
+
+## Recommendation
+
+For the tighter stepped TTFT SLO on `thinking_w20260327_1000`, use:
+
+- `tensor-parallel-size=8`
+- `data-parallel-size=1`
+- `enable-expert-parallel=false`
+- `max-num-batched-tokens=6144`
+- `max-num-seqs=48`
+- `block-size=32`
+
+Keep the rest of the `run_qwen235b.sh` baseline unchanged.

docs/qwen27b-chat-0-8k-7day-compare/README.md

@@ -0,0 +1,114 @@
+# qwen27b-chat-0-8k-7day-compare
+
+qwen3.5-27b `chat` trace, `0~8k` input bucket, tuned-best vs baseline cross-day compare on internal vLLM (`/usr/local/bin/vllm`), compared by `request_rate_per_gpu`.
+
+## Setup
+
+- Hardware: `dash1`, `8x H20`
+- Model: `/home/admin/resource/model/464482ce/qwen3.5-27b/256k-0223-internal`
+- Engine: internal vLLM
+- Baseline: empty patch over the study spec baseline, aligned to `~/run_qwen27b.sh` `TP=1, DP=1`
+- Tuned best source: `trial-0004` from `dash0-qwen27b-tight-slo-10min-run9-chat-0-8k-codex-topology`
+- Tuned best config:
+  - `tensor-parallel-size=2`
+  - `data-parallel-size=1`
+- Trace family: `chat`
+- Input bucket: `0 <= input_length <= 8192`
+- Time range scanned: `2026-03-11` to `2026-03-17`
+- Available windows in this slot: `7`
+  - `chat_w20260311_1000`
+  - `chat_w20260312_1000`
+  - `chat_w20260313_1000`
+  - `chat_w20260314_1000`
+  - `chat_w20260315_1000`
+  - `chat_w20260316_1000`
+  - `chat_w20260317_1000`
+- Window duration: `600s` (`10:00-10:10`)
+- Request mode: `chat`
+- SLO:
+  - pass target: `95%`
+  - `TTFT <= 2000ms` for `<=4096` input tokens
+  - `TTFT <= 4000ms` for `<=32768` input tokens
+  - `TTFT <= 6000ms` for `>32768` input tokens
+  - `TPOT <= 50ms`
+- Search:
+  - binary search on `sampling_u`
+  - `max_probes = 6`
+- Proposal model for tuned source: `codex / gpt-5.4`
+
+## Run assets
+
+- Compare root: `/home/admin/cpfs/wjh/aituner/aituner/.aituner-compare/dash1-qwen27b-chat-0-8k-7days-compare`
+- Summary: `/home/admin/cpfs/wjh/aituner/aituner/.aituner-compare/dash1-qwen27b-chat-0-8k-7days-compare/summary.json`
+- Report: `/home/admin/cpfs/wjh/aituner/aituner/.aituner-compare/dash1-qwen27b-chat-0-8k-7days-compare/report.md`
+- Compare spec: `/home/admin/cpfs/wjh/aituner/aituner/.aituner-compare/specs/qwen27b_chat_0_8k_compare_dash1.json`
+- Tuned study root: `/home/admin/cpfs/wjh/aituner/aituner/.aituner-tight/dash0-qwen27b-tight-slo-10min-run9-chat-0-8k-codex-topology`
+
+## Tuned-source result
+
+- Best trial: `trial-0004`
+- Best config:
+  - `tensor-parallel-size=2`
+  - `data-parallel-size=1`
+- Best `sampling_u`: `0.013061523438`
+- Best request rate: `0.405 req/s`
+- Best request rate per GPU: `0.2025 req/s/gpu`
+- Best pass rate: `0.9629629629629629`
+
+Compared with the single-day baseline on `chat_w20260311_1000`:
+
+- `trial-0001`: `0.035 req/s`, `0.035 req/s/gpu`
+- `trial-0004`: `0.405 req/s`, `0.2025 req/s/gpu`
+- Raw throughput gain: `11.57x`
+- Per-GPU throughput gain: `5.79x`
+
+## 12-trial summary
+
+| Trial | Proposed config delta | Result |
+| --- | --- | --- |
+| `trial-0001` | baseline `TP1/DP1` | `0.0350 req/s`, `0.0350 req/s/gpu`, feasible |
+| `trial-0002` | `DP=2` | `0.1233 req/s`, `0.0617 req/s/gpu`, feasible |
+| `trial-0003` | `DP=4` | `0.1567 req/s`, `0.0392 req/s/gpu`, feasible |
+| `trial-0004` | `TP=2, DP=1` | `0.4050 req/s`, `0.2025 req/s/gpu`, feasible, best |
+| `trial-0005` | `trial-0004 + max-num-batched-tokens=16384` | infeasible |
+| `trial-0006` | `trial-0004 + max-num-seqs=24` | infeasible |
+| `trial-0007` | `trial-0004 + max-num-batched-tokens=12288` | infeasible |
+| `trial-0008` | `trial-0004 + block-size=32` | infeasible |
+| `trial-0009` | `trial-0004 + gpu-memory-utilization=0.93` | infeasible |
+| `trial-0010` | `trial-0004 + max-num-seqs=16, max-num-batched-tokens=6144` | infeasible |
+| `trial-0011` | `trial-0004 + enable-prefix-caching=false` | infeasible |
+| `trial-0012` | `trial-0004 + block-size=128` | infeasible |
+
+## Aggregate result
+
+- Comparable wins: tuned `5`, baseline `0`
+- Incomparable windows: `2`
+- Baseline mean request rate: `0.046 req/s`
+- Tuned mean request rate: `0.4723809523809524 req/s`
+- Baseline mean request rate per GPU: `0.046 req/s/gpu`
+- Tuned mean request rate per GPU: `0.2361904761904762 req/s/gpu`
+
+## Per-window result
+
+| Window | Date | Baseline req/s/gpu | Tuned req/s/gpu | Winner |
+| --- | --- | ---: | ---: | --- |
+| `chat_w20260311_1000` | `2026-03-11` | `0.035` | `0.21416666666666667` | `tuned` |
+| `chat_w20260312_1000` | `2026-03-12` | `None` | `0.28` | `incomparable` |
+| `chat_w20260313_1000` | `2026-03-13` | `0.03166666666666667` | `0.265` | `tuned` |
+| `chat_w20260314_1000` | `2026-03-14` | `0.021666666666666667` | `0.24083333333333334` | `tuned` |
+| `chat_w20260315_1000` | `2026-03-15` | `0.12166666666666667` | `0.23083333333333333` | `tuned` |
+| `chat_w20260316_1000` | `2026-03-16` | `0.02` | `0.2275` | `tuned` |
+| `chat_w20260317_1000` | `2026-03-17` | `None` | `0.195` | `incomparable` |
+
+## Key insights
+
+- The tuned-source tuning itself was simple and topology-driven. The winning patch is only `TP1 -> TP2`; later runtime-only tweaks all failed to beat it.
+- This compare does not support the conclusion that the tuned config lacks generalization. Across the full 7-day slice, tuned wins every directly comparable window.
+- The two `incomparable` days are not execution failures. Baseline completed probing but never found a single feasible `sampling_u` under the target SLO, while tuned still found feasible operating points.
+- The tuned `TP=2, DP=1` shape is materially more robust than the `TP=1, DP=1` baseline for this `0~8k` chat bucket.
+- The weekend windows do not break the result. `2026-03-14` is another clear tuned win, and even on `2026-03-15`, where baseline is relatively stronger than other days, tuned still wins by about `1.90x` on `req/s/gpu`.
+- The throughput gap remains large even after normalizing by GPU count, so this is not just a raw-card-count artifact.
+
+## Recommendation
+
+For `qwen27b chat 0~8k`, keep using the tuned `TP=2, DP=1` serving shape as the default candidate over the `TP=1, DP=1` baseline, and treat cross-day robustness as confirmed on the full 7-day window set.

docs/qwen27b-chat-0-8k-current-config-fig18-20260506.md

@@ -0,0 +1,173 @@
+# qwen27b-chat-0-8k Current-Config Fig18 Plan
+
+## Question
+
+The earlier tables used best-so-far throughput. That is useful for deciding the
+best deployable incumbent, but it hides bad proposals because the curve is
+monotonic by construction. To judge whether the harness makes tuning more
+directional, the primary curve must be each iteration's measured current config
+performance.
+
+## Why Final Performance Can Be Close
+
+Harness and no-harness can converge to similar final throughput when the search
+space contains one dominant simple family. In this setup the dominant family is
+`TP=2, DP=1` over the `run_qwen27b.sh` baseline. The no-harness LLM can still
+eventually discover that family within 12 iterations, so final best performance
+can be close.
+
+The difference the harness is expected to improve is not necessarily the final
+12-iter maximum. It should improve:
+
+- iterations-to-first-good-config;
+- number of worse or infeasible proposals after an incumbent is found;
+- measured-current config oscillation;
+- early-stop behavior once adjacent harness probes no longer justify more GPU
+  trials.
+
+## Metrics
+
+- `measured-current`: each trial's own feasible `request_rate_per_gpu`.
+  Failed or no-feasible-point trials are recorded as `NA`.
+- `accepted-incumbent`: best deployable value after each trial. This is the
+  standard best-so-far curve and is monotonic by definition.
+- `iters-to-best`: first iteration where the final best value or equivalent
+  config family appears.
+- `wasted-trials-after-best`: trials after first best that are worse, infeasible,
+  or no-feasible-point.
+
+## Historical Run9 Re-Read
+
+Source:
+`.aituner-tight/dash0-qwen27b-tight-slo-10min-run9-chat-0-8k-codex-topology`
+on dash0.
+
+| Variant | Curve | Iter 1 | Iter 2 | Iter 3 | Iter 4 | Iter 5 | Iter 6 | Iter 7 | Iter 8 | Iter 9 | Iter 10 | Iter 11 | Iter 12 |
+| --- | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
+| no-harness run9 | measured-current | 0.0350 | 0.0617 | 0.0392 | 0.2025 | NA | NA | NA | NA | NA | NA | NA | NA |
+| no-harness run9 | accepted-incumbent | 0.0350 | 0.0617 | 0.0617 | 0.2025 | 0.2025 | 0.2025 | 0.2025 | 0.2025 | 0.2025 | 0.2025 | 0.2025 | 0.2025 |
+
+Interpretation: the no-harness current-config curve already has a regression at
+iter 3 and then many no-feasible-point runtime probes. The monotonic curve only
+shows the incumbent policy, not proposal quality.
+
+## New Paired Test Plan
+
+Run on dash0 with internal vLLM and the real `chat_w20260311_1000` 0-8k replay:
+
+- Base spec: `configs/examples/dash0_qwen27b_tight_slo_run4_0_8k.json`.
+- Model path:
+  `/home/admin/resource/model/464482ce/qwen3.5-27b/256k-0223-internal`.
+- Naming note: local configs and dash0 model directories expose this setup as
+  Qwen3.5-27B/Qwen35-27B, not `qwen32b`.
+- Engine: `/usr/local/bin/vllm`, baseline aligned with `~/run_qwen27b.sh`.
+- SLO: 95% pass, stepped TTFT `2s/4s/6s`, TPOT `<=50ms`.
+- Search: `low=0`, `high=0.0625`, `max_probes=6`, `tolerance=0.001`.
+- no-harness study:
+  `.aituner-tight/dash0-qwen27b-tight-slo-10min-run10-chat-0-8k-current-noharness`.
+- harness study:
+  `.aituner-tight/dash0-qwen27b-tight-slo-10min-run10-chat-0-8k-current-harness`.
+
+The result table will report both curves. The harness is considered successful
+only if it reaches the same or better incumbent in fewer iterations and reduces
+the measured-current regressions or replaces them with an explicit harness stop.
+
+## Run Status
+
+- 2026-05-06 07:05 CST: dash0 checked, 8 H20 GPUs idle.
+- 2026-05-06 07:05 CST: generated paired specs under
+  `.aituner-tight/specs/`.
+- 2026-05-06 07:05 CST: started no-harness full 12-iter run in tmux session
+  `qwen27b_run10_noharness_20260506`.
+- 2026-05-06 07:18 CST: stopped the duplicate fresh no-harness run before
+  completion. Reason: run9 is already a completed real 12-iter no-harness run
+  for the same internal vLLM 0-8k setup, while the fresh full-chat run would
+  spend a multi-hour dash0 slot duplicating that curve.
+- 2026-05-06 07:20 CST: seeded the harness study with the real run9 baseline
+  measurement as `trial-0001`, then started the harness run with
+  `--skip-baseline` in tmux session `qwen27b_run10_harness_skipbase_20260506`.
+- 2026-05-06 07:20 CST: harness generated deterministic `trial-0002`:
+  `{"tensor-parallel-size": 2}`.
+- 2026-05-06 08:11 CST: harness `trial-0002` completed:
+  `TP=2`, `0.2142 request_rate_per_gpu`.
+- 2026-05-06 08:19 CST: harness `trial-0003` failed at engine launch.
+  Root cause: the old runtime refinement coupled `gpu-memory-utilization=0.95`
+  with larger `max-num-batched-tokens`, causing speculative sampler warmup OOM.
+  This is a generic harness safety bug; fixed locally by removing the automatic
+  memory-utilization bump from runtime refinement.
+- 2026-05-06 09:24 CST: harness `trial-0004` completed:
+  `TP=4`, `0.4429 request_rate_per_gpu`. All six probes were feasible up to
+  `sampling_u=0.0615234375`, so this study is near the configured
+  `search.high=0.0625` ceiling.
+- 2026-05-06 09:25 CST: old harness repeated the same unsafe runtime refinement
+  for TP4 and `trial-0005` failed at engine launch for the same OOM reason. The
+  old process was stopped before continuing.
+- 2026-05-06 09:37 CST: pulled commit `5d96689` on dash0 and resumed. The
+  runtime-refinement OOM was fixed, but the stop guard was still too strict: it
+  did not treat a feasible high-edge probe with a small number of SLO failures
+  as saturation, even though the probe already met the 95% pass-rate target.
+- 2026-05-06 09:50 CST: stopped the unnecessary product-8 validation. The queued
+  `trial-0006`/`trial-0007` are not used for convergence claims.
+- 2026-05-06 09:56 CST: pulled commit `f653af0` on dash0. The fixed high-edge
+  stop guard produced `harness-stop-0008` without launching another GPU trial.
+
+## Current Results
+
+Unit: feasible `request_rate_per_gpu`. `NA` means the current trial did not
+produce a feasible deployable config.
+
+| Variant | Curve | Iter 1 | Iter 2 | Iter 3 | Iter 4 | Iter 5 | Iter 6 | Iter 7 | Iter 8 | Iter 9 | Iter 10 | Iter 11 | Iter 12 |
+| --- | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
+| no-harness run9 | measured-current | 0.0350 | 0.0617 | 0.0392 | 0.2025 | NA | NA | NA | NA | NA | NA | NA | NA |
+| no-harness run9 | accepted-incumbent | 0.0350 | 0.0617 | 0.0617 | 0.2025 | 0.2025 | 0.2025 | 0.2025 | 0.2025 | 0.2025 | 0.2025 | 0.2025 | 0.2025 |
+| harness run10 | measured-current | 0.0350 | 0.2142 | NA | 0.4429 | NA | skipped | skipped | stop |  |  |  |  |
+| harness run10 | accepted-incumbent | 0.0350 | 0.2142 | 0.2142 | 0.4429 | 0.4429 | 0.4429 | 0.4429 | 0.4429 stop |  |  |  |  |
+
+The harness result is stronger than the earlier strict replay. It did not merely
+reach the same TP2 region earlier; it then used the bottleneck/topology evidence
+to validate TP4 and found a much higher current config.
+
+## Interpretation
+
+- Why both variants can look close when only best-so-far is shown: no-harness can
+  eventually find a good simple topology, and best-so-far hides every bad
+  proposal after that point.
+- What the current-config curve shows: no-harness regresses at iter 3 and then
+  spends many iterations on no-feasible-point runtime probes. Harness reaches a
+  stronger TP2 config at iter 2 and a stronger TP4 config at iter 4.
+- Why harness helped: the baseline diagnostics identify TTFT/prefill as the
+  active bottleneck on low-prefix-reuse long prompts. The harness maps that to
+  adjacent TP validation before DP/runtime exploration. The no-harness LLM chose
+  DP2 then DP4 first, which diluted per-GPU throughput and delayed TP.
+- Defect fixed during the run: runtime refinement was too aggressive because it
+  combined larger MBT with higher memory utilization. It now changes batching
+  headroom without also raising memory pressure.
+- Stop defect fixed during the run: high-edge probes can have a few individual
+  latency failures and still be feasible under the configured pass-rate SLO. The
+  stop guard now keys on `feasible=true` near `search.high`, not on an empty
+  failed-reason map.
+- Search-high implication: TP4 reached `sampling_u=0.0615234375` with
+  `search.high=0.0625`, so the current spec is saturated for this topology. A
+  higher `search.high` would be required to distinguish whether TP4 can go even
+  higher in absolute throughput; it is not needed to show that harness converged
+  faster than no-harness under this spec.
+
+## Mechanism
+
+The harness contributes structured, non-testcase-specific information:
+
+- Workload features: long-prompt 0-8k distribution, low prefix reuse, and smooth
+  arrivals.
+- Bottleneck diagnosis from probes: baseline failures are TTFT/prefill-heavy, so
+  topology changes that reduce long-prefill latency should be tried before DP or
+  runtime batching.
+- Topology adjacency: validate TP1 -> TP2 -> TP4 rather than jumping randomly or
+  repeating a failing runtime family.
+- Stop condition: once the incumbent's feasible probe is within one binary-search
+  resolution of `search.high`, stop instead of spending more GPU trials.
+
+Without the harness, the LLM response in run9 chose DP2 and DP4 before TP2. That
+temporarily improved total request rate but reduced per-GPU efficiency, so the
+measured-current curve dipped at iter 3 and reached the old best only at iter 4.
+With the harness, the LLM receives the bottleneck/topology frame and chooses
+TP-oriented validation; TP2 is reached at iter 2 and TP4 at iter 4.

docs/qwen27b-chat-0-8k-harness-fig18.md

@@ -0,0 +1,58 @@
+# qwen27b-chat-0-8k Harness Fig18
+
+## Setup
+
+- Workload: `qwen3.5-27b` chat, `0 <= input_length <= 8192`.
+- Window: `chat_w20260311_1000`.
+- Engine: dash0 internal vLLM, baseline aligned to `run_qwen27b.sh`.
+- SLO: 95% pass rate, stepped TTFT `2s/4s/6s`, TPOT `<=50ms`.
+- Search metric: best-so-far feasible `request_rate_per_gpu`.
+- Before-harness source: actual 12-trial run
+  `.aituner-tight/dash0-qwen27b-tight-slo-10min-run9-chat-0-8k-codex-topology`.
+- After-harness source: strict harness replay over already measured run9 configs:
+  - Iter 1 uses the measured baseline trial.
+  - Iter 2 uses the current harness proposal after seeing only iter 1 history. It proposes `TP=2, DP=1`, whose performance is the measured run9 `trial-0004` result for the same config and spec.
+  - Iter 3 uses the current harness proposal after seeing only baseline + `TP=2, DP=1`. With the strong-incumbent guard, it returns `should_stop=true`.
+
+The replay is intentionally strict: the LLM prompt does not receive future `best_by_parallel_size` entries or later failed trials.
+
+## Fig18-Style Best-So-Far Curve
+
+Unit: feasible `request_rate_per_gpu`. Infeasible trials leave the best-so-far value unchanged.
+
+| Variant | Iter 1 | Iter 2 | Iter 3 | Iter 4 | Iter 5 | Iter 6 | Iter 7 | Iter 8 | Iter 9 | Iter 10 | Iter 11 | Iter 12 |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
+| Before harness, actual run9 | 0.0350 | 0.0617 | 0.0617 | 0.2025 | 0.2025 | 0.2025 | 0.2025 | 0.2025 | 0.2025 | 0.2025 | 0.2025 | 0.2025 |
+| After harness, strict replay | 0.0350 | 0.2025 | 0.2025 stop | 0.2025 | 0.2025 | 0.2025 | 0.2025 | 0.2025 | 0.2025 | 0.2025 | 0.2025 | 0.2025 |
+
+## Trial-Level Interpretation
+
+| Variant | Iter 1 | Iter 2 | Iter 3 | Iter 4 | Iter 5-12 |
+| --- | --- | --- | --- | --- | --- |
+| Before harness | baseline `TP1/DP1`, 0.0350 | `DP=2`, 0.0617 | `DP=4`, 0.0392, worse per GPU | `TP=2, DP=1`, 0.2025, best | runtime-only probes, all infeasible |
+| After harness | baseline `TP1/DP1`, 0.0350 | `TP=2, DP=1`, 0.2025, best | `should_stop=true` | no GPU trial | no GPU trial |
+
+## Convergence Judgment
+
+- Before harness reaches the final best value at iter 4.
+- After harness reaches the same best value at iter 2.
+- The speedup is `2x` by iterations-to-best: `4 -> 2`.
+- The harness also avoids the post-best weak proposals: before harness spent iters 5-12 on infeasible runtime-only probes; after harness stops at iter 3.
+
+## Implementation Changes From This Check
+
+- Added a strong-incumbent convergence guard:
+  - if the latest trial is the incumbent,
+  - and it improves `request_rate_per_gpu` by at least `3x` over the baseline,
+  - then runtime-only probes require direct same-topology evidence; otherwise the LLM should return `should_stop=true`.
+- Strengthened the MBT harness guard:
+  - do not raise `max-num-batched-tokens` when incumbent MBT already covers prompt p99 unless same-topology evidence proves prefill fragmentation.
+- Made early-stop engine relaunch opt-in. A real r2 run showed that default relaunch changes warm-state behavior and makes full-chat results incomparable with run9, so the default remains drain-based for comparable production measurements.
+- Added LLM empty-response retry to avoid crashing `study tune` on a transient empty streamed response.
+
+## Remote Checks
+
+- Local: `python3 -m compileall -q src tests` passed.
+- Local: `PYTHONPATH=src python3 -m unittest tests.test_core_flow` passed, 63 tests.
+- dash0: `python3 -m compileall -q src tests` passed.
+- dash0: `PYTHONPATH=src python3 -m unittest discover -s tests -p "test_*.py"` passed, 63 tests.

docs/qwen27b-chat-0-8k-setup-audit-20260506.md

@@ -0,0 +1,169 @@
+# qwen27b-chat-0-8k Setup and Result Audit
+
+## Purpose
+
+This note audits the 2026-05-06 qwen27b chat 0-8k harness result because the
+new best `0.4429 request_rate_per_gpu` is much higher than the previous
+no-harness best `0.2025`.
+
+## Setup
+
+- Host: `dash0`.
+- Hardware: 8 NVIDIA H20 GPUs.
+- Engine: internal vLLM at `/usr/local/bin/vllm`.
+- Model:
+  `/home/admin/resource/model/464482ce/qwen3.5-27b/256k-0223-internal`.
+- Served model name: `qwen35-27b-aituner`.
+- Workload window: `chat_w20260311_1000`.
+- Trace file source: `trace_windows/windows.json`.
+- Request mode: `chat`.
+- Input bucket: `0 <= input_length <= 8192`.
+- Replay scale: `1.0`.
+- Max concurrency: `32`.
+- Max requests per probe: unset, so full selected trace subset is replayed.
+- Search field: `sampling_u`.
+- Search range: `low=0.0`, `high=0.0625`.
+- Search probes: `max_probes=6`, `tolerance=0.001`.
+- Sampling seed: `20260325`.
+
+The local configs and dash0 model directories name this setup Qwen3.5-27B /
+Qwen35-27B. I did not find a `qwen32b` model/config for this internal chat
+0-8k setup.
+
+## SLO
+
+- Target pass rate: `0.95`.
+- TTFT rule: stepped by input length.
+
+| Input tokens | TTFT threshold |
+| ---: | ---: |
+| `<=4096` | `2000 ms` |
+| `<=32768` | `4000 ms` |
+| otherwise | `6000 ms` |
+
+- TPOT rule: fixed `<=50 ms`.
+
+A probe is feasible when its pass rate is at least `0.95`. Individual requests
+may still fail TTFT/TPOT while the whole probe remains feasible.
+
+## Compared Studies
+
+| Variant | Study root | Notes |
+| --- | --- | --- |
+| no-harness | `.aituner-tight/dash0-qwen27b-tight-slo-10min-run9-chat-0-8k-codex-topology` | completed 12-trial historical run |
+| harness | `.aituner-tight/dash0-qwen27b-tight-slo-10min-run10-chat-0-8k-current-harness` | seeded with run9 baseline, then ran real harness trials |
+
+The harness run reused the real run9 baseline as `trial-0001` to avoid
+duplicating a multi-hour cold-start baseline measurement. Later harness trials
+were real dash0 runs.
+
+## Metric
+
+The reported metric is `request_rate_per_gpu`:
+
+```text
+request_rate_per_gpu = best_feasible_request_rate / parallel_size
+parallel_size = tensor_parallel_size * data_parallel_size
+```
+
+The result JSON stores `best_request_rate`; `StudyStore.ingest_trial_results`
+derives `best_request_rate_per_gpu` from the trial spec topology.
+
+## Result Table
+
+Unit: feasible `request_rate_per_gpu`.
+
+| Variant | Curve | Iter 1 | Iter 2 | Iter 3 | Iter 4 | Iter 5 | Iter 6 | Iter 7 | Iter 8 |
+| --- | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
+| no-harness run9 | measured-current | 0.0350 | 0.0617 | 0.0392 | 0.2025 | NA | NA | NA | NA |
+| no-harness run9 | accepted-incumbent | 0.0350 | 0.0617 | 0.0617 | 0.2025 | 0.2025 | 0.2025 | 0.2025 | 0.2025 |
+| harness run10 | measured-current | 0.0350 | 0.2142 | NA | 0.4429 | NA | skipped | skipped | stop |
+| harness run10 | accepted-incumbent | 0.0350 | 0.2142 | 0.2142 | 0.4429 | 0.4429 | 0.4429 | 0.4429 | 0.4429 stop |
+
+## Why `0.4429` Is Plausible
+
+The new value is not the old TP2 config suddenly doubling. The comparable TP2
+results are close:
+
+| Study | Trial | Config | Best request rate | Parallel size | request_rate_per_gpu |
+| --- | --- | --- | ---: | ---: | ---: |
+| run9 | `trial-0004` | `TP=2, DP=1` | 0.4050 | 2 | 0.2025 |
+| run10 | `trial-0002` | `TP=2` | 0.4283 | 2 | 0.2142 |
+
+The large jump comes from a new topology that run9 did not evaluate:
+
+| Study | Trial | Config | Best request rate | Parallel size | request_rate_per_gpu |
+| --- | --- | --- | ---: | ---: | ---: |
+| run10 | `trial-0004` | `TP=4` | 1.7717 | 4 | 0.4429 |
+
+At the winning TP4 probe:
+
+- `sampling_u=0.0615234375`;
+- request count `1063`;
+- request rate `1.7717 req/s`;
+- pass rate `0.9680`;
+- p95 TTFT `1476.9 ms`;
+- p95 TPOT `44.4 ms`.
+
+This satisfies the configured SLO and is within one binary-search resolution of
+`search.high=0.0625`.
+
+## Correctness Audit
+
+The following fields match between run9 and run10 except for intentionally
+different identity fields such as study id and port:
+
+- model path and served model name;
+- internal vLLM executable;
+- base launch flags other than port;
+- trace window `chat_w20260311_1000`;
+- input-length filter `0-8192`;
+- replay scale `1.0`;
+- max concurrency `32`;
+- full selected trace replay, no `max_requests_per_probe`;
+- SLO target and TTFT/TPOT thresholds;
+- search `high=0.0625`, `max_probes=6`, `tolerance=0.001`, seed `20260325`;
+- metric definition `best_request_rate / (TP * DP)`.
+
+Checked differences and their impact:
+
+- Port differs: run9 used `18087`, run10 used `18082`; this should not affect
+  measured throughput.
+- run10 has explicit `restart_engine_after_early_stop=false`; chat studies
+  default to the same behavior.
+- run10 has explicit `completion_tokens_override=null`; equivalent to run9's
+  absent field.
+- run9 `trial-0004` search floor was `0.00390625` because it reused the
+  incumbent for the same parallel-size group. run10 `trial-0004` search floor
+  was `0.0` because pure `TP=4` had not been tried. Both have the same high and
+  probe budget; this does not explain the higher result.
+
+No metric-code logic error was found in the audit. The result JSONs store raw
+request rate, and the state computes per-GPU throughput by dividing by
+`TP*DP`. For run10 TP4, `1.7716666667 / 4 = 0.4429166667`.
+
+## Issues Found During The Test
+
+Two harness bugs were found and fixed:
+
+- Runtime refinement coupled larger `max-num-batched-tokens` with
+  `gpu-memory-utilization=0.95`, which caused launch-time OOM. Fixed in commit
+  `5d96689`.
+- The search-high stop guard incorrectly required no individual SLO failures at
+  a feasible high-edge probe. Fixed in commit `f653af0`; feasibility already
+  means the probe passed the configured pass-rate SLO.
+
+The queued product-8 `trial-0006` and `trial-0007` were stopped after the stop
+guard fix and are not used in the convergence claim.
+
+## Conclusion
+
+The `0.4429` result is being compared under the same workload, SLO, search
+range, and metric definition as the previous `0.2025` result. The reason it is
+higher is that no-harness run9 did not evaluate pure `TP=4`; the harness guided
+the search from the TTFT/prefill bottleneck to adjacent TP validation and found
+that topology by iter 4.
+
+Because TP4 nearly saturates the configured `search.high`, a follow-up run with
+a higher `search.high` is needed to measure the absolute ceiling. That follow-up
+is separate from the current convergence comparison.

docs/qwen27b-chat-0-8k-tpot25-16iter-20260506.md

@@ -0,0 +1,106 @@
+# qwen27b-chat-0-8k TPOT25 16-Iter Harness Compare
+
+## Goal
+
+Rerun the internal vLLM Qwen3.5-27B chat 0-8k tuning comparison under a stricter
+TPOT SLO:
+
+- no-harness: 16 tuning iterations;
+- harness: 16 tuning iterations, with permission to stop early if the harness
+  convergence guard decides no further GPU trial is needed.
+
+Both variants must be launched directly through AITuner. No state seeding,
+manual replay, or historical-result injection is allowed.
+
+## Setup
+
+- Host: `dash0`.
+- Hardware: 8 NVIDIA H20 GPUs.
+- Engine: internal vLLM at `/usr/local/bin/vllm`.
+- Model:
+  `/home/admin/resource/model/464482ce/qwen3.5-27b/256k-0223-internal`.
+- Served model name: `qwen35-27b-aituner`.
+- Workload window: `chat_w20260311_1000`.
+- Trace path source: `/home/admin/cpfs/wjh/aituner/aituner/trace_windows/windows.json`.
+- Request mode: `chat`.
+- Input bucket: `0 <= input_length <= 8192`.
+- Replay scale: `1.0`.
+- Max concurrency: `32`.
+- Max requests per probe: unset, so each probe uses the full selected trace
+  subset for its `sampling_u` threshold.
+- Restart engine after early stop: `true` for both variants. This is needed
+  under TPOT25 because very slow infeasible probes can leave live HTTP requests
+  in the engine after the SLO is already unrecoverable. Restarting keeps the
+  next binary-search probe from being contaminated by previous in-flight work.
+- Search field: `sampling_u`.
+- Search range: `low=0.0`, `high=0.0625`.
+- Search probes: `max_probes=6`, `tolerance=0.001`.
+- Sampling seed: `20260325`.
+
+## SLO
+
+- Target pass rate: `0.95`.
+- TTFT rule:
+
+| Input tokens | TTFT threshold |
+| ---: | ---: |
+| `<=4096` | `2000 ms` |
+| `<=32768` | `4000 ms` |
+| otherwise | `6000 ms` |
+
+- TPOT rule: fixed `<=25 ms`.
+
+## Specs
+
+Remote generated specs:
+
+- no-harness:
+  `.aituner-tight/specs/dash0-qwen27b-chat-0-8k-tpot25-restart-16iter-noharness.json`
+- harness:
+  `.aituner-tight/specs/dash0-qwen27b-chat-0-8k-tpot25-restart-16iter-harness.json`
+
+The two specs were generated from
+`configs/examples/dash0_qwen27b_tight_slo_run4_0_8k.json`. After normalizing
+`study_id` and `llm.use_harness`, the JSON payloads compare equal. Therefore the
+only tuning-behavior difference between the formal comparison runs is whether
+the harness is enabled.
+
+## Commands
+
+No-harness:
+
+```bash
+PYTHONPATH=src python3 -m aituner.cli study tune \
+  --spec .aituner-tight/specs/dash0-qwen27b-chat-0-8k-tpot25-restart-16iter-noharness.json \
+  --store-root .aituner-tight \
+  --max-trials 16
+```
+
+Harness:
+
+```bash
+PYTHONPATH=src python3 -m aituner.cli study tune \
+  --spec .aituner-tight/specs/dash0-qwen27b-chat-0-8k-tpot25-restart-16iter-harness.json \
+  --store-root .aituner-tight \
+  --max-trials 16
+```
+
+## Run Log
+
+- 2026-05-06 12:37 CST: generated both remote specs and verified that the only
+  normalized difference is `llm.use_harness`.
+- 2026-05-06 12:37 CST: started no-harness in tmux session
+  `qwen27b_tpot25_noharness_16iter_20260506`.
+- 2026-05-06 21:06 CST: stopped the initial no-harness pre-run before using it
+  for comparison. It used `restart_engine_after_early_stop=false`; the first
+  TP1 baseline probe already recorded `slo_pass_rate_unrecoverable`, but
+  unfinished requests remained live in vLLM and would contaminate the next probe.
+- 2026-05-06 21:07 CST: generated the formal clean specs with
+  `restart_engine_after_early_stop=true` for both variants and verified the
+  normalized diff is still only `llm.use_harness`.
+- 2026-05-06 21:09 CST: started formal no-harness run in tmux session
+  `qwen27b_tpot25_restart_noharness_16iter_20260506`.
+
+## Results
+
+Pending.

docs/qwen27b-chat-0-8k-tpot40-baseline-infeasible-20260507.md

@@ -0,0 +1,131 @@
+# Qwen27B Chat 0-8k TPOT 40ms Baseline Infeasible Run
+
+Date: 2026-05-07
+
+## Goal
+
+Re-run the internal vLLM + Qwen3.5-27B chat 0-8k tuning comparison after adding a study-level guard:
+
+- if the automatic baseline trial has no feasible probe;
+- and the lowest sampled request rate still fails the SLO target pass rate;
+- then AITuner stops the whole study and reports that the SLO is too tight for the current setup.
+
+This prevents spending the remaining tuning budget on LLM or harness proposals when the baseline itself demonstrates that the workload/SLO is infeasible at the search floor.
+
+## Implementation
+
+Commit: `f212673 Stop tuning when baseline is infeasible`
+
+Changed behavior:
+
+- `study tune` now persists `tuning_stop_reason` and `tuning_stop_diagnosis` in `state.json`.
+- `study tune` also persists `tuning_stop_details`, including the lowest sampled probe's TTFT/TPOT mean, p50, p95, and p99.
+- After the automatic baseline trial is ingested, AITuner checks the worker result:
+  - `status == completed`
+  - `best_request_rate is None`
+  - at least one probe exists
+  - all probes are infeasible
+- If true, AITuner stops before asking the LLM or harness for any proposal.
+- Re-running the same study respects the persisted stop state and does not resume tuning.
+
+Validation:
+
+```bash
+python3 -m compileall -q src tests
+PYTHONPATH=src python3 -m unittest tests.test_core_flow
+```
+
+Local and `dash0` both passed.
+
+## Setup
+
+Host: `dash0`
+
+Remote repo: `/home/admin/cpfs/wjh/aituner/aituner`
+
+Base spec: `configs/examples/dash0_qwen27b_tight_slo_run4_0_8k.json`
+
+Model: `/home/admin/resource/model/464482ce/qwen3.5-27b/256k-0223-internal`
+
+Workload: chat, 0-8k input window
+
+SLO:
+
+- TTFT: existing step rule from the base spec
+- TPOT: fixed `40ms`
+- target pass rate: `0.95`
+
+Search:
+
+- Direct AITuner command: `python3 -m aituner.cli study tune ... --max-trials 12`
+- No manual proposal/state edits during either run.
+- Both variants used `CUDA_VISIBLE_DEVICES=0,1,2,4,5,6,7`; this was identical for both specs.
+- The two specs were verified equal after normalizing only `study_id` and `llm.use_harness`.
+
+Specs:
+
+- no-harness: `.aituner-tight/specs/dash0-qwen27b-chat-0-8k-tpot40-gpu3skip-12iter-noharness-20260507.json`
+- harness: `.aituner-tight/specs/dash0-qwen27b-chat-0-8k-tpot40-gpu3skip-12iter-harness-20260507.json`
+
+## Commands
+
+No harness:
+
+```bash
+PYTHONPATH=src python3 -m aituner.cli study tune \
+  --spec .aituner-tight/specs/dash0-qwen27b-chat-0-8k-tpot40-gpu3skip-12iter-noharness-20260507.json \
+  --store-root .aituner-tight \
+  --max-trials 12
+```
+
+Harness:
+
+```bash
+PYTHONPATH=src python3 -m aituner.cli study tune \
+  --spec .aituner-tight/specs/dash0-qwen27b-chat-0-8k-tpot40-gpu3skip-12iter-harness-20260507.json \
+  --store-root .aituner-tight \
+  --max-trials 12
+```
+
+## Results
+
+Both runs stopped after the baseline trial. No LLM/harness proposal was evaluated because baseline had no feasible probe.
+
+| Variant | Trials executed | Best request rate | Best request rate / GPU | Stop reason |
+| --- | ---: | ---: | ---: | --- |
+| no-harness | 1 | - | - | `baseline_all_infeasible` |
+| harness | 1 | - | - | `baseline_all_infeasible` |
+
+Baseline probe curve:
+
+| sampling_u | request rate | pass rate | feasible | early stop reason |
+| ---: | ---: | ---: | --- | --- |
+| 0.03125 | 0.895 | 0.000000 | false | `slo_pass_rate_unrecoverable` |
+| 0.015625 | 0.483333 | 0.137931 | false | `slo_pass_rate_unrecoverable` |
+| 0.0078125 | 0.246667 | 0.236486 | false | `slo_pass_rate_unrecoverable` |
+| 0.00390625 | 0.123333 | 0.189189 | false | `slo_pass_rate_unrecoverable` |
+| 0.001953125 | 0.065000 | 0.205128 | false | `slo_pass_rate_unrecoverable` |
+| 0.0009765625 | 0.035000 | 0.142857 | false | `slo_pass_rate_unrecoverable` |
+
+Lowest request rate latency summary:
+
+| Variant | request rate | pass rate | TTFT mean | TTFT p50 | TTFT p95 | TTFT p99 | TPOT mean | TPOT p50 | TPOT p95 | TPOT p99 |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
+| no-harness | 0.035000 | 0.142857 | 1288.953ms | 446.586ms | 3011.814ms | 3011.814ms | 12.661ms | 13.141ms | 15.097ms | 15.097ms |
+| harness | 0.035000 | 0.142857 | 1268.090ms | 445.274ms | 2889.080ms | 2889.080ms | 12.658ms | 13.170ms | 15.102ms | 15.102ms |
+
+This shows that the TPOT threshold of `40ms` is not the binding constraint at the lowest sampled rate. The observed TPOT p99 is about `15.1ms`; failures are driven by TTFT and by the unrecoverable-pass-rate early stop after too many requests have already failed or been skipped.
+
+Final diagnosis written by AITuner:
+
+```text
+Baseline configuration has no feasible probe under the current SLO. Stopping tuning because even the lowest sampled request rate did not meet the target pass rate. lowest_sampled_request_rate=0.035 lowest_sampling_u=0.000976562 lowest_probe_pass_rate=0.142857 early_stop_reason=slo_pass_rate_unrecoverable
+```
+
+## Interpretation
+
+This run does not measure harness acceleration. It proves that the TPOT 40ms setup is infeasible for the current baseline and search floor: even at `0.035` aggregate request rate, only `14.29%` of requests pass the SLO, far below the `95%` target.
+
+The correct behavior is to stop the study early and report SLO infeasibility instead of spending the remaining 11 trial slots. Harness cannot accelerate convergence when there is no feasible baseline point and no incumbent for guided tuning.
+
+For a Fig. 18-style convergence comparison, the next setup must first have at least one feasible baseline or feasible low-rate point under the same metric definitions.

docs/qwen30b-community-vllm020/harness-early-stop-ablation-20260502.md

@@ -0,0 +1,175 @@
+# Qwen3-30B-A3B Community vLLM Harness Ablation, 2026-05-02
+
+## Goal
+
+Run a fresh dash0 experiment on the community vLLM latest release with the local community model:
+
+`/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B`
+
+The comparison is:
+
+| Variant | Spec | Harness |
+| --- | --- | --- |
+| no-harness | `configs/examples/dash0_qwen30b_a3b_community_vllm020_noharness.json` | disabled via `llm.use_harness=false` |
+| harness | `configs/examples/dash0_qwen30b_a3b_community_vllm020_harness.json` | enabled, including deterministic stop proposal |
+
+Both specs start from the same base vLLM configuration. The base contains only serving access fields: `host`, `port`, and `served-model-name`. It does not set performance flags such as TP, DP, EP, max model length, prefix cache, chunked prefill, max-num-seqs, max-num-batched-tokens, or gpu-memory-utilization. The first trial therefore measures community vLLM defaults for this model.
+
+The launch environment sets `HOME=/tmp/wjh` and `XDG_CACHE_HOME=/tmp/wjh/.cache` so vLLM, torch.compile, and FlashInfer build caches land on dash0 local storage instead of CPFS. This is a startup/cache placement choice, not a vLLM performance flag.
+
+## vLLM Install
+
+PyPI reports `vllm==0.20.0` as the current community release checked on 2026-05-02. The dash0 runtime venv is on local rootfs rather than CPFS, because installing torch/CUDA wheels into CPFS was I/O-bound:
+
+`/tmp/wjh/venvs/vllm-0.20.0-cu129`
+
+The first plain `pip install vllm==0.20.0` smoke pulled `torch 2.11.0+cu130` and failed on dash0's driver (`570.133.20`, CUDA 12.9). The active install uses the vLLM 0.20.0 GitHub release `+cu129` wheel and the PyTorch CUDA 12.9 index, matching the vLLM documented CUDA 12.9 install path for this driver.
+
+Install log:
+
+`/home/admin/cpfs/wjh/aituner/aituner/logs/install_vllm_0.20.0_20260502.log`
+
+## Workload
+
+The experiment reuses the 0-8k chat window that has already been used for qwen27b harness work:
+
+| Field | Value |
+| --- | --- |
+| window | `chat_w20260311_1000` |
+| source rows | 32606 |
+| input filter | 0 to 8192 tokens |
+| completion tokens | fixed 128 via `trace.completion_tokens_override` |
+| max requests per probe | 512 |
+| replay time scale | 0.1 |
+| target pass rate | 0.95 |
+| TTFT SLO | 2s up to 4k, 4s up to 32k, 6s above |
+| TPOT SLO | 50ms |
+| search high | 0.125 sampling_u |
+| max probes per trial | 4 |
+
+The `max_requests_per_probe=512` cap keeps the fresh community-vLLM ablation practical while preserving a real trace-shaped replay, SLO scoring, and binary-search threshold probe. A trace-only count check gives 31 to 65 selected requests across the six binary-search thresholds, avoiding the invalid low-cap case where early thresholds can select zero requests.
+
+The first full-output attempt showed why a bounded replay is needed for a 12-iteration ablation: at the first threshold (`0.0625`), 31 selected requests contained 14,849 output tokens with `out_max=2981`. That makes one probe too slow to finish a full no-harness/harness pair. The first out128 attempt with `replay_time_scale=1.0` was still bounded by real window time, so each probe waited close to the original window duration. The active ablation therefore fixes output length at 128 tokens, uses `replay_time_scale=0.1`, and limits each trial to four binary-search probes. `load_trace_requests` scales both request arrivals and the window duration, so reported request rates are the actual compressed replay request rates. This changes the load/decode mix, so the result should be interpreted as a community-vLLM harness convergence test under a bounded, time-compressed chat replay, not as a full-output production benchmark.
+
+## Harness Update Under Test
+
+This run tests a stricter early-stop harness:
+
+- The harness still injects L-C-A workload features, recent trial diagnostics, active bottleneck, legal topology candidates, tested signatures, and knob-family rules.
+- A strong incumbent no longer means immediate stop. It means "validate nearby alternatives".
+- Deterministic stop is allowed only after completed validation evidence says continuing is unlikely to be useful:
+  - the incumbent beats baseline by a generic large-gain ratio,
+  - at least two post-incumbent validation trials have run,
+  - those validation trials did not produce a feasible per-GPU improvement,
+  - the validation covered topology and runtime families, or accumulated at least three post-incumbent validation attempts.
+- If the stop guard fires, `study tune` writes `harness-stop-XXXX` and exits without spending another GPU trial or asking the LLM for another proposal.
+- A single-family all-infeasible plateau is not enough to stop deterministically. It only blocks repeating that family; the LLM must either justify a different family or later satisfy the validation/convergence stop rule.
+- A search-high saturation guard stops immediately when the incumbent's highest measured probe is feasible and is within the configured binary-search resolution of `search.high`. A feasible probe may still contain individual SLO failures as long as it meets the configured pass-rate target. In that case the current study cannot measure a better config without increasing the workload search range, so more config proposals only waste tuning iterations.
+
+This is a generic harness rule, not a testcase-specific threshold. It does not depend on qwen27b, qwen235b, qwen30b, a fixed TP/DP value, or a hardcoded SLO number.
+
+## Unit Tests
+
+Local test command:
+
+```bash
+PYTHONPATH=src python3 -m unittest tests.test_core_flow -q
+```
+
+Result at the time of this note: passed. The current repository test count may be higher; use the command above as the source of truth.
+
+The added coverage checks:
+
+| Test | Purpose |
+| --- | --- |
+| `test_harness_does_not_stop_immediately_after_strong_incumbent` | strong incumbent requires validation first |
+| `test_harness_stop_after_post_incumbent_validation_is_exhausted` | deterministic stop after validation exhaustion |
+| `test_cli_tune_uses_harness_stop_before_llm` | `study tune` can stop without calling the LLM or launching another GPU trial |
+| `test_prompt_can_disable_harness_for_ablation` | no-harness prompt removes structured harness context |
+| `test_harness_stop_when_incumbent_saturates_search_high` | deterministic stop when the incumbent saturates the configured workload search high |
+| `test_harness_guided_first_tp_probe_for_latency_bottleneck` | deterministic first TP probe after baseline latency bottleneck evidence |
+| `test_harness_guided_runtime_seed_preserves_tp_incumbent` | deterministic same-topology runtime refinement after a TP incumbent |
+
+## Experiment Tracking
+
+Completed dash0 runs:
+
+| Variant | tmux session | Log | Study root |
+| --- | --- | --- | --- |
+| no-harness | `qwen30b_vllm020_noharness_out128_scale01_20260502` | `logs/qwen30b_vllm020_noharness_out128_scale01_20260502.log` | `.aituner-community-vllm020/dash0-qwen30b-a3b-community-vllm020-chat-0-8k-out128-scale01-noharness` |
+| harness | `qwen30b_vllm020_harness_highstop_gpu4_7_20260502` | `logs/qwen30b_vllm020_harness_highstop_gpu4_7_20260502.log` | `.aituner-community-vllm020/dash0-qwen30b-a3b-community-vllm020-chat-0-8k-out128-scale01-harness-highstop-gpu4-7` |
+
+The harness run should be judged by best-so-far `request_rate_per_gpu` per tuning iteration, plus whether it stops only after validation evidence. The no-harness run should use the same trial budget so the ablation exposes whether the early-stop harness saves iterations without hiding a later better point.
+
+## Results
+
+Metric: best-so-far `request_rate_per_gpu` under the bounded, time-compressed replay.
+
+| Variant | Iter 1 | Iter 2 | Iter 3 | Iter 4 | Iter 5 | Iter 6 | Iter 7 | Iter 8 | Iter 9 | Iter 10 | Iter 11 | Iter 12 |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
+| no-harness | 1.0333 | 1.0333 | 1.0333 | 1.0333 | 1.0333 | 1.0333 | 1.0333 | 1.0333 | 1.0333 | 1.0333 | 1.0333 | 1.0333 |
+| harness | 1.0333 | 1.0333 stop | 1.0333 | 1.0333 | 1.0333 | 1.0333 | 1.0333 | 1.0333 | 1.0333 | 1.0333 | 1.0333 | 1.0333 |
+
+Actual per-iteration outcomes:
+
+| Variant | Iter 1 | Iter 2 | Iter 3 | Iter 4 | Iter 5 | Iter 6 | Iter 7 | Iter 8 | Iter 9 | Iter 10 | Iter 11 | Iter 12 |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
+| no-harness | 1.0333 | 0.5167 | fail | fail | fail | fail | fail | fail | fail | fail | fail | fail |
+| harness | 1.0333 | stop | stop | stop | stop | stop | stop | stop | stop | stop | stop | stop |
+
+Interpretation:
+
+- The best config is the default community vLLM config for this bounded replay. It reaches the configured search high: the last baseline probe at `sampling_u=0.1171875` is feasible, has pass rate `1.0`, and has no TTFT/TPOT SLO failures. With `search.high=0.125` and `max_probes=4`, this is exactly one binary-search resolution below the configured high.
+- The harness stops at iter 2 without calling the LLM or launching another GPU trial. This is not a claim that the engine is globally optimal; it is a claim that the current study cannot measure an improvement until `search.high` is increased.
+- No-harness spends all 12 tuning iterations anyway. Iter 2 changes to DP=2 and halves per-GPU throughput (`0.5167`). Iter 3-12 are launch failures from unguided or weakly guided proposals.
+- The harness therefore reaches the best measured config in one executed GPU trial and saves 11 tuning iterations on this setup.
+
+Operational note:
+
+- The no-harness run left driver-side orphan GPU memory on GPU0/1 after repeated launch failures. An earlier pre-high-stop harness attempt left the same kind of residue on GPU2/3. The final harness run was executed on dash0 GPU4-7 via a runtime-derived spec to avoid this contamination. Its executed GPU trial used a single H20, matching the no-harness best trial's single-GPU default configuration.
+
+## High=1.0 Rerun
+
+The `search.high=0.125` run answered only "can this config handle up to about 1.08 req/s in the compressed replay?" It could not answer "which config is best?" because the default config already reached the measurement ceiling.
+
+Trace request counts after raising `search.high` show the difference:
+
+| search.high | Near-top selected requests | Near-top request rate |
+| ---: | ---: | ---: |
+| 0.125 | 65 | 1.0833 req/s |
+| 0.25 | 141 | 2.3500 req/s |
+| 0.5 | 269 | 4.4833 req/s |
+| 1.0 | 502 | 8.3667 req/s |
+
+The high=1.0 run used the same bounded replay (`completion_tokens_override=128`, `replay_time_scale=0.1`, `max_requests_per_probe=512`) but set `search.high=1.0` and `max_probes=6`.
+
+Completed dash0 high=1.0 runs:
+
+| Variant | tmux session | Study root |
+| --- | --- | --- |
+| no-harness | `qwen30b_vllm020_noharness_high1_20260506` | `.aituner-community-vllm020/dash0-qwen30b-a3b-community-vllm020-chat-0-8k-out128-scale01-high1-noharness` |
+| harness-guided-v2 | `qwen30b_vllm020_harness_high1_guided_v2_20260506` | `.aituner-community-vllm020/dash0-qwen30b-a3b-community-vllm020-chat-0-8k-out128-scale01-high1-harness-guided-v2` |
+
+Metric: best-so-far `request_rate_per_gpu`.
+
+| Variant | Iter 1 | Iter 2 | Iter 3 | Iter 4 | Iter 5 | Iter 6 | Iter 7 | Iter 8 | Iter 9 | Iter 10 | Iter 11 | Iter 12 |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
+| no-harness | 2.2000 | 3.2583 | 3.2583 | 3.2583 | 3.2583 | 3.3000 | 3.3500 | 3.3500 | 3.3500 | 3.3500 | 3.3500 | 3.3500 |
+| harness-guided-v2 | 2.3833 | 3.2583 | 3.2833 | 3.3000 | 3.3000 stop | 3.3000 | 3.3000 | 3.3000 | 3.3000 | 3.3000 | 3.3000 | 3.3000 |
+
+Actual per-iteration outcomes:
+
+| Variant | Iter 1 | Iter 2 | Iter 3 | Iter 4 | Iter 5 | Iter 6 | Iter 7 | Iter 8-12 |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | --- |
+| no-harness | 2.2000 | 3.2583 | launch fail | infeasible | 1.1042 | 3.3000 | 3.3500 | infeasible |
+| harness-guided-v2 | 2.3833 | 3.2583 | 3.2833 | 3.3000 | stop | stop | stop | stop |
+
+Interpretation:
+
+- Raising `search.high` was necessary. The default config was not optimal under the expanded workload range; `TP=2` immediately improved per-GPU throughput from about `2.2` to `3.2583`.
+- The updated harness now provides deterministic proposals, not just early stop:
+  - iter2: adjacent TP probe (`tensor-parallel-size=2`),
+  - iter3: same-topology runtime seed (`gpu-memory-utilization=0.95`, chunked prefill, `max-num-batched-tokens=16384`),
+  - iter4: controlled MBT growth to `24576`.
+- No-harness reached the same config family at iter7, after an EP launch failure, an infeasible DP probe, a poor TP/DP probe, and then runtime refinement.
+- Harness reached the same config family at iter4 and stopped at iter5. Its measured best was `3.3000`, while no-harness measured `3.3500` for the same `TP=2 + MBT=24576` family; the 1.5% gap is within the observed boundary/noise of repeated high-load replay. The convergence claim is therefore "same config family in fewer iterations", not an exact higher single-run number.

docs/superpowers/plans/2026-05-06-repo-audit-repair.md

@@ -0,0 +1,59 @@
+# Repo Audit Repair Implementation Plan
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Repair the audit findings that affect measurement integrity, state correctness, documentation accuracy, and open-source readiness.
+
+**Architecture:** Keep changes localized to the existing stdlib-only Python package. Measurement validation lives at the HTTP/worker boundary, study state fixes remain in `StudyStore`, compare reporting gains explicit failed/no-feasible accounting, and project metadata/docs are added at repo root.
+
+**Tech Stack:** Python 3.11+ stdlib, `unittest`, setuptools `pyproject.toml`.
+
+---
+
+### Task 1: Measurement Integrity
+
+**Files:**
+- Modify: `src/aituner/http_client.py`
+- Modify: `src/aituner/slo.py`
+- Modify: `src/aituner/worker.py`
+- Test: `tests/test_core_flow.py`
+
+- [ ] Write failing tests for completion token source/mismatch failures and persisted per-request probe details.
+- [ ] Run the targeted tests and confirm they fail for the expected reason.
+- [ ] Add token source metadata to streamed metrics and request outcomes.
+- [ ] Fail requests when configured completion length cannot be verified from usage or differs from expected.
+- [ ] Persist probe outcome details under each trial artifact directory.
+- [ ] Run targeted tests and the full unittest suite.
+
+### Task 2: State, Spec, And Compare Guards
+
+**Files:**
+- Modify: `src/aituner/spec.py`
+- Modify: `src/aituner/store.py`
+- Modify: `src/aituner/compare.py`
+- Modify: `scripts/run_multi_compare.py`
+- Test: `tests/test_core_flow.py`
+
+- [ ] Write failing tests for state list isolation, invalid trace numeric bounds, and compare aggregate failure accounting.
+- [ ] Run targeted tests and confirm expected failures.
+- [ ] Deep-copy/replace trial lists when materializing trials.
+- [ ] Validate positive trace controls in `TraceSpec.from_dict`.
+- [ ] Report failed/no-feasible counts in compare aggregates without changing existing winner semantics.
+- [ ] Run targeted tests and the full unittest suite.
+
+### Task 3: Docs And Open-Source Readiness
+
+**Files:**
+- Create: `README.md`
+- Create: `LICENSE`
+- Create: `CONTRIBUTING.md`
+- Create: `SECURITY.md`
+- Modify: `pyproject.toml`
+- Modify: selected docs under `docs/`
+
+- [ ] Add concise repo usage, verification, and experiment integrity guidance.
+- [ ] Add MIT license and contribution/security notes.
+- [ ] Add project metadata and optional test extra.
+- [ ] Update stale docs about high-stop behavior and current test count.
+- [ ] Run JSON validation and full unittest suite.
+- [ ] Commit changes in logical groups.

pyproject.toml

@@ -6,8 +6,23 @@ build-backend = "setuptools.build_meta"
 name = "aituner"
 version = "0.1.0"
 description = "AITuner study orchestrator for OpenAI-compatible serving engines"
+readme = "README.md"
 requires-python = ">=3.11"
+license = {text = "MIT"}
+authors = [{name = "AITuner contributors"}]
 dependencies = []
+classifiers = [
+  "Development Status :: 3 - Alpha",
+  "Intended Audience :: Developers",
+  "Intended Audience :: Science/Research",
+  "License :: OSI Approved :: MIT License",
+  "Programming Language :: Python :: 3",
+  "Programming Language :: Python :: 3.11",
+  "Programming Language :: Python :: 3.12",
+]
+
+[project.optional-dependencies]
+test = []
 
 [project.scripts]
 aituner = "aituner.cli:main"

scripts/prepare_trace_windows.py

@@ -3,6 +3,7 @@ from __future__ import annotations
 import argparse
 import hashlib
 import json
+import os
 from pathlib import Path
 from typing import Any
 from dataclasses import dataclass
@@ -16,7 +17,15 @@ DEFAULT_THINKING_SOURCE = Path(
     "/home/admin/cpfs/wjh/bailian-trace/qwen-trace-260321-260327-formatted"
 )
 DEFAULT_OUTPUT_ROOT = REPO_ROOT / "trace_windows"
-LEGACY_TARGET_DATES = ["2026-03-11", "2026-03-12", "2026-03-13", "2026-03-16", "2026-03-17"]
+LEGACY_TARGET_DATES = [
+    "2026-03-11",
+    "2026-03-12",
+    "2026-03-13",
+    "2026-03-14",
+    "2026-03-15",
+    "2026-03-16",
+    "2026-03-17",
+]
 THINKING_WINDOWS = [
     ("2026-03-21", "1000"),
     ("2026-03-22", "1000"),
@@ -83,17 +92,39 @@ def parse_args() -> argparse.Namespace:
     return parser.parse_args()
 
 
-def stable_uniform(*, seed: int, window_id: str, index: int, row: dict[str, Any]) -> float:
+def resolve_session_root(row: dict[str, Any], root_of: dict[Any, Any]) -> Any:
+    """Resolve the session root chat_id for a trace row.
+
+    Sessions are multi-turn chains linked via parent_chat_id (turn>1 points to the
+    parent turn's chat_id, the root turn has parent_chat_id=-1). Because parent
+    turns precede their children in time, a single streaming pass that records
+    chat_id -> root resolves the full chain. Rows whose parent is not yet known
+    (e.g. it fell outside the materialized span) fall back to the parent id so
+    siblings still group together.
+    """
+    chat_id = row.get("chat_id")
+    parent = row.get("parent_chat_id")
+    parent_is_root = (
+        parent is None
+        or (isinstance(parent, (int, float)) and not isinstance(parent, bool) and int(parent) < 0)
+    )
+    root = chat_id if parent_is_root else root_of.get(parent, parent)
+    if chat_id is not None:
+        root_of[chat_id] = root
+    return root
+
+
+def session_uniform(*, seed: int, window_id: str, session_root: Any) -> float:
+    """Deterministic per-session uniform score in [0, 1).
+
+    All turns of a session share one score, so thresholding sampling_u keeps or
+    drops whole sessions and preserves intra-session prefix (KV-cache) reuse.
+    """
     payload = json.dumps(
         {
             "seed": seed,
             "window_id": window_id,
-            "index": index,
-            "timestamp": row.get("timestamp"),
-            "input_length": row.get("input_length"),
-            "output_length": row.get("output_length"),
-            "chat_id": row.get("chat_id"),
-            "turn": row.get("turn"),
+            "session_root": session_root,
         },
         sort_keys=True,
         separators=(",", ":"),
@@ -214,21 +245,34 @@ def materialize_windows(
 
     stats_by_window = {str(window["window_id"]): WindowStats() for window in windows}
     handles: dict[str, Any] = {}
+    final_paths: dict[str, Path] = {}
+    temp_paths: dict[str, Path] = {}
+    completed = False
     try:
         for window in windows:
             window_id = str(window["window_id"])
-            handles[window_id] = (traces_dir / f"{window_id}.jsonl").open("w", encoding="utf-8")
+            final_path = traces_dir / f"{window_id}.jsonl"
+            temp_path = traces_dir / f".{window_id}.jsonl.tmp.{os.getpid()}"
+            if temp_path.exists():
+                temp_path.unlink()
+            final_paths[window_id] = final_path
+            temp_paths[window_id] = temp_path
+            handles[window_id] = temp_path.open("w", encoding="utf-8")
 
         for trace_path, prompt_path in sorted(grouped.keys()):
             bucket = grouped[(trace_path, prompt_path)]
             bucket.sort(key=lambda item: (float(item["window_start"]), str(item["window_id"])))
             matched_rows = 0
+            root_of: dict[Any, Any] = {}
             with trace_path.open() as trace_handle, prompt_path.open() as prompt_handle:
                 for trace_raw, prompt_raw in zip(trace_handle, prompt_handle):
                     trace_raw = trace_raw.strip()
                     if not trace_raw:
                         continue
                     trace_row = json.loads(trace_raw)
+                    # Resolve session linkage for every row (even unmatched ones)
+                    # so multi-turn chains crossing the window edge still group.
+                    session_root = resolve_session_root(trace_row, root_of)
                     timestamp = float(trace_row.get("timestamp") or 0.0)
                     matched_window: dict[str, Any] | None = None
                     for window in bucket:
@@ -249,11 +293,11 @@ def materialize_windows(
                     start = float(matched_window["window_start"])
                     out["source_timestamp"] = timestamp
                     out["timestamp"] = timestamp - start
-                    out["sampling_u"] = stable_uniform(
+                    out["session_root"] = session_root
+                    out["sampling_u"] = session_uniform(
                         seed=sample_seed,
                         window_id=window_id,
-                        index=stats_by_window[window_id].num_requests,
-                        row=merged,
+                        session_root=session_root,
                     )
                     handles[window_id].write(json.dumps(out, ensure_ascii=False) + "\n")
                     stats_by_window[window_id].record(out)
@@ -262,9 +306,17 @@ def materialize_windows(
                 f"materialized {trace_path.name} -> matched_rows={matched_rows}",
                 flush=True,
             )
+        completed = True
     finally:
         for handle in handles.values():
             handle.close()
+        if completed:
+            for window_id, temp_path in temp_paths.items():
+                os.replace(temp_path, final_paths[window_id])
+        else:
+            for temp_path in temp_paths.values():
+                if temp_path.exists():
+                    temp_path.unlink()
     return stats_by_window
 
 
@@ -285,7 +337,7 @@ def build_output_window(
     output["num_excluded_too_long"] = 0
     output["sampling_u_field"] = "sampling_u"
     output["sampling_seed"] = int(sample_seed)
-    output["sampling_strategy"] = "fixed_uniform_score"
+    output["sampling_strategy"] = "session_coherent_uniform_score"
     output["first_request_ts"] = stats.first_request_ts
     output["last_request_ts"] = stats.last_request_ts
     output["first_request_index"] = stats.first_request_index
@@ -334,10 +386,17 @@ def main() -> int:
         "window_duration_seconds": 600.0,
         "windows": rendered_windows,
     }
-    (output_root / "windows.json").write_text(
-        json.dumps(windows_payload, ensure_ascii=False, indent=2) + "\n",
-        encoding="utf-8",
-    )
+    windows_path = output_root / "windows.json"
+    windows_tmp_path = output_root / f".windows.json.tmp.{os.getpid()}"
+    try:
+        windows_tmp_path.write_text(
+            json.dumps(windows_payload, ensure_ascii=False, indent=2) + "\n",
+            encoding="utf-8",
+        )
+        os.replace(windows_tmp_path, windows_path)
+    finally:
+        if windows_tmp_path.exists():
+            windows_tmp_path.unlink()
     print(output_root)
     print(f"windows={len(rendered_windows)}")
     return 0

scripts/run_baseline_then_llm.py

@@ -10,6 +10,7 @@ from aituner.llm import (
     load_capability_profile,
     parse_proposal_text,
 )
+from aituner.lca import build_study_workload_profile
 from aituner.spec import load_study_spec
 from aituner.store import StudyStore
 from aituner.trace import load_trace_requests, summarize_window
@@ -89,6 +90,7 @@ def main() -> int:
             window_summary=summarize_window(requests, window),
             state=state,
             capability_profile=capability_profile,
+            workload_profile=build_study_workload_profile(study, requests, window),
         )
         prompt_name = f"prompt-{state.next_trial_index:04d}"
         store.write_prompt(study.study_id, prompt_name, prompt)

scripts/run_multi_compare.py

@@ -0,0 +1,587 @@
+#!/usr/bin/env python3
+from __future__ import annotations
+
+import argparse
+import json
+from concurrent.futures import ThreadPoolExecutor, as_completed
+from dataclasses import dataclass, replace
+from pathlib import Path
+from typing import Any
+
+from aituner.spec import (
+    CompareCandidateSpec,
+    ConfigPatch,
+    SpecError,
+    TrialSpec,
+    load_study_spec,
+    load_structured_file,
+    to_jsonable,
+)
+from aituner.store import StudyStore
+from aituner.worker import run_trial
+
+
+@dataclass(frozen=True)
+class RuntimeOverride:
+    cuda_visible_devices: str
+    port: int
+    host: str = "127.0.0.1"
+
+    @classmethod
+    def from_dict(cls, data: dict[str, Any], *, context: str) -> "RuntimeOverride":
+        cuda_visible_devices = str(data.get("cuda_visible_devices") or "").strip()
+        if not cuda_visible_devices:
+            raise SpecError(f"{context}.cuda_visible_devices must be a non-empty string.")
+        port_value = data.get("port")
+        if isinstance(port_value, bool) or not isinstance(port_value, int):
+            raise SpecError(f"{context}.port must be an integer.")
+        host = str(data.get("host") or "127.0.0.1").strip()
+        if not host:
+            raise SpecError(f"{context}.host must be a non-empty string.")
+        return cls(
+            cuda_visible_devices=cuda_visible_devices,
+            port=port_value,
+            host=host,
+        )
+
+
+@dataclass(frozen=True)
+class MultiCompareCandidate:
+    name: str
+    phase: int
+    candidate: CompareCandidateSpec
+    runtime: RuntimeOverride
+
+    @classmethod
+    def from_dict(cls, data: dict[str, Any], *, context: str) -> "MultiCompareCandidate":
+        name = str(data.get("name") or "").strip()
+        if not name:
+            raise SpecError(f"{context}.name must be a non-empty string.")
+        phase_value = data.get("phase", 1)
+        if isinstance(phase_value, bool) or not isinstance(phase_value, int) or phase_value < 1:
+            raise SpecError(f"{context}.phase must be a positive integer.")
+        candidate = CompareCandidateSpec.from_dict(data, context=context)
+        runtime = RuntimeOverride.from_dict(
+            dict(data.get("runtime") or {}),
+            context=f"{context}.runtime",
+        )
+        return cls(name=name, phase=phase_value, candidate=candidate, runtime=runtime)
+
+
+@dataclass(frozen=True)
+class MultiCompareSpec:
+    compare_id: str
+    study_spec_path: str
+    output_root: str | None
+    window_ids: list[str]
+    candidates: list[MultiCompareCandidate]
+
+    @classmethod
+    def from_dict(cls, data: dict[str, Any]) -> "MultiCompareSpec":
+        compare_id = str(data.get("compare_id") or "").strip()
+        if not compare_id:
+            raise SpecError("compare_id must be a non-empty string.")
+        study_spec_path = str(data.get("study_spec_path") or "").strip()
+        if not study_spec_path:
+            raise SpecError("study_spec_path must be a non-empty string.")
+        raw_window_ids = data.get("window_ids")
+        if not isinstance(raw_window_ids, list) or not raw_window_ids:
+            raise SpecError("window_ids must be a non-empty list.")
+        window_ids = [str(item).strip() for item in raw_window_ids if str(item).strip()]
+        if not window_ids:
+            raise SpecError("window_ids must contain at least one non-empty string.")
+        raw_candidates = data.get("candidates")
+        if not isinstance(raw_candidates, list) or not raw_candidates:
+            raise SpecError("candidates must be a non-empty list.")
+        candidates = [
+            MultiCompareCandidate.from_dict(dict(item), context=f"candidates[{idx}]")
+            for idx, item in enumerate(raw_candidates)
+            if isinstance(item, dict)
+        ]
+        if len(candidates) != len(raw_candidates):
+            raise SpecError("Every candidates entry must be an object.")
+        names = [item.name for item in candidates]
+        if len(names) != len(set(names)):
+            raise SpecError("candidates names must be unique.")
+        return cls(
+            compare_id=compare_id,
+            study_spec_path=study_spec_path,
+            output_root=str(data.get("output_root") or "").strip() or None,
+            window_ids=window_ids,
+            candidates=candidates,
+        )
+
+
+def _resolve_path(raw_path: str, *, base_dir: Path) -> Path:
+    path = Path(raw_path)
+    if not path.is_absolute():
+        path = (base_dir / path).resolve()
+    return path
+
+
+def _load_windows_payload(study: Any, *, study_spec_path: Path) -> list[dict[str, Any]]:
+    windows_path = Path(study.trace.windows_path)
+    if not windows_path.is_absolute():
+        windows_path = (study_spec_path.parent / windows_path).resolve()
+    payload = json.loads(windows_path.read_text(encoding="utf-8"))
+    raw_windows = payload.get("windows") if isinstance(payload, dict) else payload
+    if not isinstance(raw_windows, list):
+        raise SpecError(f"windows payload must contain a list: {windows_path}")
+    return [
+        {str(key): value for key, value in item.items()}
+        for item in raw_windows
+        if isinstance(item, dict)
+    ]
+
+
+def _select_windows(spec: MultiCompareSpec, *, study: Any, study_spec_path: Path) -> list[dict[str, Any]]:
+    windows = _load_windows_payload(study, study_spec_path=study_spec_path)
+    indexed = {str(item.get("window_id") or "").strip(): item for item in windows}
+    selected: list[dict[str, Any]] = []
+    for window_id in spec.window_ids:
+        item = indexed.get(window_id)
+        if item is None:
+            raise SpecError(f"window_id not found in windows payload: {window_id}")
+        selected.append(item)
+    return selected
+
+
+def _load_config_patch(
+    candidate: MultiCompareCandidate,
+    *,
+    spec_path: Path,
+) -> tuple[ConfigPatch, dict[str, Any]]:
+    if candidate.candidate.config_patch is not None:
+        config_patch = candidate.candidate.config_patch
+        return config_patch, {
+            "kind": "config_patch",
+            "config_patch": {
+                "env_patch": dict(config_patch.env_patch),
+                "flag_patch": dict(config_patch.flag_patch),
+            },
+        }
+    assert candidate.candidate.trial_ref is not None
+    study_root = _resolve_path(candidate.candidate.trial_ref.study_root, base_dir=spec_path.parent)
+    trial_spec_path = study_root / "trials" / candidate.candidate.trial_ref.trial_id / "trial_spec.json"
+    if not trial_spec_path.exists():
+        raise SpecError(f"trial_ref target not found: {trial_spec_path}")
+    payload = json.loads(trial_spec_path.read_text(encoding="utf-8"))
+    config_patch = ConfigPatch.from_dict(payload.get("config_patch") or {})
+    return config_patch, {
+        "kind": "trial_ref",
+        "study_root": str(study_root),
+        "trial_id": candidate.candidate.trial_ref.trial_id,
+        "config_patch": {
+            "env_patch": dict(config_patch.env_patch),
+            "flag_patch": dict(config_patch.flag_patch),
+        },
+    }
+
+
+def _parse_int_like(value: Any, *, default: int = 1) -> int:
+    if value is None:
+        return default
+    if isinstance(value, bool):
+        raise SpecError("Topology values must be integers.")
+    if isinstance(value, int):
+        return value
+    if isinstance(value, float) and value.is_integer():
+        return int(value)
+    if isinstance(value, str) and value.strip():
+        return int(value.strip())
+    raise SpecError(f"Unable to parse integer topology value: {value!r}")
+
+
+def _parallel_size_for_candidate(*, study: Any, patch: ConfigPatch) -> int:
+    flags = dict(study.engine.base_flags)
+    flags.update(patch.flag_patch)
+    tp = _parse_int_like(flags.get("tensor-parallel-size"), default=1)
+    dp = _parse_int_like(flags.get("data-parallel-size"), default=1)
+    return tp * dp
+
+
+def _trial_snapshot(trial: TrialSpec) -> dict[str, Any]:
+    return to_jsonable(trial)
+
+
+def _study_snapshot(study: Any) -> dict[str, Any]:
+    return to_jsonable(study)
+
+
+def _run_candidate_for_window(
+    *,
+    compare_id: str,
+    compare_root: Path,
+    study: Any,
+    study_spec_path: Path,
+    window_id: str,
+    candidate: MultiCompareCandidate,
+    config_patch: ConfigPatch,
+    source: dict[str, Any],
+) -> dict[str, Any]:
+    run_root = compare_root / "runs" / window_id / candidate.name
+    run_root.mkdir(parents=True, exist_ok=True)
+    result_path = run_root / "result.json"
+    if result_path.exists():
+        result = json.loads(result_path.read_text(encoding="utf-8"))
+        parallel_size = _parallel_size_for_candidate(study=study, patch=config_patch)
+        best_rate = result.get("best_request_rate")
+        best_rate_per_gpu = (
+            float(best_rate) / float(parallel_size)
+            if isinstance(best_rate, (int, float)) and parallel_size > 0
+            else None
+        )
+        return {
+            "candidate": candidate.name,
+            "source": source,
+            "parallel_size": parallel_size,
+            "runtime": {
+                "cuda_visible_devices": candidate.runtime.cuda_visible_devices,
+                "port": candidate.runtime.port,
+                "host": candidate.runtime.host,
+            },
+            "config_patch": {
+                "env_patch": dict(config_patch.env_patch),
+                "flag_patch": dict(config_patch.flag_patch),
+            },
+            "status": result.get("status"),
+            "best_sampling_u": result.get("best_sampling_u"),
+            "best_request_rate": best_rate,
+            "best_request_rate_per_gpu": best_rate_per_gpu,
+            "best_pass_rate": result.get("best_pass_rate"),
+            "best_request_count": result.get("best_request_count"),
+            "failure_stage": result.get("failure_stage"),
+            "failure_reason": result.get("failure_reason"),
+            "artifact_dir": str(run_root),
+            "result_path": str(result_path),
+            "probe_log_path": str(run_root / "probe_history.json"),
+            "engine_log_path": str(run_root / "engine.log"),
+            "resumed": True,
+        }
+
+    engine_envs = dict(study.engine.base_envs)
+    engine_envs["CUDA_VISIBLE_DEVICES"] = candidate.runtime.cuda_visible_devices
+    engine_flags = dict(study.engine.base_flags)
+    engine_flags["port"] = candidate.runtime.port
+    runtime_study = replace(
+        study,
+        trace=replace(study.trace, window_id=window_id),
+        engine=replace(
+            study.engine,
+            host=candidate.runtime.host,
+            port=candidate.runtime.port,
+            base_envs=engine_envs,
+            base_flags=engine_flags,
+        ),
+    )
+    actual_study_path = run_root / "study_spec.json"
+    source_path = run_root / "study_spec.source"
+    actual_study_path.write_text(
+        json.dumps(_study_snapshot(runtime_study), ensure_ascii=False, indent=2) + "\n",
+        encoding="utf-8",
+    )
+    source_path.write_text(str(actual_study_path) + "\n", encoding="utf-8")
+    trial = TrialSpec(
+        study_id=compare_id,
+        trial_id=candidate.name,
+        config_patch=config_patch,
+        search=runtime_study.search,
+        study_spec_path=str(source_path),
+        artifact_dir=str(run_root),
+        probe_log_path=str(run_root / "probe_history.json"),
+        engine_log_path=str(run_root / "engine.log"),
+        result_path=str(result_path),
+    )
+    StudyStore.write_json(run_root / "trial_spec.json", _trial_snapshot(trial))
+    result = run_trial(run_root / "trial_spec.json")
+    parallel_size = _parallel_size_for_candidate(study=runtime_study, patch=config_patch)
+    best_rate = result.get("best_request_rate")
+    best_rate_per_gpu = (
+        float(best_rate) / float(parallel_size)
+        if isinstance(best_rate, (int, float)) and parallel_size > 0
+        else None
+    )
+    return {
+        "candidate": candidate.name,
+        "source": source,
+        "parallel_size": parallel_size,
+        "runtime": {
+            "cuda_visible_devices": candidate.runtime.cuda_visible_devices,
+            "port": candidate.runtime.port,
+            "host": candidate.runtime.host,
+        },
+        "config_patch": {
+            "env_patch": dict(config_patch.env_patch),
+            "flag_patch": dict(config_patch.flag_patch),
+        },
+        "status": result.get("status"),
+        "best_sampling_u": result.get("best_sampling_u"),
+        "best_request_rate": best_rate,
+        "best_request_rate_per_gpu": best_rate_per_gpu,
+        "best_pass_rate": result.get("best_pass_rate"),
+        "best_request_count": result.get("best_request_count"),
+        "failure_stage": result.get("failure_stage"),
+        "failure_reason": result.get("failure_reason"),
+        "artifact_dir": str(run_root),
+        "result_path": str(result_path),
+        "probe_log_path": str(run_root / "probe_history.json"),
+        "engine_log_path": str(run_root / "engine.log"),
+        "resumed": False,
+    }
+
+
+def _winner(candidates: dict[str, dict[str, Any]]) -> str:
+    scored = [
+        (name, float(result["best_request_rate_per_gpu"]))
+        for name, result in candidates.items()
+        if isinstance(result.get("best_request_rate_per_gpu"), (int, float))
+    ]
+    if not scored:
+        return "incomparable"
+    scored.sort(key=lambda item: item[1], reverse=True)
+    if len(scored) > 1 and scored[0][1] == scored[1][1]:
+        return "tie"
+    return scored[0][0]
+
+
+def _aggregate(rows: list[dict[str, Any]], candidates: list[MultiCompareCandidate]) -> dict[str, Any]:
+    candidate_names = [item.name for item in candidates]
+    wins = {name: 0 for name in candidate_names}
+    wins["tie"] = 0
+    wins["incomparable"] = 0
+    means: dict[str, dict[str, Any]] = {}
+    for name in candidate_names:
+        rates = [
+            float(row["candidates"][name]["best_request_rate"])
+            for row in rows
+            if isinstance(row["candidates"][name].get("best_request_rate"), (int, float))
+        ]
+        rates_per_gpu = [
+            float(row["candidates"][name]["best_request_rate_per_gpu"])
+            for row in rows
+            if isinstance(row["candidates"][name].get("best_request_rate_per_gpu"), (int, float))
+        ]
+        pass_rates = [
+            float(row["candidates"][name]["best_pass_rate"])
+            for row in rows
+            if isinstance(row["candidates"][name].get("best_pass_rate"), (int, float))
+        ]
+        means[name] = {
+            "mean_request_rate": (sum(rates) / len(rates)) if rates else None,
+            "mean_request_rate_per_gpu": (sum(rates_per_gpu) / len(rates_per_gpu))
+            if rates_per_gpu
+            else None,
+            "mean_pass_rate": (sum(pass_rates) / len(pass_rates)) if pass_rates else None,
+            **_candidate_result_counts(rows, name),
+        }
+    for row in rows:
+        wins[row["winner"]] = wins.get(row["winner"], 0) + 1
+    return {
+        "window_count": len(rows),
+        "wins": wins,
+        "candidates": means,
+    }
+
+
+def _candidate_result_counts(rows: list[dict[str, Any]], name: str) -> dict[str, int]:
+    counts = {
+        "completed_window_count": 0,
+        "failed_window_count": 0,
+        "no_feasible_window_count": 0,
+    }
+    for row in rows:
+        result = row.get("candidates", {}).get(name)
+        if not isinstance(result, dict):
+            continue
+        status = str(result.get("status") or "")
+        if status == "completed":
+            counts["completed_window_count"] += 1
+        elif status == "failed":
+            counts["failed_window_count"] += 1
+        if not isinstance(result.get("best_request_rate_per_gpu"), (int, float)):
+            counts["no_feasible_window_count"] += 1
+    return counts
+
+
+def _render_report(summary: dict[str, Any], candidates: list[MultiCompareCandidate]) -> str:
+    candidate_names = [item.name for item in candidates]
+    lines = [
+        f"# {summary['compare_id']}",
+        "",
+        "## Setup",
+        "",
+        f"- Study spec: `{summary['study_spec_path']}`",
+        f"- Compare root: `{summary['compare_root']}`",
+        f"- Windows: `{len(summary['windows'])}`",
+        "",
+        "## Candidates",
+        "",
+    ]
+    for item in candidates:
+        lines.append(
+            f"- `{item.name}`: phase=`{item.phase}`, gpus=`{item.runtime.cuda_visible_devices}`, port=`{item.runtime.port}`"
+        )
+    lines.extend(
+        [
+            "",
+            "## Aggregate",
+            "",
+            f"- Wins: `{json.dumps(summary['aggregate']['wins'], ensure_ascii=False)}`",
+        ]
+    )
+    for name in candidate_names:
+        aggregate = summary["aggregate"]["candidates"][name]
+        lines.append(
+            f"- `{name}` mean req/s=`{aggregate['mean_request_rate']}`, mean req/s/gpu=`{aggregate['mean_request_rate_per_gpu']}`, mean pass_rate=`{aggregate['mean_pass_rate']}`"
+        )
+        lines.append(
+            f"  completed/failed/no-feasible windows=`{aggregate['completed_window_count']}`/`{aggregate['failed_window_count']}`/`{aggregate['no_feasible_window_count']}`"
+        )
+    header = ["Window", "Date"]
+    for name in candidate_names:
+        header.extend([f"{name} req/s", f"{name} req/s/gpu"])
+    header.append("Winner")
+    lines.extend(
+        [
+            "",
+            "## Per Window",
+            "",
+            "| " + " | ".join(header) + " |",
+            "| " + " | ".join(["---"] * len(header)) + " |",
+        ]
+    )
+    for row in summary["windows"]:
+        cells = [f"`{row['window_id']}`", f"`{row.get('date') or ''}`"]
+        for name in candidate_names:
+            candidate = row["candidates"][name]
+            cells.append(f"`{candidate.get('best_request_rate')}`")
+            cells.append(f"`{candidate.get('best_request_rate_per_gpu')}`")
+        cells.append(f"`{row['winner']}`")
+        lines.append("| " + " | ".join(cells) + " |")
+    lines.append("")
+    return "\n".join(lines)
+
+
+def run_multi_compare(spec_path: Path) -> dict[str, Any]:
+    spec_path = spec_path.resolve()
+    spec = MultiCompareSpec.from_dict(dict(load_structured_file(spec_path)))
+    study_spec_path = _resolve_path(spec.study_spec_path, base_dir=spec_path.parent)
+    study = load_study_spec(study_spec_path)
+    compare_root = (
+        _resolve_path(spec.output_root, base_dir=spec_path.parent)
+        if spec.output_root
+        else (Path(".aituner-compare") / spec.compare_id).resolve()
+    )
+    compare_root.mkdir(parents=True, exist_ok=True)
+    windows = _select_windows(spec, study=study, study_spec_path=study_spec_path)
+    candidate_payloads = []
+    resolved_candidates: dict[str, tuple[MultiCompareCandidate, ConfigPatch, dict[str, Any]]] = {}
+    for candidate in spec.candidates:
+        config_patch, source = _load_config_patch(candidate, spec_path=spec_path)
+        resolved_candidates[candidate.name] = (candidate, config_patch, source)
+        candidate_payloads.append(
+            {
+                "name": candidate.name,
+                "phase": candidate.phase,
+                "runtime": {
+                    "cuda_visible_devices": candidate.runtime.cuda_visible_devices,
+                    "port": candidate.runtime.port,
+                    "host": candidate.runtime.host,
+                },
+                "source": source,
+            }
+        )
+    snapshot = {
+        "compare_id": spec.compare_id,
+        "study_spec_path": str(study_spec_path),
+        "window_ids": spec.window_ids,
+        "candidates": candidate_payloads,
+    }
+    StudyStore.write_json(compare_root / "compare_spec.snapshot.json", snapshot)
+
+    phases = sorted({item.phase for item in spec.candidates})
+    per_window: list[dict[str, Any]] = []
+    for window in windows:
+        window_id = str(window["window_id"])
+        row = {
+            "window_id": window_id,
+            "trace_type": window.get("trace_type"),
+            "date": window.get("date"),
+            "slot_token": window.get("slot_token"),
+            "slot_label": window.get("slot_label"),
+            "window_start": window.get("window_start"),
+            "window_end": window.get("window_end"),
+            "candidates": {},
+        }
+        for phase in phases:
+            phase_candidates = [item for item in spec.candidates if item.phase == phase]
+            with ThreadPoolExecutor(max_workers=len(phase_candidates)) as executor:
+                future_map = {
+                    executor.submit(
+                        _run_candidate_for_window,
+                        compare_id=spec.compare_id,
+                        compare_root=compare_root,
+                        study=study,
+                        study_spec_path=study_spec_path,
+                        window_id=window_id,
+                        candidate=item,
+                        config_patch=resolved_candidates[item.name][1],
+                        source=resolved_candidates[item.name][2],
+                    ): item.name
+                    for item in phase_candidates
+                }
+                for future in as_completed(future_map):
+                    result = future.result()
+                    row["candidates"][result["candidate"]] = result
+        row["winner"] = _winner(row["candidates"])
+        per_window.append(row)
+        partial_summary = {
+            "compare_id": spec.compare_id,
+            "study_spec_path": str(study_spec_path),
+            "compare_root": str(compare_root),
+            "windows": per_window,
+            "aggregate": _aggregate(per_window, spec.candidates),
+        }
+        StudyStore.write_json(compare_root / "summary.json", partial_summary)
+        (compare_root / "report.md").write_text(
+            _render_report(partial_summary, spec.candidates),
+            encoding="utf-8",
+        )
+
+    summary = {
+        "compare_id": spec.compare_id,
+        "study_spec_path": str(study_spec_path),
+        "compare_root": str(compare_root),
+        "windows": per_window,
+        "aggregate": _aggregate(per_window, spec.candidates),
+    }
+    StudyStore.write_json(compare_root / "summary.json", summary)
+    (compare_root / "report.md").write_text(
+        _render_report(summary, spec.candidates),
+        encoding="utf-8",
+    )
+    return summary
+
+
+def main() -> int:
+    parser = argparse.ArgumentParser(description="Run a multi-candidate compare over trace windows.")
+    parser.add_argument("--spec", required=True)
+    args = parser.parse_args()
+    summary = run_multi_compare(Path(args.spec))
+    print(
+        json.dumps(
+            {
+                "compare_id": summary["compare_id"],
+                "compare_root": summary["compare_root"],
+                "window_count": summary["aggregate"]["window_count"],
+                "wins": summary["aggregate"]["wins"],
+            },
+            ensure_ascii=False,
+            indent=2,
+        )
+    )
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

scripts/stop_a_calibration.py

@@ -0,0 +1,128 @@
+#!/usr/bin/env python3
+"""Stop-A calibration: print the offered-L-C-A convergence curve for a raw trace window.
+
+The convergence of prefix-vs-full L-C-A is a deterministic property of the trace
+metadata (lengths, hash_ids, arrivals), so this runs on CPU without serving the
+model. Use it to pick tau / tau_c / stable_checks and to compare how late the C
+dimension converges across workloads (e.g. low-reuse chat vs high-reuse coder).
+
+Example:
+  PYTHONPATH=src python3 scripts/stop_a_calibration.py \
+    --jsonl /dashscope/.../qwen_chat_blksz_64_032109-032111.jsonl \
+    --block-size 64 --window-start 3600 --window-end 4200 --gpu-count 8 --label chat
+"""
+from __future__ import annotations
+
+import argparse
+import json
+from pathlib import Path
+
+from aituner.lca import find_convergence_prefix, resolve_length_mode
+from aituner.trace import TraceRequest, WindowRecord
+
+
+def _session_root(row: dict, root_of: dict) -> object:
+    chat_id = row.get("chat_id")
+    parent = row.get("parent_chat_id")
+    parent_is_root = parent is None or (
+        isinstance(parent, (int, float)) and not isinstance(parent, bool) and int(parent) < 0
+    )
+    root = chat_id if parent_is_root else root_of.get(parent, parent)
+    if chat_id is not None:
+        root_of[chat_id] = root
+    return root
+
+
+def load_window(jsonl: Path, *, window_start: float, window_end: float) -> list[TraceRequest]:
+    root_of: dict = {}
+    requests: list[TraceRequest] = []
+    with jsonl.open(encoding="utf-8") as handle:
+        for line in handle:
+            line = line.strip()
+            if not line:
+                continue
+            row = json.loads(line)
+            _session_root(row, root_of)  # keep chain complete even outside the window
+            ts = float(row.get("timestamp") or 0.0)
+            if not (window_start <= ts < window_end):
+                continue
+            hash_ids = row.get("hash_ids")
+            requests.append(
+                TraceRequest(
+                    row_id=str(row.get("chat_id")),
+                    arrival_s=ts - window_start,
+                    sampling_u=1.0,
+                    body={},
+                    prompt_tokens_hint=int(row.get("input_length") or 0),
+                    completion_tokens_hint=int(row.get("output_length") or 0),
+                    metadata={"hash_ids": hash_ids if isinstance(hash_ids, list) else None},
+                )
+            )
+    requests.sort(key=lambda item: item.arrival_s)
+    return requests
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--jsonl", type=Path, required=True)
+    ap.add_argument("--block-size", type=int, required=True)
+    ap.add_argument("--window-start", type=float, default=3600.0)
+    ap.add_argument("--window-end", type=float, default=4200.0)
+    ap.add_argument("--gpu-count", type=int, default=8)
+    ap.add_argument("--length-mode", default="total")
+    ap.add_argument("--label", default="")
+    ap.add_argument("--tau", type=float, default=0.9)
+    ap.add_argument("--max-checks", type=int, default=20)
+    args = ap.parse_args()
+
+    requests = load_window(
+        args.jsonl, window_start=args.window_start, window_end=args.window_end
+    )
+    window = WindowRecord(
+        window_id=args.label or args.jsonl.stem,
+        trace_path=args.jsonl,
+        trace_type=args.label or "chat",
+        window_start=0.0,
+        window_end=float(args.window_end - args.window_start),
+        source_payload={"block_size": args.block_size},
+    )
+    mode = resolve_length_mode(length_mode=args.length_mode)
+    rows_with_hash = sum(1 for r in requests if r.metadata.get("hash_ids"))
+    print(
+        f"[{args.label}] requests={len(requests)} rows_with_hash_ids={rows_with_hash} "
+        f"window={args.window_start:.0f}-{args.window_end:.0f}s block_size={args.block_size}"
+    )
+
+    # Full curve (tau_c high so it never short-circuits; we read the curve directly).
+    point = find_convergence_prefix(
+        requests, window, gpu_count=args.gpu_count, length_mode=mode,
+        tau=args.tau, tau_c=1.01, stable_checks=10_000, max_checks=args.max_checks,
+        min_fraction=0.05,
+    )
+    print("  frac   time_s     L      C      A")
+    for c in point.checks:
+        s = c["family_similarity"]
+        print(
+            f"  {c['fraction']:.2f}  {c['time_s']:7.1f}  "
+            f"{s['L']:.3f}  {s['C']:.3f}  {s['A']:.3f}"
+        )
+
+    # Stop fraction at candidate tau_c values (L,A >= tau, C >= tau_c, stable for W=3).
+    print("  -- stop fraction (tau_L=tau_A=%.2f, W=3) --" % args.tau)
+    for tau_c in (0.85, 0.90, 0.92, 0.95):
+        p = find_convergence_prefix(
+            requests, window, gpu_count=args.gpu_count, length_mode=mode,
+            tau=args.tau, tau_c=tau_c, stable_checks=3, max_checks=args.max_checks,
+            min_fraction=0.05,
+        )
+        verdict = (
+            f"stop@frac={p.fraction:.2f} t={p.stop_time_s:.0f}s"
+            if p.converged
+            else "NEVER (replays full window)"
+        )
+        print(f"  tau_c={tau_c:.2f}: {verdict}")
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

scripts/stop_a_validate.py

@@ -0,0 +1,101 @@
+#!/usr/bin/env python3
+"""Validate Stop-A truncation fidelity from a full-replay trial's probe_details.
+
+Given a completed trial that replayed the full window (adaptive_stop disabled), for
+each probe recompute the L-C-A convergence prefix and compare the feasibility
+verdict / pass-rate of the truncated prefix against the full probe. This answers:
+"would Stop-A have changed the measured peak-sustainable-rate?" using only the one
+full run (no second GPU run needed).
+
+Example:
+  PYTHONPATH=src python3 scripts/stop_a_validate.py \
+    --spec configs/examples/dash0_qwen30b_a3b_stopA_fulldata.json \
+    --store-root .aituner/stopA-fulldata --tau 0.9 --tau-c 0.90
+"""
+from __future__ import annotations
+
+import argparse
+import json
+from pathlib import Path
+
+from aituner.lca import find_convergence_prefix, resolve_length_mode
+from aituner.spec import load_study_spec
+from aituner.trace import load_trace_requests, select_requests_for_threshold
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--spec", type=Path, required=True)
+    ap.add_argument("--store-root", type=Path, required=True)
+    ap.add_argument("--tau", type=float, default=0.9)
+    ap.add_argument("--tau-c", type=float, default=0.90)
+    ap.add_argument("--stable-checks", type=int, default=3)
+    ap.add_argument("--target-pass-rate", type=float, default=0.95)
+    args = ap.parse_args()
+
+    study = load_study_spec(args.spec)
+    window, requests = load_trace_requests(study, study_spec_path=args.spec)
+    mode = resolve_length_mode(request_mode=study.trace.request_mode)
+    gpu_count = study.hardware.gpu_count
+
+    detail_files = sorted(args.store_root.glob("*/trials/*/probe_details.jsonl"))
+    if not detail_files:
+        print(f"no probe_details.jsonl under {args.store_root}")
+        return 1
+
+    print(f"target_pass_rate={args.target_pass_rate} tau={args.tau} tau_c={args.tau_c}")
+    print(
+        "thresh   n_full  stop_idx  frac  full_pass  prefix_pass  "
+        "full_feas  prefix_feas  verdict_match"
+    )
+    mismatches = 0
+    total = 0
+    saved_fractions = []
+    for detail_file in detail_files:
+        with detail_file.open(encoding="utf-8") as handle:
+            for line in handle:
+                line = line.strip()
+                if not line:
+                    continue
+                probe = json.loads(line)
+                threshold = float(probe["threshold"])
+                outcomes = probe.get("outcomes") or []
+                # arrival-ordered outcomes that carry an arrival_s and verdict
+                ordered = sorted(
+                    (o for o in outcomes if o.get("arrival_s") is not None),
+                    key=lambda o: float(o["arrival_s"]),
+                )
+                n = len(ordered)
+                if n == 0:
+                    continue
+                selected = select_requests_for_threshold(requests, threshold=threshold)
+                cp = find_convergence_prefix(
+                    selected, window, gpu_count=gpu_count, length_mode=mode,
+                    tau=args.tau, tau_c=args.tau_c, stable_checks=args.stable_checks,
+                )
+                # Map the convergence prefix fraction onto the replayed outcomes.
+                stop_n = max(1, min(n, round(cp.fraction * n)))
+                full_pass = sum(1 for o in ordered if o.get("evaluation")) / n
+                prefix_pass = sum(1 for o in ordered[:stop_n] if o.get("evaluation")) / stop_n
+                full_feas = full_pass >= args.target_pass_rate
+                prefix_feas = prefix_pass >= args.target_pass_rate
+                match = full_feas == prefix_feas
+                total += 1
+                mismatches += 0 if match else 1
+                saved_fractions.append(1.0 - cp.fraction)
+                print(
+                    f"{threshold:.5f}  {n:6d}  {stop_n:7d}  {cp.fraction:.2f}  "
+                    f"{full_pass:.3f}      {prefix_pass:.3f}        "
+                    f"{str(full_feas):5s}      {str(prefix_feas):5s}        {match}"
+                )
+    if total:
+        avg_saved = sum(saved_fractions) / len(saved_fractions)
+        print(
+            f"\nverdict matches: {total - mismatches}/{total}  "
+            f"mean replay saved: {avg_saved*100:.0f}%"
+        )
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

src/aituner/cli.py

@@ -3,17 +3,118 @@ from __future__ import annotations
 import argparse
 import json
 import sys
+from dataclasses import replace
 from pathlib import Path
 
 from .compare import run_compare
+from .harness import (
+    build_harness_context,
+    build_harness_guided_proposal,
+    build_harness_stop_proposal,
+)
 from .job import append_job, build_trial_job
+from .lca import (
+    build_study_workload_profile,
+    build_workload_profile,
+    resolve_length_mode,
+    similarity_report,
+)
 from .llm import build_prompt, call_llm_for_proposal, load_capability_profile, parse_proposal_text
-from .spec import Proposal, SpecError, load_study_spec
+from .spec import (
+    Proposal,
+    SpecError,
+    StudySpec,
+    load_structured_file,
+    load_study_spec,
+    to_jsonable,
+)
 from .store import StudyStore
 from .trace import load_trace_requests, summarize_window
 from .worker import run_trial
 
 
+def _is_empty_config_patch(proposal: Proposal) -> bool:
+    return not proposal.config_patch.env_patch and not proposal.config_patch.flag_patch
+
+
+def _latency_percentiles(summary: object, metric: str) -> dict[str, float]:
+    if not isinstance(summary, dict):
+        return {}
+    payload = summary.get(metric)
+    if not isinstance(payload, dict):
+        return {}
+    selected: dict[str, float] = {}
+    for key in ("mean", "p50", "p95", "p99"):
+        value = payload.get(key)
+        if isinstance(value, (int, float)):
+            selected[key] = float(value)
+    return selected
+
+
+def _format_latency_percentiles(metric: str, values: dict[str, float]) -> str:
+    if not values:
+        return ""
+    ordered = ", ".join(
+        f"{key}={values[key]:.3f}"
+        for key in ("mean", "p50", "p95", "p99")
+        if key in values
+    )
+    return f"{metric}({ordered})"
+
+
+def _baseline_all_infeasible_stop(result: dict[str, object]) -> tuple[str, dict[str, object]] | None:
+    if result.get("status") != "completed":
+        return None
+    if isinstance(result.get("best_request_rate"), (int, float)):
+        return None
+    probes = result.get("probes")
+    if not isinstance(probes, list) or not probes:
+        return None
+    if any(isinstance(probe, dict) and probe.get("feasible") for probe in probes):
+        return None
+
+    diagnostics = result.get("all_infeasible_diagnostics")
+    if not isinstance(diagnostics, dict):
+        diagnostics = {}
+    lowest_rate = diagnostics.get("request_rate")
+    lowest_threshold = diagnostics.get("threshold")
+    pass_rate = diagnostics.get("pass_rate")
+    early_stop_reason = str(diagnostics.get("early_stop_reason") or "").strip()
+    latency_summary = diagnostics.get("latency_summary")
+    ttft = _latency_percentiles(latency_summary, "ttft_ms")
+    tpot = _latency_percentiles(latency_summary, "tpot_ms")
+    details: dict[str, object] = {
+        "lowest_sampled_request_rate": lowest_rate,
+        "lowest_sampling_u": lowest_threshold,
+        "lowest_probe_pass_rate": pass_rate,
+        "early_stop_reason": early_stop_reason,
+        "lowest_probe_latency_ms": {
+            "ttft": ttft,
+            "tpot": tpot,
+        },
+        "lowest_probe_latency_summary": latency_summary if isinstance(latency_summary, dict) else {},
+    }
+    pieces = [
+        "Baseline configuration has no feasible probe under the current SLO.",
+        "Stopping tuning because even the lowest sampled request rate did not meet the target pass rate.",
+    ]
+    if isinstance(lowest_rate, (int, float)):
+        pieces.append(f"lowest_sampled_request_rate={float(lowest_rate):.6g}")
+    if isinstance(lowest_threshold, (int, float)):
+        pieces.append(f"lowest_sampling_u={float(lowest_threshold):.6g}")
+    if isinstance(pass_rate, (int, float)):
+        pieces.append(f"lowest_probe_pass_rate={float(pass_rate):.6g}")
+    if early_stop_reason:
+        pieces.append(f"early_stop_reason={early_stop_reason}")
+    for item in (
+        _format_latency_percentiles("lowest_probe_ttft_ms", ttft),
+        _format_latency_percentiles("lowest_probe_tpot_ms", tpot),
+    ):
+        if item:
+            pieces.append(item)
+    return " ".join(pieces), details
+
+
 def _study_source_path(study_root: Path) -> Path:
     return Path((study_root / "study_spec.source").read_text(encoding="utf-8").strip())
 
@@ -40,6 +141,7 @@ def cmd_study_prompt(args: argparse.Namespace) -> int:
         window_summary=summarize_window(requests, window),
         state=state,
         capability_profile=capability_profile,
+        workload_profile=build_study_workload_profile(study, requests, window),
     )
     prompt_name = args.prompt_name or f"prompt-{state.next_trial_index:04d}"
     path = store.write_prompt(study.study_id, prompt_name, prompt)
@@ -60,8 +162,13 @@ def cmd_study_llm_propose(args: argparse.Namespace) -> int:
         window_summary=summarize_window(requests, window),
         state=state,
         capability_profile=capability_profile,
+        workload_profile=build_study_workload_profile(study, requests, window),
+    )
+    proposal_text = call_llm_for_proposal(
+        policy=study.llm,
+        prompt=prompt,
+        use_harness=study.llm.use_harness,
     )
-    proposal_text = call_llm_for_proposal(policy=study.llm, prompt=prompt)
     proposal = parse_proposal_text(proposal_text, study)
     name = args.proposal_name or f"proposal-{state.next_trial_index:04d}"
     path = store.write_proposal(study.study_id, name, proposal)
@@ -113,48 +220,192 @@ def cmd_study_tune(args: argparse.Namespace) -> int:
     study_root = store.init_study(spec_path=spec_path, study=study)
     capability_profile = load_capability_profile(study, study_spec_path=spec_path)
     proposal_files = [Path(item).resolve() for item in (args.proposal_file or [])]
-    max_trials = args.max_trials or (len(proposal_files) if proposal_files else 1)
+    max_trials = args.max_trials or (len(proposal_files) if proposal_files else 2)
     if max_trials <= 0:
         raise SpecError("max_trials must be positive")
     if proposal_files and max_trials > len(proposal_files):
         max_trials = len(proposal_files)
-    if not proposal_files and study.llm.endpoint is None:
-        raise SpecError("No proposal files provided and study.llm.endpoint is not configured")
-
     executed: list[dict[str, object]] = []
+    stop_vetoes = 0
+    max_llm_stop_vetoes = 1
     for idx in range(max_trials):
         state = store.load_state(study.study_id)
+        if state.tuning_stop_reason:
+            executed.append(
+                {
+                    "trial_id": None,
+                    "stopped": True,
+                    "reason": state.tuning_stop_reason,
+                    "diagnosis": state.tuning_stop_diagnosis,
+                    "details": state.tuning_stop_details,
+                    "state_best_trial_id": state.best_trial_id,
+                    "state_best_request_rate": state.best_request_rate,
+                }
+            )
+            break
+        if state.next_trial_index > max_trials:
+            break
         window, requests = load_trace_requests(study, study_spec_path=spec_path)
+        window_summary = summarize_window(requests, window)
+        workload_profile = build_study_workload_profile(study, requests, window)
+        harness_context = (
+            build_harness_context(
+                study=study,
+                window_summary=window_summary,
+                state=state,
+                workload_profile=workload_profile,
+            )
+            if study.llm.use_harness
+            else None
+        )
         prompt = build_prompt(
             study=study,
-            window_summary=summarize_window(requests, window),
+            window_summary=window_summary,
             state=state,
             capability_profile=capability_profile,
+            workload_profile=workload_profile,
         )
         prompt_name = f"prompt-{state.next_trial_index:04d}"
         store.write_prompt(study.study_id, prompt_name, prompt)
 
-        if proposal_files:
-            proposal_source = proposal_files[idx]
+        if (
+            not proposal_files
+            and not args.skip_baseline
+            and state.next_trial_index == 1
+            and not state.trials
+        ):
+            proposal_source = None
+            proposal_name = "baseline-0001"
+            proposal_text = json.dumps(
+                {
+                    "observation": "Evaluate the study's initial engine configuration before LLM-guided edits.",
+                    "diagnosis": "Baseline trial aligned with the AITuner evaluate-then-search loop.",
+                    "config_patch": {"env_patch": {}, "flag_patch": {}},
+                    "expected_effects": [
+                        "establish incumbent performance",
+                        "provide bottleneck evidence for harness-guided proposals",
+                    ],
+                    "why_not_previous_failures": "No config changes are applied.",
+                    "should_stop": False,
+                },
+                ensure_ascii=False,
+            )
+        elif proposal_files:
+            proposal_index = state.next_trial_index - 1
+            if proposal_index >= len(proposal_files):
+                break
+            proposal_source = proposal_files[proposal_index]
             proposal_text = proposal_source.read_text(encoding="utf-8")
             proposal_name = proposal_source.stem
         else:
             proposal_source = None
-            proposal_text = call_llm_for_proposal(policy=study.llm, prompt=prompt)
-            proposal_name = f"proposal-{state.next_trial_index:04d}"
+            stop_proposal = (
+                build_harness_stop_proposal(harness_context)
+                if harness_context is not None
+                else None
+            )
+            if stop_proposal is not None:
+                proposal_text = json.dumps(to_jsonable(stop_proposal), ensure_ascii=False)
+                proposal_name = f"harness-stop-{state.next_trial_index:04d}"
+            else:
+                guided_proposal = (
+                    build_harness_guided_proposal(harness_context)
+                    if harness_context is not None
+                    else None
+                )
+                if guided_proposal is not None:
+                    proposal_text = json.dumps(
+                        to_jsonable(guided_proposal),
+                        ensure_ascii=False,
+                    )
+                    proposal_name = f"harness-proposal-{state.next_trial_index:04d}"
+                else:
+                    if study.llm.endpoint is None:
+                        raise SpecError(
+                            "No proposal files provided, study.llm.endpoint is not configured, "
+                            "and the harness stop guard did not fire."
+                        )
+                    proposal_text = call_llm_for_proposal(
+                        policy=study.llm,
+                        prompt=prompt,
+                        use_harness=study.llm.use_harness,
+                    )
+                    proposal_name = f"proposal-{state.next_trial_index:04d}"
         raw_proposal_path = store.study_root(study.study_id) / "proposals" / f"{proposal_name}.raw.txt"
         raw_proposal_path.write_text(proposal_text, encoding="utf-8")
         proposal = parse_proposal_text(proposal_text, study)
         store.write_proposal(study.study_id, proposal_name, proposal)
+        if proposal.should_stop:
+            is_harness_stop = proposal_name.startswith("harness-stop-")
+            is_llm_stop = not is_harness_stop and proposal_source is None
+            stop_authority = (
+                harness_context.get("stop_authority")
+                if isinstance(harness_context, dict)
+                else None
+            )
+            authorized = stop_authority is None or bool(stop_authority.get("authorized"))
+            # Stop-B authority: the deterministic validator overrides an
+            # LLM-originated stop. Veto an unauthorized stop (bounded) so the
+            # loop does not converge prematurely on the agent's say-so alone.
+            if is_llm_stop and not authorized and stop_vetoes < max_llm_stop_vetoes:
+                stop_vetoes += 1
+                executed.append(
+                    {
+                        "trial_id": None,
+                        "proposal_name": proposal_name,
+                        "proposal_source": "llm",
+                        "stop_vetoed": True,
+                        "reason": "validator_did_not_authorize_stop",
+                        "validator_reason": (
+                            stop_authority.get("reason") if stop_authority else None
+                        ),
+                        "diagnosis": proposal.diagnosis,
+                    }
+                )
+                continue
+            if is_harness_stop:
+                proposal_source_label = "harness"
+            else:
+                proposal_source_label = str(proposal_source) if proposal_source else "llm"
+            executed.append(
+                {
+                    "trial_id": None,
+                    "proposal_name": proposal_name,
+                    "proposal_source": proposal_source_label,
+                    "stopped": True,
+                    "stop_authorized_by": (
+                        "validator"
+                        if (is_harness_stop or authorized)
+                        else "file_proposal"
+                        if proposal_source is not None
+                        else "llm_after_veto_budget"
+                    ),
+                    "diagnosis": proposal.diagnosis,
+                    "state_best_trial_id": state.best_trial_id,
+                    "state_best_request_rate": state.best_request_rate,
+                }
+            )
+            break
+        is_auto_baseline = (
+            not proposal_files
+            and not args.skip_baseline
+            and state.next_trial_index == 1
+            and not state.trials
+            and _is_empty_config_patch(proposal)
+        )
         trial, _ = store.materialize_trial(study=study, state=state, proposal=proposal)
         trial_spec_path = Path(trial.artifact_dir) / "trial_spec.json"
         result = run_trial(trial_spec_path)
         state = store.ingest_trial_results(study.study_id)
         executed.append(
-            {
-                "trial_id": trial.trial_id,
-                "proposal_name": proposal_name,
-                "proposal_source": str(proposal_source) if proposal_source else "llm",
+                {
+                    "trial_id": trial.trial_id,
+                    "proposal_name": proposal_name,
+                    "proposal_source": (
+                        "harness"
+                        if proposal_name.startswith("harness-proposal-")
+                        else str(proposal_source) if proposal_source else "llm"
+                    ),
                 "best_sampling_u": result.get("best_sampling_u"),
                 "best_request_rate": result.get("best_request_rate"),
                 "best_pass_rate": result.get("best_pass_rate"),
@@ -162,6 +413,26 @@ def cmd_study_tune(args: argparse.Namespace) -> int:
                 "state_best_request_rate": state.best_request_rate,
             }
         )
+        if is_auto_baseline:
+            stop = _baseline_all_infeasible_stop(result)
+            if stop is not None:
+                diagnosis, details = stop
+                state.tuning_stop_reason = "baseline_all_infeasible"
+                state.tuning_stop_diagnosis = diagnosis
+                state.tuning_stop_details = details
+                store.save_state(state)
+                executed.append(
+                    {
+                        "trial_id": None,
+                        "stopped": True,
+                        "reason": state.tuning_stop_reason,
+                        "diagnosis": diagnosis,
+                        "details": details,
+                        "state_best_trial_id": state.best_trial_id,
+                        "state_best_request_rate": state.best_request_rate,
+                    }
+                )
+                break
 
     final_state = store.load_state(study.study_id)
     print(
@@ -171,6 +442,9 @@ def cmd_study_tune(args: argparse.Namespace) -> int:
                 "executed_trials": executed,
                 "best_trial_id": final_state.best_trial_id,
                 "best_request_rate": final_state.best_request_rate,
+                "tuning_stop_reason": final_state.tuning_stop_reason,
+                "tuning_stop_diagnosis": final_state.tuning_stop_diagnosis,
+                "tuning_stop_details": final_state.tuning_stop_details,
             },
             ensure_ascii=False,
         )
@@ -203,6 +477,159 @@ def cmd_compare_run(args: argparse.Namespace) -> int:
     return 0
 
 
+def _resolve_profile_gpu_count(args: argparse.Namespace, study: StudySpec) -> int:
+    gpu_count = args.gpu_count
+    if gpu_count is None:
+        gpu_count = study.hardware.gpu_count
+    if gpu_count <= 0:
+        raise SpecError("--gpu-count must be > 0.")
+    return int(gpu_count)
+
+
+def _load_profile_study_spec(spec_path: Path) -> StudySpec:
+    payload = dict(load_structured_file(spec_path))
+    llm_payload = dict(payload.get("llm") or {})
+    llm_payload.pop("endpoint", None)
+    payload["llm"] = llm_payload
+    return StudySpec.from_dict(payload)
+
+
+def _profile_current_study_window(args: argparse.Namespace) -> dict[str, object]:
+    spec_path = Path(args.spec).resolve()
+    study = _load_profile_study_spec(spec_path)
+    mode = resolve_length_mode(
+        request_mode=study.trace.request_mode,
+        length_mode=args.length_mode,
+    )
+    window, requests = load_trace_requests(study, study_spec_path=spec_path)
+    profile = build_workload_profile(
+        requests,
+        window,
+        gpu_count=_resolve_profile_gpu_count(args, study),
+        length_mode=mode,
+    )
+    return {
+        "profile": profile.to_dict(),
+        "source": {
+            "study_spec_path": str(spec_path),
+            "window_id": study.trace.window_id,
+        },
+    }
+
+
+def _resolve_windows_path_for_profile(study: StudySpec, *, study_spec_path: Path) -> Path:
+    path = Path(study.trace.windows_path)
+    if not path.is_absolute():
+        path = (study_spec_path.parent / path).resolve()
+    return path
+
+
+def _load_profile_windows(
+    study: StudySpec,
+    *,
+    study_spec_path: Path,
+) -> list[dict[str, object]]:
+    windows_path = _resolve_windows_path_for_profile(study, study_spec_path=study_spec_path)
+    payload = json.loads(windows_path.read_text(encoding="utf-8"))
+    raw_windows = payload.get("windows") if isinstance(payload, dict) else payload
+    if not isinstance(raw_windows, list):
+        raise SpecError(f"windows payload must contain a list: {windows_path}")
+    return [
+        {str(key): value for key, value in item.items()}
+        for item in raw_windows
+        if isinstance(item, dict)
+    ]
+
+
+def _selected_profile_windows(
+    args: argparse.Namespace,
+    study: StudySpec,
+    *,
+    study_spec_path: Path,
+) -> list[dict[str, object]]:
+    windows = _load_profile_windows(study, study_spec_path=study_spec_path)
+    window_ids = set(args.window_id or [])
+    selected: list[dict[str, object]] = []
+    for item in windows:
+        window_id = str(item.get("window_id") or "").strip()
+        if not window_id:
+            continue
+        if window_ids and window_id not in window_ids:
+            continue
+        if not window_ids and not args.all:
+            if window_id != study.trace.window_id:
+                continue
+        trace_type = str(item.get("trace_type") or "").strip()
+        if args.trace_type and trace_type != args.trace_type:
+            continue
+        date_value = str(item.get("date") or "").strip()
+        if args.date_from and date_value and date_value < args.date_from:
+            continue
+        if args.date_to and date_value and date_value > args.date_to:
+            continue
+        if args.slot_token and str(item.get("slot_token") or "").strip() != args.slot_token:
+            continue
+        selected.append(item)
+    selected.sort(
+        key=lambda item: (
+            str(item.get("date") or ""),
+            str(item.get("slot_token") or ""),
+            str(item.get("window_id") or ""),
+        )
+    )
+    if args.limit is not None:
+        selected = selected[: args.limit]
+    if not selected:
+        raise SpecError("No trace windows selected for profile similarity.")
+    return selected
+
+
+def cmd_profile_window(args: argparse.Namespace) -> int:
+    print(json.dumps(_profile_current_study_window(args), ensure_ascii=False, indent=2))
+    return 0
+
+
+def cmd_profile_similarity(args: argparse.Namespace) -> int:
+    spec_path = Path(args.spec).resolve()
+    study = _load_profile_study_spec(spec_path)
+    mode = resolve_length_mode(
+        request_mode=study.trace.request_mode,
+        length_mode=args.length_mode,
+    )
+    gpu_count = _resolve_profile_gpu_count(args, study)
+    profiles = []
+    selected = _selected_profile_windows(args, study, study_spec_path=spec_path)
+    for item in selected:
+        window_id = str(item["window_id"])
+        window_study = replace(study, trace=replace(study.trace, window_id=window_id))
+        window, requests = load_trace_requests(window_study, study_spec_path=spec_path)
+        profiles.append(
+            build_workload_profile(
+                requests,
+                window,
+                gpu_count=gpu_count,
+                length_mode=mode,
+            )
+        )
+    print(
+        json.dumps(
+            {
+                "source": {
+                    "study_spec_path": str(spec_path),
+                    "selected_window_count": len(profiles),
+                    "length_mode": mode,
+                    "gpu_count": gpu_count,
+                },
+                "profiles": [profile.to_dict() for profile in profiles],
+                "similarity": similarity_report(profiles),
+            },
+            ensure_ascii=False,
+            indent=2,
+        )
+    )
+    return 0
+
+
 def build_parser() -> argparse.ArgumentParser:
     parser = argparse.ArgumentParser(description="AITuner CLI")
     subparsers = parser.add_subparsers(dest="command", required=True)
@@ -251,6 +678,11 @@ def build_parser() -> argparse.ArgumentParser:
     tune.add_argument("--store-root")
     tune.add_argument("--proposal-file", action="append")
     tune.add_argument("--max-trials", type=int)
+    tune.add_argument(
+        "--skip-baseline",
+        action="store_true",
+        help="Do not automatically evaluate the initial config before LLM proposals.",
+    )
     tune.set_defaults(func=cmd_study_tune)
 
     worker = subparsers.add_parser("worker")
@@ -266,6 +698,50 @@ def build_parser() -> argparse.ArgumentParser:
     compare_run.add_argument("--output-root")
     compare_run.set_defaults(func=cmd_compare_run)
 
+    profile = subparsers.add_parser("profile")
+    profile_sub = profile.add_subparsers(dest="profile_command", required=True)
+
+    profile_window = profile_sub.add_parser("window")
+    profile_window.add_argument("--spec", required=True)
+    profile_window.add_argument(
+        "--length-mode",
+        default="auto",
+        choices=["auto", "total", "input", "output"],
+        help="Token length basis for the L vector. auto uses output for decode_only and total otherwise.",
+    )
+    profile_window.add_argument(
+        "--gpu-count",
+        type=int,
+        help="GPU denominator for per-GPU arrival rate. Defaults to hardware.gpu_count.",
+    )
+    profile_window.set_defaults(func=cmd_profile_window)
+
+    profile_similarity = profile_sub.add_parser("similarity")
+    profile_similarity.add_argument("--spec", required=True)
+    profile_similarity.add_argument("--window-id", action="append")
+    profile_similarity.add_argument("--trace-type")
+    profile_similarity.add_argument("--date-from")
+    profile_similarity.add_argument("--date-to")
+    profile_similarity.add_argument("--slot-token")
+    profile_similarity.add_argument("--limit", type=int)
+    profile_similarity.add_argument(
+        "--all",
+        action="store_true",
+        help="Profile all windows selected by filters. Without this or --window-id, only the study window is used.",
+    )
+    profile_similarity.add_argument(
+        "--length-mode",
+        default="auto",
+        choices=["auto", "total", "input", "output"],
+        help="Token length basis for the L vector. auto uses output for decode_only and total otherwise.",
+    )
+    profile_similarity.add_argument(
+        "--gpu-count",
+        type=int,
+        help="GPU denominator for per-GPU arrival rate. Defaults to hardware.gpu_count.",
+    )
+    profile_similarity.set_defaults(func=cmd_profile_similarity)
+
     return parser
 
 

src/aituner/compare.py

@@ -382,6 +382,8 @@ def _aggregate_summary(rows: list[dict[str, Any]]) -> dict[str, Any]:
     wins = {"baseline": 0, "tuned": 0, "tie": 0, "incomparable": 0}
     for row in rows:
         wins[row["delta"]["winner"]] += 1
+    baseline_counts = _candidate_result_counts(rows, "baseline")
+    tuned_counts = _candidate_result_counts(rows, "tuned")
     return {
         "window_count": len(rows),
         "wins": wins,
@@ -389,9 +391,31 @@ def _aggregate_summary(rows: list[dict[str, Any]]) -> dict[str, Any]:
         "tuned_mean_request_rate": _mean_or_none(tuned_rates),
         "baseline_mean_request_rate_per_gpu": _mean_or_none(baseline_per_gpu),
         "tuned_mean_request_rate_per_gpu": _mean_or_none(tuned_per_gpu),
+        "baseline_completed_window_count": baseline_counts["completed"],
+        "baseline_failed_window_count": baseline_counts["failed"],
+        "baseline_no_feasible_window_count": baseline_counts["no_feasible"],
+        "tuned_completed_window_count": tuned_counts["completed"],
+        "tuned_failed_window_count": tuned_counts["failed"],
+        "tuned_no_feasible_window_count": tuned_counts["no_feasible"],
     }
 
 
+def _candidate_result_counts(rows: list[dict[str, Any]], name: str) -> dict[str, int]:
+    counts = {"completed": 0, "failed": 0, "no_feasible": 0}
+    for row in rows:
+        result = row.get(name)
+        if not isinstance(result, dict):
+            continue
+        status = str(result.get("status") or "")
+        if status == "completed":
+            counts["completed"] += 1
+        elif status == "failed":
+            counts["failed"] += 1
+        if not isinstance(result.get("best_request_rate_per_gpu"), (int, float)):
+            counts["no_feasible"] += 1
+    return counts
+
+
 def _mean_or_none(values: list[float]) -> float | None:
     if not values:
         return None
@@ -417,6 +441,8 @@ def _render_report(summary: dict[str, Any]) -> str:
         f"- Tuned mean request rate: `{summary['aggregate']['tuned_mean_request_rate']}`",
         f"- Baseline mean request rate per GPU: `{summary['aggregate']['baseline_mean_request_rate_per_gpu']}`",
         f"- Tuned mean request rate per GPU: `{summary['aggregate']['tuned_mean_request_rate_per_gpu']}`",
+        f"- Baseline completed/failed/no-feasible windows: `{summary['aggregate']['baseline_completed_window_count']}`/`{summary['aggregate']['baseline_failed_window_count']}`/`{summary['aggregate']['baseline_no_feasible_window_count']}`",
+        f"- Tuned completed/failed/no-feasible windows: `{summary['aggregate']['tuned_completed_window_count']}`/`{summary['aggregate']['tuned_failed_window_count']}`/`{summary['aggregate']['tuned_no_feasible_window_count']}`",
         "",
         "## Per Window",
         "",

src/aituner/http_client.py

@@ -179,6 +179,9 @@ def chat_completion(
     except urllib.error.HTTPError as exc:
         detail = exc.read().decode("utf-8", errors="replace")
         raise HttpClientError(f"llm_completion failed: {exc.code} {detail}") from exc
+    except OSError as exc:
+        # TimeoutError (socket.timeout), URLError, ConnectionError all subclass OSError.
+        raise HttpClientError(f"llm_completion failed: {exc}") from exc
 
 
 def stream_text_completion(
@@ -232,6 +235,8 @@ def stream_text_completion(
     except urllib.error.HTTPError as exc:
         detail = exc.read().decode("utf-8", errors="replace")
         raise HttpClientError(f"stream_text_completion failed: {exc.code} {detail}") from exc
+    except OSError as exc:
+        raise HttpClientError(f"stream_text_completion failed: {exc}") from exc
     return "".join(parts)
 
 
@@ -240,6 +245,8 @@ class StreamMetrics:
     ttft_ms: float | None
     tpot_ms: float | None
     completion_tokens: int | None
+    completion_tokens_source: str = "usage"
+    streamed_chunk_count: int = 0
 
 
 def stream_chat_completion(
@@ -260,6 +267,7 @@ def stream_chat_completion(
     last_token_at: float | None = None
     chunk_token_count = 0
     completion_tokens: int | None = None
+    completion_tokens_source = "none"
     try:
         with _urlopen(request, timeout=timeout_s) as response:
             for raw in _iter_sse_lines(response):
@@ -273,6 +281,7 @@ def stream_chat_completion(
                     comp = usage.get("completion_tokens")
                     if isinstance(comp, int) and comp >= 0:
                         completion_tokens = comp
+                        completion_tokens_source = "usage"
                 choices = payload.get("choices")
                 if not isinstance(choices, list) or not choices:
                     continue
@@ -289,8 +298,15 @@ def stream_chat_completion(
     except urllib.error.HTTPError as exc:
         detail = exc.read().decode("utf-8", errors="replace")
         raise HttpClientError(f"stream_chat_completion failed: {exc.code} {detail}") from exc
+    except OSError as exc:
+        # A request that exceeds request_timeout_s raises TimeoutError mid-stream;
+        # treat it as a failed request (SLO miss), not a crashed trial.
+        raise HttpClientError(f"stream_chat_completion failed: {exc}") from exc
     ttft_ms = None if first_token_at is None else (first_token_at - start) * 1000.0
-    used_tokens = completion_tokens if completion_tokens is not None else chunk_token_count
+    if completion_tokens is None and chunk_token_count > 0:
+        completion_tokens = chunk_token_count
+        completion_tokens_source = "stream_chunks"
+    used_tokens = completion_tokens
     if (
         first_token_at is None
         or last_token_at is None
@@ -303,7 +319,9 @@ def stream_chat_completion(
     return StreamMetrics(
         ttft_ms=ttft_ms,
         tpot_ms=tpot_ms,
-        completion_tokens=used_tokens if used_tokens > 0 else None,
+        completion_tokens=used_tokens if used_tokens is not None and used_tokens > 0 else None,
+        completion_tokens_source=completion_tokens_source,
+        streamed_chunk_count=chunk_token_count,
     )
 
 

src/aituner/lca.py

@@ -0,0 +1,577 @@
+from __future__ import annotations
+
+import json
+import math
+import statistics
+from dataclasses import dataclass
+from typing import TYPE_CHECKING, Any, Sequence
+
+from .trace import TraceRequest, WindowRecord
+
+if TYPE_CHECKING:
+    from .spec import StudySpec
+
+
+EPSILON = 1e-9
+
+FEATURE_NAMES = [
+    "L.log_mean_length",
+    "L.log_p95_over_mean_length",
+    "L.cv_length",
+    "C.log_mean_hit_length",
+    "C.log_p95_over_mean_hit_length",
+    "C.cv_hit_length",
+    "C.hit_rate",
+    "A.log_request_rate_per_gpu",
+    "A.cv_interarrival",
+    "A.log_fano_1s",
+]
+
+FAMILY_SLICES = {
+    "L": slice(0, 3),
+    "C": slice(3, 7),
+    "A": slice(7, 10),
+}
+
+LENGTH_MODES = {"total", "input", "output"}
+
+
+@dataclass(frozen=True)
+class WorkloadProfile:
+    window_id: str
+    trace_type: str
+    request_count: int
+    duration_s: float
+    gpu_count: int
+    length_mode: str
+    feature_names: list[str]
+    vector: list[float]
+    stats: dict[str, Any]
+
+    def to_dict(self) -> dict[str, Any]:
+        return {
+            "window_id": self.window_id,
+            "trace_type": self.trace_type,
+            "request_count": self.request_count,
+            "duration_s": self.duration_s,
+            "gpu_count": self.gpu_count,
+            "length_mode": self.length_mode,
+            "feature_names": self.feature_names,
+            "vector": self.vector,
+            "stats": self.stats,
+        }
+
+
+@dataclass(frozen=True)
+class RobustScale:
+    feature_names: list[str]
+    center: list[float]
+    scale: list[float]
+
+    def transform(self, vector: Sequence[float]) -> list[float]:
+        return [
+            (float(value) - self.center[idx]) / self.scale[idx]
+            for idx, value in enumerate(vector)
+        ]
+
+    def to_dict(self) -> dict[str, Any]:
+        return {
+            "feature_names": self.feature_names,
+            "center": self.center,
+            "scale": self.scale,
+        }
+
+
+def resolve_length_mode(*, request_mode: str | None = None, length_mode: str = "auto") -> str:
+    normalized = str(length_mode or "auto").strip().lower()
+    if normalized == "auto":
+        return (
+            "output"
+            if str(request_mode or "").strip().lower() == "decode_only"
+            else "total"
+        )
+    if normalized not in LENGTH_MODES:
+        raise ValueError(
+            "length_mode must be one of: auto, total, input, output."
+        )
+    return normalized
+
+
+def build_workload_profile(
+    requests: list[TraceRequest],
+    window: WindowRecord,
+    *,
+    gpu_count: int,
+    length_mode: str = "total",
+) -> WorkloadProfile:
+    if gpu_count <= 0:
+        raise ValueError("gpu_count must be > 0.")
+    if length_mode not in LENGTH_MODES:
+        raise ValueError(f"Unsupported length_mode: {length_mode}")
+
+    duration_s = _duration_s(requests, window)
+    input_lengths = [float(item.prompt_tokens_hint or 0) for item in requests]
+    output_lengths = [float(item.completion_tokens_hint or 0) for item in requests]
+    profile_lengths = [
+        _profile_length(input_len, output_len, length_mode=length_mode)
+        for input_len, output_len in zip(input_lengths, output_lengths)
+    ]
+    hit_lengths, cache_stats = _ideal_cache_hit_lengths(
+        requests,
+        input_lengths=input_lengths,
+        block_size=_block_size(window),
+    )
+    arrival_stats = _arrival_stats(requests, duration_s=duration_s, gpu_count=gpu_count)
+
+    length_stats = _series_stats(profile_lengths)
+    hit_stats = _series_stats(hit_lengths)
+    total_profile_length = sum(profile_lengths)
+    total_input_length = sum(input_lengths)
+    total_hit_length = sum(hit_lengths)
+    feature_hit_rate = (
+        float(total_hit_length / max(total_profile_length, EPSILON))
+        if total_profile_length > 0
+        else 0.0
+    )
+    input_hit_rate = (
+        float(total_hit_length / max(total_input_length, EPSILON))
+        if total_input_length > 0
+        else 0.0
+    )
+
+    vector = [
+        math.log1p(length_stats["mean"]),
+        math.log1p(length_stats["p95"] / max(length_stats["mean"], EPSILON)),
+        length_stats["cv"],
+        math.log1p(hit_stats["mean"]),
+        math.log1p(hit_stats["p95"] / max(hit_stats["mean"], EPSILON)),
+        hit_stats["cv"],
+        feature_hit_rate,
+        math.log1p(arrival_stats["request_rate_per_gpu"]),
+        arrival_stats["interarrival_cv"],
+        math.log1p(arrival_stats["fano_1s"]),
+    ]
+
+    return WorkloadProfile(
+        window_id=window.window_id,
+        trace_type=window.trace_type,
+        request_count=len(requests),
+        duration_s=duration_s,
+        gpu_count=int(gpu_count),
+        length_mode=length_mode,
+        feature_names=list(FEATURE_NAMES),
+        vector=[float(item) for item in vector],
+        stats={
+            "length": {
+                **length_stats,
+                "mode": length_mode,
+                "total": total_profile_length,
+                "input_total": total_input_length,
+                "output_total": sum(output_lengths),
+            },
+            "cache": {
+                **hit_stats,
+                **cache_stats,
+                "total_hit_length": total_hit_length,
+                "hit_rate": feature_hit_rate,
+                "input_hit_rate": input_hit_rate,
+            },
+            "arrival": arrival_stats,
+        },
+    )
+
+
+def build_study_workload_profile(
+    study: "StudySpec",
+    requests: list[TraceRequest],
+    window: WindowRecord,
+) -> WorkloadProfile:
+    """Canonical L-C-A profile for a study's loaded window.
+
+    This is the single source of truth for the paper's 10-dimensional L-C-A
+    feature vector used by the harness prompt and (later) by Stop-A.
+    """
+    mode = resolve_length_mode(
+        request_mode=study.trace.request_mode,
+        length_mode="auto",
+    )
+    return build_workload_profile(
+        requests,
+        window,
+        gpu_count=study.hardware.gpu_count,
+        length_mode=mode,
+    )
+
+
+def fit_robust_scale(profiles: Sequence[WorkloadProfile]) -> RobustScale:
+    if not profiles:
+        raise ValueError("At least one profile is required to fit a robust scale.")
+    centers: list[float] = []
+    scales: list[float] = []
+    for idx in range(len(FEATURE_NAMES)):
+        values = [float(profile.vector[idx]) for profile in profiles]
+        median = _percentile(values, 50.0)
+        iqr = _percentile(values, 75.0) - _percentile(values, 25.0)
+        centers.append(float(median))
+        scales.append(float(iqr if abs(iqr) > EPSILON else 1.0))
+    return RobustScale(feature_names=list(FEATURE_NAMES), center=centers, scale=scales)
+
+
+def profile_similarity(
+    left: WorkloadProfile,
+    right: WorkloadProfile,
+    *,
+    scale: RobustScale | None = None,
+) -> float:
+    scaler = scale or fit_robust_scale([left, right])
+    z_left = scaler.transform(left.vector)
+    z_right = scaler.transform(right.vector)
+    return _similarity_from_z(z_left, z_right)
+
+
+def similarity_report(profiles: Sequence[WorkloadProfile]) -> dict[str, Any]:
+    if not profiles:
+        raise ValueError("At least one profile is required.")
+    scale = fit_robust_scale(profiles)
+    transformed = [scale.transform(profile.vector) for profile in profiles]
+    rows: list[dict[str, Any]] = []
+    matrix: list[list[float]] = []
+    for i, left in enumerate(profiles):
+        row_values: list[float] = []
+        for j, right in enumerate(profiles):
+            sim = _similarity_from_z(transformed[i], transformed[j])
+            row_values.append(sim)
+            rows.append(
+                {
+                    "left": left.window_id,
+                    "right": right.window_id,
+                    "similarity": sim,
+                    "family_similarity": _family_similarity(transformed[i], transformed[j]),
+                }
+            )
+        matrix.append(row_values)
+    return {
+        "feature_names": list(FEATURE_NAMES),
+        "scaler": scale.to_dict(),
+        "windows": [profile.window_id for profile in profiles],
+        "matrix": matrix,
+        "pairs": rows,
+    }
+
+
+@dataclass(frozen=True)
+class ConvergencePoint:
+    converged: bool
+    stop_index: int
+    stop_time_s: float
+    fraction: float
+    family_similarity: dict[str, float]
+    checks: list[dict[str, Any]]
+
+    def to_dict(self) -> dict[str, Any]:
+        return {
+            "converged": self.converged,
+            "stop_index": self.stop_index,
+            "stop_time_s": self.stop_time_s,
+            "fraction": self.fraction,
+            "family_similarity": self.family_similarity,
+            "checks": self.checks,
+        }
+
+
+def find_convergence_prefix(
+    requests: list[TraceRequest],
+    window: WindowRecord,
+    *,
+    gpu_count: int,
+    length_mode: str = "total",
+    tau: float = 0.9,
+    tau_c: float = 0.92,
+    stable_checks: int = 3,
+    max_checks: int = 20,
+    min_fraction: float = 0.1,
+) -> ConvergencePoint:
+    """Earliest arrival-ordered prefix whose offered L-C-A converges to the full set.
+
+    The L-C-A vector is a deterministic function of the trace metadata, so the
+    convergence of prefix-vs-full is itself deterministic (the paper's Fig. 9
+    curve). Stop-A replays only up to this prefix. A prefix counts as converged
+    when the L and A family similarities reach ``tau`` and the (slowest) C family
+    similarity reaches the stricter ``tau_c`` for ``stable_checks`` consecutive
+    checkpoints. If that never happens within the window the point reports the
+    full set (converged=False), which keeps the C-gate honest: an unconverged C
+    means the probe must replay the whole window rather than stop early.
+    """
+    total = len(requests)
+    if total == 0:
+        return ConvergencePoint(
+            converged=False,
+            stop_index=0,
+            stop_time_s=0.0,
+            fraction=1.0,
+            family_similarity={"L": 1.0, "C": 1.0, "A": 1.0},
+            checks=[],
+        )
+    # Compare each arrival-ordered prefix to the whole set, both measured over
+    # their own elapsed span so the A (rate) dimension is comparable rather than
+    # diluted by the fixed window length.
+    target = _prefix_profile(
+        requests, total, window, gpu_count=gpu_count, length_mode=length_mode
+    )
+    indices = _checkpoint_indices(
+        total, max_checks=max_checks, min_fraction=min_fraction
+    )
+    checks: list[dict[str, Any]] = []
+    consecutive = 0
+    converged_index: int | None = None
+    converged_sims: dict[str, float] | None = None
+    for index in indices:
+        prefix = _prefix_profile(
+            requests, index, window, gpu_count=gpu_count, length_mode=length_mode
+        )
+        sims = _family_similarity(target.vector, prefix.vector)
+        checks.append(
+            {
+                "index": index,
+                "fraction": float(index / total),
+                "time_s": float(requests[index - 1].arrival_s),
+                "family_similarity": sims,
+            }
+        )
+        passed = sims["L"] >= tau and sims["A"] >= tau and sims["C"] >= tau_c
+        consecutive = consecutive + 1 if passed else 0
+        if consecutive >= stable_checks and converged_index is None:
+            converged_index = index
+            converged_sims = sims
+            break
+    if converged_index is None:
+        last_sims = checks[-1]["family_similarity"] if checks else {"L": 1.0, "C": 1.0, "A": 1.0}
+        return ConvergencePoint(
+            converged=False,
+            stop_index=total,
+            stop_time_s=float(requests[-1].arrival_s),
+            fraction=1.0,
+            family_similarity=last_sims,
+            checks=checks,
+        )
+    return ConvergencePoint(
+        converged=True,
+        stop_index=converged_index,
+        stop_time_s=float(requests[converged_index - 1].arrival_s),
+        fraction=float(converged_index / total),
+        family_similarity=converged_sims or {},
+        checks=checks,
+    )
+
+
+def _prefix_profile(
+    requests: list[TraceRequest],
+    index: int,
+    window: WindowRecord,
+    *,
+    gpu_count: int,
+    length_mode: str,
+) -> WorkloadProfile:
+    prefix = requests[:index]
+    start = float(prefix[0].arrival_s) if prefix else float(window.window_start)
+    end = float(prefix[-1].arrival_s) if prefix else float(window.window_start)
+    prefix_window = WindowRecord(
+        window_id=window.window_id,
+        trace_path=window.trace_path,
+        trace_type=window.trace_type,
+        window_start=start,
+        window_end=end,
+        source_payload=window.source_payload,
+    )
+    return build_workload_profile(
+        prefix, prefix_window, gpu_count=gpu_count, length_mode=length_mode
+    )
+
+
+def _checkpoint_indices(total: int, *, max_checks: int, min_fraction: float) -> list[int]:
+    start = max(1, int(math.ceil(min_fraction * total)))
+    if total <= max_checks:
+        candidates = range(start, total + 1)
+    else:
+        step = max(1, total // max_checks)
+        candidates = list(range(start, total + 1, step))
+        if candidates and candidates[-1] != total:
+            candidates.append(total)
+    seen: list[int] = []
+    for value in candidates:
+        clamped = min(total, max(1, int(value)))
+        if not seen or seen[-1] != clamped:
+            seen.append(clamped)
+    return seen
+
+
+def dumps_profile(profile: WorkloadProfile) -> str:
+    return json.dumps(profile.to_dict(), ensure_ascii=False, indent=2) + "\n"
+
+
+def _duration_s(requests: list[TraceRequest], window: WindowRecord) -> float:
+    duration = max(float(window.window_end) - float(window.window_start), 0.0)
+    if duration > 0:
+        return duration
+    if len(requests) >= 2:
+        return max(0.0, float(requests[-1].arrival_s) - float(requests[0].arrival_s))
+    return 0.0
+
+
+def _profile_length(input_length: float, output_length: float, *, length_mode: str) -> float:
+    if length_mode == "input":
+        return max(input_length, 0.0)
+    if length_mode == "output":
+        return max(output_length, 0.0)
+    return max(input_length, 0.0) + max(output_length, 0.0)
+
+
+def _block_size(window: WindowRecord) -> int:
+    value = window.source_payload.get("block_size")
+    if isinstance(value, bool):
+        return 1
+    if isinstance(value, (int, float)) and value > 0:
+        return int(value)
+    if isinstance(value, str) and value.strip():
+        try:
+            parsed = int(value)
+        except ValueError:
+            return 1
+        return parsed if parsed > 0 else 1
+    return 1
+
+
+def _ideal_cache_hit_lengths(
+    requests: list[TraceRequest],
+    *,
+    input_lengths: list[float],
+    block_size: int,
+) -> tuple[list[float], dict[str, Any]]:
+    seen_hashes: set[Any] = set()
+    hit_lengths: list[float] = []
+    total_blocks = 0
+    repeated_blocks = 0
+    rows_with_hash_ids = 0
+    for request, input_length in zip(requests, input_lengths):
+        hash_ids = request.metadata.get("hash_ids")
+        if not isinstance(hash_ids, list):
+            hit_lengths.append(0.0)
+            continue
+        rows_with_hash_ids += 1
+        repeated_for_request = 0
+        for hash_id in hash_ids:
+            total_blocks += 1
+            if hash_id in seen_hashes:
+                repeated_blocks += 1
+                repeated_for_request += 1
+            else:
+                seen_hashes.add(hash_id)
+        hit_lengths.append(float(min(max(input_length, 0.0), repeated_for_request * block_size)))
+    return hit_lengths, {
+        "block_size": block_size,
+        "rows_with_hash_ids": rows_with_hash_ids,
+        "total_blocks": total_blocks,
+        "repeated_blocks": repeated_blocks,
+        "repeated_block_ratio": (
+            float(repeated_blocks / total_blocks) if total_blocks else 0.0
+        ),
+    }
+
+
+def _arrival_stats(
+    requests: list[TraceRequest],
+    *,
+    duration_s: float,
+    gpu_count: int,
+) -> dict[str, Any]:
+    arrivals = [float(item.arrival_s) for item in requests]
+    interarrivals = [
+        max(0.0, arrivals[idx] - arrivals[idx - 1])
+        for idx in range(1, len(arrivals))
+    ]
+    per_second_counts = _per_second_counts(arrivals, duration_s=duration_s)
+    qps = float(len(requests) / duration_s) if duration_s > 0 else 0.0
+    return {
+        "request_rate": qps,
+        "request_rate_per_gpu": float(qps / gpu_count) if gpu_count > 0 else 0.0,
+        "interarrival_cv": _cv(interarrivals),
+        "fano_1s": _fano(per_second_counts),
+        "one_second_count_mean": statistics.fmean(per_second_counts)
+        if per_second_counts
+        else 0.0,
+        "one_second_count_variance": statistics.pvariance(per_second_counts)
+        if len(per_second_counts) >= 2
+        else 0.0,
+        "one_second_bin_count": len(per_second_counts),
+    }
+
+
+def _per_second_counts(arrivals: list[float], *, duration_s: float) -> list[float]:
+    if duration_s <= 0:
+        return [float(len(arrivals))] if arrivals else []
+    bin_count = max(1, int(math.ceil(duration_s)))
+    counts = [0.0 for _ in range(bin_count)]
+    for arrival in arrivals:
+        if arrival < 0:
+            continue
+        idx = int(math.floor(arrival))
+        if 0 <= idx < bin_count:
+            counts[idx] += 1.0
+    return counts
+
+
+def _series_stats(values: list[float]) -> dict[str, float]:
+    return {
+        "count": float(len(values)),
+        "mean": statistics.fmean(values) if values else 0.0,
+        "p50": _percentile(values, 50.0),
+        "p95": _percentile(values, 95.0),
+        "cv": _cv(values),
+    }
+
+
+def _cv(values: list[float]) -> float:
+    if not values:
+        return 0.0
+    mean = statistics.fmean(values)
+    if abs(mean) <= EPSILON:
+        return 0.0
+    return float(statistics.pstdev(values) / mean) if len(values) >= 2 else 0.0
+
+
+def _fano(values: list[float]) -> float:
+    if not values:
+        return 0.0
+    mean = statistics.fmean(values)
+    if abs(mean) <= EPSILON:
+        return 0.0
+    return float(statistics.pvariance(values) / mean) if len(values) >= 2 else 0.0
+
+
+def _percentile(values: Sequence[float], p: float) -> float:
+    if not values:
+        return 0.0
+    ordered = sorted(float(item) for item in values)
+    if len(ordered) == 1:
+        return ordered[0]
+    rank = (p / 100.0) * (len(ordered) - 1)
+    lower = int(math.floor(rank))
+    upper = int(math.ceil(rank))
+    if lower == upper:
+        return ordered[lower]
+    weight = rank - lower
+    return float(ordered[lower] * (1.0 - weight) + ordered[upper] * weight)
+
+
+def _similarity_from_z(left: Sequence[float], right: Sequence[float]) -> float:
+    distance = math.sqrt(
+        sum((float(lval) - float(rval)) ** 2 for lval, rval in zip(left, right))
+    )
+    return float(math.exp(-distance))
+
+
+def _family_similarity(left: Sequence[float], right: Sequence[float]) -> dict[str, float]:
+    result: dict[str, float] = {}
+    for family, family_slice in FAMILY_SLICES.items():
+        result[family] = _similarity_from_z(left[family_slice], right[family_slice])
+    return result

src/aituner/llm.py

@@ -1,12 +1,17 @@
 from __future__ import annotations
 
 import json
+import time
 from pathlib import Path
-from typing import Any
+from typing import TYPE_CHECKING, Any
 
+from .harness import build_harness_context, render_harness_context
 from .http_client import chat_completion, stream_text_completion
 from .spec import LLMPolicySpec, Proposal, SpecError, StudySpec, StudyState
 
+if TYPE_CHECKING:
+    from .lca import WorkloadProfile
+
 
 def _parse_bool_like(value: Any, *, context: str) -> bool:
     if isinstance(value, bool):
@@ -176,6 +181,7 @@ def build_prompt(
     window_summary: dict[str, Any],
     state: StudyState,
     capability_profile: dict[str, Any] | None,
+    workload_profile: "WorkloadProfile | None" = None,
 ) -> str:
     objective_notes: list[str] = []
     if study.trace.request_mode == "decode_only":
@@ -210,15 +216,102 @@ def build_prompt(
         )
     launch_failures = _launch_failure_history(state)
     parallel_candidates = _enumerate_parallel_candidates(study)
-    sections = [
+    common_preamble = [
         "You are tuning an OpenAI-compatible serving engine.",
-        "Return exactly one JSON object with keys: observation, diagnosis, config_patch, expected_effects, why_not_previous_failures.",
+        "Return exactly one JSON object with keys: observation, diagnosis, config_patch, expected_effects, why_not_previous_failures, should_stop.",
         "config_patch must contain env_patch and flag_patch.",
         "expected_effects must be a JSON array of short strings, not an object.",
+        (
+            "should_stop must be a boolean. Use true only when the harness convergence guard says another adjacent probe is not justified."
+            if study.llm.use_harness
+            else "should_stop must be a boolean. Use false unless no valid config can be proposed."
+        ),
         "Only use allowed tunable env keys and allowed tunable flag keys.",
         "Do not wrap the JSON in markdown fences or any extra text.",
         "Do not repeat a config that previously failed to launch unless the new patch explicitly removes the failing knob.",
         "Treat previous engine launch failures as hard negative evidence. If you touch TP/DP/EP, keep the proposal inside the topology constraints exactly.",
+    ]
+    if not study.llm.use_harness:
+        sections = [
+            *common_preamble,
+            "",
+            "Study context:",
+            json.dumps(
+                {
+                    "study_id": study.study_id,
+                    "objective": "maximize feasible request_rate_per_gpu at the SLO target",
+                    "current_best": {
+                        "trial_id": state.best_trial_id,
+                        "best_parallel_size": state.best_parallel_size,
+                        "best_sampling_u": state.best_sampling_u,
+                        "best_request_rate": state.best_request_rate,
+                        "best_request_rate_per_gpu": state.best_request_rate_per_gpu,
+                    },
+                    "hardware": {
+                        "gpu_count": study.hardware.gpu_count,
+                        "gpu_model": study.hardware.gpu_model,
+                    },
+                    "model": {
+                        "model_id": study.model.model_id,
+                        "served_model_name": study.model.served_model_name,
+                    },
+                    "trace": {
+                        "window_id": study.trace.window_id,
+                        "request_mode": study.trace.request_mode,
+                        "completion_tokens_override": study.trace.completion_tokens_override,
+                        "input_length_filter": (
+                            {
+                                "min_input_tokens": study.trace.input_length_filter.min_input_tokens,
+                                "max_input_tokens": study.trace.input_length_filter.max_input_tokens,
+                            }
+                            if study.trace.input_length_filter is not None
+                            else None
+                        ),
+                    },
+                    "engine": {
+                        "engine_name": study.engine.engine_name,
+                        "engine_version": study.engine.engine_version,
+                        "base_flags": study.engine.base_flags,
+                        "base_envs": study.engine.base_envs,
+                        "allowed_flag_keys": study.engine.tunable_flags,
+                        "allowed_env_keys": study.engine.tunable_envs,
+                        "topology_constraints": (
+                            study.engine.topology_constraints.__dict__
+                            if study.engine.topology_constraints is not None
+                            else None
+                        ),
+                    },
+                },
+                ensure_ascii=False,
+                indent=2,
+            ),
+            "",
+            "SLO:",
+            json.dumps(
+                {
+                    "target_pass_rate": study.slo.target_pass_rate,
+                    "ttft_rule": study.slo.ttft_rule,
+                    "tpot_rule": study.slo.tpot_rule,
+                    "objective_notes": objective_notes,
+                },
+                default=lambda value: value.__dict__,
+                ensure_ascii=False,
+                indent=2,
+            ),
+            "",
+            "Trial history:",
+            json.dumps(history, ensure_ascii=False, indent=2),
+            "",
+            "Known launch failures:",
+            json.dumps(launch_failures, ensure_ascii=False, indent=2),
+            "",
+            "Tested config signatures:",
+            json.dumps(_tested_config_signatures(state), ensure_ascii=False, indent=2),
+        ]
+        return "\n".join(sections)
+
+    sections = [
+        *common_preamble,
         (
             "TP/DP/EP are part of the tunable space for this study. Prioritize exploring legal topology changes in parallel space before runtime-only knobs unless recent history already proves a topology variant is worse or fails to launch."
             if parallel_candidates
@@ -249,6 +342,9 @@ def build_prompt(
                     "window_id": study.trace.window_id,
                     "request_mode": study.trace.request_mode,
                     "completion_tokens_override": study.trace.completion_tokens_override,
+                    "early_stop_max_lag_s": study.trace.early_stop_max_lag_s,
+                    "early_stop_max_elapsed_s": study.trace.early_stop_max_elapsed_s,
+                    "restart_engine_after_early_stop": study.trace.restart_engine_after_early_stop,
                     "input_length_filter": (
                         {
                             "min_input_tokens": study.trace.input_length_filter.min_input_tokens,
@@ -305,15 +401,60 @@ def build_prompt(
         "Parallel space candidates:",
         json.dumps(parallel_candidates, ensure_ascii=False, indent=2),
         "",
+        "Tested config signatures:",
+        json.dumps(_tested_config_signatures(state), ensure_ascii=False, indent=2),
+    ]
+    sections.extend(
+        [
+            "",
+            "Harnesses:",
+            render_harness_context(
+                build_harness_context(
+                    study=study,
+                    window_summary=window_summary,
+                    state=state,
+                    workload_profile=workload_profile,
+                )
+            ),
+            "",
+        ]
+    )
+    sections.extend(
+        [
         "The primary cross-topology comparison metric is request_rate_per_gpu, not raw request_rate.",
         "The proposal should beat the incumbent on request_rate_per_gpu under the 95%+ SLO target.",
-        "The evaluator uses the best feasible sampling_u from the same tp_dp_product group when it exists.",
-        "If a tp_dp_product group has no history yet, the evaluator starts from the study's original search.low and runs a full binary search for that group.",
+        (
+            "The evaluator may use the same tp_dp_product incumbent as the search floor when search.inherit_incumbent_floor=true."
+            if study.search.inherit_incumbent_floor
+            else "The evaluator runs each proposal over the full configured search range so raw per-iteration performance is measured directly."
+        ),
         "Do not assume a configuration with fewer GPUs should inherit the global incumbent sampling_u.",
-    ]
+        "Follow the active harness. Prefer stop over a weak exploratory proposal once a good incumbent has converged.",
+        ]
+    )
     return "\n".join(sections)
 
 
+def _tested_config_signatures(state: StudyState) -> list[dict[str, Any]]:
+    signatures: list[dict[str, Any]] = []
+    seen: set[str] = set()
+    for trial in state.trials:
+        config_patch = trial.config_patch or {}
+        signature = json.dumps(config_patch, sort_keys=True, ensure_ascii=False)
+        if signature in seen:
+            continue
+        seen.add(signature)
+        signatures.append(
+            {
+                "trial_id": trial.trial_id,
+                "status": trial.status,
+                "best_request_rate_per_gpu": trial.best_request_rate_per_gpu,
+                "config_patch": config_patch,
+            }
+        )
+    return signatures
+
+
 def load_capability_profile(study: StudySpec, *, study_spec_path: Path) -> dict[str, Any] | None:
     if not study.capability_profile_path:
         return None
@@ -540,30 +681,46 @@ def call_llm_for_proposal(
     *,
     policy: LLMPolicySpec,
     prompt: str,
+    use_harness: bool = True,
 ) -> str:
     if policy.endpoint is None:
         raise RuntimeError("study.llm.endpoint is not configured")
-    if policy.endpoint.stream:
-        return stream_text_completion(
-            base_url=policy.endpoint.base_url,
-            api_key_env=policy.endpoint.api_key_env,
-            provider=policy.endpoint.provider,
-            wire_api=policy.endpoint.wire_api,
-            model=policy.endpoint.model,
-            messages=[{"role": "user", "content": prompt}],
-            timeout_s=policy.endpoint.timeout_s,
-            system_prompt=policy.system_prompt,
-            reasoning_effort=policy.endpoint.reasoning_effort,
-        )
-    response = chat_completion(
-        base_url=policy.endpoint.base_url,
-        api_key_env=policy.endpoint.api_key_env,
-        provider=policy.endpoint.provider,
-        wire_api=policy.endpoint.wire_api,
-        model=policy.endpoint.model,
-        messages=[{"role": "user", "content": prompt}],
-        timeout_s=policy.endpoint.timeout_s,
-        system_prompt=policy.system_prompt,
-        reasoning_effort=policy.endpoint.reasoning_effort,
-    )
-    return _extract_response_text(response)
+    last_error: Exception | None = None
+    max_attempts = 4
+    for attempt in range(max_attempts):
+        try:
+            system_prompt = policy.system_prompt if use_harness else ""
+            if policy.endpoint.stream:
+                text = stream_text_completion(
+                    base_url=policy.endpoint.base_url,
+                    api_key_env=policy.endpoint.api_key_env,
+                    provider=policy.endpoint.provider,
+                    wire_api=policy.endpoint.wire_api,
+                    model=policy.endpoint.model,
+                    messages=[{"role": "user", "content": prompt}],
+                    timeout_s=policy.endpoint.timeout_s,
+                    system_prompt=system_prompt,
+                    reasoning_effort=policy.endpoint.reasoning_effort,
+                )
+            else:
+                response = chat_completion(
+                    base_url=policy.endpoint.base_url,
+                    api_key_env=policy.endpoint.api_key_env,
+                    provider=policy.endpoint.provider,
+                    wire_api=policy.endpoint.wire_api,
+                    model=policy.endpoint.model,
+                    messages=[{"role": "user", "content": prompt}],
+                    timeout_s=policy.endpoint.timeout_s,
+                    system_prompt=system_prompt,
+                    reasoning_effort=policy.endpoint.reasoning_effort,
+                )
+                text = _extract_response_text(response)
+            if text.strip():
+                return text
+            last_error = RuntimeError("LLM response content is empty")
+        except Exception as exc:  # noqa: BLE001
+            last_error = exc
+        if attempt < max_attempts - 1:
+            time.sleep(min(30.0, 2.0 * (2**attempt)))
+            continue
+    raise RuntimeError(f"LLM proposal failed after retry: {last_error}") from last_error

src/aituner/slo.py

@@ -15,6 +15,7 @@ class RequestOutcome:
     prompt_tokens: int | None
     completion_tokens: int | None
     error: str = ""
+    completion_tokens_source: str = ""
 
 
 @dataclass(frozen=True)
@@ -28,6 +29,9 @@ def _rule_threshold_ms(rule: ThresholdRule, prompt_tokens: int | None) -> float:
     if rule.kind == "fixed_ms":
         assert rule.threshold_ms is not None
         return rule.threshold_ms
+    if rule.kind == "linear_ms":
+        assert rule.intercept_ms is not None and rule.per_token_ms is not None
+        return float(rule.intercept_ms) + float(rule.per_token_ms) * float(prompt_tokens or 0)
     if rule.kind != "step_ms":
         raise ValueError(f"Unsupported threshold rule: {rule.kind}")
     prompt = float(prompt_tokens or 0)

src/aituner/spec.py

@@ -63,6 +63,16 @@ def _coerce_str_list(value: Any, *, context: str) -> list[str]:
     return result
 
 
+def _coerce_text(value: Any, *, context: str) -> str:
+    if isinstance(value, str) and value.strip():
+        return value.strip()
+    if isinstance(value, Mapping) or isinstance(value, list):
+        text = json.dumps(value, ensure_ascii=False, sort_keys=True)
+        if text:
+            return text
+    raise SpecError(f"{context} must be a non-empty string.")
+
+
 def _coerce_int_list(value: Any, *, context: str) -> list[int]:
     if value is None:
         return []
@@ -311,6 +321,71 @@ class InputLengthFilterSpec:
         return spec
 
 
+@dataclass(frozen=True)
+class AdaptiveStopSpec:
+    """Stop-A: truncate per-probe replay once the offered L-C-A converges.
+
+    Disabled by default; the thresholds are calibrated per workload (Phase 3)
+    before being switched on, so existing studies are unaffected.
+    """
+
+    enabled: bool = False
+    tau: float = 0.9
+    tau_c: float = 0.92
+    stable_checks: int = 3
+    max_checks: int = 20
+    min_fraction: float = 0.1
+    boundary_delta: float = 0.02
+
+    @classmethod
+    def from_dict(cls, data: Any) -> "AdaptiveStopSpec":
+        if data is None:
+            return cls()
+        m = _require_mapping(data, context="trace.adaptive_stop")
+        enabled = (
+            _require_bool(m.get("enabled"), context="trace.adaptive_stop.enabled")
+            if m.get("enabled") is not None
+            else False
+        )
+        tau = _require_float(m.get("tau", 0.9), context="trace.adaptive_stop.tau")
+        tau_c = _require_float(m.get("tau_c", 0.92), context="trace.adaptive_stop.tau_c")
+        stable_checks = _require_int(
+            m.get("stable_checks", 3), context="trace.adaptive_stop.stable_checks"
+        )
+        max_checks = _require_int(
+            m.get("max_checks", 20), context="trace.adaptive_stop.max_checks"
+        )
+        min_fraction = _require_float(
+            m.get("min_fraction", 0.1), context="trace.adaptive_stop.min_fraction"
+        )
+        boundary_delta = _require_float(
+            m.get("boundary_delta", 0.02), context="trace.adaptive_stop.boundary_delta"
+        )
+        for name, value in (("tau", tau), ("tau_c", tau_c), ("min_fraction", min_fraction)):
+            if not 0.0 < value <= 1.0:
+                raise SpecError(f"trace.adaptive_stop.{name} must be in (0, 1].")
+        if not 0.0 <= boundary_delta < 1.0:
+            raise SpecError("trace.adaptive_stop.boundary_delta must be in [0, 1).")
+        if stable_checks <= 0 or max_checks <= 0:
+            raise SpecError(
+                "trace.adaptive_stop.stable_checks and max_checks must be > 0."
+            )
+        if stable_checks > max_checks:
+            raise SpecError(
+                "trace.adaptive_stop.stable_checks must be <= max_checks, "
+                "otherwise convergence can never be detected."
+            )
+        return cls(
+            enabled=enabled,
+            tau=tau,
+            tau_c=tau_c,
+            stable_checks=stable_checks,
+            max_checks=max_checks,
+            min_fraction=min_fraction,
+            boundary_delta=boundary_delta,
+        )
+
+
 @dataclass(frozen=True)
 class TraceSpec:
     windows_path: str
@@ -327,6 +402,8 @@ class TraceSpec:
     replay_time_scale: float = 1.0
     early_stop_max_lag_s: float | None = None
     early_stop_max_elapsed_s: float | None = None
+    restart_engine_after_early_stop: bool = False
+    adaptive_stop: AdaptiveStopSpec = AdaptiveStopSpec()
 
     @classmethod
     def from_dict(cls, data: Mapping[str, Any]) -> "TraceSpec":
@@ -343,6 +420,33 @@ class TraceSpec:
             )
             if completion_tokens_override < 0:
                 raise SpecError("trace.completion_tokens_override must be >= 0.")
+        max_requests_value = (
+            _require_int(max_requests, context="trace.max_requests_per_probe")
+            if max_requests is not None
+            else None
+        )
+        if max_requests_value is not None and max_requests_value <= 0:
+            raise SpecError("trace.max_requests_per_probe must be > 0.")
+        synthetic_prompt_cap_value = (
+            _require_int(
+                synthetic_prompt_cap,
+                context="trace.synthetic_prompt_cap_tokens",
+            )
+            if synthetic_prompt_cap is not None
+            else None
+        )
+        if synthetic_prompt_cap_value is not None and synthetic_prompt_cap_value < 0:
+            raise SpecError("trace.synthetic_prompt_cap_tokens must be >= 0.")
+        replay_time_scale = _require_float(
+            data.get("replay_time_scale", 1.0), context="trace.replay_time_scale"
+        )
+        if replay_time_scale <= 0:
+            raise SpecError("trace.replay_time_scale must be > 0.")
+        max_concurrency = _require_int(
+            data.get("max_concurrency", 64), context="trace.max_concurrency"
+        )
+        if max_concurrency <= 0:
+            raise SpecError("trace.max_concurrency must be > 0.")
         return cls(
             windows_path=_require_str(data.get("windows_path"), context="trace.windows_path"),
             window_id=_require_str(data.get("window_id"), context="trace.window_id"),
@@ -353,9 +457,7 @@ class TraceSpec:
             completion_tokens_override=completion_tokens_override,
             u_field=str(data.get("u_field") or "sampling_u").strip(),
             timestamp_field=str(data.get("timestamp_field") or "timestamp").strip(),
-            max_concurrency=_require_int(
-                data.get("max_concurrency", 64), context="trace.max_concurrency"
-            ),
+            max_concurrency=max_concurrency,
             input_length_filter=(
                 InputLengthFilterSpec.from_dict(
                     _require_mapping(
@@ -367,13 +469,9 @@ class TraceSpec:
                 if data.get("input_length_filter") is not None
                 else None
             ),
-            max_requests_per_probe=int(max_requests) if max_requests is not None else None,
-            synthetic_prompt_cap_tokens=(
-                int(synthetic_prompt_cap) if synthetic_prompt_cap is not None else None
-            ),
-            replay_time_scale=_require_float(
-                data.get("replay_time_scale", 1.0), context="trace.replay_time_scale"
-            ),
+            max_requests_per_probe=max_requests_value,
+            synthetic_prompt_cap_tokens=synthetic_prompt_cap_value,
+            replay_time_scale=replay_time_scale,
             early_stop_max_lag_s=(
                 _require_float(
                     data.get("early_stop_max_lag_s"), context="trace.early_stop_max_lag_s"
@@ -389,6 +487,15 @@ class TraceSpec:
                 if data.get("early_stop_max_elapsed_s") is not None
                 else None
             ),
+            restart_engine_after_early_stop=(
+                _require_bool(
+                    data.get("restart_engine_after_early_stop"),
+                    context="trace.restart_engine_after_early_stop",
+                )
+                if data.get("restart_engine_after_early_stop") is not None
+                else request_mode == "decode_only"
+            ),
+            adaptive_stop=AdaptiveStopSpec.from_dict(data.get("adaptive_stop")),
         )
 
 
@@ -397,6 +504,8 @@ class ThresholdRule:
     kind: str
     threshold_ms: float | None = None
     buckets: list[dict[str, float]] = field(default_factory=list)
+    intercept_ms: float | None = None
+    per_token_ms: float | None = None
 
     @classmethod
     def from_dict(cls, data: Mapping[str, Any], *, context: str) -> "ThresholdRule":
@@ -408,6 +517,18 @@ class ThresholdRule:
                     data.get("threshold_ms"), context=f"{context}.threshold_ms"
                 ),
             )
+        if kind == "linear_ms":
+            # threshold = intercept_ms + per_token_ms * input_tokens
+            # e.g. "4s + L_in/8k" -> intercept_ms=4000, per_token_ms=0.125
+            intercept_ms = _require_float(
+                data.get("intercept_ms"), context=f"{context}.intercept_ms"
+            )
+            per_token_ms = _require_float(
+                data.get("per_token_ms"), context=f"{context}.per_token_ms"
+            )
+            if intercept_ms < 0 or per_token_ms < 0:
+                raise SpecError(f"{context}.intercept_ms/per_token_ms must be >= 0.")
+            return cls(kind=kind, intercept_ms=intercept_ms, per_token_ms=per_token_ms)
         if kind == "step_ms":
             raw = data.get("buckets")
             if not isinstance(raw, list) or not raw:
@@ -471,6 +592,7 @@ class SamplingSearchSpec:
     tolerance: float
     max_probes: int
     sample_seed: int
+    inherit_incumbent_floor: bool = False
 
     @classmethod
     def from_dict(cls, data: Mapping[str, Any]) -> "SamplingSearchSpec":
@@ -484,6 +606,10 @@ class SamplingSearchSpec:
             sample_seed=_require_int(
                 data.get("sample_seed", 20260325), context="search.sample_seed"
             ),
+            inherit_incumbent_floor=_require_bool(
+                data.get("inherit_incumbent_floor", False),
+                context="search.inherit_incumbent_floor",
+            ),
         )
 
 
@@ -557,6 +683,7 @@ class LLMPolicySpec:
     endpoint: LLMEndpointSpec | None
     system_prompt: str
     max_history_trials: int
+    use_harness: bool = True
 
     @classmethod
     def from_dict(cls, data: Mapping[str, Any] | None) -> "LLMPolicySpec":
@@ -574,6 +701,11 @@ class LLMPolicySpec:
             max_history_trials=_require_int(
                 payload.get("max_history_trials", 8), context="llm.max_history_trials"
             ),
+            use_harness=(
+                _require_bool(payload.get("use_harness"), context="llm.use_harness")
+                if payload.get("use_harness") is not None
+                else True
+            ),
         )
 
 
@@ -642,6 +774,7 @@ class Proposal:
     config_patch: ConfigPatch
     expected_effects: list[str]
     why_not_previous_failures: str = ""
+    should_stop: bool = False
 
     @classmethod
     def from_dict(cls, data: Mapping[str, Any]) -> "Proposal":
@@ -664,13 +797,18 @@ class Proposal:
                 expected_effects, context="proposal.expected_effects"
             )
         return cls(
-            observation=_require_str(data.get("observation"), context="proposal.observation"),
-            diagnosis=_require_str(data.get("diagnosis"), context="proposal.diagnosis"),
+            observation=_coerce_text(data.get("observation"), context="proposal.observation"),
+            diagnosis=_coerce_text(data.get("diagnosis"), context="proposal.diagnosis"),
             config_patch=ConfigPatch.from_dict(
                 _require_mapping(data.get("config_patch"), context="proposal.config_patch")
             ),
             expected_effects=expected_effects_value,
             why_not_previous_failures=str(data.get("why_not_previous_failures") or "").strip(),
+            should_stop=(
+                _require_bool(data.get("should_stop"), context="proposal.should_stop")
+                if data.get("should_stop") is not None
+                else False
+            ),
         )
 
 
@@ -712,6 +850,9 @@ class StudyState:
     best_request_rate: float | None = None
     best_request_rate_per_gpu: float | None = None
     next_trial_index: int = 1
+    tuning_stop_reason: str = ""
+    tuning_stop_diagnosis: str = ""
+    tuning_stop_details: dict[str, Any] = field(default_factory=dict)
     best_by_parallel_size: dict[str, dict[str, Any]] = field(default_factory=dict)
     trials: list[TrialSummary] = field(default_factory=list)
 

src/aituner/store.py

@@ -5,7 +5,15 @@ from dataclasses import replace
 from pathlib import Path
 from typing import Any
 
-from .spec import Proposal, StudySpec, StudyState, TrialSpec, TrialSummary, to_jsonable
+from .spec import ConfigPatch, Proposal, StudySpec, StudyState, TrialSpec, TrialSummary, to_jsonable
+
+
+_TOPOLOGY_FLAG_KEYS = {
+    "tensor-parallel-size",
+    "data-parallel-size",
+    "expert-parallel-size",
+    "enable-expert-parallel",
+}
 
 
 class StudyStore:
@@ -37,6 +45,9 @@ class StudyStore:
             best_request_rate=payload.get("best_request_rate"),
             best_request_rate_per_gpu=payload.get("best_request_rate_per_gpu"),
             next_trial_index=int(payload.get("next_trial_index", 1)),
+            tuning_stop_reason=str(payload.get("tuning_stop_reason") or ""),
+            tuning_stop_diagnosis=str(payload.get("tuning_stop_diagnosis") or ""),
+            tuning_stop_details=dict(payload.get("tuning_stop_details") or {}),
             best_by_parallel_size={
                 str(key): value
                 for key, value in (payload.get("best_by_parallel_size") or {}).items()
@@ -65,19 +76,30 @@ class StudyStore:
         state: StudyState,
         proposal: Proposal,
     ) -> tuple[TrialSpec, StudyState]:
+        proposal = _inherit_incumbent_topology_for_runtime_patch(
+            study=study,
+            state=state,
+            proposal=proposal,
+        )
         trial_id = f"trial-{state.next_trial_index:04d}"
         trial_root = self.study_root(study.study_id) / "trials" / trial_id
         trial_root.mkdir(parents=True, exist_ok=True)
         parallel_size = _parallel_size_for_proposal(study=study, proposal=proposal)
-        search_low = _derive_search_floor(study=study, state=state, parallel_size=parallel_size)
+        search = study.search
+        if study.search.inherit_incumbent_floor:
+            search = replace(
+                study.search,
+                low=_derive_search_floor(
+                    study=study,
+                    state=state,
+                    parallel_size=parallel_size,
+                ),
+            )
         spec = TrialSpec(
             study_id=study.study_id,
             trial_id=trial_id,
             config_patch=proposal.config_patch,
-            search=replace(
-                study.search,
-                low=search_low,
-            ),
+            search=search,
             study_spec_path=str((self.study_root(study.study_id) / "study_spec.source").resolve()),
             artifact_dir=str(trial_root),
             probe_log_path=str(trial_root / "probe_history.json"),
@@ -85,8 +107,7 @@ class StudyStore:
             result_path=str(trial_root / "result.json"),
         )
         self.write_json(trial_root / "trial_spec.json", to_jsonable(spec))
-        next_state = replace(state, next_trial_index=state.next_trial_index + 1)
-        next_state.trials.append(
+        next_trial = (
             TrialSummary(
                 trial_id=trial_id,
                 status="queued",
@@ -95,6 +116,11 @@ class StudyStore:
                 config_patch=to_jsonable(proposal.config_patch),
             )
         )
+        next_state = replace(
+            state,
+            next_trial_index=state.next_trial_index + 1,
+            trials=[*state.trials, next_trial],
+        )
         self.save_state(next_state)
         return spec, next_state
 
@@ -225,6 +251,47 @@ def _parallel_size_for_proposal(*, study: StudySpec, proposal: Proposal) -> int:
     return _parallel_size_for_config(study=study, flag_patch=proposal.config_patch.flag_patch)
 
 
+def _inherit_incumbent_topology_for_runtime_patch(
+    *,
+    study: StudySpec,
+    state: StudyState,
+    proposal: Proposal,
+) -> Proposal:
+    flag_patch = dict(proposal.config_patch.flag_patch)
+    env_patch = dict(proposal.config_patch.env_patch)
+    if not flag_patch and not env_patch:
+        return proposal
+    if _TOPOLOGY_FLAG_KEYS.intersection(flag_patch):
+        return proposal
+    if not state.best_trial_id:
+        return proposal
+    incumbent = next(
+        (trial for trial in state.trials if trial.trial_id == state.best_trial_id),
+        None,
+    )
+    if incumbent is None or not isinstance(incumbent.config_patch, dict):
+        return proposal
+    incumbent_patch = incumbent.config_patch.get("flag_patch")
+    if not isinstance(incumbent_patch, dict):
+        return proposal
+    inherited_topology = {
+        key: value
+        for key, value in incumbent_patch.items()
+        if key in _TOPOLOGY_FLAG_KEYS and study.engine.base_flags.get(key) != value
+    }
+    if not inherited_topology:
+        return proposal
+    merged_flag_patch = dict(inherited_topology)
+    merged_flag_patch.update(flag_patch)
+    return replace(
+        proposal,
+        config_patch=ConfigPatch(
+            env_patch=env_patch,
+            flag_patch=merged_flag_patch,
+        ),
+    )
+
+
 def _parallel_size_for_trial_id(*, study: StudySpec, study_root: Path, trial_id: str) -> int | None:
     trial_spec_path = study_root / "trials" / trial_id / "trial_spec.json"
     if not trial_spec_path.exists():

src/aituner/trace.py

@@ -2,7 +2,8 @@ from __future__ import annotations
 
 import json
 import math
-from dataclasses import dataclass
+import statistics
+from dataclasses import dataclass, field
 from pathlib import Path
 from typing import Any, Mapping
 
@@ -39,6 +40,7 @@ class TraceRequest:
     body: dict[str, Any]
     prompt_tokens_hint: int | None
     completion_tokens_hint: int | None
+    metadata: dict[str, Any] = field(default_factory=dict)
 
 
 def resolve_window_record(study: StudySpec, *, study_spec_path: Path) -> WindowRecord:
@@ -223,6 +225,12 @@ def load_trace_requests(study: StudySpec, *, study_spec_path: Path) -> tuple[Win
                     body=body,
                     prompt_tokens_hint=prompt_tokens_hint,
                     completion_tokens_hint=completion_tokens,
+                    metadata={
+                        "hash_ids": row.get("hash_ids") if isinstance(row.get("hash_ids"), list) else None,
+                        "turn": row.get("turn"),
+                        "parent_chat_id": row.get("parent_chat_id"),
+                        "type": row.get("type"),
+                    },
                 )
             )
     requests.sort(key=lambda item: item.arrival_s)
@@ -241,6 +249,23 @@ def summarize_window(requests: list[TraceRequest], window: WindowRecord) -> dict
         requests[-1].arrival_s - requests[0].arrival_s if len(requests) >= 2 else 0.0
     )
     qps = (len(requests) / duration) if duration > 0 else 0.0
+    interarrivals = [
+        max(0.0, requests[idx].arrival_s - requests[idx - 1].arrival_s)
+        for idx in range(1, len(requests))
+    ]
+    mean_interarrival = statistics.fmean(interarrivals) if interarrivals else 0.0
+    stdev_interarrival = statistics.pstdev(interarrivals) if len(interarrivals) >= 2 else 0.0
+    interarrival_cv = (
+        float(stdev_interarrival / mean_interarrival) if mean_interarrival > 0 else 0.0
+    )
+    one_second_bins: dict[int, int] = {}
+    for request in requests:
+        bin_id = int(math.floor(request.arrival_s))
+        one_second_bins[bin_id] = one_second_bins.get(bin_id, 0) + 1
+    one_second_counts = [float(value) for value in one_second_bins.values()]
+    cache_summary = _cache_summary(requests, window)
+    p50_prompt = _percentile(prompt_tokens, 50.0)
+    p95_prompt = _percentile(prompt_tokens, 95.0)
     return {
         "window_id": window.window_id,
         "trace_path": str(window.trace_path),
@@ -248,10 +273,66 @@ def summarize_window(requests: list[TraceRequest], window: WindowRecord) -> dict
         "request_count": len(requests),
         "duration_s": duration,
         "request_rate": qps,
-        "prompt_tokens_p50": _percentile(prompt_tokens, 50.0),
-        "prompt_tokens_p95": _percentile(prompt_tokens, 95.0),
+        "prompt_tokens_p50": p50_prompt,
+        "prompt_tokens_p95": p95_prompt,
+        "prompt_tokens_p99": _percentile(prompt_tokens, 99.0),
+        "prompt_tail_ratio_p95_p50": (
+            float(p95_prompt / max(p50_prompt, 1.0)) if prompt_tokens else 0.0
+        ),
         "completion_tokens_p50": _percentile(completion_tokens, 50.0),
         "completion_tokens_p95": _percentile(completion_tokens, 95.0),
+        "arrival_interarrival_cv": interarrival_cv,
+        "arrival_qps_1s_p50": _percentile(one_second_counts, 50.0),
+        "arrival_qps_1s_p95": _percentile(one_second_counts, 95.0),
+        "arrival_burst_ratio_p95_to_mean": (
+            float(_percentile(one_second_counts, 95.0) / max(qps, 1e-9))
+            if one_second_counts and qps > 0
+            else 0.0
+        ),
+        "prefix_cache": cache_summary,
+    }
+
+
+def _cache_summary(requests: list[TraceRequest], window: WindowRecord) -> dict[str, Any]:
+    block_size = int(window.source_payload.get("block_size") or 1)
+    seen_hashes: set[Any] = set()
+    repeated_blocks = 0
+    total_blocks = 0
+    repeated_token_estimate = 0
+    total_token_estimate = 0
+    multi_turn_count = 0
+    rows_with_hashes = 0
+    for request in requests:
+        prompt_tokens = int(request.prompt_tokens_hint or 0)
+        total_token_estimate += prompt_tokens
+        turn = request.metadata.get("turn")
+        if isinstance(turn, (int, float)) and turn > 1:
+            multi_turn_count += 1
+        hash_ids = request.metadata.get("hash_ids")
+        if not isinstance(hash_ids, list):
+            continue
+        rows_with_hashes += 1
+        request_repeated_blocks = 0
+        for hash_id in hash_ids:
+            total_blocks += 1
+            if hash_id in seen_hashes:
+                repeated_blocks += 1
+                request_repeated_blocks += 1
+            else:
+                seen_hashes.add(hash_id)
+        repeated_token_estimate += min(prompt_tokens, request_repeated_blocks * block_size)
+    return {
+        "block_size": block_size,
+        "rows_with_hash_ids": rows_with_hashes,
+        "multi_turn_request_ratio": (
+            float(multi_turn_count / len(requests)) if requests else 0.0
+        ),
+        "repeated_block_ratio": float(repeated_blocks / total_blocks) if total_blocks else 0.0,
+        "repeated_token_ratio_estimate": (
+            float(repeated_token_estimate / total_token_estimate)
+            if total_token_estimate
+            else 0.0
+        ),
     }
 
 

src/aituner/worker.py

@@ -8,6 +8,7 @@ import statistics
 import subprocess
 import threading
 import time
+import traceback
 from concurrent.futures import FIRST_COMPLETED, Future, ThreadPoolExecutor, wait
 from dataclasses import dataclass
 from pathlib import Path
@@ -15,6 +16,7 @@ from typing import Any, Callable
 
 from .engine import build_launch_recipe
 from .http_client import HttpClientError, stream_chat_completion, wait_for_server
+from .lca import find_convergence_prefix, resolve_length_mode
 from .search import ThresholdProbe, binary_search_max_feasible
 from .slo import RequestOutcome, evaluate_request, summarize_evaluations
 from .spec import ConfigPatch, SamplingSearchSpec, TrialSpec, load_study_spec, to_jsonable
@@ -105,13 +107,49 @@ def _run_one_request(
 ) -> RequestOutcome:
     try:
         metrics = stream_chat_completion(base_url=base_url, body=request.body, timeout_s=timeout_s)
+        expected_completion_tokens = request.completion_tokens_hint
+        actual_completion_tokens = metrics.completion_tokens
+        completion_tokens_source = getattr(metrics, "completion_tokens_source", "")
+        if expected_completion_tokens is not None:
+            if completion_tokens_source != "usage":
+                return RequestOutcome(
+                    request_id=request.row_id,
+                    success=False,
+                    ttft_ms=metrics.ttft_ms,
+                    tpot_ms=metrics.tpot_ms,
+                    prompt_tokens=request.prompt_tokens_hint,
+                    completion_tokens=actual_completion_tokens,
+                    error=(
+                        "completion_tokens_unverified "
+                        f"source={completion_tokens_source or 'unknown'} "
+                        f"expected={expected_completion_tokens} "
+                        f"actual={actual_completion_tokens}"
+                    ),
+                    completion_tokens_source=completion_tokens_source,
+                )
+            if actual_completion_tokens != expected_completion_tokens:
+                return RequestOutcome(
+                    request_id=request.row_id,
+                    success=False,
+                    ttft_ms=metrics.ttft_ms,
+                    tpot_ms=metrics.tpot_ms,
+                    prompt_tokens=request.prompt_tokens_hint,
+                    completion_tokens=actual_completion_tokens,
+                    error=(
+                        "completion_tokens_mismatch "
+                        f"expected={expected_completion_tokens} "
+                        f"actual={actual_completion_tokens}"
+                    ),
+                    completion_tokens_source=completion_tokens_source,
+                )
         return RequestOutcome(
             request_id=request.row_id,
             success=True,
             ttft_ms=metrics.ttft_ms,
             tpot_ms=metrics.tpot_ms,
             prompt_tokens=request.prompt_tokens_hint,
-            completion_tokens=metrics.completion_tokens or request.completion_tokens_hint,
+            completion_tokens=actual_completion_tokens or request.completion_tokens_hint,
+            completion_tokens_source=completion_tokens_source,
         )
     except HttpClientError as exc:
         return RequestOutcome(
@@ -125,6 +163,170 @@ def _run_one_request(
         )
 
 
+def _probe_outcome_details(
+    *,
+    threshold: float,
+    selected: list[TraceRequest],
+    outcomes: list[RequestOutcome],
+    evaluations: list[Any],
+    early_stopped: bool,
+    early_stop_reason: str,
+) -> dict[str, Any]:
+    selected_by_id = {request.row_id: request for request in selected}
+    return {
+        "threshold": threshold,
+        "early_stopped": early_stopped,
+        "early_stop_reason": early_stop_reason,
+        "outcomes": [
+            {
+                "request_id": outcome.request_id,
+                "sampling_u": (
+                    selected_by_id[outcome.request_id].sampling_u
+                    if outcome.request_id in selected_by_id
+                    else None
+                ),
+                "arrival_s": (
+                    selected_by_id[outcome.request_id].arrival_s
+                    if outcome.request_id in selected_by_id
+                    else None
+                ),
+                "success": outcome.success,
+                "ttft_ms": outcome.ttft_ms,
+                "tpot_ms": outcome.tpot_ms,
+                "prompt_tokens": outcome.prompt_tokens,
+                "expected_completion_tokens": (
+                    selected_by_id[outcome.request_id].completion_tokens_hint
+                    if outcome.request_id in selected_by_id
+                    else None
+                ),
+                "completion_tokens": outcome.completion_tokens,
+                "completion_tokens_source": outcome.completion_tokens_source,
+                "error": outcome.error,
+                "evaluation": evaluation.passed,
+                "reasons": evaluation.reasons,
+            }
+            for outcome, evaluation in zip(outcomes, evaluations)
+        ],
+    }
+
+
+_SIGTERM_NOT_INSTALLED = object()
+
+
+def _install_sigterm_as_keyboardinterrupt() -> Any:
+    """Make SIGTERM raise KeyboardInterrupt so the engine-teardown finally runs.
+
+    When `study tune` is killed, a default SIGTERM skips the finally blocks and
+    orphans the vLLM engine (and its EngineCore workers) on the GPUs. Converting
+    SIGTERM to KeyboardInterrupt lets _terminate_process_tree run. Only installable
+    from the main thread; returns the previous handler (or a sentinel).
+    """
+    if threading.current_thread() is not threading.main_thread():
+        return _SIGTERM_NOT_INSTALLED
+
+    def _handler(signum: int, frame: Any) -> None:
+        raise KeyboardInterrupt()
+
+    try:
+        return signal.signal(signal.SIGTERM, _handler)
+    except (ValueError, OSError):
+        return _SIGTERM_NOT_INSTALLED
+
+
+def _restore_sigterm(previous: Any) -> None:
+    if previous is _SIGTERM_NOT_INSTALLED:
+        return
+    if threading.current_thread() is not threading.main_thread():
+        return
+    try:
+        signal.signal(signal.SIGTERM, previous)
+    except (ValueError, OSError):
+        pass
+
+
+def _ignore_sigterm_if_main() -> None:
+    """Ignore SIGTERM during teardown so a second signal cannot orphan the engine."""
+    if threading.current_thread() is not threading.main_thread():
+        return
+    try:
+        signal.signal(signal.SIGTERM, signal.SIG_IGN)
+    except (ValueError, OSError):
+        pass
+
+
+def _adaptive_replay_set(
+    selected: list[TraceRequest],
+    *,
+    study: Any,
+    window: Any,
+) -> tuple[list[TraceRequest], dict[str, Any] | None]:
+    """Stop-A: truncate the replay to the offered-L-C-A convergence prefix.
+
+    Returns the (possibly shortened) request list to replay and a certificate of
+    the convergence decision. When Stop-A is disabled, or C never converges, the
+    full selected set is replayed (the C-gate: no early stop on a cold cache).
+    """
+    spec = study.trace.adaptive_stop
+    if not getattr(spec, "enabled", False) or not selected:
+        return selected, None
+    point = find_convergence_prefix(
+        selected,
+        window,
+        gpu_count=study.hardware.gpu_count,
+        length_mode=resolve_length_mode(request_mode=study.trace.request_mode),
+        tau=spec.tau,
+        tau_c=spec.tau_c,
+        stable_checks=spec.stable_checks,
+        max_checks=spec.max_checks,
+        min_fraction=spec.min_fraction,
+    )
+    replay = selected[: point.stop_index] if point.stop_index > 0 else selected
+    certificate = {
+        "enabled": True,
+        "converged": point.converged,
+        "stop_index": point.stop_index,
+        "total_selected": len(selected),
+        "fraction": point.fraction,
+        "stop_time_s": point.stop_time_s,
+        "family_similarity": point.family_similarity,
+    }
+    return replay, certificate
+
+
+def _should_extend_on_boundary(
+    *,
+    pass_rate: float,
+    target_pass_rate: float,
+    certificate: dict[str, Any] | None,
+    truncated: bool,
+    boundary_delta: float,
+) -> bool:
+    """SLO-boundary guard: re-measure on the full window when a truncated probe
+    lands within +/- boundary_delta of the SLO target.
+
+    Offered-L-C-A convergence cannot see engine-state drift in the window's tail,
+    so a near-boundary truncated verdict is untrustworthy. This fires only on
+    probes sitting on the feasibility knee, so non-boundary probes keep the Stop-A
+    time saving.
+    """
+    if certificate is None or not certificate.get("converged"):
+        return False
+    if not truncated or boundary_delta <= 0:
+        return False
+    return abs(float(pass_rate) - float(target_pass_rate)) <= float(boundary_delta)
+
+
+def _best_feasible_probe_record(probe_history: list[dict[str, Any]]) -> dict[str, Any] | None:
+    feasible = [
+        item
+        for item in probe_history
+        if item.get("feasible") and isinstance(item.get("request_rate"), (int, float))
+    ]
+    if not feasible:
+        return None
+    return max(feasible, key=lambda item: float(item["request_rate"]))
+
+
 def _replay_requests(
     requests: list[TraceRequest],
     *,
@@ -135,6 +337,7 @@ def _replay_requests(
     max_lag_s: float | None,
     max_elapsed_s: float | None,
     evaluate_outcome: Callable[[RequestOutcome], Any],
+    drain_inflight_on_early_stop: bool = True,
 ) -> tuple[list[RequestOutcome], bool, str]:
     outcomes_by_id: dict[str, RequestOutcome] = {}
     lock = threading.Lock()
@@ -209,18 +412,47 @@ def _replay_requests(
                 if sleep_for > 0:
                     time.sleep(min(sleep_for, 0.1))
         if early_stopped:
-            for future in list(futures_by_request):
-                future.cancel()
-            for request in futures_by_request.values():
-                outcomes_by_id[request.row_id] = RequestOutcome(
-                    request_id=request.row_id,
-                    success=False,
-                    ttft_ms=None,
-                    tpot_ms=None,
-                    prompt_tokens=request.prompt_tokens_hint,
-                    completion_tokens=request.completion_tokens_hint,
-                    error=early_stop_reason or "probe_early_stopped",
-                )
+            if drain_inflight_on_early_stop and futures_by_request:
+                done, not_done = wait(list(futures_by_request), timeout=timeout_s)
+                for future in done:
+                    request = futures_by_request[future]
+                    try:
+                        outcomes_by_id[request.row_id] = future.result(timeout=0)
+                    except Exception:  # noqa: BLE001
+                        outcomes_by_id[request.row_id] = RequestOutcome(
+                            request_id=request.row_id,
+                            success=False,
+                            ttft_ms=None,
+                            tpot_ms=None,
+                            prompt_tokens=request.prompt_tokens_hint,
+                            completion_tokens=request.completion_tokens_hint,
+                            error=early_stop_reason or "probe_early_stopped",
+                        )
+                for future in not_done:
+                    future.cancel()
+                    request = futures_by_request[future]
+                    outcomes_by_id[request.row_id] = RequestOutcome(
+                        request_id=request.row_id,
+                        success=False,
+                        ttft_ms=None,
+                        tpot_ms=None,
+                        prompt_tokens=request.prompt_tokens_hint,
+                        completion_tokens=request.completion_tokens_hint,
+                        error=early_stop_reason or "probe_early_stopped",
+                    )
+            else:
+                for future in list(futures_by_request):
+                    future.cancel()
+                for request in futures_by_request.values():
+                    outcomes_by_id[request.row_id] = RequestOutcome(
+                        request_id=request.row_id,
+                        success=False,
+                        ttft_ms=None,
+                        tpot_ms=None,
+                        prompt_tokens=request.prompt_tokens_hint,
+                        completion_tokens=request.completion_tokens_hint,
+                        error=early_stop_reason or "probe_early_stopped",
+                    )
             for request in requests:
                 if request.row_id in submitted_ids:
                     continue
@@ -276,27 +508,76 @@ def _wait_for_server_or_exit(
     raise HttpClientError(f"Timed out waiting for {base_url}{healthcheck_path}: {last_error}")
 
 
-def _terminate_process_tree(process: subprocess.Popen[str], *, timeout_s: float = 30.0) -> None:
-    if process.poll() is not None:
-        return
+def _process_group_exists(pgid: int) -> bool:
+    try:
+        os.killpg(pgid, 0)
+        return True
+    except ProcessLookupError:
+        return False
+
+
+def _pids_matching_env(marker_env: dict[str, str] | None) -> list[int]:
+    if not marker_env:
+        return []
+    expected = {
+        f"{key}={value}".encode()
+        for key, value in marker_env.items()
+    }
+    pids: list[int] = []
+    proc_root = Path("/proc")
+    for entry in proc_root.iterdir():
+        if not entry.name.isdigit():
+            continue
+        pid = int(entry.name)
+        if pid == os.getpid():
+            continue
+        try:
+            environ = (entry / "environ").read_bytes()
+        except (FileNotFoundError, PermissionError, ProcessLookupError):
+            continue
+        if expected.issubset(set(environ.split(b"\0"))):
+            pids.append(pid)
+    return sorted(pids)
+
+
+def _signal_pids(pids: list[int], sig: signal.Signals) -> None:
+    for pid in pids:
+        try:
+            os.kill(pid, sig)
+        except (ProcessLookupError, PermissionError):
+            continue
+
+
+def _terminate_process_tree(
+    process: subprocess.Popen[str],
+    *,
+    timeout_s: float = 30.0,
+    marker_env: dict[str, str] | None = None,
+) -> None:
     try:
         pgid = os.getpgid(process.pid)
     except ProcessLookupError:
-        return
+        # Children can keep the session/process group alive after the vLLM API
+        # server exits.  In that case the group id is still the original pid
+        # because the process was launched with start_new_session=True.
+        pgid = process.pid
     try:
         os.killpg(pgid, signal.SIGTERM)
     except ProcessLookupError:
-        return
+        pass
+    _signal_pids(_pids_matching_env(marker_env), signal.SIGTERM)
     deadline = time.monotonic() + timeout_s
     while time.monotonic() < deadline:
-        if process.poll() is not None:
+        if not _process_group_exists(pgid) and not _pids_matching_env(marker_env):
             return
         time.sleep(0.1)
     try:
         os.killpg(pgid, signal.SIGKILL)
     except ProcessLookupError:
-        return
-    process.wait(timeout=timeout_s)
+        pass
+    _signal_pids(_pids_matching_env(marker_env), signal.SIGKILL)
+    if process.poll() is None:
+        process.wait(timeout=timeout_s)
 
 
 def run_trial(trial_spec_path: Path) -> dict[str, Any]:
@@ -310,16 +591,28 @@ def run_trial(trial_spec_path: Path) -> dict[str, Any]:
     artifact_dir = Path(trial.artifact_dir)
     artifact_dir.mkdir(parents=True, exist_ok=True)
     engine_log_path = Path(trial.engine_log_path)
+    probe_details_path = artifact_dir / "probe_details.jsonl"
+    if probe_details_path.exists():
+        probe_details_path.unlink()
+    trial_marker_env = {
+        "AITUNER_STUDY_ID": trial.study_id,
+        "AITUNER_TRIAL_ID": trial.trial_id,
+    }
     with engine_log_path.open("w", encoding="utf-8") as engine_log:
-        process = subprocess.Popen(  # noqa: S603
-            recipe.argv,
-            cwd=recipe.cwd,
-            env=recipe.env,
-            stdout=engine_log,
-            stderr=subprocess.STDOUT,
-            text=True,
-            start_new_session=True,
-        )
+        def launch_process() -> subprocess.Popen[str]:
+            launch_env = {**recipe.env, **trial_marker_env}
+            return subprocess.Popen(  # noqa: S603
+                recipe.argv,
+                cwd=recipe.cwd,
+                env=launch_env,
+                stdout=engine_log,
+                stderr=subprocess.STDOUT,
+                text=True,
+                start_new_session=True,
+            )
+
+        process = launch_process()
+        previous_sigterm = _install_sigterm_as_keyboardinterrupt()
         probe_history: list[dict[str, Any]] = []
         failure_stage = "engine_launch"
         try:
@@ -332,18 +625,55 @@ def run_trial(trial_spec_path: Path) -> dict[str, Any]:
             failure_stage = "probe_search"
 
             def evaluator(threshold: float) -> ThresholdProbe[ProbePayload]:
+                nonlocal process
                 selected = select_requests_for_threshold(requests, threshold=threshold)
-                outcomes, early_stopped, early_stop_reason = _replay_requests(
-                    selected,
-                    base_url=recipe.base_url,
-                    timeout_s=recipe.request_timeout_s,
-                    max_concurrency=study.trace.max_concurrency,
-                    target_pass_rate=study.slo.target_pass_rate,
-                    max_lag_s=study.trace.early_stop_max_lag_s,
-                    max_elapsed_s=study.trace.early_stop_max_elapsed_s,
-                    evaluate_outcome=lambda outcome: evaluate_request(outcome, study.slo),
+                replay_set, adaptive_stop_certificate = _adaptive_replay_set(
+                    selected, study=study, window=window
                 )
+                restart_after_early_stop = study.trace.restart_engine_after_early_stop
+
+                def _run(reqs: list[TraceRequest]) -> tuple[list[RequestOutcome], bool, str]:
+                    return _replay_requests(
+                        reqs,
+                        base_url=recipe.base_url,
+                        timeout_s=recipe.request_timeout_s,
+                        max_concurrency=study.trace.max_concurrency,
+                        target_pass_rate=study.slo.target_pass_rate,
+                        max_lag_s=study.trace.early_stop_max_lag_s,
+                        max_elapsed_s=study.trace.early_stop_max_elapsed_s,
+                        evaluate_outcome=lambda outcome: evaluate_request(outcome, study.slo),
+                        drain_inflight_on_early_stop=not restart_after_early_stop,
+                    )
+
+                outcomes, early_stopped, early_stop_reason = _run(replay_set)
                 evaluations, summary = summarize_evaluations(outcomes, study.slo)
+                if _should_extend_on_boundary(
+                    pass_rate=summary["slo_pass_rate"],
+                    target_pass_rate=study.slo.target_pass_rate,
+                    certificate=adaptive_stop_certificate,
+                    truncated=len(replay_set) < len(selected),
+                    boundary_delta=study.trace.adaptive_stop.boundary_delta,
+                ):
+                    # On the feasibility knee the truncated verdict is untrustworthy;
+                    # re-measure the full window and use that result.
+                    replay_set = selected
+                    outcomes, early_stopped, early_stop_reason = _run(selected)
+                    evaluations, summary = summarize_evaluations(outcomes, study.slo)
+                    if adaptive_stop_certificate is not None:
+                        adaptive_stop_certificate["boundary_extended"] = True
+                probe_details = _probe_outcome_details(
+                    threshold=threshold,
+                    selected=replay_set,
+                    outcomes=outcomes,
+                    evaluations=evaluations,
+                    early_stopped=early_stopped,
+                    early_stop_reason=early_stop_reason,
+                )
+                probe_details["adaptive_stop"] = adaptive_stop_certificate
+                with probe_details_path.open("a", encoding="utf-8") as details_handle:
+                    details_handle.write(
+                        json.dumps(probe_details, ensure_ascii=False) + "\n"
+                    )
                 request_rate = (
                     len(selected) / max(window.window_end - window.window_start, 1e-9)
                     if selected
@@ -370,6 +700,7 @@ def run_trial(trial_spec_path: Path) -> dict[str, Any]:
                             "tpot_ms": outcome.tpot_ms,
                             "prompt_tokens": outcome.prompt_tokens,
                             "completion_tokens": outcome.completion_tokens,
+                            "completion_tokens_source": outcome.completion_tokens_source,
                             "evaluation": evaluation.passed,
                             "reasons": evaluation.reasons,
                         }
@@ -379,56 +710,118 @@ def run_trial(trial_spec_path: Path) -> dict[str, Any]:
                 probe_record = {
                     "threshold": threshold,
                     "request_count": payload.request_count,
+                    "replayed_request_count": len(replay_set),
                     "pass_rate": payload.pass_rate,
                     "request_rate": payload.request_rate,
                     "feasible": payload.feasible,
                     "early_stopped": payload.early_stopped,
                     "early_stop_reason": payload.early_stop_reason,
                     "latency_summary": payload.latency_summary,
+                    "adaptive_stop": adaptive_stop_certificate,
                 }
                 probe_history.append(probe_record)
                 StudyStore.write_json(Path(trial.probe_log_path), probe_history)
+                if early_stopped and restart_after_early_stop:
+                    _terminate_process_tree(
+                        process,
+                        timeout_s=30.0,
+                        marker_env=trial_marker_env,
+                    )
+                    process = launch_process()
+                    _wait_for_server_or_exit(
+                        process,
+                        base_url=recipe.base_url,
+                        healthcheck_path=recipe.healthcheck_path,
+                        ready_timeout_s=recipe.ready_timeout_s,
+                    )
                 return ThresholdProbe(
                     threshold=threshold,
                     feasible=payload.feasible,
                     payload=payload,
                 )
 
-            search = binary_search_max_feasible(
+            primary_search = binary_search_max_feasible(
                 low=trial.search.low,
                 high=trial.search.high,
                 tolerance=trial.search.tolerance,
                 max_probes=trial.search.max_probes,
                 evaluator=evaluator,
             )
-            best = search.best_feasible_payload
+            search_for_best = primary_search
+            best = primary_search.best_feasible_payload
+            best_source = "primary_search"
+            fallback_search = None
+            original_search_low = float(study.search.low)
+            inherited_search_floor = float(trial.search.low)
+            if best is None and inherited_search_floor > original_search_low:
+                fallback_search = binary_search_max_feasible(
+                    low=original_search_low,
+                    high=inherited_search_floor,
+                    tolerance=trial.search.tolerance,
+                    max_probes=trial.search.max_probes,
+                    evaluator=evaluator,
+                )
+                if fallback_search.best_feasible_payload is not None:
+                    search_for_best = fallback_search
+                    best = fallback_search.best_feasible_payload
+                    best_source = "lower_range_fallback"
+
+            def serialize_probe(probe: ThresholdProbe[ProbePayload]) -> dict[str, Any]:
+                return {
+                    "threshold": probe.threshold,
+                    "feasible": probe.feasible,
+                    "payload": {
+                        "request_count": probe.payload.request_count,
+                        "pass_rate": probe.payload.pass_rate,
+                        "request_rate": probe.payload.request_rate,
+                        "early_stopped": probe.payload.early_stopped,
+                        "early_stop_reason": probe.payload.early_stop_reason,
+                        "latency_summary": probe.payload.latency_summary,
+                    },
+                }
+
+            all_probes = [
+                *primary_search.probes,
+                *((fallback_search.probes if fallback_search is not None else [])),
+            ]
             result = {
                 "study_id": trial.study_id,
                 "trial_id": trial.trial_id,
                 "status": "completed",
                 "config_patch": to_jsonable(trial.config_patch),
-                "best_sampling_u": search.best_threshold if best is not None else None,
+                "best_source": best_source,
+                "best_sampling_u": search_for_best.best_threshold if best is not None else None,
                 "best_request_rate": best.request_rate if best is not None else None,
                 "best_pass_rate": best.pass_rate if best is not None else None,
                 "best_request_count": best.request_count if best is not None else None,
-                "probes": [
-                    {
-                        "threshold": probe.threshold,
-                        "feasible": probe.feasible,
-                        "payload": {
-                            "request_count": probe.payload.request_count,
-                            "pass_rate": probe.payload.pass_rate,
-                            "request_rate": probe.payload.request_rate,
-                            "early_stopped": probe.payload.early_stopped,
-                            "early_stop_reason": probe.payload.early_stop_reason,
-                            "latency_summary": probe.payload.latency_summary,
-                        },
-                    }
-                    for probe in search.probes
-                ],
+                "probes": [serialize_probe(probe) for probe in all_probes],
             }
-            if best is None and search.probes:
-                last_probe = search.probes[-1]
+            if fallback_search is not None:
+                result["primary_search"] = {
+                    "low": inherited_search_floor,
+                    "high": trial.search.high,
+                    "best_sampling_u": primary_search.best_threshold
+                    if primary_search.best_feasible_payload is not None
+                    else None,
+                    "best_request_rate": primary_search.best_feasible_payload.request_rate
+                    if primary_search.best_feasible_payload is not None
+                    else None,
+                    "probes": [serialize_probe(probe) for probe in primary_search.probes],
+                }
+                result["lower_range_fallback"] = {
+                    "triggered": True,
+                    "low": original_search_low,
+                    "high": inherited_search_floor,
+                    "best_sampling_u": fallback_search.best_threshold
+                    if fallback_search.best_feasible_payload is not None
+                    else None,
+                    "best_request_rate": fallback_search.best_feasible_payload.request_rate
+                    if fallback_search.best_feasible_payload is not None
+                    else None,
+                    "probes": [serialize_probe(probe) for probe in fallback_search.probes],
+                }
+            if best is None and all_probes:
+                last_probe = all_probes[-1]
                 result["all_infeasible_diagnostics"] = {
                     "threshold": last_probe.threshold,
                     "request_count": last_probe.payload.request_count,
@@ -441,6 +834,26 @@ def run_trial(trial_spec_path: Path) -> dict[str, Any]:
             StudyStore.write_json(Path(trial.result_path), result)
             return result
         except Exception as exc:  # noqa: BLE001
+            partial_best = _best_feasible_probe_record(probe_history)
+            if partial_best is not None:
+                result = {
+                    "study_id": trial.study_id,
+                    "trial_id": trial.trial_id,
+                    "status": "completed",
+                    "config_patch": to_jsonable(trial.config_patch),
+                    "best_source": "partial_probe_before_failure",
+                    "best_sampling_u": partial_best.get("threshold"),
+                    "best_request_rate": partial_best.get("request_rate"),
+                    "best_pass_rate": partial_best.get("pass_rate"),
+                    "best_request_count": partial_best.get("request_count"),
+                    "completed_with_probe_failure": True,
+                    "failure_stage": failure_stage,
+                    "failure_reason": str(exc),
+                    "failure_traceback": traceback.format_exc(),
+                    "probes": probe_history,
+                }
+                StudyStore.write_json(Path(trial.result_path), result)
+                return result
             result = {
                 "study_id": trial.study_id,
                 "trial_id": trial.trial_id,
@@ -452,9 +865,12 @@ def run_trial(trial_spec_path: Path) -> dict[str, Any]:
                 "best_request_count": None,
                 "failure_stage": failure_stage,
                 "failure_reason": str(exc),
+                "failure_traceback": traceback.format_exc(),
                 "probes": probe_history,
             }
             StudyStore.write_json(Path(trial.result_path), result)
             return result
         finally:
-            _terminate_process_tree(process, timeout_s=30.0)
+            _ignore_sigterm_if_main()
+            _terminate_process_tree(process, timeout_s=30.0, marker_env=trial_marker_env)
+            _restore_sigterm(previous_sigterm)

tests/test_prepare_trace_windows.py

@@ -0,0 +1,100 @@
+from __future__ import annotations
+
+import importlib.util
+import sys
+import unittest
+from pathlib import Path
+
+REPO_ROOT = Path(__file__).resolve().parents[1]
+_SPEC = importlib.util.spec_from_file_location(
+    "prepare_trace_windows",
+    REPO_ROOT / "scripts" / "prepare_trace_windows.py",
+)
+assert _SPEC and _SPEC.loader
+ptw = importlib.util.module_from_spec(_SPEC)
+# Register before exec so dataclasses can resolve the module's annotations.
+sys.modules[_SPEC.name] = ptw
+_SPEC.loader.exec_module(ptw)
+
+
+class SessionCoherentSamplingTests(unittest.TestCase):
+    def test_multi_hop_chain_resolves_to_root(self) -> None:
+        root_of: dict[object, object] = {}
+        # turn1 root, turn2 -> turn1, turn3 -> turn2 (multi-hop), streamed in order.
+        self.assertEqual(
+            ptw.resolve_session_root({"chat_id": 1, "parent_chat_id": -1, "turn": 1}, root_of),
+            1,
+        )
+        self.assertEqual(
+            ptw.resolve_session_root({"chat_id": 2, "parent_chat_id": 1, "turn": 2}, root_of),
+            1,
+        )
+        self.assertEqual(
+            ptw.resolve_session_root({"chat_id": 3, "parent_chat_id": 2, "turn": 3}, root_of),
+            1,
+        )
+
+    def test_unknown_parent_falls_back_to_parent_id(self) -> None:
+        root_of: dict[object, object] = {}
+        # parent never seen (fell outside the span): group siblings under the parent.
+        self.assertEqual(
+            ptw.resolve_session_root({"chat_id": 50, "parent_chat_id": 9, "turn": 2}, root_of),
+            9,
+        )
+        self.assertEqual(
+            ptw.resolve_session_root({"chat_id": 51, "parent_chat_id": 9, "turn": 2}, root_of),
+            9,
+        )
+
+    def test_all_turns_of_a_session_share_one_u(self) -> None:
+        root_of: dict[object, object] = {}
+        rows = [
+            {"chat_id": 1, "parent_chat_id": -1, "turn": 1},
+            {"chat_id": 2, "parent_chat_id": 1, "turn": 2},
+            {"chat_id": 3, "parent_chat_id": 2, "turn": 3},
+        ]
+        us = {
+            ptw.session_uniform(
+                seed=7,
+                window_id="w",
+                session_root=ptw.resolve_session_root(row, root_of),
+            )
+            for row in rows
+        }
+        self.assertEqual(len(us), 1)
+        only = next(iter(us))
+        self.assertGreaterEqual(only, 0.0)
+        self.assertLess(only, 1.0)
+
+    def test_thresholding_keeps_or_drops_whole_sessions(self) -> None:
+        # Two distinct sessions get distinct scores; a threshold either keeps a
+        # session's every turn or none of them.
+        seed, window_id = 20260325, "chat_w_x"
+        sessions = {
+            "A": [
+                {"chat_id": 10, "parent_chat_id": -1},
+                {"chat_id": 11, "parent_chat_id": 10},
+            ],
+            "B": [
+                {"chat_id": 20, "parent_chat_id": -1},
+                {"chat_id": 21, "parent_chat_id": 20},
+            ],
+        }
+        root_of: dict[object, object] = {}
+        scored: list[tuple[str, float]] = []
+        for name, rows in sessions.items():
+            for row in rows:
+                root = ptw.resolve_session_root(row, root_of)
+                u = ptw.session_uniform(seed=seed, window_id=window_id, session_root=root)
+                scored.append((name, u))
+        for name in sessions:
+            us = {u for n, u in scored if n == name}
+            self.assertEqual(len(us), 1, f"session {name} turns must share one u")
+        for threshold in (0.0, 0.25, 0.5, 0.75, 1.0):
+            for name in sessions:
+                kept = {u <= threshold for n, u in scored if n == name}
+                self.assertEqual(len(kept), 1, "a session must be kept/dropped as a whole")
+
+
+if __name__ == "__main__":
+    unittest.main()