# OpProf Phase 6 pre-registered cross-version churn protocol Status: **ACCEPTED FOR EXECUTION — ALL FIVE ORCHESTRATOR DECISIONS RESOLVED**. Date frozen: 2026-07-12 (Asia/Singapore). Phase 6 asks whether the vLLM 0.20.0 best configuration and ranking on the C1 `TP={1,2,4} x max-num-seqs={8,16,32,64}` surface survive an upgrade to patched vLLM 0.24.0. This is a paired historical-anchor experiment, not a fresh tuner search. It measures churn at the exact recorded C1 request sets and reports where the recorded local frontier moved. ## Approved dispositions (orchestrator; 2026-07-12) All five open decisions are approved exactly as proposed, without changing an anchor, threshold, estimator, validity gate, or budget: 1. The upgrade-path estimand is approved. Resolved default changes are part of observed churn; dash1->dash0 and co-location remain explicit limitations and the result is not described as a pure engine-version causal effect. 2. The adaptive 25-anchor subset is approved: every cell receives its old peak plus the direction-relevant adjacent anchor, and TP4/MNS16 receives both neighbors. 3. The 5% material-frontier threshold, `tau-b>=0.8` ranking-survival threshold, and zero-anchored floor-bucket argmax/trap rules are approved. 4. One primary observation per historical anchor plus the 0.35-H20-hour confirmation reserve is approved. 5. The A-P5-1-class warm-up long tier is approved as raw input `>4096` tokens for this 0-8192 workload. This approval authorizes the later execution only after pinned-input, detached ownership, cold-start, echo-before-launch, placement, privacy, cleanup, and projected-budget gates pass. The 3.0-H20-hour cap remains absolute. ## Amendment A-P6-1 — checkpoint-sidecar accounting and conditional cap The first Phase-6 W1 request replays were incorrectly rejected by requiring a clean in-stream footer and `final=true` sidecar after a non-graceful shutdown. That contradicts the already accepted A-P3-5/P5 accounting rule. Before any resume, all four W1 cell streams and their eight measured anchor intervals must be CPU-re-adjudicated against the latest atomic checkpoint sidecar. With no in-stream footer, the sidecar is authoritative when every complete JSONL line decodes, step indices are contiguous, on-disk data-record count equals `written_records`, final data-line step equals `last_step_index`, `encoded_records = written_records + dropped_records`, `dropped_records=0`, and the checkpoint is within the configured one-second flush interval (plus the accepted 100-ms scheduling tolerance) of the recorded shutdown boundary. At most one post-checkpoint flush interval may be absent. A balanced stream accepts all covered W1 replay intervals without a GPU rerun. All subsequent waves use the graceful P5 path: start `vllm serve` with positive `--shutdown-timeout`, signal each API parent with SIGINT, wait up to 150 seconds for EngineCore drain/finalization, and use process-group SIGTERM/SIGKILL only as fallback. The preferred result is one in-stream footer plus an agreeing `final=true` sidecar; the atomic checkpoint rule remains the crash fallback. Only if any W1 sidecar fails the CPU balance/coverage gate does A-P6-1c activate: the hard cap becomes **3.5 H20-hours**, the activation is logged before launch, and W1 is rerun under graceful shutdown. If all W1 sidecars balance, A-P6-1c is not activated, W1 is accepted in place, its already charged 0.299393 H20-hours is retained, and execution resumes at W2 under the original 3.0-hour cap. ## Amendment A-P6-2 — authoritative solo frontier tier and 6.0-hour cap The user raises the cumulative Phase-6 hard cap from **3.0 to 6.0 H20-hours**. This amendment responds to the W2/W3 confirmation evidence: when neighboring wave clients became idle, pass rate changed from `0.412 -> 1.000` and `0.028 -> 0.957` at TP2/MNS32 and from `0.036 -> 1.000` at TP4/MNS16, while Layer-1 waiting means collapsed from `1.22 -> 0.00`, `8.50 -> 0.35`, and `28.33 -> 0.06`. Because the v0.20 C1 baseline ran one cell at a time on dash1, co-located SLO feasibility is not a valid authoritative tier for the paired frontier. Co-location remains an explicitly reported, metric-dependent methodological ablation; its throughput-only observations are indicative. Every SLO-frontier-decisive Phase-6 anchor is therefore run **solo**: exactly one vLLM server and one replay client are active on dash0, no other Phase-6 server/client is resident, and only the cell's `TP` GPUs are allocated. Each solo cell gets a fresh server process, the frozen exact-selection client, an independent 16-request long-tier warm-up, A-P5-1 cold-start gates, Layer-1 telemetry, and graceful A-P6-1 shutdown. The solo result is authoritative for feasibility, frontier, floor bucket, argmax, tau-b, and trap decisions. A co-located-only result may not fill a missing solo frontier or support a paper claim. The mandatory solo set has no cell exclusions: | Cell set | Mandatory solo anchors | Inclusion reason | |---|---|---| | TP1/MNS8 | existing L+P | co-located peak failed | | TP1/MNS16,32,64 | existing P+U | co-located U passed and censored the frontier | | TP2/MNS8,16 | existing L+P | co-located peak failed; L distinguishes bracket vs left censoring | | TP2/MNS32 | existing L+P | old argmax; both anchors split primary/confirmation | | TP2/MNS64 | existing L+P | the indicative `>29.4% down` bound is unquotable until solo-confirmed | | TP4/MNS8 | existing L+P | co-located peak failed; trap-neighbor comparison | | TP4/MNS16 | existing L+P+U | named trap; peak split and L failed co-located | | TP4/MNS32,64 | P plus direction-relevant L/U | W4 completion; unmeasured under the 3.0-hour cap | After these 25 anchors, the controller may crawl only the pinned 92-anchor history: upward while the highest solo anchor passes or downward while the lowest solo anchor fails, stopping at the first opposite-feasibility anchor. There is no interpolation and no unrecorded request set. A pass rate in `[0.93,0.97]`, or a solo/co-located disagreement that changes argmax/trap or a material-frontier claim, triggers a same-solo-placement confirmation; a split gets a third trial for the frozen 2-of-3 rule if projected budget permits. Unresolved or history-edge frontiers remain explicitly censored. Priority is W4, TP2/MNS32, TP2/MNS64, TP4/MNS16, then the remaining failed or censored cells, so a budget stop preserves the most decision-relevant evidence. All twelve cells are nevertheless included. The planning envelope is 3.44 new H20-hours for serialized startup, warm-up, 25 mandatory anchors, bounded crawl, graceful drain, and a 0.20-hour safety reserve. Added to the already charged 2.291173 hours, the launch projection is **5.731173/6.0 H20-hours**. Before each one-cell wave, the controller echoes cell, anchors, GPU placement, input paths, charged spend, remaining projection, and cap. Projected or actual spend reaching 6.0 stops further GPU work. Final analysis preserves the complete W1-W3 co-located attempt history and reports a solo-versus-co-located delta table for every exact anchor pair. ARGMAX, RANKING, and TRAP use solo values only. RANKING remains inconclusive and tau-b non-evaluable unless all 12 solo frontiers are bounded under the pinned history. ## Headline claim and falsifiable outcomes The paper-facing question is: > Does the C1 vLLM 0.20.0 per-GPU SLO-feasible ranking, especially global best > TP2/MNS32 and local trap TP4/MNS16, remain valid on vLLM 0.24.0 when the exact > historical request-selection and SLO semantics are replayed? Phase 6 can produce four distinct outcomes: 1. **SURVIVES:** TP2/MNS32 remains in the top floor bucket, the full-surface Kendall tau-b is at least 0.8, and the TP4/MNS16 trap persists. 2. **ARGMAX MOVED:** TP2/MNS32 is not in the top v0.24 measured-frontier bucket. 3. **RANKING MOVED WITHOUT ARGMAX MOVEMENT:** the old best survives but tau-b is below 0.8 or the trap changes status. 4. **INCONCLUSIVE/PARTIAL:** a decision-critical frontier is unbracketed, an anchor is not reproducible, repeated boundary verdicts disagree, the 12-cell surface is incomplete, or the 3.0-H20-hour cap stops required work. The experiment quantifies observed upgrade churn. Because engine version, host, resolved defaults, and co-location differ together, it does not identify a pure causal effect of one vLLM commit. ## Frozen vLLM 0.20.0 surface The historical objective is the maximum measured SLO-feasible offered rate divided by TP: ```text f20(c) = max feasible selected_request_count / 60 seconds / TP ``` | TP | MNS8 | MNS16 | MNS32 | MNS64 | |---:|---:|---:|---:|---:| | 1 | 2.1000 | 2.3500 | 2.2833 | 2.2833 | | 2 | 2.2750 | 2.2750 | **3.2833** | 3.2583 | | 4 | 1.2833 | **2.4417** | 2.4417 | 2.4417 | Values are request/s/GPU. The global best is TP2/MNS32. TP4/MNS16 is the registered local trap: it is the first point on a TP4 plateau, has no improving adjacent MNS move, but is below the global best. The pinned ground-truth asset is `/home/gahow/phd/replayserve/docs/assets/simfid_s2r/ground_truth.json`, SHA-256 `23ff3e6f6b88df8632bf37d37c0ff87bfef9d1676db81592948c08e898f7a670`. It contains 92 probe observations: eight anchors for each TP1/TP2 cell and seven for each TP4 cell. ## Comparability contract ### Exact workload and selection semantics The private materialized workload is `trace_windows/traces/chat_w20260311_1000.jsonl`, 32,606 rows, SHA-256 `f539f38eb0ee0f750e3c23ff47df6eed3faf723a25f1444d55665a85871750b9`. Its window record is `trace_windows/windows.json`, SHA-256 `23c432e7439508b07991cabfe1db9977ebb55f8ddc39ca214a627fc5f5ae4725`: window `[0,600]` seconds becomes `[0,60]` under replay scale 0.1. The later helper must call or byte-for-byte reproduce the following pinned AITuner implementations, rather than use the Phase-3/5 client: | Source | SHA-256 | Frozen role | |---|---|---| | `src/aituner/trace.py` | `6bb19f0e3eb42b1aecc6969a5f20d65eb505a91564dd117dd149155a6114eb35` | load, stable time sort, cap/downsample, threshold selection | | `src/aituner/worker.py` | `18337fe273a48d1cbb2bc364f774d100f8ffa2be5f9cd6a7c58c8ed4717c5e48` | rate-following submit loop, early stop, denominator accounting | | `src/aituner/slo.py` | `1d2e5b1ed9d0d6d6c07f8b5b6fa56d36ed9bb62710be54c125e9b2890f4f96d5` | stepped SLO evaluation | | `src/aituner/http_client.py` | `213b80d5b37cc74333517f46ab4e8cdba1a274bdbfabdfc14207b314f61799ac` | urllib/SSE TTFT, TPOT, usage accounting | The exact order is normative: 1. Read all JSONL rows in file order. Retain raw `input_length` in `[0,8192]`. 2. Build the same chat body and stable-sort by scaled arrival time only. Stable ties retain source order; sorting by `(timestamp,sampling_u,...)` is forbidden. 3. For TP1/TP2, even-downsample the full filtered, time-sorted list to 512 entries **before** threshold selection, using index `floor(i*N/512)` for `i=0..511`. TP4 is uncapped. 4. Select exactly the requests with `sampling_u <= anchor`. Preserve their selected service order. Do not resample, wrap, shuffle, or interpolate. 5. Submit at each row's materialized `timestamp*0.1`; do not uniformize, rebase individual gaps, or replace the recorded burst process. 6. Send the original messages through `/v1/chat/completions` with `stream=true`, `stream_options.include_usage=true`, and the row's temperature when present. Set both `min_tokens=128` and `max_tokens=128`. 7. Use maximum concurrency 64. Client request timeout remains 900 seconds. 8. Verify `usage.completion_tokens==128`; missing usage, a different token count, HTTP failure, timeout, and every request not submitted because the SLO verdict became unrecoverable are failures in the original selected-request denominator. 9. Preserve original early-stop logic: after more than `N-ceil(0.95*N)` failed evaluations, synthesize failures for the remaining selected requests. Disable adaptive Stop-A; it was absent in C1. An offline, prompt-free preflight must regenerate all 92 historical request counts exactly. The local reconstruction already yields 17,710 filtered rows, 512 capped rows, and **92/92 count matches**. Any future mismatch in count, request-ID hash, arrival hash, raw-length hash, or requested-token sum stops the GPU launch. ### Exact SLO and score Request feasibility uses the original raw trace input length, not tokenizer usage after chat templating: ```text raw input <= 4096: TTFT <= 2000 ms raw input <= 32768: TTFT <= 4000 ms otherwise: TTFT <= 6000 ms all requests: TPOT <= 50 ms anchor feasible: SLO-passing selected requests / all selected requests >= 0.95 ``` Although the C1 filter limits inputs to 8192, the complete three-step rule is retained. TTFT is client wall time from request start to first nonempty content. TPOT is `(last_content_time-first_content_time)/(usage_completion_tokens-1)`. The paired surface score is always selected count divided by 60 seconds and TP; completed throughput is a diagnostic and never replaces the historical score. ### Serving configuration The v0.24 command repeats the explicit C1 launch surface: ```bash MODEL=/home/admin/cpfs/wjh/models/Qwen/Qwen3-30B-A3B VENV=/tmp/wjh-opprof-phase2-dash0-20260711/.venv RUN_ROOT=/home/admin/cpfs/wjh/opprof-phase6-dash0-20260712/runs/phase6 taskset -c "$CPUSET" env CUDA_VISIBLE_DEVICES="$GPU_IDS" \ VLLM_OPPROF_DIR="$RUN_DIR/opprof" \ "$VENV/bin/vllm" serve "$MODEL" \ --host 127.0.0.1 --port "$PORT" \ --served-model-name qwen3-30b-a3b-community \ --max-num-batched-tokens 8192 \ --max-num-seqs "$MNS" \ --tensor-parallel-size "$TP" \ --shutdown-timeout 120 ``` `--shutdown-timeout` is operational and outside the measured interval. As in C1, do **not** explicitly set block size, GPU-memory utilization, prefix caching, chunked prefill, CUDA-graph capture sizes, DP, or EP. Their resolved v0.24 values must be parsed from startup logs and reported. Allowing version defaults to change is intentional: this experiment asks whether an operator's old explicit configuration remains good after upgrading, not whether internal defaults can be manually made identical. Preflight records `vllm.__version__`, source commit, patch checksums, Python, PyTorch, CUDA, NCCL, Triton, driver, model/tokenizer file hashes, MoE backend, resolved scheduler/cache/graph config, compile-cache key, and exact command. The expected source is the existing Phase-3/5 patched vLLM 0.24 environment; any missing OpProf patch or unexpected model path is a stop condition. ### Client equivalence, not a new client covariate The later `scripts/opprof_phase6_client.py` is permitted only as a thin anchor driver around the four pinned AITuner modules. Its interface is normative: ```bash P6C='python scripts/opprof_phase6_client.py' $P6C preflight \ --trace trace_windows/traces/chat_w20260311_1000.jsonl \ --windows trace_windows/windows.json --window-id chat_w20260311_1000 \ --ground-truth /home/gahow/phd/replayserve/docs/assets/simfid_s2r/ground_truth.json $P6C run-anchor --cell "$CELL" --anchor "$U" \ --base-url "http://127.0.0.1:$PORT" \ --model qwen3-30b-a3b-community --completion-tokens 128 \ --replay-time-scale 0.1 --max-concurrency 64 \ --max-requests "$CAP_OR_NONE" --target-pass-rate 0.95 \ --ttft-step-ms '4096:2000,32768:4000,inf:6000' --tpot-ms 50 \ --request-timeout-seconds 900 --probe-deadline-ceiling-seconds 900 \ --result-dir "$ANCHOR_DIR" ``` Before GPU execution, a no-server golden test must prove identical selected IDs, order, arrivals, raw input lengths, serialized bodies excluding model/URL, and SLO decisions for synthetic outcomes. The wrapper may add interval markers and sanity JSON, but may not change HTTP transport, scheduling, completion checks, early stop, or latency formulas. ### Covariate ledger and estimand limit | Covariate | C1 v0.20 | Phase 6 v0.24 | Treatment/handling | |---|---|---|---| | Engine | community vLLM 0.20.0 | patched vLLM 0.24.0 | Primary intended churn dimension; exact builds recorded | | Host | dash1 | dash0 | Both 8x H20 96 GB class; CPU/NUMA/clocks/driver/kernel recorded, not assumed identical | | GPU visibility | TP1 C1 env exposed GPUs `0,1`, TP1 consumed one | expose exactly TP GPUs | Declared operational covariate; H20-hour charge follows TP | | Trial placement | recovered C1 harness trials | up to four co-located servers | CPU/NUMA pinned; reject outside GPU processes and report co-location limitation | | HTTP client | pinned AITuner urllib/SSE | same code paths through thin wrapper | Eliminated as a semantic covariate by golden tests | | Workload | materialized C1 chat window | byte-identical file/hash | Fixed | | Selection/cap | cap-before-threshold; TP4 uncapped | exact same | Fixed and 92-count preflighted | | SLO length | raw input length | raw input length | Fixed; server usage length is diagnostic only | | Explicit flags | TP, MNS, MBT8192 | same | Fixed | | Unspecified defaults | v0.20 defaults | v0.24 defaults | Intentionally allowed to churn; resolved values reported | | Telemetry | no Layer 1 | zero-overhead OpProf Layer 1 | Added measurement; overhead evidence -0.04196%, CI [-0.17443%, 0.04550%] | Therefore “version churn” means the observed paired upgrade path under these documented platform covariates. It must not be rewritten as a same-host causal microbenchmark. ## Budget-constrained anchor subset ### Adaptive adjacent-anchor design The primary subset improves on measuring next-infeasible anchors only for a chosen top four: 1. Measure the recorded v0.20 peak anchor for all 12 cells. 2. For every non-trap cell, select exactly one adjacent recorded anchor by a result-independent rule fixed now: - if the v0.20 peak remains feasible on v0.24, measure the nearest higher recorded anchor, which was infeasible on v0.20; - if the peak becomes infeasible, measure the nearest lower recorded anchor, which was feasible on v0.20. 3. For TP4/MNS16, measure lower, peak, and higher anchors unconditionally. This directly brackets the registered trap. Thus exactly 25 primary anchors are measured: TP1 eight, TP2 eight, and TP4 nine. The design covers every cell and spends the second anchor in the direction where its frontier could have moved. It also includes the suggested top-four test. Under the floor-bucket rule, the nominal top-four cutoff expands to five cells because TP4/MNS16/32/64 share one bucket; all five receive an upward test whenever their old peak still passes. If an adjacent anchor does not bracket the transition—higher also passes or lower also fails—the cell is censored. Remaining budget may crawl one recorded anchor farther outward, in this priority order: old global best TP2/MNS32, TP2/MNS64, the TP4 plateau MNS16/32/64, then other cells by old score. Crawling uses only that cell's recorded history and stops at the first transition. No new `sampling_u` value may be invented. Failure to close a decision-critical bracket before the budget stop makes the applicable argmax/ranking verdict inconclusive. ### Boundary confirmations The original surface has one observation per anchor. Phase 6 keeps that paired unit but reserves at most 0.35 H20-hours for decision-triggered confirmation. Repeat an anchor if its v0.24 pass rate lies in `[0.93,0.97]`, or if its single verdict alone changes argmax or trap status. Priority is argmax, trap, material frontier calls, then other cells. Agreement across two runs freezes the verdict; disagreement requests a third run if budget permits and uses 2-of-3. Without a third run, that anchor and every dependent headline verdict are inconclusive. Request outcomes are arrival-correlated, so a binomial CI is descriptive only and cannot replace these repeat rules. ## Execution plan ### Preflight and detached ownership The Phase-6 controller follows A-P3-2 detached ownership: one controller PID, per-stage atomic state, process-group/environment markers, resumable completed anchors, exact command logs, and cleanup restricted to its own markers. Before launch it must verify dash0 has eight H20s, no outside compute processes on assigned GPUs, at least 100 GB CPFS free, pinned source/workload/helper hashes, all 92 selection counts, legal TP placement, and a projected total below 3.0 H20-hours. The autonomous launch log must echo one resolved line before starting: ```text LAUNCH_ECHO host=dash0 engine=vllm-0.24.0-patched model=Qwen3-30B-A3B trace_sha=f539f38e... cells=12 primary_anchors=25 waves=4 gpus=0-7 warmup=16 clean_horizon=60s drain=derived<=120s est_wall=25-35min est_gpu=2.65_H20h confirm_reserve=0.35_H20h hard_cap=3.0_H20h ``` ### Rate-following cold-start and drain gates The throughput-drift stationarity gate is forbidden for these rate-following anchors. A-P5-1-class cold-start gates are applied instead: 1. Every torch.compile/CUDA-graph first-occurrence event must precede the first measured anchor, verified from server logs and Layer 1. 2. A deterministic unmeasured warm set must complete at least 16 exact-output requests, including at least one request with raw input `>4096` tokens. The peak-selected sets contain 43-184 such requests, so this gate is feasible. 3. No first-occurrence compile/capture event may occur in any measured anchor; all graph-hit `(runtime_mode,bucket_tokens)` descriptors must be covered by startup capture logs. A violation invalidates that anchor only. 4. Before the next anchor, all selected requests must be accounted for and three consecutive controller samples must show running/waiting/deferred queues at zero. Prefix cache is not explicitly enabled; nevertheless a full drain is required. The probe deadline reproduces `_probe_drain_deadline`: last selected arrival + raw-length TTFT budget + `128*50 ms` + 30 seconds, capped by the historical 900-second ceiling. With raw inputs <=8192 this is about 100.4 seconds from probe start; the controller has a 120-second class watchdog for cleanup. A deadline miss is an SLO-infeasible anchor, not permission to omit denominator requests. The server then shuts down through the official 120-second path. ### Four-wave packing | Wave | Cells | Placement | Per-server measured anchors | Planned wall | |---|---|---|---:|---:| | W1 | TP1/MNS8,16,32,64 | four servers on GPUs 0,1,2,3; GPUs 4-7 idle | 2 each | 4-6 min | | W2 | TP2/MNS8,16,32,64 | four servers on GPU pairs 0-1,2-3,4-5,6-7 | 2 each | 4-6 min | | W3 | TP4/MNS8 and TP4/MNS16 trap | GPU quads 0-3 and 4-7 | 2 and 3 | 5-7 min | | W4 | TP4/MNS32 and TP4/MNS64 | GPU quads 0-3 and 4-7 | 2 each | 4-6 min | Each server starts once, passes warm-up once, runs its peak first, drains, then runs the predeclared adjacent anchor. Trap lower/upper order after peak is counterbalanced by execution seed 20260717. Anchor confirmation occurs before that server is released when possible. The controller records GPU clocks, power, utilization, memory, CPU load, NUMA placement, and outside processes through every wave. Any contamination invalidates the whole co-located wave. Actual H20-hours are charged from server launch through verified cleanup for every allocated GPU, including idle time while a paired TP4 cell finishes. Before each wave or confirmation, the controller recomputes `spent + current + conservative_remaining`; it stops before launch if the projection can reach 3.0. Optional crawling and confirmations are sacrificed before any hard-cap violation. ## Measurement and mechanism notes For every anchor, report selected/admitted/completed/exact-output/SLO-pass counts; early-stop reason; offered and completed req/s; TTFT/TPOT mean and p50/p90/p95/p99; failures by raw-length TTFT bucket; elapsed/drain time; and peak GPU memory/utilization. The surface objective remains offered req/s/GPU at the highest measured feasible recorded anchor. Layer 1 is active over the complete anchor service interval, from the first submission through the last selected-request accounting event. Aggregate: - prefill, decode, and mixed-step counts and token shares; - decode-batch distribution and 5-second CV; - FULL/PIECEWISE/NONE graph-mode shares, bucket padding, and uncovered support; - running/waiting/deferred queues and queue CV; - preemption count/rate; - KV blocks used/total, mean/max usage, and allocation pressure; and - prefix queries/hits as a resolved-default diagnostic. Mechanism notes compare v0.24 cells/anchors and may relate a frontier change to preemptions, KV pressure, batching, or graph modes. C1 lacks Layer-1 telemetry, so these notes cannot claim that a measured composition field itself changed from v0.20 or caused the version drift. Historical v0.20 TTFT/TPOT failure reasons may be paired directly; device-mechanism attribution remains descriptive. ## Statistical and ranking analysis ### Floor buckets For each unrounded score vector `s`, use the SimFid inventory rule: ```text tol(s) = max(1e-9, 1e-6 * max_c(abs(s(c)))) bucket_s(c) = floor(s(c) / tol(s)) ``` Two scores tie only when their integer buckets match. Bucketing occurs before display rounding, pair signs, tau-b, top selection, or trap evaluation. Report `tol`, bucket integers, and occupied intervals `[j*tol,(j+1)*tol)`. For v0.20, `tol20=3.283333333333333e-6`; the TP4 MNS16/32/64 tie makes nominal top-4 an effective top-5 set. ### Per-cell frontier For each cell, define the v0.24 measured recorded-anchor frontier: ```text f24(c) = max_{measured u with accepted feasible verdict} N_c(u) / 60 / TP drift(c) = (f24(c) - f20(c)) / f20(c) ``` The predeclared material threshold is **X=5% relative**, large enough to avoid calling one-request anchor quantization a paper-level churn effect. Report two orthogonal labels: - **BOUNDARY DOWN/UP:** old peak flips feasible->infeasible, or the old adjacent infeasible anchor flips to feasible, respectively; - **MATERIAL FRONTIER MOVED:** `abs(drift)>5%`; otherwise report the boundary flip as sub-material. If peak passes and higher fails, and `abs(drift)<=5%`, the local recorded frontier is **STABLE**. If peak and lower both fail or peak and every affordable higher anchor pass, report a directional bound and **UNBRACKETED**, never an invented peak. ### Argmax, ranking, and trap - **ARGMAX MOVED** when TP2/MNS32 is not in the top `bucket_f24` among all 12 decision-bounded cells. **ARGMAX SURVIVED** requires it to share the top bucket and no censored competitor capable of overtaking it. Otherwise it is inconclusive. - Compute Kendall tau-b over all 12 floor-bucketed `f20/f24` scores and report concordant, discordant, v0.20-tied, v0.24-tied, and comparable-pair counts. **RANKING SURVIVES** requires an evaluable 12-cell surface, argmax survival, `tau-b>=0.8`, and no >5% top-bucket pair reversal. `tau-b<0.8` is **RANKING MOVED**. A partial surface receives no suite-level tau claim. - **TRAP PERSISTS** when TP4/MNS16 is in a bucket at least as high as both adjacent TP4 cells MNS8 and MNS32, yet below the global-best bucket. It **ESCAPES** if an adjacent TP4 move is strictly better, and **CEASES TO BE A TRAP** if it joins the global-best bucket. Missing/unbracketed inputs make trap status inconclusive. No arithmetic average of normalized scores is reported. Absolute rates, relative drift, the paired table, and every excluded/censored anchor accompany all ranking summaries. ## Later deliverables and artifact contract Execution, after approval, must produce: - `docs/opprof/phase6-results.md`; - `runs/opprof-phase6/phase6/metrics.json` containing the complete paired surface, anchor/run values, selection hashes, feasibility, floor buckets, frontier/argmax/ranking/trap verdicts, confirmation history, Layer-1 notes, covariates, and GPU accounting; - exact commands, controller state, environment/machine fingerprints, server logs, client outcomes, Layer-1 JSONL/footer/sidecar, monitor samples, and per-anchor sanity JSON; and - an operational-findings section covering startup/capture gates, drains, early stops, co-location, default changes, retries, contamination, and budget. The results begin with data sanity and stop inferential claims on any red flag. Every numeric family ends with `n`, finite/missing, min, max, distinct count, and applicable invariants. Prompt text, messages, generated content, source substrings, and private request bodies stay in mode-0700 storage. Public files contain only hashes, IDs, lengths, timestamps, outcomes, counters, and summaries. ## Anchor-subset table Notation is `u / selected N / recorded req/s/GPU`. `P` is always measured. For non-trap cells, measure `U` if P passes on v0.24, otherwise `L`. TP4/MNS16 measures all `L+P+U`. Planning cost per anchor includes the 60-second replay and 20 seconds expected drain/accounting; shared startup/warm-up is budgeted below. | Cell | L: nearest recorded feasible below P | P: recorded peak | U: nearest recorded infeasible above P | Primary selection | H20-hour cost each | |---|---|---|---|---|---:| | TP1/MNS8 | 0.21875 / 121 / 2.0167 | 0.2265625 / 126 / 2.1000 | 0.23046875 / 130 / 2.1667 | P + (U if pass else L) | 0.0222 | | TP1/MNS16 | 0.2421875 / 137 / 2.2833 | 0.24609375 / 141 / 2.3500 | 0.25 / 143 / 2.3833 | P + (U if pass else L) | 0.0222 | | TP1/MNS32 | 0.234375 / 132 / 2.2000 | 0.2421875 / 137 / 2.2833 | 0.24609375 / 141 / 2.3500 | P + (U if pass else L) | 0.0222 | | TP1/MNS64 | 0.234375 / 132 / 2.2000 | 0.2421875 / 137 / 2.2833 | 0.24609375 / 141 / 2.3500 | P + (U if pass else L) | 0.0222 | | TP2/MNS8 | 0.4921875 / 269 / 2.2417 | 0.49609375 / 273 / 2.2750 | 0.5 / 276 / 2.3000 | P + (U if pass else L) | 0.0444 | | TP2/MNS16 | 0.4921875 / 269 / 2.2417 | 0.49609375 / 273 / 2.2750 | 0.5 / 276 / 2.3000 | P + (U if pass else L) | 0.0444 | | TP2/MNS32 | 0.75 / 391 / 3.2583 | 0.75390625 / 394 / **3.2833** | 0.7578125 / 396 / 3.3000 | P + (U if pass else L) | 0.0444 | | TP2/MNS64 | 0.5 / 276 / 2.3000 | 0.75 / 391 / 3.2583 | 0.75390625 / 394 / 3.2833 | P + (U if pass else L) | 0.0444 | | TP4/MNS8 | 0.016055910008 / 301 / 1.2542 | 0.016591107009 / 308 / 1.2833 | 0.017126304009 / 317 / 1.3208 | P + (U if pass else L) | 0.0889 | | TP4/MNS16 trap | 0.033182214016 / 575 / 2.3958 | 0.033717411016 / 586 / **2.4417** | 0.034252608017 / 600 / 2.5000 | **L + P + U** | 0.0889 | | TP4/MNS32 | 0.033182214016 / 575 / 2.3958 | 0.033717411016 / 586 / 2.4417 | 0.034252608017 / 600 / 2.5000 | P + (U if pass else L) | 0.0889 | | TP4/MNS64 | 0.033182214016 / 575 / 2.3958 | 0.033717411016 / 586 / 2.4417 | 0.034252608017 / 600 / 2.5000 | P + (U if pass else L) | 0.0889 | ## Total GPU estimate versus 3.0 cap The 25 primary anchors contain 8 TP1, 8 TP2, and 9 TP4 replays. At the 80-second per-anchor planning charge they consume `0.1778 + 0.3556 + 0.8000 = 1.3334` H20-hours. Four shared startup/warm-up/cleanup envelopes plus runtime variance are budgeted at 1.3166 hours, for a **2.65-H20-hour primary plan**. The remaining **0.35 H20-hours** is reserved for boundary confirmations or decision-critical outward crawling. The absolute launch cap is **3.0 H20-hours**; projected or actual spend reaching the cap stops all further work. Expected wall time is 25-35 minutes over four waves. Expected public disk is 0.3-0.6 GB and must remain below 3 GB. ## Decision rules 1. Anchor feasibility is exact-output request pass rate `>=0.95` using raw-length stepped TTFT and 50-ms TPOT; absent/early-stopped requests remain failures. 2. Boundary movement is an old peak feasible->infeasible flip or an old adjacent infeasible->feasible flip. Material frontier churn is `abs(drift)>5%`. 3. Apply zero-anchored floor buckets with set-wide `tol=max(1e-9,1e-6*max|s|)` before every tie/rank decision. 4. ARGMAX MOVED iff TP2/MNS32 is outside the v0.24 top bucket on a bounded 12-cell subset; censored contenders make survival inconclusive. 5. RANKING SURVIVES only with all 12 cells, argmax survival, tau-b >=0.8, and no >5% top-bucket pair reversal. Tau-b <0.8 is RANKING MOVED. 6. TP4/MNS16 trap persists only when it is no worse than adjacent MNS8/MNS32 and remains below the global best; an improving neighbor means escape. 7. Boundary `[0.93,0.97]` or verdict-changing anchors require confirmation when budget permits. Disagreement without a 2-of-3 resolution is inconclusive. ## Open decisions for orchestrator 1. Approve the **upgrade-path estimand**: resolved default changes are part of churn, while dash1->dash0 and co-location remain explicit limitations rather than pretending this is a pure engine-version causal effect. 2. Approve the **adaptive 25-anchor subset**, which gives every cell a direction-relevant neighbor and the trap both neighbors, instead of spending next-infeasible anchors only on a tie-ambiguous top four. 3. Approve **5% material frontier drift**, `tau-b>=0.8`, and the stated argmax/trap floor-bucket rules. 4. Approve one primary observation per historical anchor with the 0.35-hour reserve for `[0.93,0.97]`/verdict-changing confirmations, rather than reducing surface coverage to replicate every anchor. 5. Approve the A-P5-1-class warm-up adaptation from `input>=8192` to **raw input >4096**, the longest SLO-relevant tier available reliably in this 0-8192 C1 workload. ## Sanity block | Numeric family | n | Min | Max | Distinct | Checked invariant/result | |---|---:|---:|---:|---:|---| | C1 cells | 12 | TP=1,MNS=8 | TP=4,MNS=64 | 12 | Exact 3x4 Cartesian surface | | Historical probe observations | 92 | 7/cell | 8/cell | 2 per-cell counts | `8*8 + 4*7 = 92` | | Materialized trace rows | 1 file | 32,606 | 32,606 | 1 expected | SHA pinned; prompt content not emitted | | Raw-length-filtered rows | 1 reconstruction | 17,710 | 17,710 | 1 expected | Raw length 22-8192; non-negative | | Capped TP1/TP2 source rows | 1 set | 512 | 512 | 1 expected | Stable time sort, then even downsample | | Reconstructed historical counts | 92 | 66 | 600 | 34 | 92/92 match ground truth | | C1 peak scores (req/s/GPU) | 12 | 1.283333 | 3.283333 | 8 | Finite, positive, not all identical | | v0.20 floor tolerance | 1 vector | 3.283333e-6 | 3.283333e-6 | 1 | Applied before display rounding | | Nominal top-4 effective size | 1 cutoff | 5 | 5 | 1 | TP4 three-way cutoff tie retained | | Primary peak anchors | 12 | 1/cell | 1/cell | 1 expected | No cell omitted | | Realized adjacent/trap anchors | 13 | 1/non-trap | 2/trap | 2 | Predeclared direction rule; total 25 | | Primary anchors by TP | 25 | 8 at TP1/TP2 | 9 at TP4 | 2 counts | `8+8+9=25` | | Selected requests in anchor candidates | 36 candidates | 121 | 600 | >1 expected | TP1/2 cap-before-threshold; TP4 uncapped | | Per-anchor planning cost (H20-hour) | 3 TP classes | 0.0222 | 0.0889 | 3 | Proportional to TP for 80 seconds | | Primary planned H20-hours | 1 plan | 2.65 | 2.65 | 1 | Non-negative; below 3.0 | | Confirmation/crawl reserve | 1 plan | 0.35 | 0.35 | 1 | Primary plus reserve equals hard cap | | GPU work in this protocol turn | 1 turn | 0 | 0 | 1 expected | Protocol-only requirement satisfied | Checked invariants: `12=3*4`; `92=8*8+4*7`; all 92 local selection counts match; cap precedes threshold; arrivals span the fixed 60-second replay; raw input length, not retokenized usage, selects TTFT thresholds; completion is exactly 128; SLO denominator includes all selected requests; surface scores use 60 seconds and divide by TP; floor buckets precede ranking; the top-four cutoff expands to five; `25=12+11+2`; planned `2.65+0.35=3.0`; ratios and counters have their declared domains; and no GPU run, helper, manifest, remote artifact, or results file was created in this protocol-only turn.