gahow/aituner
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Generalize oracle transport retry rule

Browse Source
This commit is contained in:
Gahow Wang
2026-07-14 00:00:24 +08:00
parent d3bc63a972
commit 0f891d99c9
3 changed files with 30 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

26
docs/opprof/oracle-gap-protocol.md
View File

@@ -1,11 +1,35 @@
# Static-policy oracle-gap protocol
Status: **FROZEN WITH A-OG-1 AND A-OG-2 AMENDMENTS**.
Status: **FROZEN WITH A-OG-1, A-OG-2, AND A-OG-3 AMENDMENTS**.
Date frozen: 2026-07-13 (Asia/Singapore). Existing Phase-3 measurements were
inspected only to choose the workload pair and rate brackets. They are
exploratory calibration data, not primary observations in this protocol.
### A-OG-3 — freeze a per-logical-trial transport retry rule (after 62 scores)
The first attempt at `P06-C10-r1.9-rep0` reproduced the same isolated local
transport signature seen in A-OG-2: one clean request raised
`ClientOSError` 3.56 ms after admission, with HTTP status 0, no first token,
and no output. The other 343 requests produced exactly 512 tokens, the server
stayed healthy, drain took 4.00 s, and every other client invariant passed.
The controller again rejected the attempt before creating a `score.json`.
For the remainder of this experiment, an attempt may be quarantined and the
identical logical trial retried once on a fresh server only if all of the
following hold: the sole failed client invariant is `clean_failures_zero`;
exactly one clean request failed; its error is `ClientOSError`, HTTP status is
0, it produced neither a first token nor output, and it completed within 10
ms of admission; every successful request has exact output; and the server
has no crash or error. The attempt is never scored. A second invalid attempt
for the same logical key, more than one failed request, or any other error
signature is a stop condition and makes the experiment inconclusive.
This rule is independent of config, rate, and observed performance, and
supersedes the one-key wording in A-OG-2 without changing how that retry was
executed. Existing scores remain immutable. No client, grid, policy,
threshold, order, SLO, or oracle calculation changes.
### A-OG-2 — retry one transport-invalid attempt (after 49 scored trials)
The first attempt at `P06-C01-r2.2-rep0` produced one local

6
scripts/oracle_gap/run_frontier.py
View File

@@ -20,10 +20,10 @@ from analyze import CONFIGS, PHASES, atomic_json, score_trial, summarize_trials
SCHEMA = 1
AMENDMENT = "A-OG-2"
AMENDMENT = "A-OG-3"
AMENDMENT_REASON = (
"retry one quarantined P06-C01-r2.2-rep0 transport-invalid attempt "
"without changing its logical trial"
"apply the frozen per-logical-trial quarantine rule to an isolated "
"local transport-invalid attempt"
)
REMOTE_ROOT = Path("/home/admin/cpfs/wjh/oracle-gap-20260713")
RUN_ROOT = REMOTE_ROOT / "runs"

4
tests/test_oracle_gap.py
View File

@@ -175,7 +175,7 @@ def test_a_og_1_extends_only_mutable_resume_fields() -> None:
)
def test_a_og_2_requires_the_amended_grid_to_stay_fixed() -> None:
def test_a_og_3_requires_the_amended_grid_to_stay_fixed() -> None:
old = {
"analyzer_sha256": "analyzer",
"p5_client_sha256": "p5",
@@ -190,7 +190,7 @@ def test_a_og_2_requires_the_amended_grid_to_stay_fixed() -> None:
"up_extensions": {"P01": [38.0], "P06": [2.1, 2.2, 2.3, 2.4]},
}
assert AMENDMENT == "A-OG-2"
assert AMENDMENT == "A-OG-3"
assert resume_compatible(old, {**old, "controller_sha256": "new"})
assert not resume_compatible(
old,