Generalize oracle transport retry rule
This commit is contained in:
@@ -1,11 +1,35 @@
|
|||||||
# Static-policy oracle-gap protocol
|
# Static-policy oracle-gap protocol
|
||||||
|
|
||||||
Status: **FROZEN WITH A-OG-1 AND A-OG-2 AMENDMENTS**.
|
Status: **FROZEN WITH A-OG-1, A-OG-2, AND A-OG-3 AMENDMENTS**.
|
||||||
|
|
||||||
Date frozen: 2026-07-13 (Asia/Singapore). Existing Phase-3 measurements were
|
Date frozen: 2026-07-13 (Asia/Singapore). Existing Phase-3 measurements were
|
||||||
inspected only to choose the workload pair and rate brackets. They are
|
inspected only to choose the workload pair and rate brackets. They are
|
||||||
exploratory calibration data, not primary observations in this protocol.
|
exploratory calibration data, not primary observations in this protocol.
|
||||||
|
|
||||||
|
### A-OG-3 — freeze a per-logical-trial transport retry rule (after 62 scores)
|
||||||
|
|
||||||
|
The first attempt at `P06-C10-r1.9-rep0` reproduced the same isolated local
|
||||||
|
transport signature seen in A-OG-2: one clean request raised
|
||||||
|
`ClientOSError` 3.56 ms after admission, with HTTP status 0, no first token,
|
||||||
|
and no output. The other 343 requests produced exactly 512 tokens, the server
|
||||||
|
stayed healthy, drain took 4.00 s, and every other client invariant passed.
|
||||||
|
The controller again rejected the attempt before creating a `score.json`.
|
||||||
|
|
||||||
|
For the remainder of this experiment, an attempt may be quarantined and the
|
||||||
|
identical logical trial retried once on a fresh server only if all of the
|
||||||
|
following hold: the sole failed client invariant is `clean_failures_zero`;
|
||||||
|
exactly one clean request failed; its error is `ClientOSError`, HTTP status is
|
||||||
|
0, it produced neither a first token nor output, and it completed within 10
|
||||||
|
ms of admission; every successful request has exact output; and the server
|
||||||
|
has no crash or error. The attempt is never scored. A second invalid attempt
|
||||||
|
for the same logical key, more than one failed request, or any other error
|
||||||
|
signature is a stop condition and makes the experiment inconclusive.
|
||||||
|
|
||||||
|
This rule is independent of config, rate, and observed performance, and
|
||||||
|
supersedes the one-key wording in A-OG-2 without changing how that retry was
|
||||||
|
executed. Existing scores remain immutable. No client, grid, policy,
|
||||||
|
threshold, order, SLO, or oracle calculation changes.
|
||||||
|
|
||||||
### A-OG-2 — retry one transport-invalid attempt (after 49 scored trials)
|
### A-OG-2 — retry one transport-invalid attempt (after 49 scored trials)
|
||||||
|
|
||||||
The first attempt at `P06-C01-r2.2-rep0` produced one local
|
The first attempt at `P06-C01-r2.2-rep0` produced one local
|
||||||
|
|||||||
@@ -20,10 +20,10 @@ from analyze import CONFIGS, PHASES, atomic_json, score_trial, summarize_trials
|
|||||||
|
|
||||||
|
|
||||||
SCHEMA = 1
|
SCHEMA = 1
|
||||||
AMENDMENT = "A-OG-2"
|
AMENDMENT = "A-OG-3"
|
||||||
AMENDMENT_REASON = (
|
AMENDMENT_REASON = (
|
||||||
"retry one quarantined P06-C01-r2.2-rep0 transport-invalid attempt "
|
"apply the frozen per-logical-trial quarantine rule to an isolated "
|
||||||
"without changing its logical trial"
|
"local transport-invalid attempt"
|
||||||
)
|
)
|
||||||
REMOTE_ROOT = Path("/home/admin/cpfs/wjh/oracle-gap-20260713")
|
REMOTE_ROOT = Path("/home/admin/cpfs/wjh/oracle-gap-20260713")
|
||||||
RUN_ROOT = REMOTE_ROOT / "runs"
|
RUN_ROOT = REMOTE_ROOT / "runs"
|
||||||
|
|||||||
@@ -175,7 +175,7 @@ def test_a_og_1_extends_only_mutable_resume_fields() -> None:
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
def test_a_og_2_requires_the_amended_grid_to_stay_fixed() -> None:
|
def test_a_og_3_requires_the_amended_grid_to_stay_fixed() -> None:
|
||||||
old = {
|
old = {
|
||||||
"analyzer_sha256": "analyzer",
|
"analyzer_sha256": "analyzer",
|
||||||
"p5_client_sha256": "p5",
|
"p5_client_sha256": "p5",
|
||||||
@@ -190,7 +190,7 @@ def test_a_og_2_requires_the_amended_grid_to_stay_fixed() -> None:
|
|||||||
"up_extensions": {"P01": [38.0], "P06": [2.1, 2.2, 2.3, 2.4]},
|
"up_extensions": {"P01": [38.0], "P06": [2.1, 2.2, 2.3, 2.4]},
|
||||||
}
|
}
|
||||||
|
|
||||||
assert AMENDMENT == "A-OG-2"
|
assert AMENDMENT == "A-OG-3"
|
||||||
assert resume_compatible(old, {**old, "controller_sha256": "new"})
|
assert resume_compatible(old, {**old, "controller_sha256": "new"})
|
||||||
assert not resume_compatible(
|
assert not resume_compatible(
|
||||||
old,
|
old,
|
||||||
|
|||||||
Reference in New Issue
Block a user