Strengthen fidelity calibration baseline

docs/fidelity-aware-harness-protocol-20260714.md

@@ -52,6 +52,35 @@ difference is Z.  The initial family is intentionally simple: a positive result
 then demonstrates value in the engine signal rather than capacity in a larger
 learner.  A sequence model is admissible only as a later, paired ablation.
 
+### Amendment A1: strengthen the calibration baseline before P2
+
+Frozen 2026-07-14 13:08 Asia/Singapore, after P1 launch but before P1
+completion or analysis.  A baseline audit found that the first frozen P1
+models use the simulator only to define candidate order; their feature vectors
+do not contain the simulator's per-anchor prediction.  This is insufficient
+for the stronger term **outcome-only calibration**.  P1 therefore remains a
+prospective test of the originally frozen cross-workload predictor, but cannot
+by itself open a contribution claim.
+
+For P2/P3, both nested models must additionally receive the identical frozen
+simulator outputs available at that decision: predicted completed throughput
+per GPU, predicted SLO pass rate, and predicted feasibility.  The comparison
+is consequently `sim + config + workload + real outcome prefix` versus that
+exact vector plus real engine state.  Simulator features, regularization,
+cutoff, and thresholds are frozen before any P2 task.  If telemetry does not
+improve this stronger baseline, the harness has no independent contribution.
+
+The same audit also separates algorithm cost from benchmark-oracle cost.
+Headline method cost includes every action the method would execute online:
+simulator profiling/calibration, model onboarding, server startup, warm-up,
+real prefix, continuation after abstention, method-requested confirmation,
+logging overhead, failures, and cleanup.  Exhaustive real-oracle runs and the
+extra repetitions used only to construct 2-of-3 evaluation labels are common
+benchmark annotation cost; they are reported separately and charged to no
+method.  A second, deliberately conservative table adds that common cost to
+all methods.  This prevents both hiding real method cost and making the
+percentage gate mathematically depend on offline ground-truth annotation.
+
 The frozen first policy uses a 5-second prefix, L2 regularization 1.0, and a
 two-sided abstaining threshold of 0.95: accept at `p(feasible)>=0.95`, reject at
 `p(feasible)<=0.05`, otherwise continue the exact same trial to completion.
@@ -64,15 +93,15 @@ therefore not evidence; all claims come from subsequent held-out tasks.
 |---|---:|---:|---:|---:|---:|
 | Real-only oracle | no | no | full | optional diagnostic | every candidate/anchor |
 | Sim top-k + real final | yes | included in full run | full | no decision use | every shortlisted candidate/anchor |
-| Outcome-only calibration | yes | yes | yes | no | only on abstention |
-| Instrumentation-aware | yes | yes | yes | yes | only on abstention |
+| Outcome-only calibration | yes, including its prediction features | yes | yes | no | only on abstention |
+| Instrumentation-aware | same prediction features | yes | yes | yes | only on abstention |
 
 Tie buckets are expanded before top-k.  `k` is selected on training tasks and
 is fixed on held-out tasks; an oracle per-task k is forbidden.  Outcome-only
-receives all information available outside the engine, including config and
-workload features.  Instrumentation cannot use any record submitted after the
-cutoff.  The full label, confirmation votes, simulator error, and later
-requests are never model features.
+receives all information available outside the engine, including config,
+workload, and frozen simulator-prediction features.  Instrumentation cannot use
+any record submitted after the cutoff.  The full label, confirmation votes,
+realized simulator error, and later requests are never model features.
 
 ## Staged experiment