""" Do the overnight sleeve's backtest prices match the auctions it would trade in? The sleeve buys market-on-close and sells market-on-open, so a real night earns the primary exchange's closing cross to its next opening cross. The backtest (research/lab/overnight_only.py) earns Yahoo open_t / close_t-1 and charges 1 bp a side. This measures the gap. PYTHONPATH=. uv run python -m research.lab.auction_audit fetch # sample, SIP trades, daily bars (resumable) PYTHONPATH=. uv run python -m research.lab.auction_audit # compare (refuses to overwrite results) Sample: 800 nights drawn at random (seed 20261012) from 2016-01-05..2026-09-24 (Alpaca SIP history starts 2016). On each night it takes every name the sleeve holds (sleeve_score, the live score, top 10 by score at close t-2) plus the overnight_12_1 names (reported). Auction prints (Alpaca SIP trades, feed=sip): the cross is the largest-size print with condition 'O' (open) or '6' (close). That is the primary listing exchange's cross: Nasdaq x=Q (x=T in older data), NYSE x=N. close d-1, 15:59:59-16:10:00 New York; NYSE close prints can arrive after 16:03 open d, 09:29:59-09:30:03; names still missing 09:30:03-09:31:00, then 09:31:00-09:40:00 (NYSE opens can come seconds to minutes late) Adjustment: F = close(adjustment=all) / close(raw) from Alpaca daily bars, so real night R_real = open_x(d) F(d) / (close_x(d-1) F(d-1)) - 1 backtest R_bt = Yahoo open(d) / Yahoo close(d-1) - 1 (research panel) shortfall R_bt - R_real, per name-night; the night mean is the unit of observation (SE over nights) split R_bt - R_bar (Yahoo vs Alpaca daily-bar open/close) and R_bar - R_real (bar open/close vs the crosses) Fees on top: SEC Section 31 fee about 0.28 bp on sells (0.14 bp a side on average); FINRA TAF is negligible. Commissions are $0 at Alpaca. Rule (fixed before any comparison was computed): cost per side = night-mean shortfall / 2 + 0.14 bp, for sleeve_score names <= 1.0 bp PASS: the backtest's 1 bp a side holds; the Reg T-cap ranges stand (research/lab/option_overlay.py: about 36% three_sleeve_overnight, 41% 12_1) 1.0-2.5 bp plan with the measured cost (about 2 bp: 32% / 37% at the cap) > 2.5 bp the overnight sleeve's edge over the all-day sleeve is gone (overnight_book: 4 bp a side is the baseline's Sharpe); no live overnight sleeve Reported: by year, 2016-20 vs 2021-26, Nasdaq vs NYSE listing, overnight_12_1 names, trimmed mean, the share of name-nights without a cross print. Not measured: whether orders reach the auction (Alpaca's MOC cutoff 15:50, MOO before 09:28), impact at larger sizes, partial fills (a market order in a cross fills unless the stock is halted). Result (2026-10-12): PASS. The cost per side is +0.20 bp for the sleeve_score names and +0.28 bp for overnight_12_1, including the SEC fee. sleeve_score: 7,856 name-nights over 795 nights, 2016-01..2026-09 R_bt - R_real +0.12 bp/night (SE 0.24), median 0.00, trimmed +0.23 Yahoo vs bar -0.02 (SE 0.09) bar vs cross +0.14 (SE 0.24): the closing cross equals the bar close (0.00); the bar open vs the opening cross is +0.16 by year within +-1 bp except 2020, -1.6 (SE 0.8); 2016-20 -0.24, 2021-26 +0.48 by listing NYSE +0.18, Nasdaq -0.02 coverage 122 / 74 of 8,000 name-nights had no close / open cross print (halts, symbol changes); 2 outliers over 5% were dropped - Yahoo's open and close are the primary exchange's crosses (median difference 0). The backtest's prices are the prices a MOC/MOO order gets, so the 1 bp a side the books assumed was conservative. About 0.2 bp a side remains, which is the SEC fee. - Recomputed after the run at 0.2 bp a side, 2006-10..2026-09 with margin at T-bill + 1.5% (CAGR / MaxDD): book 1x Reg T cap three_sleeve_overnight 23.9% 39.2% at 1.7x (-26.7%) overnight_12_1 26.6% 44.3% at 1.7x (-26.2%) two_sleeve_overnight_trend 32.0% 50.6% at 1.6x (-28.7%) two_sleeve_overnight_trend makes 64% a year in 2017-26 and 39% in 2006-16. It leans on TrendRider's last decade, which the 1990-2026 synthetic test does not support. - Not measured: - whether a live MOC/MOO order fills at the cross. Alpaca's paper engine does not run auctions, so only a small live account can check this. - impact at larger size. Orders here are thousands of dollars against crosses of millions. - whole-share sizing in small accounts. """ from __future__ import annotations import argparse import datetime as dt import json import os import numpy as np import pandas as pd from research.lab.factor_scan import RESULTS, load OUT = os.path.join(RESULTS, "auction_audit") RAW = os.path.join("data", "research", "auction_audit") TRADES_URL = "https://data.alpaca.markets/v2/stocks/trades" BARS_URL = "https://data.alpaca.markets/v2/stocks/bars" NY = "America/New_York" SAMPLE_START, SAMPLE_END, N_NIGHTS, SEED = "2016-01-05", "2026-09-24", 800, 20261012 TOP_N = 10 SEC_FEE_SIDE_BP = 0.14 PASS_BP, STOP_BP = 1.0, 2.5 PREREG = { "question": "How far is the overnight sleeve's backtest night return (Yahoo open/close) from the closing-cross " "to opening-cross return it would actually earn?", "sample": {"nights": N_NIGHTS, "from": SAMPLE_START, "to": SAMPLE_END, "seed": SEED, "names": "sleeve_score top 10 (score at close t-2); overnight_12_1 names reported"}, "cross": "largest-size SIP print with condition 'O' (open) / '6' (close)", "adjustment": "Alpaca daily bars: F = close(all) / close(raw)", "metric": "cost per side = night-mean (R_bt - R_real) / 2 + 0.14 bp SEC fee", "rule": {"PASS": "<= 1.0 bp/side: the 1 bp backtest cost holds", "MEASURED": "1.0-2.5 bp/side: re-plan returns with the measured cost", "STOP": "> 2.5 bp/side: no live overnight sleeve"}, "data_status": "no auction prints compared before this rule; the 09:30 5-min bar check (overnight_only N5) " "was seen: Yahoo vs bar overnight returns corr 0.996 on selected names, mean diff 0.03-0.3 bp", } # --------------------------------------------------------------------------- # sample # --------------------------------------------------------------------------- def selections(p) -> dict[str, pd.DataFrame]: from research.lab.factors import overnight_mom, rec_126 from research.lab.overnight_only import top_mask from research.lab.us_sweep import _blend mem = p.member scores = {"sleeve_score": _blend(*[f.where(mem).rank(axis=1, pct=True) for f in (overnight_mom(p), rec_126(p))]).where(mem), "overnight_12_1": overnight_mom(p).where(mem)} return {k: top_mask(sc.shift(2), TOP_N) for k, sc in scores.items()} def sample_nights(p, sels: dict) -> pd.DataFrame: dates = p.dates[(p.dates >= SAMPLE_START) & (p.dates <= SAMPLE_END)] dates = [d for d in dates if sels["sleeve_score"].loc[d].sum() == TOP_N] rng = np.random.default_rng(SEED) nights = sorted(rng.choice(np.array(dates, dtype="datetime64[ns]"), size=N_NIGHTS, replace=False)) rows = [] for d in pd.DatetimeIndex(nights): prev = p.dates[p.dates.get_loc(d) - 1] for k, sel in sels.items(): for s in sel.columns[sel.loc[d].values]: rows.append({"night": d, "prev": prev, "symbol": s, "candidate": k}) return pd.DataFrame(rows) # --------------------------------------------------------------------------- # fetch # --------------------------------------------------------------------------- def _ts(day: pd.Timestamp, hhmmss: str) -> str: t = pd.Timestamp(f"{day.date()} {hhmmss}").tz_localize(NY) return t.isoformat() def alp(sym: str) -> str: return sym.replace("-", ".") def crosses(api, syms: list[str], day: pd.Timestamp, a: str, b: str, cond: str) -> dict[str, dict]: """Largest-size print with condition ``cond`` per symbol in [a, b] New York time.""" best: dict[str, dict] = {} params = {"symbols": ",".join(alp(s) for s in syms), "start": _ts(day, a), "end": _ts(day, b), "feed": "sip", "limit": 10000} for page in api.pages(TRADES_URL, params, "trades"): if not page: continue for s, ts in page.items(): for t in ts: if cond in (t.get("c") or []) and t["s"] > best.get(s, {}).get("s", -1): best[s] = {"p": t["p"], "s": t["s"], "x": t["x"], "t": t["t"]} return {s.replace(".", "-"): v for s, v in best.items()} def fetch() -> None: import research.lab.fetch_options as fo os.makedirs(RAW, exist_ok=True) os.makedirs(OUT, exist_ok=True) prereg = os.path.join(OUT, "prereg.json") if not os.path.exists(prereg): with open(prereg, "w") as fh: json.dump(PREREG, fh, indent=2) p, _ = load("us") sample_path = os.path.join(RAW, "sample.csv") if os.path.exists(sample_path): S = pd.read_csv(sample_path, parse_dates=["night", "prev"]) else: S = sample_nights(p, selections(p)) S.to_csv(sample_path, index=False) api = fo.Api(min_interval=0.4, yield_to_snapshot=True) prints_path = os.path.join(RAW, "prints.csv") done = set() if os.path.exists(prints_path): done = set(pd.read_csv(prints_path, usecols=["night"])["night"].astype(str)) nights = sorted(S["night"].unique()) print(f"{len(nights)} nights, {S['symbol'].nunique()} symbols, {len(done)} done", flush=True) for i, d in enumerate(nights): d = pd.Timestamp(d) if str(d.date()) in done: continue g = S[S["night"] == d] syms, prev = sorted(g["symbol"].unique()), pd.Timestamp(g["prev"].iloc[0]) close = crosses(api, syms, prev, "15:59:59", "16:10:00", "6") opn = crosses(api, syms, d, "09:29:59", "09:30:03", "O") for a, b in (("09:30:03", "09:31:00"), ("09:31:00", "09:40:00")): missing = [s for s in syms if s not in opn] if not missing: break opn.update(crosses(api, missing, d, a, b, "O")) rows = [] for s in syms: c, o = close.get(s, {}), opn.get(s, {}) rows.append({"night": str(d.date()), "prev": str(prev.date()), "symbol": s, "close_p": c.get("p"), "close_s": c.get("s"), "close_x": c.get("x"), "close_t": c.get("t"), "open_p": o.get("p"), "open_s": o.get("s"), "open_x": o.get("x"), "open_t": o.get("t")}) pd.DataFrame(rows).to_csv(prints_path, mode="a", header=not os.path.exists(prints_path), index=False) if (i + 1) % 25 == 0: print(f" [{i + 1}/{len(nights)}] {d.date()} {api.n} requests", flush=True) bars_path = os.path.join(RAW, "bars.pkl") if not os.path.exists(bars_path): syms = sorted(S["symbol"].unique()) out = {} for adj in ("raw", "all"): rows = [] for k in range(0, len(syms), 100): params = {"symbols": ",".join(alp(s) for s in syms[k:k + 100]), "timeframe": "1Day", "start": "2015-12-01", "end": "2026-09-30", "adjustment": adj, "feed": "sip", "limit": 10000} for page in api.pages(BARS_URL, params, "bars"): if not page: continue for s, bl in page.items(): rows += [{"symbol": s.replace(".", "-"), "date": b["t"][:10], "o": b["o"], "c": b["c"]} for b in bl] out[adj] = pd.DataFrame(rows) print(f"bars {adj}: {len(rows)} rows", flush=True) pd.to_pickle(out, bars_path) print(f"done, {api.n} requests", flush=True) # --------------------------------------------------------------------------- # compare # --------------------------------------------------------------------------- def night_stats(x: pd.Series, nights: pd.Series) -> dict: """Mean over nights of the night-mean, its SE, and name-level median.""" nm = x.groupby(nights).mean().dropna() return {"nights": int(len(nm)), "name_nights": int(x.notna().sum()), "mean_bp": float(nm.mean() * 1e4), "se_bp": float(nm.std() / np.sqrt(len(nm)) * 1e4), "median_bp": float(x.median() * 1e4), "trimmed_mean_bp": float(x.clip(x.quantile(0.025), x.quantile(0.975)).mean() * 1e4)} def main() -> None: res_path = os.path.join(OUT, "results.json") if os.path.exists(res_path): raise SystemExit(f"{res_path} exists; this study does not overwrite results") S = pd.read_csv(os.path.join(RAW, "sample.csv"), parse_dates=["night", "prev"]) P = pd.read_csv(os.path.join(RAW, "prints.csv"), parse_dates=["night", "prev"]).drop_duplicates(["night", "symbol"]) bars = pd.read_pickle(os.path.join(RAW, "bars.pkl")) p, _ = load("us") def wide(df, col): return df.pivot_table(index=pd.to_datetime(df["date"]), columns="symbol", values=col, aggfunc="last") braw_c, braw_o, ball_c = wide(bars["raw"], "c"), wide(bars["raw"], "o"), wide(bars["all"], "c") F = ball_c / braw_c X = S.merge(P, on=["night", "prev", "symbol"], how="left") def at(df, dates, syms): return np.array([df.at[d, s] if (d in df.index and s in df.columns) else np.nan for d, s in zip(dates, syms)], dtype=float) X["F_prev"], X["F_night"] = at(F, X["prev"], X["symbol"]), at(F, X["night"], X["symbol"]) X["bar_c_prev"], X["bar_o_night"] = at(braw_c, X["prev"], X["symbol"]), at(braw_o, X["night"], X["symbol"]) X["y_c_prev"] = at(p.close, X["prev"], X["symbol"]) X["y_o_night"] = at(p.open, X["night"], X["symbol"]) X["R_bt"] = X["y_o_night"] / X["y_c_prev"] - 1 X["R_real"] = (X["open_p"] * X["F_night"]) / (X["close_p"] * X["F_prev"]) - 1 X["R_bar"] = (X["bar_o_night"] * X["F_night"]) / (X["bar_c_prev"] * X["F_prev"]) - 1 X["short"] = X["R_bt"] - X["R_real"] X["yahoo_vs_bar"] = X["R_bt"] - X["R_bar"] X["bar_vs_cross"] = X["R_bar"] - X["R_real"] X["close_cross_vs_bar"] = X["close_p"] / X["bar_c_prev"] - 1 # same day, raw: + = paid more than the bar close X["open_bar_vs_cross"] = X["bar_o_night"] / X["open_p"] - 1 # same day, raw: + = sold below the bar open X["listing"] = np.where(X["open_x"].isin(["N"]) | X["close_x"].isin(["N"]), "NYSE", np.where(X["open_x"].isin(["Q", "T"]) | X["close_x"].isin(["Q", "T"]), "Nasdaq", "other")) ok = X[["R_bt", "R_real"]].notna().all(axis=1) & (X["short"].abs() < 0.05) res = {"coverage": {}, "candidates": {}} for k in ("sleeve_score", "overnight_12_1"): g = X[X["candidate"] == k] res["coverage"][k] = {"name_nights": int(len(g)), "no_close_cross": int(g["close_p"].isna().sum()), "no_open_cross": int(g["open_p"].isna().sum()), "no_yahoo_or_factor": int((g[["R_bt", "F_prev", "F_night"]].isna().any(axis=1)).sum()), "outliers_over_5pct": int((g["short"].abs() >= 0.05).sum())} print("coverage:", json.dumps(res["coverage"])) for k in ("sleeve_score", "overnight_12_1"): g = X[(X["candidate"] == k) & ok] d = {c: night_stats(g[c], g["night"]) for c in ("short", "yahoo_vs_bar", "bar_vs_cross", "close_cross_vs_bar", "open_bar_vs_cross")} d["cost_per_side_bp"] = d["short"]["mean_bp"] / 2 + SEC_FEE_SIDE_BP d["by_year"] = {int(y): night_stats(h["short"], h["night"]) for y, h in g.groupby(g["night"].dt.year)} d["by_period"] = {lab: night_stats(h["short"], h["night"]) for lab, h in (("2016-20", g[g["night"] < "2021-01-01"]), ("2021-26", g[g["night"] >= "2021-01-01"]))} d["by_listing"] = {lab: night_stats(h["short"], h["night"]) for lab, h in g.groupby("listing")} res["candidates"][k] = d s = d["short"] print(f"\n== {k}: {s['name_nights']} name-nights over {s['nights']} nights") print(f" shortfall R_bt - R_real: mean {s['mean_bp']:+.2f} bp/night (SE {s['se_bp']:.2f}), median " f"{s['median_bp']:+.2f}, trimmed {s['trimmed_mean_bp']:+.2f} -> cost per side {d['cost_per_side_bp']:+.2f} bp") for c in ("yahoo_vs_bar", "bar_vs_cross", "close_cross_vs_bar", "open_bar_vs_cross"): print(f" {c:20s} mean {d[c]['mean_bp']:+.2f} bp (SE {d[c]['se_bp']:.2f}) median {d[c]['median_bp']:+.2f}") print(" by year:", " ".join(f"{y} {v['mean_bp']:+.1f}±{v['se_bp']:.1f}" for y, v in d["by_year"].items())) print(" by period:", {a: round(v["mean_bp"], 2) for a, v in d["by_period"].items()}, " by listing:", {a: (round(v["mean_bp"], 2), v["name_nights"]) for a, v in d["by_listing"].items()}) cost = res["candidates"]["sleeve_score"]["cost_per_side_bp"] verdict = "PASS" if cost <= PASS_BP else ("MEASURED" if cost <= STOP_BP else "STOP") res["verdict"] = {"cost_per_side_bp": cost, "decision": verdict} print(f"\nverdict: cost per side {cost:+.2f} bp -> {verdict}") with open(res_path, "w") as fh: json.dump(res, fh, indent=2) X.to_csv(os.path.join(OUT, "name_nights.csv.gz"), index=False) if __name__ == "__main__": ap = argparse.ArgumentParser() ap.add_argument("cmd", nargs="?", default="run", choices=["run", "fetch"]) if ap.parse_args().cmd == "fetch": fetch() else: main()