Gahow Wang
aa24ed8c31
The overnight sleeve buys at the closing cross and sells at the opening cross; the backtest used Yahoo open/close and 1 bp a side. On 800 random nights in 2016-26 (7,856 name-nights), the backtest night return exceeds the cross-to-cross return by +0.12 bp (SE 0.24, median 0): Yahoo's prices are the crosses. With the SEC fee, cost is +0.20 bp a side, so 1 bp was conservative. At the Reg T cap this lifts three_sleeve_overnight to 39%/yr (MaxDD -27%) and the 12_1 variant to 44%. Live fills at the cross remain unverified (paper does not run auctions). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
337 lines
18 KiB
Python
337 lines
18 KiB
Python
"""
|
|
Do the overnight sleeve's backtest prices match the auctions it would trade
|
|
in? The sleeve buys market-on-close and sells market-on-open, so a real
|
|
night earns the primary exchange's closing cross to its next opening cross.
|
|
The backtest (research/lab/overnight_only.py) earns Yahoo open_t / close_t-1
|
|
and charges 1 bp a side. This measures the gap.
|
|
|
|
PYTHONPATH=. uv run python -m research.lab.auction_audit fetch # sample, SIP trades, daily bars (resumable)
|
|
PYTHONPATH=. uv run python -m research.lab.auction_audit # compare (refuses to overwrite results)
|
|
|
|
Sample: 800 nights drawn at random (seed 20261012) from 2016-01-05..2026-09-24
|
|
(Alpaca SIP history starts 2016). On each night it takes every name the sleeve
|
|
holds (sleeve_score, the live score, top 10 by score at close t-2) plus the
|
|
overnight_12_1 names (reported).
|
|
|
|
Auction prints (Alpaca SIP trades, feed=sip): the cross is the largest-size
|
|
print with condition 'O' (open) or '6' (close). That is the primary listing
|
|
exchange's cross: Nasdaq x=Q (x=T in older data), NYSE x=N.
|
|
close d-1, 15:59:59-16:10:00 New York; NYSE close prints can arrive after 16:03
|
|
open d, 09:29:59-09:30:03; names still missing 09:30:03-09:31:00, then
|
|
09:31:00-09:40:00 (NYSE opens can come seconds to minutes late)
|
|
Adjustment: F = close(adjustment=all) / close(raw) from Alpaca daily bars, so
|
|
real night R_real = open_x(d) F(d) / (close_x(d-1) F(d-1)) - 1
|
|
backtest R_bt = Yahoo open(d) / Yahoo close(d-1) - 1 (research panel)
|
|
shortfall R_bt - R_real, per name-night; the night mean is the unit of
|
|
observation (SE over nights)
|
|
split R_bt - R_bar (Yahoo vs Alpaca daily-bar open/close) and
|
|
R_bar - R_real (bar open/close vs the crosses)
|
|
Fees on top: SEC Section 31 fee about 0.28 bp on sells (0.14 bp a side on
|
|
average); FINRA TAF is negligible. Commissions are $0 at Alpaca.
|
|
|
|
Rule (fixed before any comparison was computed):
|
|
cost per side = night-mean shortfall / 2 + 0.14 bp, for sleeve_score names
|
|
<= 1.0 bp PASS: the backtest's 1 bp a side holds; the Reg T-cap ranges stand
|
|
(research/lab/option_overlay.py: about 36% three_sleeve_overnight, 41% 12_1)
|
|
1.0-2.5 bp plan with the measured cost (about 2 bp: 32% / 37% at the cap)
|
|
> 2.5 bp the overnight sleeve's edge over the all-day sleeve is gone
|
|
(overnight_book: 4 bp a side is the baseline's Sharpe); no live overnight sleeve
|
|
Reported: by year, 2016-20 vs 2021-26, Nasdaq vs NYSE listing, overnight_12_1
|
|
names, trimmed mean, the share of name-nights without a cross print.
|
|
|
|
Not measured: whether orders reach the auction (Alpaca's MOC cutoff 15:50,
|
|
MOO before 09:28), impact at larger sizes, partial fills (a market order in
|
|
a cross fills unless the stock is halted).
|
|
|
|
Result (2026-10-12): PASS. The cost per side is +0.20 bp for the
|
|
sleeve_score names and +0.28 bp for overnight_12_1, including the SEC fee.
|
|
|
|
sleeve_score: 7,856 name-nights over 795 nights, 2016-01..2026-09
|
|
R_bt - R_real +0.12 bp/night (SE 0.24), median 0.00, trimmed +0.23
|
|
Yahoo vs bar -0.02 (SE 0.09)
|
|
bar vs cross +0.14 (SE 0.24): the closing cross equals the bar close
|
|
(0.00); the bar open vs the opening cross is +0.16
|
|
by year within +-1 bp except 2020, -1.6 (SE 0.8);
|
|
2016-20 -0.24, 2021-26 +0.48
|
|
by listing NYSE +0.18, Nasdaq -0.02
|
|
coverage 122 / 74 of 8,000 name-nights had no close / open
|
|
cross print (halts, symbol changes); 2 outliers over
|
|
5% were dropped
|
|
|
|
- Yahoo's open and close are the primary exchange's crosses (median
|
|
difference 0). The backtest's prices are the prices a MOC/MOO order
|
|
gets, so the 1 bp a side the books assumed was conservative. About
|
|
0.2 bp a side remains, which is the SEC fee.
|
|
- Recomputed after the run at 0.2 bp a side, 2006-10..2026-09 with
|
|
margin at T-bill + 1.5% (CAGR / MaxDD):
|
|
|
|
book 1x Reg T cap
|
|
three_sleeve_overnight 23.9% 39.2% at 1.7x (-26.7%)
|
|
overnight_12_1 26.6% 44.3% at 1.7x (-26.2%)
|
|
two_sleeve_overnight_trend 32.0% 50.6% at 1.6x (-28.7%)
|
|
|
|
two_sleeve_overnight_trend makes 64% a year in 2017-26 and 39% in
|
|
2006-16. It leans on TrendRider's last decade, which the 1990-2026
|
|
synthetic test does not support.
|
|
- Not measured:
|
|
- whether a live MOC/MOO order fills at the cross. Alpaca's paper engine
|
|
does not run auctions, so only a small live account can check this.
|
|
- impact at larger size. Orders here are thousands of dollars against
|
|
crosses of millions.
|
|
- whole-share sizing in small accounts.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import argparse
|
|
import datetime as dt
|
|
import json
|
|
import os
|
|
|
|
import numpy as np
|
|
import pandas as pd
|
|
|
|
from research.lab.factor_scan import RESULTS, load
|
|
|
|
OUT = os.path.join(RESULTS, "auction_audit")
|
|
RAW = os.path.join("data", "research", "auction_audit")
|
|
TRADES_URL = "https://data.alpaca.markets/v2/stocks/trades"
|
|
BARS_URL = "https://data.alpaca.markets/v2/stocks/bars"
|
|
NY = "America/New_York"
|
|
SAMPLE_START, SAMPLE_END, N_NIGHTS, SEED = "2016-01-05", "2026-09-24", 800, 20261012
|
|
TOP_N = 10
|
|
SEC_FEE_SIDE_BP = 0.14
|
|
PASS_BP, STOP_BP = 1.0, 2.5
|
|
|
|
PREREG = {
|
|
"question": "How far is the overnight sleeve's backtest night return (Yahoo open/close) from the closing-cross "
|
|
"to opening-cross return it would actually earn?",
|
|
"sample": {"nights": N_NIGHTS, "from": SAMPLE_START, "to": SAMPLE_END, "seed": SEED,
|
|
"names": "sleeve_score top 10 (score at close t-2); overnight_12_1 names reported"},
|
|
"cross": "largest-size SIP print with condition 'O' (open) / '6' (close)",
|
|
"adjustment": "Alpaca daily bars: F = close(all) / close(raw)",
|
|
"metric": "cost per side = night-mean (R_bt - R_real) / 2 + 0.14 bp SEC fee",
|
|
"rule": {"PASS": "<= 1.0 bp/side: the 1 bp backtest cost holds",
|
|
"MEASURED": "1.0-2.5 bp/side: re-plan returns with the measured cost",
|
|
"STOP": "> 2.5 bp/side: no live overnight sleeve"},
|
|
"data_status": "no auction prints compared before this rule; the 09:30 5-min bar check (overnight_only N5) "
|
|
"was seen: Yahoo vs bar overnight returns corr 0.996 on selected names, mean diff 0.03-0.3 bp",
|
|
}
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# sample
|
|
# ---------------------------------------------------------------------------
|
|
|
|
def selections(p) -> dict[str, pd.DataFrame]:
|
|
from research.lab.factors import overnight_mom, rec_126
|
|
from research.lab.overnight_only import top_mask
|
|
from research.lab.us_sweep import _blend
|
|
mem = p.member
|
|
scores = {"sleeve_score": _blend(*[f.where(mem).rank(axis=1, pct=True)
|
|
for f in (overnight_mom(p), rec_126(p))]).where(mem),
|
|
"overnight_12_1": overnight_mom(p).where(mem)}
|
|
return {k: top_mask(sc.shift(2), TOP_N) for k, sc in scores.items()}
|
|
|
|
|
|
def sample_nights(p, sels: dict) -> pd.DataFrame:
|
|
dates = p.dates[(p.dates >= SAMPLE_START) & (p.dates <= SAMPLE_END)]
|
|
dates = [d for d in dates if sels["sleeve_score"].loc[d].sum() == TOP_N]
|
|
rng = np.random.default_rng(SEED)
|
|
nights = sorted(rng.choice(np.array(dates, dtype="datetime64[ns]"), size=N_NIGHTS, replace=False))
|
|
rows = []
|
|
for d in pd.DatetimeIndex(nights):
|
|
prev = p.dates[p.dates.get_loc(d) - 1]
|
|
for k, sel in sels.items():
|
|
for s in sel.columns[sel.loc[d].values]:
|
|
rows.append({"night": d, "prev": prev, "symbol": s, "candidate": k})
|
|
return pd.DataFrame(rows)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# fetch
|
|
# ---------------------------------------------------------------------------
|
|
|
|
def _ts(day: pd.Timestamp, hhmmss: str) -> str:
|
|
t = pd.Timestamp(f"{day.date()} {hhmmss}").tz_localize(NY)
|
|
return t.isoformat()
|
|
|
|
|
|
def alp(sym: str) -> str:
|
|
return sym.replace("-", ".")
|
|
|
|
|
|
def crosses(api, syms: list[str], day: pd.Timestamp, a: str, b: str, cond: str) -> dict[str, dict]:
|
|
"""Largest-size print with condition ``cond`` per symbol in [a, b] New York time."""
|
|
best: dict[str, dict] = {}
|
|
params = {"symbols": ",".join(alp(s) for s in syms), "start": _ts(day, a), "end": _ts(day, b),
|
|
"feed": "sip", "limit": 10000}
|
|
for page in api.pages(TRADES_URL, params, "trades"):
|
|
if not page:
|
|
continue
|
|
for s, ts in page.items():
|
|
for t in ts:
|
|
if cond in (t.get("c") or []) and t["s"] > best.get(s, {}).get("s", -1):
|
|
best[s] = {"p": t["p"], "s": t["s"], "x": t["x"], "t": t["t"]}
|
|
return {s.replace(".", "-"): v for s, v in best.items()}
|
|
|
|
|
|
def fetch() -> None:
|
|
import research.lab.fetch_options as fo
|
|
os.makedirs(RAW, exist_ok=True)
|
|
os.makedirs(OUT, exist_ok=True)
|
|
prereg = os.path.join(OUT, "prereg.json")
|
|
if not os.path.exists(prereg):
|
|
with open(prereg, "w") as fh:
|
|
json.dump(PREREG, fh, indent=2)
|
|
p, _ = load("us")
|
|
sample_path = os.path.join(RAW, "sample.csv")
|
|
if os.path.exists(sample_path):
|
|
S = pd.read_csv(sample_path, parse_dates=["night", "prev"])
|
|
else:
|
|
S = sample_nights(p, selections(p))
|
|
S.to_csv(sample_path, index=False)
|
|
api = fo.Api(min_interval=0.4, yield_to_snapshot=True)
|
|
prints_path = os.path.join(RAW, "prints.csv")
|
|
done = set()
|
|
if os.path.exists(prints_path):
|
|
done = set(pd.read_csv(prints_path, usecols=["night"])["night"].astype(str))
|
|
nights = sorted(S["night"].unique())
|
|
print(f"{len(nights)} nights, {S['symbol'].nunique()} symbols, {len(done)} done", flush=True)
|
|
for i, d in enumerate(nights):
|
|
d = pd.Timestamp(d)
|
|
if str(d.date()) in done:
|
|
continue
|
|
g = S[S["night"] == d]
|
|
syms, prev = sorted(g["symbol"].unique()), pd.Timestamp(g["prev"].iloc[0])
|
|
close = crosses(api, syms, prev, "15:59:59", "16:10:00", "6")
|
|
opn = crosses(api, syms, d, "09:29:59", "09:30:03", "O")
|
|
for a, b in (("09:30:03", "09:31:00"), ("09:31:00", "09:40:00")):
|
|
missing = [s for s in syms if s not in opn]
|
|
if not missing:
|
|
break
|
|
opn.update(crosses(api, missing, d, a, b, "O"))
|
|
rows = []
|
|
for s in syms:
|
|
c, o = close.get(s, {}), opn.get(s, {})
|
|
rows.append({"night": str(d.date()), "prev": str(prev.date()), "symbol": s,
|
|
"close_p": c.get("p"), "close_s": c.get("s"), "close_x": c.get("x"), "close_t": c.get("t"),
|
|
"open_p": o.get("p"), "open_s": o.get("s"), "open_x": o.get("x"), "open_t": o.get("t")})
|
|
pd.DataFrame(rows).to_csv(prints_path, mode="a", header=not os.path.exists(prints_path), index=False)
|
|
if (i + 1) % 25 == 0:
|
|
print(f" [{i + 1}/{len(nights)}] {d.date()} {api.n} requests", flush=True)
|
|
bars_path = os.path.join(RAW, "bars.pkl")
|
|
if not os.path.exists(bars_path):
|
|
syms = sorted(S["symbol"].unique())
|
|
out = {}
|
|
for adj in ("raw", "all"):
|
|
rows = []
|
|
for k in range(0, len(syms), 100):
|
|
params = {"symbols": ",".join(alp(s) for s in syms[k:k + 100]), "timeframe": "1Day",
|
|
"start": "2015-12-01", "end": "2026-09-30", "adjustment": adj, "feed": "sip",
|
|
"limit": 10000}
|
|
for page in api.pages(BARS_URL, params, "bars"):
|
|
if not page:
|
|
continue
|
|
for s, bl in page.items():
|
|
rows += [{"symbol": s.replace(".", "-"), "date": b["t"][:10], "o": b["o"], "c": b["c"]}
|
|
for b in bl]
|
|
out[adj] = pd.DataFrame(rows)
|
|
print(f"bars {adj}: {len(rows)} rows", flush=True)
|
|
pd.to_pickle(out, bars_path)
|
|
print(f"done, {api.n} requests", flush=True)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# compare
|
|
# ---------------------------------------------------------------------------
|
|
|
|
def night_stats(x: pd.Series, nights: pd.Series) -> dict:
|
|
"""Mean over nights of the night-mean, its SE, and name-level median."""
|
|
nm = x.groupby(nights).mean().dropna()
|
|
return {"nights": int(len(nm)), "name_nights": int(x.notna().sum()),
|
|
"mean_bp": float(nm.mean() * 1e4), "se_bp": float(nm.std() / np.sqrt(len(nm)) * 1e4),
|
|
"median_bp": float(x.median() * 1e4),
|
|
"trimmed_mean_bp": float(x.clip(x.quantile(0.025), x.quantile(0.975)).mean() * 1e4)}
|
|
|
|
|
|
def main() -> None:
|
|
res_path = os.path.join(OUT, "results.json")
|
|
if os.path.exists(res_path):
|
|
raise SystemExit(f"{res_path} exists; this study does not overwrite results")
|
|
S = pd.read_csv(os.path.join(RAW, "sample.csv"), parse_dates=["night", "prev"])
|
|
P = pd.read_csv(os.path.join(RAW, "prints.csv"), parse_dates=["night", "prev"]).drop_duplicates(["night", "symbol"])
|
|
bars = pd.read_pickle(os.path.join(RAW, "bars.pkl"))
|
|
p, _ = load("us")
|
|
|
|
def wide(df, col):
|
|
return df.pivot_table(index=pd.to_datetime(df["date"]), columns="symbol", values=col, aggfunc="last")
|
|
|
|
braw_c, braw_o, ball_c = wide(bars["raw"], "c"), wide(bars["raw"], "o"), wide(bars["all"], "c")
|
|
F = ball_c / braw_c
|
|
X = S.merge(P, on=["night", "prev", "symbol"], how="left")
|
|
|
|
def at(df, dates, syms):
|
|
return np.array([df.at[d, s] if (d in df.index and s in df.columns) else np.nan for d, s in zip(dates, syms)],
|
|
dtype=float)
|
|
|
|
X["F_prev"], X["F_night"] = at(F, X["prev"], X["symbol"]), at(F, X["night"], X["symbol"])
|
|
X["bar_c_prev"], X["bar_o_night"] = at(braw_c, X["prev"], X["symbol"]), at(braw_o, X["night"], X["symbol"])
|
|
X["y_c_prev"] = at(p.close, X["prev"], X["symbol"])
|
|
X["y_o_night"] = at(p.open, X["night"], X["symbol"])
|
|
X["R_bt"] = X["y_o_night"] / X["y_c_prev"] - 1
|
|
X["R_real"] = (X["open_p"] * X["F_night"]) / (X["close_p"] * X["F_prev"]) - 1
|
|
X["R_bar"] = (X["bar_o_night"] * X["F_night"]) / (X["bar_c_prev"] * X["F_prev"]) - 1
|
|
X["short"] = X["R_bt"] - X["R_real"]
|
|
X["yahoo_vs_bar"] = X["R_bt"] - X["R_bar"]
|
|
X["bar_vs_cross"] = X["R_bar"] - X["R_real"]
|
|
X["close_cross_vs_bar"] = X["close_p"] / X["bar_c_prev"] - 1 # same day, raw: + = paid more than the bar close
|
|
X["open_bar_vs_cross"] = X["bar_o_night"] / X["open_p"] - 1 # same day, raw: + = sold below the bar open
|
|
X["listing"] = np.where(X["open_x"].isin(["N"]) | X["close_x"].isin(["N"]), "NYSE",
|
|
np.where(X["open_x"].isin(["Q", "T"]) | X["close_x"].isin(["Q", "T"]), "Nasdaq", "other"))
|
|
ok = X[["R_bt", "R_real"]].notna().all(axis=1) & (X["short"].abs() < 0.05)
|
|
|
|
res = {"coverage": {}, "candidates": {}}
|
|
for k in ("sleeve_score", "overnight_12_1"):
|
|
g = X[X["candidate"] == k]
|
|
res["coverage"][k] = {"name_nights": int(len(g)), "no_close_cross": int(g["close_p"].isna().sum()),
|
|
"no_open_cross": int(g["open_p"].isna().sum()),
|
|
"no_yahoo_or_factor": int((g[["R_bt", "F_prev", "F_night"]].isna().any(axis=1)).sum()),
|
|
"outliers_over_5pct": int((g["short"].abs() >= 0.05).sum())}
|
|
print("coverage:", json.dumps(res["coverage"]))
|
|
for k in ("sleeve_score", "overnight_12_1"):
|
|
g = X[(X["candidate"] == k) & ok]
|
|
d = {c: night_stats(g[c], g["night"]) for c in ("short", "yahoo_vs_bar", "bar_vs_cross",
|
|
"close_cross_vs_bar", "open_bar_vs_cross")}
|
|
d["cost_per_side_bp"] = d["short"]["mean_bp"] / 2 + SEC_FEE_SIDE_BP
|
|
d["by_year"] = {int(y): night_stats(h["short"], h["night"]) for y, h in g.groupby(g["night"].dt.year)}
|
|
d["by_period"] = {lab: night_stats(h["short"], h["night"]) for lab, h in
|
|
(("2016-20", g[g["night"] < "2021-01-01"]), ("2021-26", g[g["night"] >= "2021-01-01"]))}
|
|
d["by_listing"] = {lab: night_stats(h["short"], h["night"]) for lab, h in g.groupby("listing")}
|
|
res["candidates"][k] = d
|
|
s = d["short"]
|
|
print(f"\n== {k}: {s['name_nights']} name-nights over {s['nights']} nights")
|
|
print(f" shortfall R_bt - R_real: mean {s['mean_bp']:+.2f} bp/night (SE {s['se_bp']:.2f}), median "
|
|
f"{s['median_bp']:+.2f}, trimmed {s['trimmed_mean_bp']:+.2f} -> cost per side {d['cost_per_side_bp']:+.2f} bp")
|
|
for c in ("yahoo_vs_bar", "bar_vs_cross", "close_cross_vs_bar", "open_bar_vs_cross"):
|
|
print(f" {c:20s} mean {d[c]['mean_bp']:+.2f} bp (SE {d[c]['se_bp']:.2f}) median {d[c]['median_bp']:+.2f}")
|
|
print(" by year:", " ".join(f"{y} {v['mean_bp']:+.1f}±{v['se_bp']:.1f}" for y, v in d["by_year"].items()))
|
|
print(" by period:", {a: round(v["mean_bp"], 2) for a, v in d["by_period"].items()},
|
|
" by listing:", {a: (round(v["mean_bp"], 2), v["name_nights"]) for a, v in d["by_listing"].items()})
|
|
cost = res["candidates"]["sleeve_score"]["cost_per_side_bp"]
|
|
verdict = "PASS" if cost <= PASS_BP else ("MEASURED" if cost <= STOP_BP else "STOP")
|
|
res["verdict"] = {"cost_per_side_bp": cost, "decision": verdict}
|
|
print(f"\nverdict: cost per side {cost:+.2f} bp -> {verdict}")
|
|
with open(res_path, "w") as fh:
|
|
json.dump(res, fh, indent=2)
|
|
X.to_csv(os.path.join(OUT, "name_nights.csv.gz"), index=False)
|
|
|
|
|
|
if __name__ == "__main__":
|
|
ap = argparse.ArgumentParser()
|
|
ap.add_argument("cmd", nargs="?", default="run", choices=["run", "fetch"])
|
|
if ap.parse_args().cmd == "fetch":
|
|
fetch()
|
|
else:
|
|
main()
|