""" Hold the momentum sleeve's names only overnight (buy at the close, sell at the next open) and stay flat during the day. Intraday trading is allowed since 2026-10-09, so an overnight-only book is tradable with market-on-close and market-on-open orders, and it leaves the capital free for an intraday strategy during the session. PYTHONPATH=. uv run python -m research.lab.overnight_only Background: Lou, Polk & Skouras, "A tug of war: Overnight versus intraday expected returns" (JFE 2019) find that momentum profits accrue overnight and partly reverse intraday. overnight_recovery_top10 ranks on 12-1 month overnight returns, so its own return may sit mostly in the overnight leg. Rules (one book per candidate, equal weight, rebuilt every night): night t from the close of t-1 to the open of t selection top 10 by the candidate score read at the close of t-2 (one day of slack so the MOC order can be sent before the close of t-1), PIT S&P 500 members on t-2 return open_t / close_{t-1} - 1 per name (Yahoo adjusted bars, so dividends and splits net out); a missing open falls back to close_t costs every night trades the whole book twice: c per side, baseline c = 1 bp (auction fills, $0 commission, SEC fee ~0.3 bp on sells), stress c = 3 bp cash earns nothing; Sharpe = mean / sd of the book's own returns Candidates: sleeve_score overnight_mom (12-1) + rec_126 percentile blend, the live sleeve's score overnight_12_1 overnight_mom alone overnight_1m last month's overnight return (LPS: overnight returns persist at short horizons) Reported only: the overnight / intraday split of the sleeve's names held for 10 days (the live cadence, no hysteresis), and the equal-weight member universe held overnight only. Open-price fidelity (N5): Yahoo opens against the 09:30 bar of the Alpaca 5-min SIP files on disk at run time (2016+), on all name-days and on the name-days the candidates select. If N5 fails the verdict is DATA. Every period here was examined before (the sleeve's score was chosen on this panel). 2019+ is after the LPS publication, not independent OOS. The rule is written to prereg.json before the run; the script refuses to overwrite results.json. Result (2026-10-09): STOP. No candidate reaches N1 (full-span net Sharpe 1.5). N5 passes: on 288 names, 2016+, Yahoo and Alpaca opens give overnight returns with corr 0.96 on all name-days and 0.996 on selected ones; the mean difference is 0.03-0.3 bp. candidate (net, 1 bp) CAGR Sharpe halves 2019+ 3 bp MaxDD sleeve_score 30.4% 1.29 1.34 / 1.24 1.17 0.84 -31% overnight_12_1 38.0% 1.42 1.59 / 1.24 1.16 1.02 -31% overnight_1m 25.9% 1.08 1.47 / 0.53 0.54 0.66 -33% Reported only: - The live sleeve's names held 10 days (gross) earn the whole return overnight: overnight leg 13.7 bp/day, Sharpe 1.57 (halves 1.62 / 1.50, 2019+ 1.43); intraday leg -0.8 bp/day, Sharpe -0.07; close to close Sharpe 0.87. - Nightly re-selection adds nothing over the 10-day list (13.5 vs 13.7 bp/day). The equal-weight universe earns 3.5 bp/day overnight (Sharpe 0.73). - Costs decide the result: 2 sides a night at 1 bp take 2 bp of the 13.5 bp/day gross. Breakeven is about 6.7 bp a side. """ from __future__ import annotations import json import os import numpy as np import pandas as pd from research.lab.factor_scan import RESULTS, load from research.lab.factors import overnight_mom, overnight_mom_1, rec_126 from research.lab.us_sweep import _blend OUT = os.path.join(RESULTS, "overnight_only") BARS = os.path.join("data", "research", "intraday", "alpaca", "5Min") SPAN = ("2006-10-02", "2026-09-24") HALVES = [("2006-10-02", "2016-12-30"), ("2017-01-03", "2026-09-24")] POST_PUB = ("2019-01-01", "2026-09-24") TOP_N, HOLD_DAYS = 10, 10 COST_BASE, COST_STRESS = 0.0001, 0.0003 PREREG = { "question": "Does holding the momentum sleeve's names only overnight (MOC buy, MOO sell) earn a net " "Sharpe high enough to be a stand-alone stream that frees intraday capital?", "rules": "top 10 equal weight by score at close t-2, held close t-1 -> open t, rebuilt nightly; " "costs 2 x c per night on the whole book, c = 1 bp base / 3 bp stress; no cash interest", "candidates": ["sleeve_score", "overnight_12_1", "overnight_1m"], "span": SPAN, "halves": HALVES, "post_publication": POST_PUB, "gates": {"N1": "full-span net Sharpe (1 bp) >= 1.5", "N2": "net Sharpe (1 bp) >= 1.2 in both halves", "N3": "net Sharpe (1 bp) >= 1.0 in 2019-01 .. 2026-09", "N4": "full-span net Sharpe at 3 bp/side >= 1.0", "N5": "open fidelity vs Alpaca 09:30 bar (2016+): corr of overnight returns >= 0.9 on all " "name-days and |mean diff| <= 3 bp on selected name-days"}, "decision": {"CONTINUE": "a candidate passes N1-N4 and N5 passes -> check MOC/MOO auction fills on Alpaca " "intraday data and the combination with an intraday stream on the same capital", "DATA": "N5 fails -> rebuild overnight returns from Alpaca opens before any verdict", "STOP": "otherwise"}, "multiplicity": "3 candidates; a lone pass by one is weaker evidence than the sleeve score passing", "data_status": "all periods previously examined; 2019+ is post-publication, not independent OOS", } def describe(r: pd.Series) -> dict: r = r.dropna() eq = (1 + r).cumprod() yrs = len(r) / 252 return {"cagr": float(eq.iloc[-1] ** (1 / yrs) - 1), "vol": float(r.std() * np.sqrt(252)), "sharpe": float(r.mean() / r.std() * np.sqrt(252)), "mdd": float((eq / eq.cummax() - 1).min()), "mean_bp_per_day": float(r.mean() * 1e4)} def legs(p) -> tuple[pd.DataFrame, pd.DataFrame]: """Per name-day overnight (open_t / close_{t-1}) and intraday (close_t / open_t) returns.""" c = p.close.ffill() o = p.open.where(p.open.notna(), p.close) return o / c.shift(1) - 1, p.close / o - 1 def top_mask(score: pd.DataFrame, n: int) -> pd.DataFrame: rk = score.rank(axis=1, ascending=False, method="first") return rk <= n def nightly(sel: pd.DataFrame, on: pd.DataFrame) -> pd.Series: """Equal-weight overnight return of the selected names (selection already aligned to night t).""" x = on.where(sel) return x.mean(axis=1).where(sel.sum(axis=1) > 0) def held_split(score: pd.DataFrame, on: pd.DataFrame, intra: pd.DataFrame) -> dict: """Sleeve names held 10 days (refresh at close t-1 with score through t-2), daily reset to equal weight, averaged over the 10 phases: overnight leg, intraday leg, close-to-close.""" rows = {"overnight": [], "intraday": [], "close_to_close": []} sel_daily = top_mask(score.shift(2), TOP_N) idx = np.arange(len(score)) for off in range(HOLD_DAYS): sel = sel_daily.astype(float) sel.loc[(idx % HOLD_DAYS) != off] = np.nan sel = sel.ffill().fillna(0.0).astype(bool) o = on.where(sel).mean(axis=1) i = intra.where(sel).mean(axis=1) cc = ((1 + on) * (1 + intra) - 1).where(sel).mean(axis=1) rows["overnight"].append(o); rows["intraday"].append(i); rows["close_to_close"].append(cc) return {k: pd.concat(v, axis=1).mean(axis=1) for k, v in rows.items()} def alpaca_opens(columns) -> tuple[pd.DataFrame, pd.DataFrame, int]: """Daily 09:30 bar open and last-bar close per symbol from the 5-min files on disk.""" opens, closes = {}, {} files = [f[:-4] for f in os.listdir(BARS) if f.endswith(".pkl")] if os.path.isdir(BARS) else [] for s in files: if s not in columns: continue df = pd.read_pickle(os.path.join(BARS, f"{s}.pkl")) day = df.index.tz_localize(None).normalize() first = df.groupby(day).head(1) first = first[(first.index.hour == 9) & (first.index.minute == 30)] opens[s] = pd.Series(first["o"].values.astype(float), index=first.index.tz_localize(None).normalize()) closes[s] = df["c"].groupby(day).last().astype(float) return pd.DataFrame(opens), pd.DataFrame(closes), len(opens) def fidelity(p, sels: dict[str, pd.DataFrame]) -> dict: ao, ac, n = alpaca_opens(set(p.close.columns)) if n == 0: return {"symbols": 0} cols = list(ao.columns) dates = p.dates[p.dates >= "2016-01-04"] ao, ac = ao.reindex(index=dates, columns=cols), ac.reindex(index=dates, columns=cols) # overnight with each source's own prior close (adjustment factors cancel within a source) on_y = (p.open / p.close.ffill().shift(1) - 1).reindex(index=dates, columns=cols) on_a = ao / ac.shift(1) - 1 mem = p.member.reindex(index=dates, columns=cols).fillna(False) ok = mem & on_y.notna() & on_a.notna() y, a = on_y.where(ok), on_a.where(ok) out = {"symbols": n, "name_days": int(ok.sum().sum()), "corr_all": float(np.corrcoef(y.stack().values, a.stack().values)[0, 1]), "mean_diff_all_bp": float((y - a).stack().mean() * 1e4), "median_abs_diff_all_bp": float((y - a).abs().stack().median() * 1e4)} for name, sel in sels.items(): s = sel.reindex(index=dates, columns=cols).fillna(False).astype(bool) & ok d = (y - a).where(s).stack() out[f"{name}_name_days"] = int(s.sum().sum()) out[f"{name}_mean_diff_bp"] = float(d.mean() * 1e4) if len(d) else float("nan") out[f"{name}_corr"] = (float(np.corrcoef(y.where(s).stack().values, a.where(s).stack().values)[0, 1]) if len(d) > 10 else float("nan")) return out def main() -> None: os.makedirs(OUT, exist_ok=True) res_path = os.path.join(OUT, "results.json") if os.path.exists(res_path): raise SystemExit(f"{res_path} exists; this study does not overwrite results") with open(os.path.join(OUT, "prereg.json"), "w") as fh: json.dump(PREREG, fh, indent=2) p, _ = load("us") mem = p.member on, intra = legs(p) scores = { "sleeve_score": _blend(*[f.where(mem).rank(axis=1, pct=True) for f in (overnight_mom(p), rec_126(p))]).where(mem), "overnight_12_1": overnight_mom(p).where(mem), "overnight_1m": overnight_mom_1(p).where(mem), } a, b = SPAN res, daily, sels = {"candidates": {}}, {}, {} for name, sc in scores.items(): sel = top_mask(sc.shift(2), TOP_N) sels[name] = sel gross = nightly(sel, on).loc[a:b] net, stress = gross - 2 * COST_BASE, gross - 2 * COST_STRESS clipped = nightly(sel, on.clip(-0.2, 0.2)).loc[a:b] - 2 * COST_BASE big = on.where(sel & (on.abs() > 0.10)).sum(axis=1).loc[a:b] / TOP_N daily[name] = net d = {"gross": describe(gross), "net": describe(net), "stress": describe(stress), "halves": [describe(net.loc[x:y]) for x, y in HALVES], "post_pub": describe(net.loc[POST_PUB[0]:]), "net_clipped_20pct": describe(clipped), "share_of_gross_from_moves_over_10pct": float(big.sum() / gross.sum()), "yearly": {int(k): float(v) for k, v in ((1 + net).groupby(net.index.year).prod() - 1).items()}} res["candidates"][name] = d print(f"{name:15s} gross {d['gross']['mean_bp_per_day']:5.1f}bp/d sh {d['gross']['sharpe']:5.2f} | net " f"cagr {d['net']['cagr']*100:5.1f}% vol {d['net']['vol']*100:4.1f}% sh {d['net']['sharpe']:5.2f} " f"mdd {d['net']['mdd']*100:6.1f}% | halves {d['halves'][0]['sharpe']:5.2f} {d['halves'][1]['sharpe']:5.2f}" f" | 2019+ {d['post_pub']['sharpe']:5.2f} | stress {d['stress']['sharpe']:5.2f} | clip " f"{d['net_clipped_20pct']['sharpe']:5.2f} | >10% moves {d['share_of_gross_from_moves_over_10pct']:.2f}", flush=True) print(" yearly net %:", {k: round(v * 100, 1) for k, v in d["yearly"].items()}) uni = on.where(mem.shift(1).fillna(False).astype(bool)).mean(axis=1).loc[a:b] res["universe_overnight_gross"] = {"full": describe(uni), "halves": [describe(uni.loc[x:y]) for x, y in HALVES], "post_pub": describe(uni.loc[POST_PUB[0]:])} u = res["universe_overnight_gross"] print(f"EW members overnight gross: {u['full']['mean_bp_per_day']:.1f}bp/d sh {u['full']['sharpe']:.2f} " f"halves {u['halves'][0]['sharpe']:.2f} {u['halves'][1]['sharpe']:.2f} 2019+ {u['post_pub']['sharpe']:.2f}") split = held_split(scores["sleeve_score"], on, intra) res["sleeve_held_10d_split_gross"] = {} for k, r in split.items(): r = r.loc[a:b] res["sleeve_held_10d_split_gross"][k] = {"full": describe(r), "halves": [describe(r.loc[x:y]) for x, y in HALVES], "post_pub": describe(r.loc[POST_PUB[0]:])} s = res["sleeve_held_10d_split_gross"][k] print(f"sleeve held 10d, {k:14s}: {s['full']['mean_bp_per_day']:5.1f}bp/d cagr {s['full']['cagr']*100:5.1f}% " f"sh {s['full']['sharpe']:5.2f} halves {s['halves'][0]['sharpe']:5.2f} {s['halves'][1]['sharpe']:5.2f} " f"2019+ {s['post_pub']['sharpe']:5.2f}") fid = fidelity(p, sels) res["fidelity"] = fid print("fidelity:", {k: (round(v, 3) if isinstance(v, float) else v) for k, v in fid.items()}) n5 = (fid.get("symbols", 0) > 0 and fid["corr_all"] >= 0.9 and all(abs(fid[f"{c}_mean_diff_bp"]) <= 3 for c in scores if fid.get(f"{c}_name_days", 0) > 0)) checks = {} for name, d in res["candidates"].items(): checks[name] = {"N1": d["net"]["sharpe"] >= 1.5, "N2": all(h["sharpe"] >= 1.2 for h in d["halves"]), "N3": d["post_pub"]["sharpe"] >= 1.0, "N4": d["stress"]["sharpe"] >= 1.0} passing = [n for n, c in checks.items() if all(c.values())] decision = "DATA" if not n5 else ("CONTINUE" if passing else "STOP") res["verdict"] = {"checks": checks, "N5": n5, "passing": passing, "decision": decision} print(f"checks {checks}\nN5 {n5} passing {passing} -> {decision}") with open(res_path, "w") as fh: json.dump(res, fh, indent=2) pd.DataFrame(daily).to_pickle(os.path.join(OUT, "daily.pkl")) from research.lab.trials import append_trials append_trials(os.path.join(RESULTS, "trials_us.jsonl"), [{"name": f"overnight_only:{n}", **res["candidates"][n]["net"]} for n in scores], {"study": "overnight_only", "role": "preregistered quick check"}) if __name__ == "__main__": main()