Gahow Wang
6b24bedcbd
Buying the biggest overnight losers at 09:35 and selling at the close earns 1-6 bp a day gross on 2016-26 Alpaca 5-min bars; net Sharpe ~0, and stacking it on the overnight sleeve lowers the Sharpe (1.27 -> ~0.8). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
224 lines
11 KiB
Python
224 lines
11 KiB
Python
"""
|
|
Daytime reversal of the overnight gap, as the daytime half of the
|
|
overnight-only book: the overnight sleeve (research/lab/overnight_only.py)
|
|
holds stocks from the close to the open, so the same capital is idle from
|
|
09:30 to 16:00. Does buying the S&P 500 names that gapped down the most, at
|
|
09:35, and selling them at the close earn enough to stack on it?
|
|
|
|
PYTHONPATH=. uv run python -m research.lab.fetch_alpaca stocks # once: 5-min bars
|
|
PYTHONPATH=. uv run python -m research.lab.daytime_reversal
|
|
|
|
Background: Akbas, Boehmer, Jiang & Koch, "Overnight returns, daytime
|
|
reversals, and future stock returns" (JFE 2022); Lou, Polk & Skouras (JFE
|
|
2019). In the overnight study the momentum sleeve's names lost 0.8 bp a day
|
|
intraday.
|
|
|
|
Rules (Alpaca 5-min SIP bars, adjusted; regular session):
|
|
gap close of the 09:30-09:35 bar / previous session's last-bar close - 1
|
|
universe PIT S&P 500 members on the previous session (research panel),
|
|
with both bars present
|
|
entry the close of the 09:30 bar (09:35); exit at the session's last
|
|
bar close (MOC proxy). Early-close sessions (Alpaca calendar)
|
|
are skipped.
|
|
weights equal, 1x long (and 1x short for long_short); flat overnight
|
|
costs entry 2 bp (half spread at 09:35), exit 1 bp (closing auction);
|
|
stress 5 bp each side
|
|
candidates long_losers_10 the 10 lowest gaps, long
|
|
long_losers_z_10 the 10 lowest gap / sd of the last 20
|
|
daily close-to-close returns, long
|
|
long_short_10 long the 10 lowest, short the 10 highest gaps
|
|
Stacked (reported and gated): the overnight sleeve's night return (night
|
|
ending at the open of day t) compounded with the day-t candidate return:
|
|
(1 + night) x (1 + day) - 1, on the same capital.
|
|
|
|
Span 2016-02 .. 2026-09. All of it lies after the papers' samples end, but
|
|
the overnight sleeve was seen before this rule was written. The rule is
|
|
written to prereg.json before the run; the script refuses to overwrite
|
|
results.json.
|
|
|
|
Result (2026-10-09): STOP. No candidate passes any gate.
|
|
|
|
candidate gross bp/day gross Sharpe net Sharpe halves stress stacked (night alone 1.27)
|
|
long_losers_10 3.2 0.34 0.02 0.16 / -0.08 -0.70 0.86
|
|
long_losers_z_10 1.3 0.17 -0.22 0.02 / -0.40 -1.11 0.79
|
|
long_short_10 6.4 0.60 0.04 0.37 / -0.19 -1.28 0.83
|
|
|
|
- The equal-weight S&P 500 earns 1.5 bp/day from 09:35 to the close. The
|
|
losers' excess is about 2 bp, and 3 bp of round-trip cost removes it.
|
|
- The reversal is weaker in 2021-26 than in 2016-20.
|
|
- Stacking on the overnight sleeve lowers the Sharpe (1.27 -> 0.79-0.86).
|
|
The day leg adds variance with no mean: corr with the night leg is ~0,
|
|
and the long-only legs have corr 0.66-0.69 with the market's day move.
|
|
Interpretation: in S&P 500 names after 2016, the daytime reversal of the
|
|
overnight gap is too small to pay for trading it at 09:35. The idle daytime
|
|
capital stays idle. The daily summary cache (daily_from_5min.pkl) is reusable.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import json
|
|
import os
|
|
|
|
import numpy as np
|
|
import pandas as pd
|
|
|
|
from research.lab.factor_scan import RESULTS, load
|
|
|
|
OUT = os.path.join(RESULTS, "daytime_reversal")
|
|
BARS = os.path.join("data", "research", "intraday", "alpaca", "5Min")
|
|
DAILY_CACHE = os.path.join("data", "research", "intraday", "alpaca", "daily_from_5min.pkl")
|
|
CAL_CACHE = os.path.join("data", "research", "intraday", "alpaca", "calendar.json")
|
|
OVERNIGHT = os.path.join(RESULTS, "overnight_only", "daily.pkl")
|
|
SPAN = ("2016-02-01", "2026-09-24")
|
|
HALVES = [("2016-02-01", "2020-12-31"), ("2021-01-01", "2026-09-24")]
|
|
TOP = 10
|
|
ENTRY_COST, EXIT_COST, STRESS = 0.0002, 0.0001, 0.0005
|
|
|
|
PREREG = {
|
|
"question": "Does buying the S&P 500's biggest overnight losers at 09:35 and selling at the close earn a "
|
|
"net Sharpe that is worth stacking on the overnight sleeve's idle daytime capital?",
|
|
"rules": "gap = 09:35 price / prior last-bar close - 1; PIT S&P 500; equal weight; entry 09:35 bar close, "
|
|
"exit last bar close; early closes skipped; costs 2 bp in + 1 bp out, stress 5 + 5",
|
|
"candidates": ["long_losers_10", "long_losers_z_10", "long_short_10"],
|
|
"span": SPAN, "halves": HALVES,
|
|
"gates": {"D1": "full-span net Sharpe >= 1.0",
|
|
"D2": "net Sharpe >= 0.7 in both halves",
|
|
"D3": "stress (5 bp a side) full-span Sharpe >= 0.5",
|
|
"D4": "stacked night+day Sharpe >= overnight sleeve alone + 0.2 (2016-02..2026-09)"},
|
|
"decision": {"CONTINUE": "a candidate passes D1-D4 -> add the daytime leg to the paper test "
|
|
"(overnight_paper.py) and measure 09:35 fills",
|
|
"STOP": "otherwise; the daytime capital stays idle"},
|
|
"multiplicity": "3 candidates",
|
|
"data_status": "post-sample for the papers; overnight sleeve inspected before; Alpaca SIP 2016+",
|
|
}
|
|
|
|
|
|
def calendar_early_closes() -> set[pd.Timestamp]:
|
|
if not os.path.exists(CAL_CACHE):
|
|
from overnight_paper import Alpaca
|
|
cal = Alpaca().get("/calendar", start="2016-01-01", end="2026-12-31")
|
|
with open(CAL_CACHE, "w") as fh:
|
|
json.dump(cal, fh)
|
|
cal = json.load(open(CAL_CACHE))
|
|
return {pd.Timestamp(c["date"]) for c in cal if c["close"] != "16:00"}
|
|
|
|
|
|
def daily_from_bars() -> dict[str, pd.DataFrame]:
|
|
"""Per symbol and session: 09:30-bar open/close, last-bar close (cached)."""
|
|
if os.path.exists(DAILY_CACHE):
|
|
return pd.read_pickle(DAILY_CACHE)
|
|
o930, c935, clast = {}, {}, {}
|
|
files = sorted(f for f in os.listdir(BARS) if f.endswith(".pkl"))
|
|
for i, f in enumerate(files, 1):
|
|
df = pd.read_pickle(os.path.join(BARS, f))
|
|
day = df.index.tz_localize(None).normalize()
|
|
first = df.groupby(day).head(1)
|
|
ok = (first.index.hour == 9) & (first.index.minute == 30)
|
|
first = first[ok]
|
|
idx = first.index.tz_localize(None).normalize()
|
|
s = f[:-4]
|
|
o930[s] = pd.Series(first["o"].values.astype(float), index=idx)
|
|
c935[s] = pd.Series(first["c"].values.astype(float), index=idx)
|
|
clast[s] = df["c"].groupby(day).last().astype(float)
|
|
if i % 400 == 0:
|
|
print(f" daily summary {i}/{len(files)}", flush=True)
|
|
out = {"o930": pd.DataFrame(o930), "c935": pd.DataFrame(c935), "clast": pd.DataFrame(clast)}
|
|
pd.to_pickle(out, DAILY_CACHE)
|
|
return out
|
|
|
|
|
|
def describe(r: pd.Series) -> dict:
|
|
r = r.dropna()
|
|
eq = (1 + r).cumprod()
|
|
yrs = len(r) / 252
|
|
return {"cagr": float(eq.iloc[-1] ** (1 / yrs) - 1), "vol": float(r.std() * np.sqrt(252)),
|
|
"sharpe": float(r.mean() / r.std() * np.sqrt(252)), "mdd": float((eq / eq.cummax() - 1).min()),
|
|
"mean_bp_per_day": float(r.mean() * 1e4)}
|
|
|
|
|
|
def main() -> None:
|
|
os.makedirs(OUT, exist_ok=True)
|
|
res_path = os.path.join(OUT, "results.json")
|
|
if os.path.exists(res_path):
|
|
raise SystemExit(f"{res_path} exists; this study does not overwrite results")
|
|
with open(os.path.join(OUT, "prereg.json"), "w") as fh:
|
|
json.dump(PREREG, fh, indent=2)
|
|
|
|
D = daily_from_bars()
|
|
early = calendar_early_closes()
|
|
p, _ = load("us")
|
|
cols = sorted(set(D["c935"].columns) & set(p.close.columns))
|
|
days = D["c935"].index.union(D["clast"].index).sort_values()
|
|
days = days[(days >= SPAN[0]) & (days <= SPAN[1])]
|
|
c935 = D["c935"].reindex(index=days, columns=cols)
|
|
clast_all = D["clast"].reindex(columns=cols).sort_index()
|
|
prev_last = clast_all.shift(1).reindex(days) # previous session's last close (per symbol rows)
|
|
clast = clast_all.reindex(days)
|
|
mem_prev = p.member.reindex(columns=cols).shift(1).reindex(days).fillna(False).astype(bool)
|
|
gap = (c935 / prev_last - 1).where(mem_prev & c935.notna() & clast.notna() & prev_last.notna())
|
|
gap = gap[~gap.index.isin(early)]
|
|
day_ret = (clast / c935 - 1).reindex(gap.index)
|
|
vol20 = p.close.reindex(columns=cols).pct_change(fill_method=None).rolling(20, min_periods=15).std() \
|
|
.shift(1).reindex(gap.index)
|
|
z = gap / vol20
|
|
|
|
def pick(score, n, low=True):
|
|
rk = score.rank(axis=1, ascending=low, method="first")
|
|
return rk <= n
|
|
|
|
legs = {}
|
|
lo = pick(gap, TOP)
|
|
hi = pick(gap, TOP, low=False)
|
|
loz = pick(z.where(gap.notna()), TOP)
|
|
gross = {
|
|
"long_losers_10": day_ret.where(lo).mean(axis=1),
|
|
"long_losers_z_10": day_ret.where(loz).mean(axis=1),
|
|
"long_short_10": day_ret.where(lo).mean(axis=1) - day_ret.where(hi).mean(axis=1),
|
|
}
|
|
sides = {"long_losers_10": 1, "long_losers_z_10": 1, "long_short_10": 2}
|
|
uni = day_ret.mean(axis=1)
|
|
night = pd.read_pickle(OVERNIGHT)["sleeve_score"].reindex(gap.index)
|
|
res = {"candidates": {}, "n_days": int(len(gap)), "early_closes_skipped": int(len(early))}
|
|
daily = {}
|
|
for k, g in gross.items():
|
|
net = g - sides[k] * (ENTRY_COST + EXIT_COST)
|
|
stress = g - sides[k] * 2 * STRESS
|
|
stacked = (1 + night.fillna(0.0)) * (1 + net.fillna(0.0)) - 1
|
|
daily[k] = net
|
|
d = {"gross": describe(g), "net": describe(net), "stress": describe(stress),
|
|
"halves": [describe(net.loc[a:b]) for a, b in HALVES],
|
|
"stacked": describe(stacked), "night_alone": describe(night.fillna(0.0)),
|
|
"corr_with_night": float(net.corr(night)),
|
|
"corr_with_universe_day": float(net.corr(uni)),
|
|
"yearly": {int(y): float(v) for y, v in ((1 + net).groupby(net.index.year).prod() - 1).items()}}
|
|
res["candidates"][k] = d
|
|
print(f"{k:17s} gross {d['gross']['mean_bp_per_day']:5.1f}bp/d sh {d['gross']['sharpe']:5.2f} | net cagr "
|
|
f"{d['net']['cagr']*100:6.1f}% sh {d['net']['sharpe']:5.2f} mdd {d['net']['mdd']*100:6.1f}% | halves "
|
|
f"{d['halves'][0]['sharpe']:5.2f} {d['halves'][1]['sharpe']:5.2f} | stress {d['stress']['sharpe']:5.2f} | "
|
|
f"stacked sh {d['stacked']['sharpe']:5.2f} (night {d['night_alone']['sharpe']:.2f}) corr night "
|
|
f"{d['corr_with_night']:.2f} mkt {d['corr_with_universe_day']:.2f}", flush=True)
|
|
print(" yearly net %:", {y: round(v * 100, 1) for y, v in d["yearly"].items()})
|
|
res["universe_day"] = describe(uni)
|
|
print(f"EW S&P 500 09:35->close: {res['universe_day']['mean_bp_per_day']:.1f} bp/d sh {res['universe_day']['sharpe']:.2f}")
|
|
|
|
checks = {}
|
|
for k, d in res["candidates"].items():
|
|
checks[k] = {"D1": d["net"]["sharpe"] >= 1.0, "D2": all(h["sharpe"] >= 0.7 for h in d["halves"]),
|
|
"D3": d["stress"]["sharpe"] >= 0.5,
|
|
"D4": d["stacked"]["sharpe"] >= d["night_alone"]["sharpe"] + 0.2}
|
|
passing = [k for k, c in checks.items() if all(c.values())]
|
|
decision = "CONTINUE" if passing else "STOP"
|
|
res["verdict"] = {"checks": checks, "passing": passing, "decision": decision}
|
|
print(f"checks {checks}\n-> {decision} {passing}")
|
|
with open(res_path, "w") as fh:
|
|
json.dump(res, fh, indent=2)
|
|
pd.DataFrame(daily).to_pickle(os.path.join(OUT, "daily.pkl"))
|
|
from research.lab.trials import append_trials
|
|
append_trials(os.path.join(RESULTS, "trials_us.jsonl"),
|
|
[{"name": f"daytime_reversal:{k}", **res["candidates"][k]["net"]} for k in gross],
|
|
{"study": "daytime_reversal", "role": "preregistered quick check"})
|
|
|
|
|
|
if __name__ == "__main__":
|
|
main()
|