gahow/quant-trading
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

research: daytime reversal of the overnight gap on S&P 500 (STOP)

Browse Source 
Buying the biggest overnight losers at 09:35 and selling at the close earns
1-6 bp a day gross on 2016-26 Alpaca 5-min bars; net Sharpe ~0, and stacking
it on the overnight sleeve lowers the Sharpe (1.27 -> ~0.8).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Gahow Wang
2026-10-09 22:07:26 -07:00
parent 4899bb9702
commit 6b24bedcbd
2 changed files with 226 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
CLAUDE.md
View File

@@ -102,6 +102,9 @@ uv run python -m research.lab.fetch_us_fundamentals --universe mid # EDGAR
uv run python -m research.lab.fetch_us_fundamentals --universe small # the same for S&P 600 (IJR)
PYTHONPATH=. uv run python -m research.lab.overnight_book # three-sleeve book with the overnight-only momentum sleeve, leverage for 50%/yr (preregistered; CONTINUE 2026-10-09: Sharpe 0.98 -> 1.38, MaxDD -37% -> -17% at 1 bp/side; 4 bp/side erases it)
PYTHONPATH=. uv run python -m research.lab.overnight_book_mf [--no-proxy] # the overnight books + a DBMF/CTA sleeve, levered to 50%/yr (preregistered; KEEP 2026-10-09: levered MaxDD unchanged, no CTA sleeve)
PYTHONPATH=. uv run python -m research.lab.daytime_reversal # buy the S&P 500's biggest overnight losers at 09:35, sell at the close (preregistered; STOP 2026-10-09: gross 1-6 bp/day, net Sharpe ~0)
uv run python -m research.lab.fetch_us_fundamentals --universe mid # EDGAR PIT facts for S&P 400 ever-members (name-checked CIKs)
uv run python -m research.lab.fetch_us_fundamentals --universe small # the same for S&P 600 (IJR)
PYTHONPATH=. uv run python -m research.lab.midcap_sleeve [--only h1..h4] [--universe small] # S&P 400 sleeves vs IJH; S&P 600 replication vs IJR (all REJECT 2026-10-08)
PYTHONPATH=. uv run python -m research.lab.fetch_shortvolume # FINRA REG SHO daily short volume (~8y archive, resume-safe)
PYTHONPATH=. uv run python -m research.lab.short_volume [us|midsmall|all] # SVR (short-volume ratio) IC / quintile scan on PIT panels

223
research/lab/daytime_reversal.py Normal file
View File

@@ -0,0 +1,223 @@
"""
Daytime reversal of the overnight gap, as the daytime half of the
overnight-only book: the overnight sleeve (research/lab/overnight_only.py)
holds stocks from the close to the open, so the same capital is idle from
09:30 to 16:00. Does buying the S&P 500 names that gapped down the most, at
09:35, and selling them at the close earn enough to stack on it?
PYTHONPATH=. uv run python -m research.lab.fetch_alpaca stocks # once: 5-min bars
PYTHONPATH=. uv run python -m research.lab.daytime_reversal
Background: Akbas, Boehmer, Jiang & Koch, "Overnight returns, daytime
reversals, and future stock returns" (JFE 2022); Lou, Polk & Skouras (JFE
2019). In the overnight study the momentum sleeve's names lost 0.8 bp a day
intraday.
Rules (Alpaca 5-min SIP bars, adjusted; regular session):
gap close of the 09:30-09:35 bar / previous session's last-bar close - 1
universe PIT S&P 500 members on the previous session (research panel),
with both bars present
entry the close of the 09:30 bar (09:35); exit at the session's last
bar close (MOC proxy). Early-close sessions (Alpaca calendar)
are skipped.
weights equal, 1x long (and 1x short for long_short); flat overnight
costs entry 2 bp (half spread at 09:35), exit 1 bp (closing auction);
stress 5 bp each side
candidates long_losers_10 the 10 lowest gaps, long
long_losers_z_10 the 10 lowest gap / sd of the last 20
daily close-to-close returns, long
long_short_10 long the 10 lowest, short the 10 highest gaps
Stacked (reported and gated): the overnight sleeve's night return (night
ending at the open of day t) compounded with the day-t candidate return:
(1 + night) x (1 + day) - 1, on the same capital.
Span 2016-02 .. 2026-09. All of it lies after the papers' samples end, but
the overnight sleeve was seen before this rule was written. The rule is
written to prereg.json before the run; the script refuses to overwrite
results.json.
Result (2026-10-09): STOP. No candidate passes any gate.
candidate gross bp/day gross Sharpe net Sharpe halves stress stacked (night alone 1.27)
long_losers_10 3.2 0.34 0.02 0.16 / -0.08 -0.70 0.86
long_losers_z_10 1.3 0.17 -0.22 0.02 / -0.40 -1.11 0.79
long_short_10 6.4 0.60 0.04 0.37 / -0.19 -1.28 0.83
- The equal-weight S&P 500 earns 1.5 bp/day from 09:35 to the close. The
losers' excess is about 2 bp, and 3 bp of round-trip cost removes it.
- The reversal is weaker in 2021-26 than in 2016-20.
- Stacking on the overnight sleeve lowers the Sharpe (1.27 -> 0.79-0.86).
The day leg adds variance with no mean: corr with the night leg is ~0,
and the long-only legs have corr 0.66-0.69 with the market's day move.
Interpretation: in S&P 500 names after 2016, the daytime reversal of the
overnight gap is too small to pay for trading it at 09:35. The idle daytime
capital stays idle. The daily summary cache (daily_from_5min.pkl) is reusable.
"""
from __future__ import annotations
import json
import os
import numpy as np
import pandas as pd
from research.lab.factor_scan import RESULTS, load
OUT = os.path.join(RESULTS, "daytime_reversal")
BARS = os.path.join("data", "research", "intraday", "alpaca", "5Min")
DAILY_CACHE = os.path.join("data", "research", "intraday", "alpaca", "daily_from_5min.pkl")
CAL_CACHE = os.path.join("data", "research", "intraday", "alpaca", "calendar.json")
OVERNIGHT = os.path.join(RESULTS, "overnight_only", "daily.pkl")
SPAN = ("2016-02-01", "2026-09-24")
HALVES = [("2016-02-01", "2020-12-31"), ("2021-01-01", "2026-09-24")]
TOP = 10
ENTRY_COST, EXIT_COST, STRESS = 0.0002, 0.0001, 0.0005
PREREG = {
"question": "Does buying the S&P 500's biggest overnight losers at 09:35 and selling at the close earn a "
"net Sharpe that is worth stacking on the overnight sleeve's idle daytime capital?",
"rules": "gap = 09:35 price / prior last-bar close - 1; PIT S&P 500; equal weight; entry 09:35 bar close, "
"exit last bar close; early closes skipped; costs 2 bp in + 1 bp out, stress 5 + 5",
"candidates": ["long_losers_10", "long_losers_z_10", "long_short_10"],
"span": SPAN, "halves": HALVES,
"gates": {"D1": "full-span net Sharpe >= 1.0",
"D2": "net Sharpe >= 0.7 in both halves",
"D3": "stress (5 bp a side) full-span Sharpe >= 0.5",
"D4": "stacked night+day Sharpe >= overnight sleeve alone + 0.2 (2016-02..2026-09)"},
"decision": {"CONTINUE": "a candidate passes D1-D4 -> add the daytime leg to the paper test "
"(overnight_paper.py) and measure 09:35 fills",
"STOP": "otherwise; the daytime capital stays idle"},
"multiplicity": "3 candidates",
"data_status": "post-sample for the papers; overnight sleeve inspected before; Alpaca SIP 2016+",
}
def calendar_early_closes() -> set[pd.Timestamp]:
if not os.path.exists(CAL_CACHE):
from overnight_paper import Alpaca
cal = Alpaca().get("/calendar", start="2016-01-01", end="2026-12-31")
with open(CAL_CACHE, "w") as fh:
json.dump(cal, fh)
cal = json.load(open(CAL_CACHE))
return {pd.Timestamp(c["date"]) for c in cal if c["close"] != "16:00"}
def daily_from_bars() -> dict[str, pd.DataFrame]:
"""Per symbol and session: 09:30-bar open/close, last-bar close (cached)."""
if os.path.exists(DAILY_CACHE):
return pd.read_pickle(DAILY_CACHE)
o930, c935, clast = {}, {}, {}
files = sorted(f for f in os.listdir(BARS) if f.endswith(".pkl"))
for i, f in enumerate(files, 1):
df = pd.read_pickle(os.path.join(BARS, f))
day = df.index.tz_localize(None).normalize()
first = df.groupby(day).head(1)
ok = (first.index.hour == 9) & (first.index.minute == 30)
first = first[ok]
idx = first.index.tz_localize(None).normalize()
s = f[:-4]
o930[s] = pd.Series(first["o"].values.astype(float), index=idx)
c935[s] = pd.Series(first["c"].values.astype(float), index=idx)
clast[s] = df["c"].groupby(day).last().astype(float)
if i % 400 == 0:
print(f" daily summary {i}/{len(files)}", flush=True)
out = {"o930": pd.DataFrame(o930), "c935": pd.DataFrame(c935), "clast": pd.DataFrame(clast)}
pd.to_pickle(out, DAILY_CACHE)
return out
def describe(r: pd.Series) -> dict:
r = r.dropna()
eq = (1 + r).cumprod()
yrs = len(r) / 252
return {"cagr": float(eq.iloc[-1] ** (1 / yrs) - 1), "vol": float(r.std() * np.sqrt(252)),
"sharpe": float(r.mean() / r.std() * np.sqrt(252)), "mdd": float((eq / eq.cummax() - 1).min()),
"mean_bp_per_day": float(r.mean() * 1e4)}
def main() -> None:
os.makedirs(OUT, exist_ok=True)
res_path = os.path.join(OUT, "results.json")
if os.path.exists(res_path):
raise SystemExit(f"{res_path} exists; this study does not overwrite results")
with open(os.path.join(OUT, "prereg.json"), "w") as fh:
json.dump(PREREG, fh, indent=2)
D = daily_from_bars()
early = calendar_early_closes()
p, _ = load("us")
cols = sorted(set(D["c935"].columns) & set(p.close.columns))
days = D["c935"].index.union(D["clast"].index).sort_values()
days = days[(days >= SPAN[0]) & (days <= SPAN[1])]
c935 = D["c935"].reindex(index=days, columns=cols)
clast_all = D["clast"].reindex(columns=cols).sort_index()
prev_last = clast_all.shift(1).reindex(days) # previous session's last close (per symbol rows)
clast = clast_all.reindex(days)
mem_prev = p.member.reindex(columns=cols).shift(1).reindex(days).fillna(False).astype(bool)
gap = (c935 / prev_last - 1).where(mem_prev & c935.notna() & clast.notna() & prev_last.notna())
gap = gap[~gap.index.isin(early)]
day_ret = (clast / c935 - 1).reindex(gap.index)
vol20 = p.close.reindex(columns=cols).pct_change(fill_method=None).rolling(20, min_periods=15).std() \
.shift(1).reindex(gap.index)
z = gap / vol20
def pick(score, n, low=True):
rk = score.rank(axis=1, ascending=low, method="first")
return rk <= n
legs = {}
lo = pick(gap, TOP)
hi = pick(gap, TOP, low=False)
loz = pick(z.where(gap.notna()), TOP)
gross = {
"long_losers_10": day_ret.where(lo).mean(axis=1),
"long_losers_z_10": day_ret.where(loz).mean(axis=1),
"long_short_10": day_ret.where(lo).mean(axis=1) - day_ret.where(hi).mean(axis=1),
}
sides = {"long_losers_10": 1, "long_losers_z_10": 1, "long_short_10": 2}
uni = day_ret.mean(axis=1)
night = pd.read_pickle(OVERNIGHT)["sleeve_score"].reindex(gap.index)
res = {"candidates": {}, "n_days": int(len(gap)), "early_closes_skipped": int(len(early))}
daily = {}
for k, g in gross.items():
net = g - sides[k] * (ENTRY_COST + EXIT_COST)
stress = g - sides[k] * 2 * STRESS
stacked = (1 + night.fillna(0.0)) * (1 + net.fillna(0.0)) - 1
daily[k] = net
d = {"gross": describe(g), "net": describe(net), "stress": describe(stress),
"halves": [describe(net.loc[a:b]) for a, b in HALVES],
"stacked": describe(stacked), "night_alone": describe(night.fillna(0.0)),
"corr_with_night": float(net.corr(night)),
"corr_with_universe_day": float(net.corr(uni)),
"yearly": {int(y): float(v) for y, v in ((1 + net).groupby(net.index.year).prod() - 1).items()}}
res["candidates"][k] = d
print(f"{k:17s} gross {d['gross']['mean_bp_per_day']:5.1f}bp/d sh {d['gross']['sharpe']:5.2f} | net cagr "
f"{d['net']['cagr']*100:6.1f}% sh {d['net']['sharpe']:5.2f} mdd {d['net']['mdd']*100:6.1f}% | halves "
f"{d['halves'][0]['sharpe']:5.2f} {d['halves'][1]['sharpe']:5.2f} | stress {d['stress']['sharpe']:5.2f} | "
f"stacked sh {d['stacked']['sharpe']:5.2f} (night {d['night_alone']['sharpe']:.2f}) corr night "
f"{d['corr_with_night']:.2f} mkt {d['corr_with_universe_day']:.2f}", flush=True)
print(" yearly net %:", {y: round(v * 100, 1) for y, v in d["yearly"].items()})
res["universe_day"] = describe(uni)
print(f"EW S&P 500 09:35->close: {res['universe_day']['mean_bp_per_day']:.1f} bp/d sh {res['universe_day']['sharpe']:.2f}")
checks = {}
for k, d in res["candidates"].items():
checks[k] = {"D1": d["net"]["sharpe"] >= 1.0, "D2": all(h["sharpe"] >= 0.7 for h in d["halves"]),
"D3": d["stress"]["sharpe"] >= 0.5,
"D4": d["stacked"]["sharpe"] >= d["night_alone"]["sharpe"] + 0.2}
passing = [k for k, c in checks.items() if all(c.values())]
decision = "CONTINUE" if passing else "STOP"
res["verdict"] = {"checks": checks, "passing": passing, "decision": decision}
print(f"checks {checks}\n-> {decision} {passing}")
with open(res_path, "w") as fh:
json.dump(res, fh, indent=2)
pd.DataFrame(daily).to_pickle(os.path.join(OUT, "daily.pkl"))
from research.lab.trials import append_trials
append_trials(os.path.join(RESULTS, "trials_us.jsonl"),
[{"name": f"daytime_reversal:{k}", **res["candidates"][k]["net"]} for k in gross],
{"study": "daytime_reversal", "role": "preregistered quick check"})
if __name__ == "__main__":
main()