Gahow Wang
dc14a541b0
After S&P 400 names failed inside the large-cap momentum top 10, test them as their own sleeve: top 30, monthly, buffer 1, 20 bp, judged by alpha vs IJH and by whether 1/3 of the book in the sleeve beats 1/3 in IJH. Four hypotheses and a control, definitions and rule written before the run. Holdout 2017-01..2026-09: mid-cap momentum -1.0%/yr vs IJH (corr 0.83 with the large-cap book), lottery-excluded momentum -1.7%, SUE earnings momentum -1.7%, quality-screened momentum +3.5% (t 1.28), quality composite +3.4% (t 1.23; book 0.91 vs 0.85 with IJH). Quality is positive in both halves but underpowered; nothing is adopted. fetch_us_fundamentals gains --universe mid: EDGAR facts for S&P 400 ever-members, CIKs accepted only when an SEC name matches the iShares holding name. fundamentals.load_mid builds the same factors on that panel. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
221 lines
11 KiB
Python
221 lines
11 KiB
Python
"""
|
|
A separate S&P 400 sleeve: does any of four preregistered signals earn
|
|
alpha over IJH and improve the book more than holding IJH would?
|
|
|
|
PYTHONPATH=. uv run python -m research.lab.midcap_sleeve --only h4
|
|
PYTHONPATH=. uv run python -m research.lab.midcap_sleeve --only h1 h2 h3 # needs us_mid_fund_raw.pkl
|
|
|
|
Background (research/lab/universe_midcap.py): ranking S&P 400 names inside
|
|
the large-cap momentum top 10 lost (Sharpe 0.86 vs 0.98, 2019-26); the S&P
|
|
400 winners were a few lottery tickets among many small losers. Here the
|
|
mid caps get their own book and their own benchmark.
|
|
|
|
Every sleeve: PIT S&P 400 members (research/lab/panel_mid.py, Tiingo-
|
|
extended), top 30, equal weight, rebalance every 21 sessions with buffer 1
|
|
(a holding stays while its rank is within 60), next-open fills, 20 bp a
|
|
side, averaged over the 21 rebalance phases. Definitions and the decision
|
|
rule live in PREREG and are written to disk before the first run; they
|
|
are textbook choices, not fitted (price coverage before 2017 is 46-78% and
|
|
XBRL starts in 2009-11, so there is no clean design period to fit on).
|
|
|
|
The book used for the portfolio test is the lab replay of
|
|
overnight_recovery_top10 (S&P 500, 20 bp) saved by universe_midcap.py.
|
|
|
|
Result (2026-10-08): all four REJECT. Holdout 2017-01..2026-09 (book alone
|
|
Sharpe 0.88; 2/3 book + 1/3 IJH 0.85):
|
|
|
|
sleeve CAGR Sharpe alpha/IJH (t) halves corr book book+sleeve
|
|
c0 momentum (control) 7.1% 0.46 -1.0% (-0.39) -0.9 / -1.0 0.83 0.80
|
|
h1 earnings momentum 7.8% 0.45 -1.7% (-0.74) +1.0 / -4.3 0.60 0.83
|
|
h2 quality-screened momentum 14.0% 0.65 +3.5% (1.28) +5.8 / +1.6 0.70 0.86
|
|
h3 quality composite 13.2% 0.67 +3.4% (1.23) +4.3 / +2.4 0.52 0.91
|
|
h4 lottery-excluded momentum 6.2% 0.46 -1.7% (-1.07) -0.1 / -3.8 0.71 0.83
|
|
EW members 11.3% 0.59 +1.0% (1.18)
|
|
IJH 10.1% 0.56
|
|
|
|
Price momentum among S&P 400 names loses to the index and moves with the
|
|
large-cap book (corr 0.83), so it diversifies nothing. Quality (h2, h3) is
|
|
the only family with positive alpha in both halves and the only one that
|
|
helps the book (h3: 0.91 vs 0.85, corr 0.52), but at ~3.5%/yr with ~10%
|
|
tracking error t is 1.2-1.3; t 2 would need about 25 years at this effect
|
|
size. Boundary: gp_a covers 55-70% of members (no gross profit for banks,
|
|
REITs), so h2's median screen also drops most financials. EDGAR CIKs are
|
|
name-checked against iShares holding names (723/845 tickers; members
|
|
covered 84-98% a year from 2017).
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import argparse
|
|
import json
|
|
import os
|
|
import warnings
|
|
|
|
import numpy as np
|
|
import pandas as pd
|
|
|
|
from research.lab.alpha import alpha_stats
|
|
from research.lab.factors import overnight_mom, rec_126
|
|
from research.lab.panel_mid import load_us_mid
|
|
from research.lab.us_final import monthly_blend
|
|
from research.lab.us_sweep import _blend, run
|
|
|
|
warnings.filterwarnings("ignore")
|
|
OUT = os.path.join("data", "research", "results", "midcap_sleeve")
|
|
BOOK = os.path.join("data", "research", "results", "universe_midcap", "returns.pkl")
|
|
HOLDOUT = ("2017-01-03", "2026-09-24")
|
|
HALVES = [("2017-01-03", "2021-12-31"), ("2022-01-03", "2026-09-24")]
|
|
REFERENCE = ("2010-01-04", "2016-12-30")
|
|
KW = dict(top_n=30, buffer=1.0, every=21)
|
|
PREREG = {
|
|
"family": "S&P 400 sleeve, 4 hypotheses + 1 control; all run once, results reported whatever they are",
|
|
"construction": "PIT S&P 400 members, top 30, equal weight, every 21 sessions, buffer 1, next-open fills, "
|
|
"20 bp a side, 21 phases averaged",
|
|
"signals": {
|
|
"c0_momentum": "control: mean pct rank of overnight_mom (252d sum of log open/prev close, skip 21) "
|
|
"and rec_126 (close / 126d low - 1) among members (the live overnight_recovery score)",
|
|
"h1_earnings_momentum": "mean pct rank of sue and sue_rev (research/lab/fundamentals.py: seasonal "
|
|
"change scaled by the std of the last 8, visible from the filing date)",
|
|
"h2_quality_screened_momentum": "c0 score, eligible only if roa (TTM net income / assets) > 0 and "
|
|
"gp_a >= the members' median that day",
|
|
"h3_quality_composite": "mean pct rank of gp_a, roa, accruals (-(NI-CFO)/assets), net_issuance "
|
|
"(-12m log change in shares)",
|
|
"h4_lottery_excluded_momentum": "c0 score, eligible only if MAX (largest daily return, last 21 "
|
|
"sessions) and idiosyncratic vol (63d, residual to IJH) are both "
|
|
"below the members' 80th percentile that day",
|
|
},
|
|
"holdout": HOLDOUT,
|
|
"accept_if_all": [
|
|
"A1: alpha vs IJH over the holdout with Newey-West t >= 2.0, or t >= 1.5 and alpha > 0 in both "
|
|
"2017-2021 and 2022-2026",
|
|
"A2: Sharpe(2/3 book + 1/3 sleeve) > Sharpe(2/3 book + 1/3 IJH), monthly reset, holdout",
|
|
"A3 (h2, h4 only): Sharpe(sleeve) > Sharpe(c0_momentum), holdout",
|
|
],
|
|
"on_accept": "forward-only validation book; no production change",
|
|
"multiple_testing": "4 hypotheses: a lone t just above 2 is weak evidence; report all of them",
|
|
"reference_only": f"{REFERENCE}: price coverage 57-78%, survivor-biased",
|
|
"status": "holdout overlaps periods seen in earlier mid/small scans; not independent OOS",
|
|
}
|
|
|
|
|
|
def lottery_ok(p, members: pd.DataFrame) -> pd.DataFrame:
|
|
"""True where MAX(21) and idiosyncratic vol(63) are both under the
|
|
members' 80th percentile. Idio variance = var(r) - beta^2 var(m)."""
|
|
r = p.close.pct_change(fill_method=None)
|
|
m = p.extra["etf"]["IJH"].pct_change(fill_method=None).reindex(p.dates)
|
|
mx = r.rolling(21, min_periods=15).max()
|
|
var_r = r.rolling(63, min_periods=40).var()
|
|
var_m = m.rolling(63, min_periods=40).var()
|
|
cov = r.rolling(63, min_periods=40).cov(m)
|
|
idio = np.sqrt((var_r - cov.pow(2).div(var_m, axis=0)).clip(lower=0))
|
|
rank = lambda x: x.where(members).rank(axis=1, pct=True)
|
|
return (rank(mx) <= 0.8) & (rank(idio) <= 0.8)
|
|
|
|
|
|
def score(factors: list[pd.DataFrame], eligible: pd.DataFrame) -> pd.DataFrame:
|
|
return _blend(*[f.where(eligible).rank(axis=1, pct=True) for f in factors]).where(eligible)
|
|
|
|
|
|
def sleeves(p, which: list[str]) -> dict[str, pd.DataFrame]:
|
|
mem = p.member
|
|
mom = [overnight_mom(p), rec_126(p)]
|
|
out = {"c0_momentum": score(mom, mem)}
|
|
if "h4" in which:
|
|
out["h4_lottery_excluded_momentum"] = score(mom, mem & lottery_ok(p, mem))
|
|
if {"h1", "h2", "h3"} & set(which):
|
|
from research.lab import fundamentals as fu
|
|
F = fu.load_mid(p)
|
|
if "h1" in which:
|
|
out["h1_earnings_momentum"] = score([F["sue"], F["sue_rev"]], mem)
|
|
if "h2" in which:
|
|
gp = F["gp_a"].where(mem)
|
|
ok = (F["roa"] > 0) & gp.ge(gp.median(axis=1), axis=0)
|
|
out["h2_quality_screened_momentum"] = score(mom, mem & ok)
|
|
if "h3" in which:
|
|
out["h3_quality_composite"] = score([F["gp_a"], F["roa"], F["accruals"], F["net_issuance"]], mem)
|
|
return out
|
|
|
|
|
|
def sharpe(r: pd.Series) -> float:
|
|
return float(r.mean() / r.std() * np.sqrt(252))
|
|
|
|
|
|
def describe(r: pd.Series, ijh: pd.Series, book: pd.Series) -> dict:
|
|
h = r.loc[HOLDOUT[0]:HOLDOUT[1]]
|
|
e = (1 + h).cumprod()
|
|
a = alpha_stats(h, ijh)
|
|
halves = [alpha_stats(r.loc[x:y], ijh)["alpha"] for x, y in HALVES]
|
|
R = pd.DataFrame({"book": book, "s": r, "ijh": ijh}).loc[HOLDOUT[0]:HOLDOUT[1]].dropna()
|
|
with_s = monthly_blend(R, {"book": 2 / 3, "s": 1 / 3})
|
|
with_i = monthly_blend(R, {"book": 2 / 3, "ijh": 1 / 3})
|
|
ref = r.loc[REFERENCE[0]:REFERENCE[1]]
|
|
return {"cagr": float(e.iloc[-1] ** (252 / len(h)) - 1), "sharpe": sharpe(h),
|
|
"mdd": float((e / e.cummax() - 1).min()),
|
|
"alpha_ijh": float(a["alpha"]), "alpha_t": float(a["t"]), "beta_ijh": float(a["beta"]),
|
|
"alpha_halves": [float(x) for x in halves],
|
|
"corr_book": float(R["s"].corr(R["book"])),
|
|
"book_sharpe_with_sleeve": sharpe(with_s), "book_sharpe_with_ijh": sharpe(with_i),
|
|
"reference_sharpe": sharpe(ref),
|
|
"years": {int(k): float(v) for k, v in ((1 + h).groupby(h.index.year).prod() - 1).items()}}
|
|
|
|
|
|
def verdict(name: str, d: dict, c0: dict) -> dict:
|
|
a1 = d["alpha_t"] >= 2.0 or (d["alpha_t"] >= 1.5 and min(d["alpha_halves"]) > 0)
|
|
a2 = d["book_sharpe_with_sleeve"] > d["book_sharpe_with_ijh"]
|
|
checks = {"A1": a1, "A2": a2}
|
|
if name.startswith(("h2", "h4")):
|
|
checks["A3"] = d["sharpe"] > c0["sharpe"]
|
|
return {"checks": {k: bool(v) for k, v in checks.items()},
|
|
"verdict": "ACCEPT" if all(checks.values()) else "REJECT"}
|
|
|
|
|
|
def main() -> None:
|
|
ap = argparse.ArgumentParser()
|
|
ap.add_argument("--only", nargs="+", default=["h1", "h2", "h3", "h4"])
|
|
which = ap.parse_args().only
|
|
os.makedirs(OUT, exist_ok=True)
|
|
pre = os.path.join(OUT, "prereg.json")
|
|
if not os.path.exists(pre):
|
|
with open(pre, "w") as fh:
|
|
json.dump(PREREG, fh, indent=2)
|
|
res_path = os.path.join(OUT, "results.json")
|
|
done = json.load(open(res_path)) if os.path.exists(res_path) else {}
|
|
clash = [k for k in done if k != "c0_momentum" and k.split("_")[0] in which]
|
|
if clash:
|
|
raise SystemExit(f"already run: {clash}; this study does not overwrite results")
|
|
|
|
p = load_us_mid("mid")
|
|
ijh = p.extra["etf"]["IJH"].pct_change(fill_method=None)
|
|
book = pd.read_pickle(BOOK)["base"]
|
|
S = sleeves(p, which)
|
|
span = [(REFERENCE[0], HOLDOUT[1])]
|
|
rets = {}
|
|
for name, sc in S.items():
|
|
rets[name] = run(p, sc, span, **KW)["_rets"]
|
|
print(f"ran {name}", flush=True)
|
|
ew = p.close.pct_change(fill_method=None).where(p.member.shift(1, fill_value=False)).mean(axis=1)
|
|
rets["bench_ew_members"] = ew
|
|
rets["bench_ijh"] = ijh
|
|
for name, r in rets.items():
|
|
d = describe(r.reindex(p.dates).fillna(0.0), ijh, book)
|
|
if name.startswith("h"):
|
|
d.update(verdict(name, d, done.get("c0_momentum") or describe(rets["c0_momentum"], ijh, book)))
|
|
done[name] = d
|
|
print(f"{name:32s} cagr {d['cagr']*100:5.1f}% sh {d['sharpe']:.2f} mdd {d['mdd']*100:6.1f}% "
|
|
f"alpha {d['alpha_ijh']*100:5.1f}% t {d['alpha_t']:.2f} halves "
|
|
f"{[round(x*100, 1) for x in d['alpha_halves']]} corr_book {d['corr_book']:.2f} "
|
|
f"book+sleeve {d['book_sharpe_with_sleeve']:.2f} vs book+IJH {d['book_sharpe_with_ijh']:.2f} "
|
|
f"ref {d['reference_sharpe']:.2f} {d.get('verdict', '')} {d.get('checks', '')}")
|
|
with open(res_path, "w") as fh:
|
|
json.dump(done, fh, indent=2)
|
|
pd.DataFrame(rets).to_pickle(os.path.join(OUT, f"returns_{'_'.join(which)}.pkl"))
|
|
from research.lab.trials import append_trials
|
|
append_trials(os.path.join("data", "research", "results", "trials_us.jsonl"),
|
|
[{"name": f"midcap_sleeve:{k}", **{x: y for x, y in v.items() if x != "years"}}
|
|
for k, v in done.items() if k in rets],
|
|
{"study": "midcap_sleeve", "role": "preregistered family; holdout partly seen"})
|
|
|
|
|
|
if __name__ == "__main__":
|
|
main()
|