dip_buying: SPY/QQQ RSI(2) dip is real after publication (SPY 2008-26 46 bp/trade, t 3.4) but in the market 14% of days, so 3x earns the same Sharpe as SPY; trend-filtered stock RSI(2) breaks even at 10 bp/side and lowers the three-sleeve book (1.14 -> 0.95); stocks far below their high rebound only in market-wide crashes. dip_leverage_overlay: +1x SPY inside the dip windows on the overnight_book books leaves Sharpe flat or lower (0.98 -> 0.99, 1.38 -> 1.33); the gain ranks 77-88% against trend-matched random windows. The books already earn the bounce at about 2x SPY's beta. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
494 lines
24 KiB
Python
494 lines
24 KiB
Python
"""
|
|
Quick check: does buying dips pay? The repo's reversal tests so far were
|
|
cross-sectional (rev_5/rev_21/residual/sector reversal top 20, the hedged
|
|
5-day S&P 400/600 reversal, the 09:35 gap-down buy). This one tests the two
|
|
dip-buying rules the project has not tried, plus descriptive tables of what
|
|
follows a drawdown.
|
|
|
|
PYTHONPATH=. uv run --with numba python -m research.lab.dip_buying
|
|
|
|
A. Index dip (Connors & Alvarez 2008, "RSI(2)"), SPY (gated) and QQQ (reported):
|
|
enter close > 200-day mean and RSI(2) < 10 (Wilder smoothing)
|
|
exit the first close above the 5-day mean
|
|
fills next_open (gated): act at the open after the signal close;
|
|
close (reported): act at the signal close, i.e. an MOC order sent
|
|
with a 15:40 estimate of the signal
|
|
otherwise T-bills (FRED DTB3)
|
|
costs 1 bp a side (SPY/QQQ/UPRO spreads); 3 bp stress reported
|
|
3x in-trade return 3 r - 2 rf - 0.9%/yr (a UPRO-like fund)
|
|
pre-publication check (reported): ^GSPC 1962-1993, close fills, no costs
|
|
|
|
B. Stock dip with a trend filter, PIT S&P 500, $5+:
|
|
enter close > 200-day mean and RSI(2) < 5; the lowest RSI(2) first,
|
|
10 slots of 10% of equity
|
|
exit the first close above the 5-day mean, 10 sessions held, or the
|
|
name leaves the index; next_open fills (gated) / close (reported)
|
|
costs 10 bp a side (the lab convention); breakeven reported
|
|
also the repo's MeanReversionStrategy rule on the lab engine (12-month
|
|
return > 0, the 20 biggest 1-month losers, monthly), reported only
|
|
2006-2013 of this panel is survivor-biased (CLAUDE.md), which flatters a
|
|
dip buyer most of all: the gate uses 2014-2026.
|
|
|
|
C. Descriptive, no gate: forward returns by drawdown from the 252-day high,
|
|
for ^GSPC (1928-2026) and ^NDX (1986-2026), and for PIT S&P 500 stocks
|
|
(excess over the member mean, entry at the next open).
|
|
|
|
The rule is written to prereg.json before the run; the script refuses to
|
|
overwrite results.json.
|
|
|
|
Result (2026-10-12): STOP. C1 and C2 pass; C3, C4 and C5 fail.
|
|
- Index RSI(2) is real but small. It is invested 14% of days, about 8 trades a year.
|
|
SPY next_open trade bp t win 1x cagr/xSh 3x cagr/xSh/MaxDD SPY cagr/xSh
|
|
1994-2007 73 4.7 78% 9.6% / 0.81 20.3% / 0.82 / -23% 10.4% / 0.43
|
|
2008-2026 46 3.4 72% 4.9% / 0.56 10.9% / 0.57 / -41% 11.3% / 0.57
|
|
2017-2026 39 2.4 74% 5.6% / 0.50 10.9% / 0.51 / -33% 15.3% / 0.74
|
|
Close fills: 2008-26 53 bp (t 4.1), 3x 13.2% / 0.67. QQQ is similar
|
|
(46 bp, t 3.4; 3x 0.49 vs QQQ 0.72). ^GSPC 1962-1993: 3.7 bp, t 0.36,
|
|
so the effect is absent before the 1990s.
|
|
- Reported only: always 1x, switched to 3x inside the dip trades. SPY
|
|
2008-26: 17.5% / 0.69 / -52%, vs 11.3% / 0.57 / -52% for SPY. 2017-26:
|
|
21.0% / 0.78 vs 15.3% / 0.74. QQQ 2008-26: 22.5% / 0.75 vs 16.3% / 0.72.
|
|
- Stock RSI(2), 2014-26, next_open: -0.2%/yr, xSh -0.07. There were 5730
|
|
trades with a 60% win rate and +0.8 bp net each; breakeven is 10.4 bp a
|
|
side. With close fills: 2.1% / 0.09, breakeven 13 bp. 2006-13 with close
|
|
fills (survivor-biased): 15.7% / 0.58. Adding it as a fourth sleeve
|
|
lowers the book from 1.14 to 0.95. MeanReversionStrategy (monthly):
|
|
2014-26 7.8% / 0.38, 2006-13 -1.5%, corr 0.89 with SPY.
|
|
- Drawdown tables:
|
|
- ^GSPC 1988-2026, 252-day forward return: 10.3% unconditional, 4.4%
|
|
from 10-20% off the high, 10.8% from 20-30% off, and 27.9% from more
|
|
than 30% off. The last bucket is 11 episodes from about 3 bear markets.
|
|
- 1928-2026, more than 30% off: 3.6% mean, p5 -61%.
|
|
- NDX more than 30% off: 7.7% vs 17.4% unconditional.
|
|
- Stocks: results.json averages each bucket over days, and every bucket
|
|
comes out negative, which is an aggregation artifact. Pooled stock-days
|
|
(stock_dd_pooled.json; stock_dd_table now reports pooled_excess): names more than 30% off their high beat the
|
|
member mean by 0.7% over 63 days. That is +3.0% in 2008-09 and 2020
|
|
and -0.5% on all other days.
|
|
Interpretation: a short-term index dip is followed by a bounce that is still
|
|
measurable after publication. Because the rule is in the market so rarely,
|
|
it is worth something only as a leverage timer, not as a sleeve. A stock
|
|
dip earns ~20 bp gross per 4-day trade, about what trading it costs. A
|
|
stock that is deep in its own drawdown rebounds only when the whole market
|
|
crashed with it. Otherwise it keeps lagging, which is momentum.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import json
|
|
import os
|
|
import pickle
|
|
import warnings
|
|
|
|
import numpy as np
|
|
import pandas as pd
|
|
|
|
from research.lab.factor_scan import RESULTS, load
|
|
from research.lab.levered_book import stats, tbill
|
|
from research.lab.us_final import monthly_blend
|
|
|
|
warnings.filterwarnings("ignore")
|
|
|
|
OUT = os.path.join(RESULTS, "dip_buying")
|
|
END = "2026-09-24"
|
|
FEE3 = 0.009
|
|
IDX_COST, IDX_STRESS, STK_COST = 0.0001, 0.0003, 0.001
|
|
IDX_PERIODS = {"1994-2007": ("1994-01-01", "2007-12-31"), "2008-2026": ("2008-01-01", END),
|
|
"2017-2026": ("2017-01-01", END)}
|
|
STK_SPAN = ("2006-10-02", END)
|
|
STK_GATE = ("2014-01-02", END)
|
|
BUCKETS = [(0.0, -0.05), (-0.05, -0.10), (-0.10, -0.20), (-0.20, -0.30), (-0.30, -1.01)]
|
|
|
|
PREREG = {
|
|
"question": "Does buying short-term dips (index RSI(2) or trend-filtered stock RSI(2)) earn a "
|
|
"return worth trading or adding to the three-sleeve book?",
|
|
"continue_if": {
|
|
"index (all of)": [
|
|
"C1 SPY rsi2 next_open 2008-2026: mean net trade return > 0 with t >= 2.0",
|
|
"C2 SPY rsi2 next_open 2017-2026: mean net trade return > 0 with t >= 1.5",
|
|
"C3 SPY rsi2 next_open 3x 2008-2026: excess Sharpe >= 0.6 and above SPY buy-and-hold",
|
|
],
|
|
"stock (all of)": [
|
|
"C4 stock rsi2 next_open 2014-2026: excess Sharpe >= 0.8 and above the EW S&P 500 member book",
|
|
"C5 2014-2026: momentum/lowrisk/trend + stock rsi2 at 1/4 each raises the three-sleeve "
|
|
"book's excess Sharpe by >= 0.05",
|
|
],
|
|
},
|
|
"decision": {
|
|
"CONTINUE (index)": "preregister an index-dip leverage overlay; signals from 15:40 Alpaca bars",
|
|
"CONTINUE (stock)": "register the stock dip sleeve in the web engine and replay it",
|
|
"STOP": "drop dip buying; the descriptive tables stay as the answer",
|
|
},
|
|
}
|
|
|
|
|
|
def rsi(c, n: int = 2):
|
|
d = c.diff()
|
|
up = d.clip(lower=0).ewm(alpha=1 / n, adjust=False).mean()
|
|
dn = (-d).clip(lower=0).ewm(alpha=1 / n, adjust=False).mean()
|
|
return 100 - 100 / (1 + up / dn)
|
|
|
|
|
|
def tstat(x) -> float:
|
|
x = np.asarray(x, float)
|
|
return float(x.mean() / x.std(ddof=1) * np.sqrt(len(x))) if len(x) > 2 else float("nan")
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# A. index dip
|
|
# ---------------------------------------------------------------------------
|
|
|
|
def index_rule(c: pd.Series, o: pd.Series, th: float = 10.0, fill: str = "next_open"):
|
|
"""Per-day in-market return (NaN when flat), trade flags and the trade list."""
|
|
enter = ((c > c.rolling(200).mean()) & (rsi(c) < th)).to_numpy()
|
|
leave = (c > c.rolling(5).mean()).to_numpy()
|
|
cv, ov = c.to_numpy(), o.to_numpy()
|
|
n = len(c)
|
|
m = np.full(n, np.nan)
|
|
traded = np.zeros(n)
|
|
trades, held, pend, t0, legs = [], False, None, None, []
|
|
for i in range(1, n):
|
|
if fill == "next_open":
|
|
if pend == "buy":
|
|
held, t0, legs = True, i, [cv[i] / ov[i] - 1]
|
|
m[i], traded[i] = legs[0], 1
|
|
elif pend == "sell":
|
|
m[i], traded[i] = ov[i] / cv[i - 1] - 1, 1
|
|
legs.append(m[i])
|
|
trades.append((c.index[t0], c.index[i], float(np.prod(1 + np.array(legs)) - 1), i - t0))
|
|
held = False
|
|
elif held:
|
|
m[i] = cv[i] / cv[i - 1] - 1
|
|
legs.append(m[i])
|
|
pend = None
|
|
if held and leave[i]:
|
|
pend = "sell"
|
|
elif not held and enter[i]:
|
|
pend = "buy"
|
|
else:
|
|
if held:
|
|
m[i] = cv[i] / cv[i - 1] - 1
|
|
legs.append(m[i])
|
|
if leave[i]:
|
|
traded[i] = 1
|
|
trades.append((c.index[t0], c.index[i], float(np.prod(1 + np.array(legs)) - 1), i - t0))
|
|
held = False
|
|
continue
|
|
if not held and enter[i]:
|
|
held, t0, legs, traded[i] = True, i, [], 1
|
|
return pd.Series(m, c.index), pd.Series(traded, c.index), trades
|
|
|
|
|
|
def wrap(m: pd.Series, traded: pd.Series, rf: pd.Series, lev: float, cost: float) -> pd.Series:
|
|
rfd = rf / 252
|
|
fee = FEE3 / 252 if lev > 1 else 0.0
|
|
inside = lev * m - (lev - 1) * rfd - fee
|
|
return inside.where(m.notna(), rfd) - cost * traded
|
|
|
|
|
|
def trade_stats(trades, a, b, cost) -> dict:
|
|
r = np.array([x[2] - 2 * cost for x in trades if pd.Timestamp(a) <= x[0] <= pd.Timestamp(b)])
|
|
days = [x[3] for x in trades if pd.Timestamp(a) <= x[0] <= pd.Timestamp(b)]
|
|
if len(r) < 3:
|
|
return {"n": int(len(r))}
|
|
return {"n": int(len(r)), "win": float((r > 0).mean()), "mean_bp": float(r.mean() * 1e4),
|
|
"t": tstat(r), "avg_days": float(np.mean(days)), "per_year": float(len(r) / ((pd.Timestamp(b) - pd.Timestamp(a)).days / 365.25))}
|
|
|
|
|
|
def brief(r: pd.Series, rf: pd.Series) -> dict:
|
|
s = stats(r, rf)
|
|
return {k: float(s[k]) for k in ("cagr", "sharpe", "vol", "mdd", "worst_yr")}
|
|
|
|
|
|
def part_a(L: dict) -> dict:
|
|
out = {}
|
|
for tk in ("SPY", "QQQ"):
|
|
c = L["close"][tk].dropna().loc[:END]
|
|
o = L["open"][tk].reindex(c.index)
|
|
rf = tbill(c.index).fillna(0.0)
|
|
r_bh = c.pct_change().fillna(0.0)
|
|
bh3 = 3 * r_bh - 2 * rf / 252 - FEE3 / 252
|
|
for fill in ("next_open", "close"):
|
|
m, tr, trades = index_rule(c, o, fill=fill)
|
|
key = f"{tk}_rsi2_{fill}"
|
|
rows = {}
|
|
for per, (a, b) in IDX_PERIODS.items():
|
|
if c.index[0] > pd.Timestamp(a) - pd.Timedelta(days=300):
|
|
continue
|
|
sl = slice(a, b)
|
|
rows[per] = {
|
|
"trades": trade_stats(trades, a, b, IDX_COST),
|
|
"time_in": float(m.loc[sl].notna().mean()),
|
|
"x1": brief(wrap(m, tr, rf, 1, IDX_COST).loc[sl], rf),
|
|
"x3": brief(wrap(m, tr, rf, 3, IDX_COST).loc[sl], rf),
|
|
"x3_stress": brief(wrap(m, tr, rf, 3, IDX_STRESS).loc[sl], rf),
|
|
"bh": brief(r_bh.loc[sl], rf), "bh3": brief(bh3.loc[sl], rf),
|
|
# always 1x, 3x inside dips (switch SPY -> UPRO): reported only
|
|
"bh_plus_dip3": brief((r_bh + (2 * (m - rf / 252) - FEE3 / 252).where(m.notna(), 0.0)
|
|
- IDX_COST * tr).loc[sl], rf),
|
|
}
|
|
out[key] = rows
|
|
if tk == "SPY" and fill == "next_open":
|
|
out["_spy_dip_x1"] = wrap(m, tr, rf, 1, IDX_COST)
|
|
out["_spy_dip_x3"] = wrap(m, tr, rf, 3, IDX_COST)
|
|
g = L["close"]["^GSPC"].dropna().loc[:"1993-12-31"]
|
|
m, tr, trades = index_rule(g, g, fill="close")
|
|
rf = tbill(g.index).fillna(0.0)
|
|
out["GSPC_rsi2_close_1962-1993"] = {"trades": trade_stats(trades, "1962-01-01", "1993-12-31", 0.0),
|
|
"time_in": float(m.loc["1962":].notna().mean())}
|
|
return out
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# B. stock dip
|
|
# ---------------------------------------------------------------------------
|
|
|
|
def stock_rule(p, fill: str = "next_open", slots: int = 10, th: float = 5.0, max_hold: int = 10,
|
|
cost: float = STK_COST):
|
|
cf = p.close.ffill(limit=5)
|
|
enter = p.member & p.close.notna() & (p.close >= 5) & (cf > cf.rolling(200, min_periods=200).mean()) \
|
|
& (rsi(cf) < th)
|
|
score = rsi(cf).where(enter).to_numpy()
|
|
leave = (cf > cf.rolling(5).mean()).to_numpy()
|
|
C, O, M = p.close.ffill().to_numpy(), p.open.to_numpy(), p.member.to_numpy()
|
|
raw_c = p.close.to_numpy()
|
|
cash_r = p.cash_ret.reindex(p.dates).fillna(0.0).to_numpy()
|
|
n = len(p.dates)
|
|
cash, pos = 1.0, {} # j -> [shares, entry index, entry value]
|
|
eq = np.ones(n)
|
|
pend_sell, pend_buy, trades = set(), [], []
|
|
stale = {}
|
|
|
|
def sell(j, px, i):
|
|
nonlocal cash
|
|
sh, t0, v0 = pos.pop(j)
|
|
cash += sh * px * (1 - cost)
|
|
trades.append((p.dates[t0], i - t0, sh * px * (1 - cost) / v0 - 1))
|
|
|
|
def buy(j, px, budget, i):
|
|
nonlocal cash
|
|
amt = min(budget, cash)
|
|
if amt < 0.5 * budget or not np.isfinite(px) or px <= 0:
|
|
return
|
|
cash -= amt
|
|
pos[j] = [amt / (px * (1 + cost)), i, amt]
|
|
|
|
for i in range(1, n):
|
|
cash *= 1 + cash_r[i]
|
|
if fill == "next_open":
|
|
for j in list(pend_sell):
|
|
if np.isfinite(O[i, j]):
|
|
sell(j, O[i, j], i)
|
|
pend_sell.discard(j)
|
|
budget = eq[i - 1] / slots
|
|
for j in pend_buy:
|
|
if j not in pos and len(pos) < slots:
|
|
buy(j, O[i, j], budget, i)
|
|
pend_buy = []
|
|
for j in list(pos): # settle names with no print for 5 sessions
|
|
stale[j] = 0 if np.isfinite(raw_c[i, j]) else stale.get(j, 0) + 1
|
|
if stale[j] >= 5:
|
|
sell(j, C[i, j], i)
|
|
pend_sell.discard(j)
|
|
val = cash + sum(sh * C[i, j] for j, (sh, _, _) in pos.items())
|
|
exits = [j for j, (_, t0, _) in pos.items()
|
|
if leave[i, j] or i - t0 >= max_hold or not M[i, j]]
|
|
if fill == "close":
|
|
for j in exits:
|
|
if np.isfinite(raw_c[i, j]):
|
|
sell(j, raw_c[i, j], i)
|
|
else:
|
|
pend_sell.update(exits)
|
|
free = slots - (len(pos) - len(pend_sell & set(pos)))
|
|
row = score[i]
|
|
cand = np.where(np.isfinite(row))[0]
|
|
cand = [j for j in cand[np.argsort(row[cand])] if j not in pos][:max(free, 0)]
|
|
if fill == "close":
|
|
for j in cand:
|
|
buy(j, raw_c[i, j], val / slots, i)
|
|
else:
|
|
pend_buy = cand
|
|
eq[i] = cash + sum(sh * C[i, j] for j, (sh, _, _) in pos.items())
|
|
eqs = pd.Series(eq, p.dates)
|
|
return eqs.pct_change().fillna(0.0), trades
|
|
|
|
|
|
def part_b(p, sleeves: pd.DataFrame) -> dict:
|
|
from research.lab.us_sweep import run
|
|
sp = p.slice("2005-06-01", END)
|
|
rf = tbill(sp.dates).fillna(0.0)
|
|
out, R = {}, {}
|
|
for fill in ("next_open", "close"):
|
|
r, trades = stock_rule(sp, fill=fill)
|
|
R[f"stock_rsi2_{fill}"] = r.loc[STK_SPAN[0]:]
|
|
rows = {}
|
|
for per, (a, b) in {"2006-2013": (STK_SPAN[0], "2013-12-31"), "2014-2026": STK_GATE}.items():
|
|
ts = [x for x in trades if pd.Timestamp(a) <= x[0] <= pd.Timestamp(b)]
|
|
net = np.array([x[2] for x in ts])
|
|
gross = net + 2 * STK_COST
|
|
rows[per] = {**brief(r.loc[a:b], rf), "n_trades": len(ts), "win": float((net > 0).mean()),
|
|
"mean_net_bp": float(net.mean() * 1e4), "t": tstat(net),
|
|
"breakeven_bp_side": float(gross.mean() / 2 * 1e4),
|
|
"avg_days": float(np.mean([x[1] for x in ts]))}
|
|
out[f"stock_rsi2_{fill}"] = rows
|
|
c = sp.close.ffill(limit=5)
|
|
mr = (-(c / c.shift(21) - 1)).where((c / c.shift(252) - 1 > 0) & sp.member)
|
|
st = run(sp, mr, [(STK_SPAN[0], "2013-12-31"), STK_GATE], top_n=20, buffer=0.0, every=21)
|
|
R["mean_reversion_monthly"] = st["_rets"]
|
|
out["mean_reversion_monthly"] = {"2006-2013": brief(st["_rets"].loc[:"2013-12-31"], rf),
|
|
"2014-2026": brief(st["_rets"].loc[STK_GATE[0]:], rf),
|
|
"turn": float(st["turn"])}
|
|
R = pd.DataFrame(R)
|
|
S = sleeves.join(R, how="left")
|
|
gate = S.loc[STK_GATE[0]:STK_GATE[1]]
|
|
rfg = tbill(gate.index).fillna(0.0)
|
|
out["ew_sp500_2014-2026"] = brief(gate["ew_sp500"], rfg)
|
|
out["spy_2014-2026"] = brief(gate["spy"], rfg)
|
|
base = gate[["momentum", "lowrisk", "trend"]].fillna(0.0)
|
|
three = monthly_blend(base, {"momentum": 1 / 3, "lowrisk": 1 / 3, "trend": 1 / 3})
|
|
out["book3_2014-2026"] = brief(three, rfg)
|
|
mon = lambda x: (1 + x).resample("ME").prod() - 1
|
|
for k in R.columns:
|
|
four = monthly_blend(gate[["momentum", "lowrisk", "trend", k]].fillna(0.0),
|
|
{"momentum": .25, "lowrisk": .25, "trend": .25, k: .25})
|
|
out[f"book4_{k}_2014-2026"] = brief(four, rfg)
|
|
out[f"corr_{k}"] = {s: float(mon(gate[k]).corr(mon(gate[s]))) for s in ("momentum", "lowrisk", "trend", "spy")}
|
|
out["_R"] = R
|
|
return out
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# C. descriptive drawdown tables
|
|
# ---------------------------------------------------------------------------
|
|
|
|
def bucket_of(dd: pd.DataFrame | pd.Series):
|
|
lab = dd * np.nan
|
|
for k, (hi, lo) in enumerate(BUCKETS):
|
|
lab = lab.mask((dd <= hi) & (dd > lo), k)
|
|
return lab
|
|
|
|
|
|
def index_dd_table(c: pd.Series, a: str, b: str) -> list[dict]:
|
|
c = c.dropna()
|
|
dd = c / c.rolling(252, min_periods=200).max() - 1
|
|
lab = bucket_of(dd).loc[a:b]
|
|
rows = []
|
|
for k, (hi, lo) in enumerate(BUCKETS + [(None, None)]):
|
|
sel = lab.notna() if hi is None else lab == k
|
|
row = {"bucket": "all" if hi is None else f"{-hi:.0%}..{-lo if lo > -1 else 1:.0%}",
|
|
"share_days": float(sel.mean()),
|
|
"episodes": int((sel & ~sel.shift(1, fill_value=False)).sum())}
|
|
for h in (21, 63, 252):
|
|
f = (c.shift(-h) / c - 1).loc[a:b][sel].dropna()
|
|
row[f"fwd{h}"] = {"mean": float(f.mean()), "median": float(f.median()), "p_up": float((f > 0).mean()),
|
|
"p5": float(f.quantile(0.05))} if len(f) else {}
|
|
rows.append(row)
|
|
return rows
|
|
|
|
|
|
def stock_dd_table(p, a: str, b: str) -> list[dict]:
|
|
c = p.close.ffill(limit=5)
|
|
dd = c / c.rolling(252, min_periods=200).max() - 1
|
|
lab = bucket_of(dd).where(p.member & (p.close >= 5))
|
|
later = p.close.ffill() # a delisted name keeps its last print
|
|
entry = p.open.shift(-1)
|
|
rows = []
|
|
for h in (21, 63):
|
|
fwd = later.shift(-h) / entry - 1
|
|
fwd = fwd.where(lab.notna())
|
|
ex = fwd.sub(fwd.mean(axis=1), axis=0)
|
|
for k, (hi, lo) in enumerate(BUCKETS):
|
|
daily = ex.where(lab == k).mean(axis=1).loc[a:b].dropna()
|
|
spaced = daily.iloc[::h] # non-overlapping for the t-stat
|
|
rows.append({"h": h, "bucket": f"{-hi:.0%}..{-lo if lo > -1 else 1:.0%}",
|
|
"share": float((lab == k).loc[a:b].sum().sum() / lab.loc[a:b].notna().sum().sum()),
|
|
"excess_mean": float(daily.mean()), "t_nonoverlap": tstat(spaced),
|
|
"pooled_excess": float(ex.where(lab == k).loc[a:b].stack().mean()),
|
|
"p_beats_mean": float((ex.where(lab == k).loc[a:b].stack() > 0).mean())})
|
|
return rows
|
|
|
|
|
|
def main() -> None:
|
|
os.makedirs(OUT, exist_ok=True)
|
|
res_path = os.path.join(OUT, "results.json")
|
|
if os.path.exists(res_path):
|
|
raise SystemExit(f"{res_path} exists; this study does not overwrite results")
|
|
with open(os.path.join(OUT, "prereg.json"), "w") as fh:
|
|
json.dump(PREREG, fh, indent=2)
|
|
|
|
with open(os.path.join("data", "research", "long_history.pkl"), "rb") as fh:
|
|
L = pickle.load(fh)
|
|
res = {"index": part_a(L)}
|
|
spy_dip = pd.DataFrame({"spy_dip_x1": res["index"].pop("_spy_dip_x1"),
|
|
"spy_dip_x3": res["index"].pop("_spy_dip_x3")})
|
|
for k, rows in res["index"].items():
|
|
if k.startswith("_"):
|
|
continue
|
|
if "trades" in rows:
|
|
print(f"{k}: {rows['trades']} time in {rows['time_in']:.1%}")
|
|
continue
|
|
for per, r in rows.items():
|
|
t = r["trades"]
|
|
print(f"{k:22s} {per}: n {t['n']:3d} ({t.get('per_year', 0):.1f}/yr) win {t.get('win', 0):.0%} "
|
|
f"{t.get('mean_bp', 0):6.1f} bp/trade t {t.get('t', 0):4.2f} days {t.get('avg_days', 0):.1f} "
|
|
f"in {r['time_in']:.0%}")
|
|
for s in ("x1", "x3", "x3_stress", "bh", "bh3", "bh_plus_dip3"):
|
|
x = r[s]
|
|
print(f" {s:13s} cagr {x['cagr']*100:5.1f}% xsh {x['sharpe']:5.2f} vol {x['vol']*100:4.1f}% "
|
|
f"mdd {x['mdd']*100:6.1f}% worst yr {x['worst_yr']*100:6.1f}%")
|
|
|
|
res["index_dd"] = {"GSPC_1928-2026": index_dd_table(L["close"]["^GSPC"], "1928-12-31", END),
|
|
"GSPC_1988-2026": index_dd_table(L["close"]["^GSPC"], "1988-01-01", END),
|
|
"NDX_1986-2026": index_dd_table(L["close"]["^NDX"], "1986-10-01", END)}
|
|
for k, rows in res["index_dd"].items():
|
|
print(f"\n{k}: forward return by drawdown from the 252-day high")
|
|
for r in rows:
|
|
f = " ".join(f"{h}d {r[f'fwd{h}']['mean']*100:6.1f}%/{r[f'fwd{h}']['median']*100:5.1f}% "
|
|
f"up {r[f'fwd{h}']['p_up']:.0%} p5 {r[f'fwd{h}']['p5']*100:6.1f}%"
|
|
for h in (21, 63, 252) if r[f"fwd{h}"])
|
|
print(f" {r['bucket']:>9s} days {r['share_days']:5.1%} ep {r['episodes']:4d} | {f}")
|
|
|
|
p, _ = load("us")
|
|
sleeves = pd.read_pickle(os.path.join(RESULTS, "us_final_sleeves.pkl"))
|
|
b = part_b(p, sleeves)
|
|
R = b.pop("_R")
|
|
res["stock"] = b
|
|
for k, v in b.items():
|
|
print(f"{k}: {json.dumps(v, default=float)}")
|
|
res["stock_dd"] = {"2006-2026": stock_dd_table(p, STK_SPAN[0], END), "2014-2026": stock_dd_table(p, *STK_GATE)}
|
|
for k, rows in res["stock_dd"].items():
|
|
print(f"\nstocks {k}: excess return over the member mean by drawdown from the 252-day high")
|
|
for r in rows:
|
|
print(f" h{r['h']:2d} {r['bucket']:>9s} share {r['share']:5.1%} excess {r['excess_mean']*100:6.2f}% "
|
|
f"t {r['t_nonoverlap']:5.2f} beats mean {r['p_beats_mean']:.0%}")
|
|
|
|
spy = res["index"]["SPY_rsi2_next_open"]
|
|
gate = b["stock_rsi2_next_open"]["2014-2026"]
|
|
checks = {
|
|
"C1": spy["2008-2026"]["trades"]["mean_bp"] > 0 and spy["2008-2026"]["trades"]["t"] >= 2.0,
|
|
"C2": spy["2017-2026"]["trades"]["mean_bp"] > 0 and spy["2017-2026"]["trades"]["t"] >= 1.5,
|
|
"C3": spy["2008-2026"]["x3"]["sharpe"] >= 0.6 and spy["2008-2026"]["x3"]["sharpe"] > spy["2008-2026"]["bh"]["sharpe"],
|
|
"C4": gate["sharpe"] >= 0.8 and gate["sharpe"] > b["ew_sp500_2014-2026"]["sharpe"],
|
|
"C5": b["book4_stock_rsi2_next_open_2014-2026"]["sharpe"] >= b["book3_2014-2026"]["sharpe"] + 0.05,
|
|
}
|
|
idx_ok = checks["C1"] and checks["C2"] and checks["C3"]
|
|
stk_ok = checks["C4"] and checks["C5"]
|
|
decision = "CONTINUE (index)" if idx_ok else ""
|
|
decision += (" + " if decision and stk_ok else "") + ("CONTINUE (stock)" if stk_ok else "")
|
|
res["verdict"] = {"checks": checks, "decision": decision or "STOP"}
|
|
print(f"\nchecks {checks} -> {res['verdict']['decision']}")
|
|
pd.concat([R, spy_dip], axis=1).to_pickle(os.path.join(OUT, "returns.pkl"))
|
|
with open(res_path, "w") as fh:
|
|
json.dump(res, fh, indent=2, default=float)
|
|
from research.lab.trials import append_trials
|
|
rows = [{"name": f"dip_buying:{k}:{per}:{s}", **v[per][s]} for k, v in res["index"].items()
|
|
if "trades" not in v for per in v for s in ("x1", "x3")]
|
|
rows += [{"name": f"dip_buying:{k}:{per}", **{kk: vv for kk, vv in v[per].items()}}
|
|
for k, v in b.items() if k.startswith("stock_rsi2") for per in v]
|
|
append_trials(os.path.join(RESULTS, "trials_us.jsonl"), rows,
|
|
{"study": "dip_buying", "role": "preregistered quick check"})
|
|
|
|
|
|
if __name__ == "__main__":
|
|
main()
|