""" A risk-budgeted book of low-correlation return streams: does weighting five streams by risk, then levering the book to a volatility target, beat the best single stream and the live three-sleeve book in both halves of 2006-2026? PYTHONPATH=. uv run python -m research.lab.fetch_long_history # once PYTHONPATH=. uv run python -m research.lab.letf_history # once (synthetic 3x ETFs) PYTHONPATH=. uv run --with numba python -m research.lab.stream_portfolio Streams (daily returns, each a separate cash account, costs inside): momentum overnight_recovery_top10 on the PIT S&P 500 panel (Tiingo overlay), top 10, every 10 days, buffer 2, lab engine (10 bp a side), phase-averaged lowrisk low idiosyncratic vol + low beta, top 20, monthly, buffer 2 trend TrendRider V10 (registry trend_rider_v10) on synthetic 3x ETFs (research/lab/letf_history.py), weights traded one session late ("lag", the pessimistic fill), $10k, 5 bp, $2/order xtrend multi-asset time-series momentum on ETFs (Moskowitz, Ooi & Pedersen 2012): SPY EFA EEM VNQ TLT IEF GLD DBC, long while the 252-day return is positive, each sized to 10% vol (EWMA, 30-day half-life, at most 3x), gross capped at 1x, rest in T-bills, 10 bp per unit traded btc long BTC while above its 20-day average, else T-bills (the median rule of research/lab/diversifiers.py's grid); exists from 2015-06 Weighting (stream shares reset at each month start, drift inside the month, 10 bp on the capital moved between streams; a stream enters once it has 63 days of returns): equal 1/N of live streams invvol share proportional to 1 / (126-day vol of the stream) erc equal risk contribution on the 252-day covariance Leverage: levered_book.managed on the blended book, EWMA-20 vol, cap 2x (Reg-T), borrow at T-bill + 1.5%, 10 bp per unit of exposure changed. The decision rule is written to prereg.json before the streams are built; the script refuses to run when results.json exists. Both halves were inspected in earlier studies (TrendRider V9/V10 knobs were tuned on 2010-2026), so neither is independent out-of-sample data. Result (2026-10-08): REJECT. A2 and A3 pass, A1 fails. Excess Sharpe, 2006-10..2026-09, halves 2006-16 / 2017-26: | book / stream | CAGR | Sharpe | halves | MaxDD | worst yr | |---------------------------------|-------|--------|-----------|-------|----------| | invvol5 (primary, 1x) | 12.6% | 1.07 | 0.82/1.38 | -18% | -5.5% | | invvol5 vt25 (avg 1.8x) | 19.6% | 0.98 | 0.74/1.30 | -32% | -16% | | erc5 | 13.3% | 1.12 | 0.82/1.49 | -17% | | | equal5 (reported only) | 22.2% | 1.23 | 0.80/1.70 | -31% | -15% | | three_sleeve_v7 (live analog) | 18.4% | 0.97 | 0.70/1.33 | -38% | -21% | | three_sleeve_v10 | 20.5% | 1.02 | 0.68/1.45 | -39% | | | trend (V10, synthetic, lag) | 30.7% | 1.01 | 0.46/1.65 | -54% | -38% | | momentum | 20.3% | 0.69 | 0.59/0.81 | -69% | -55% | | btc (2015-06-) | 64.9% | 1.25 | 1.91/1.15 | -59% | | | xtrend | 4.8% | 0.39 | 0.37/0.42 | -20% | | A1 fails in both readings. The first half's best stream is btc over only 2015-06..2016-12, a weak yardstick. Without btc, lowrisk (0.67) sets the bar and invvol5 (0.82) clears it. In the second half, trend (1.65) beats invvol5 (1.38) either way. Interpretation: - Monthly correlations are low: btc 0.09-0.17, trend/momentum 0.30, xtrend 0.27-0.41. Diversification halves the drawdown (2008: -5% vs -21%). - Inverse-vol puts 72% in the two weakest streams, xtrend (0.39) and lowrisk (0.54), so the Sharpe gain over three_sleeve is only about +0.1. - Levering the book to 25% vol costs more than that gain: Sharpe 1.07 at 1x vs 0.98 levered, down to the three_sleeve vt25 level of 0.96-0.99. - Exploration afterwards (not preregistered): with futures-like financing (T-bill + 0.4% instead of + 1.5%), invvol5 vt25 is 1.03-1.04. So financing explains about 0.05; the rest is vol timing, which does not pay. - equal5's edge rests on btc (11% share; invvol4 without btc is 0.92) and on V10's tuned 2017-26. - TrendRider V10 before 2017 is a 0.46-Sharpe strategy (1990-2026 on synthetic ETFs: 15.9% / 0.65 / -62%). Decision: keep the current sleeves; no stream book is registered. No weighting of these streams reaches Sharpe 1.3 over 2006-26, so "stable 50%" needs better streams, not a better weighting. """ from __future__ import annotations import json import os import pickle import warnings import numpy as np import pandas as pd from research.lab.diversifiers import btc_trend from research.lab.factor_scan import RESULTS, load from research.lab.factors import ivol, low_beta, overnight_mom, rec_126 from research.lab.levered_book import managed, stats, tbill from research.lab.letf_history import run as run_letf from research.lab.us_sweep import _blend, run warnings.filterwarnings("ignore") OUT = os.path.join(RESULTS, "stream_portfolio") SPAN = ("2006-10-02", "2026-09-24") HALVES = [("2006-10-02", "2016-12-30"), ("2017-01-03", "2026-09-24")] BLOCKS = [("2006-10-02", "2009-12-31"), ("2010-01-01", "2014-12-31"), ("2015-01-01", "2019-12-31"), ("2020-01-01", "2026-09-24")] XTREND = ["SPY", "EFA", "EEM", "VNQ", "TLT", "IEF", "GLD", "DBC"] COST = 0.001 MIN_OBS = 63 PREREG = { "question": "Does a risk-budgeted book of five return streams beat its best single stream and " "the live three-sleeve book in both halves, with a tolerable drawdown when levered?", "streams": ["momentum", "lowrisk", "trend (TrendRider V10, synthetic ETFs, lag fills)", "xtrend (8-ETF TSMOM 252d, 10% vol each, gross <= 1)", "btc (sma20, 2015-06-)"], "span": SPAN, "halves": HALVES, "primary": "invvol5: inverse-vol (126d) shares of the five streams, monthly; levered to a 25% " "vol target (EWMA-20, cap 2x) for the drawdown check", "sharpe": "excess over the 3-month T-bill, daily, annualized", "accept_if_all": [ "A1 invvol5 (1x) Sharpe >= the highest single-stream Sharpe in each half", "A2 invvol5 (1x) Sharpe >= three_sleeve_v7 (1/3 momentum, lowrisk, TrendRider V7 vt24 " "synthetic lag; the live us_three_sleeve) in each half", "A3 invvol5 at vt25 full-span MaxDD >= -35%", ], "reported_not_decisive": ["equal5", "erc5", "invvol4 (no btc: spot ETFs only from 2024)", "invvol5 with qqq_trend or TrendRider V7 vt24 in place of V10", "three_sleeve_v10", "vt20 / vt30", "block Sharpes, yearly returns"], "decision": {"ACCEPT": "build the book as a registered sleeve portfolio and forward-test it next " "to us_three_sleeve and trend_rider_v10", "REJECT": "keep the current sleeves; report which check failed"}, } # --------------------------------------------------------------------------- # streams # --------------------------------------------------------------------------- def _score(factors: list[pd.DataFrame], mem: pd.DataFrame) -> pd.DataFrame: return _blend(*[f.where(mem).rank(axis=1, pct=True) for f in factors]).where(mem) def stock_streams() -> dict[str, pd.Series]: p, _ = load("us") mem = p.member out = {} for name, facs, kw in (("momentum", [overnight_mom(p), rec_126(p)], dict(top_n=10, buffer=2.0, every=10)), ("lowrisk", [ivol(p), low_beta(p)], dict(top_n=20, buffer=2.0, every=21))): out[name] = run(p, _score(facs, mem), [SPAN], **kw)["_rets"] print(f"ran {name}", flush=True) out["spy"] = p.benchmark.ffill().pct_change(fill_method=None).reindex(p.dates) return out def tsmom(close: pd.DataFrame, rf: pd.Series, assets: list[str], look: int = 252, target: float = 0.10) -> pd.Series: close = close[assets].ffill() r = close.pct_change(fill_method=None) on = (close / close.shift(look) - 1 > 0).astype(float) vol = r.ewm(halflife=30, min_periods=60).std() * np.sqrt(252) W = (on * (target / vol).clip(upper=3.0)).shift(1).fillna(0.0) W = W.div(W.sum(axis=1).clip(lower=1.0), axis=0) cash = 1 - W.sum(axis=1) return (W * r.fillna(0.0)).sum(axis=1) + cash * rf.reindex(W.index) / 252 \ - W.diff().abs().sum(axis=1) * COST def etf_streams(days: pd.DatetimeIndex, rf: pd.Series) -> dict[str, pd.Series]: from trader import STRATEGY_REGISTRY with open(os.path.join("data", "research", "letf_history.pkl"), "rb") as fh: syn = pickle.load(fh)["close"].loc["1989-01-01":] out = {} for name, key in (("trend", "trend_rider_v10"), ("trend_v7", "trend_rider_v7_vt24")): out[name] = run_letf(STRATEGY_REGISTRY[key](), syn, "lag").pct_change() print(f"ran {name}", flush=True) with open(os.path.join("data", "research", "long_history.pkl"), "rb") as fh: src = pickle.load(fh) c = src["close"] out["xtrend"] = tsmom(c.loc["2002-01-01":], rf, XTREND).loc["2004-01-01":] btc = c["BTC-USD"].reindex(c["SPY"].dropna().loc["2014-12-01":].index, method="ffill").dropna() out["btc"] = btc_trend(btc, rf, "sma", 20)[0].loc["2015-06-01":] with open(os.path.join(RESULTS, "vol_trend.pkl"), "rb") as fh: out["qqq_trend"] = pickle.load(fh)["returns"]["QQQ ma200 vt20 cap3"] return {k: v.reindex(days) for k, v in out.items()} # --------------------------------------------------------------------------- # weighting # --------------------------------------------------------------------------- def _erc(cov: np.ndarray, iters: int = 500) -> np.ndarray: w = 1 / np.sqrt(np.diag(cov)) w /= w.sum() for _ in range(iters): rc = w * (cov @ w) w = w * np.sqrt(rc.mean() / rc) w /= w.sum() return w def month_weights(R: pd.DataFrame, scheme: str) -> pd.DataFrame: """Shares for each month from returns through the previous month end.""" months = R.index.to_period("M").unique() rows = {} for m in months: hist = R.loc[:m.start_time - pd.Timedelta(days=1)] live = [c for c in R.columns if hist[c].dropna().shape[0] >= MIN_OBS] w = pd.Series(0.0, index=R.columns) if not live: rows[m] = w continue if scheme == "equal": w[live] = 1 / len(live) elif scheme == "invvol": v = hist[live].iloc[-126:].std() w[live] = (1 / v) / (1 / v).sum() elif scheme == "erc": h = hist[live].iloc[-252:].dropna() w[live] = _erc(h.cov().to_numpy()) if len(h) >= 126 else (1 / h.std()) / (1 / h.std()).sum() rows[m] = w return pd.DataFrame(rows).T def blend(R: pd.DataFrame, W: pd.DataFrame, rf: pd.Series) -> pd.Series: """Shares reset at month starts and drift inside; uninvested capital earns T-bills.""" out, drifted = [], pd.Series(0.0, index=R.columns) for m, g in R.groupby(R.index.to_period("M")): w = W.loc[m] cost = (w - drifted).abs().sum() * COST grow = (1 + g.fillna(0.0)).cumprod() cash = (1 - w.sum()) * (1 + rf.reindex(g.index).fillna(0.0) / 252).cumprod() val = grow.mul(w, axis=1).sum(axis=1) + cash prev = np.r_[1.0, val.values[:-1]] r = pd.Series(val.values / prev - 1, index=g.index) r.iloc[0] -= cost out.append(r) end = grow.iloc[-1] * w drifted = end / val.iloc[-1] return pd.concat(out) # --------------------------------------------------------------------------- # report # --------------------------------------------------------------------------- def _sharpe(r: pd.Series, rf: pd.Series) -> float: r = r.dropna() return float(stats(r, rf)["sharpe"]) if len(r) >= 126 else float("nan") def summary(r: pd.Series, rf: pd.Series) -> dict: s = stats(r.loc[SPAN[0]:SPAN[1]], rf) d = {k: float(v) for k, v in s.items()} d["halves"] = [_sharpe(r.loc[a:b], rf) for a, b in HALVES] d["blocks"] = [_sharpe(r.loc[a:b], rf) for a, b in BLOCKS] return d def line(name: str, d: dict) -> str: return (f"{name:28s} cagr {d['cagr']*100:5.1f}% xsh {d['sharpe']:5.2f} vol {d['vol']*100:4.1f}% " f"mdd {d['mdd']*100:6.1f}% worst yr {d['worst_yr']*100:6.1f}% | halves " f"{d['halves'][0]:5.2f} {d['halves'][1]:5.2f} | blocks " + " ".join(f"{x:5.2f}" for x in d["blocks"])) def main() -> None: os.makedirs(OUT, exist_ok=True) res_path = os.path.join(OUT, "results.json") if os.path.exists(res_path): raise SystemExit(f"{res_path} exists; this study does not overwrite results") with open(os.path.join(OUT, "prereg.json"), "w") as fh: json.dump(PREREG, fh, indent=2) cache = os.path.join(OUT, "streams.pkl") # inputs only; results are never reused if os.path.exists(cache): R = pd.read_pickle(cache) else: S = stock_streams() days = S["momentum"].index S.update(etf_streams(days, tbill(days))) R = pd.DataFrame(S).loc[:SPAN[1]] R.to_pickle(cache) rf = tbill(R.index) five = ["momentum", "lowrisk", "trend", "xtrend", "btc"] books = { "invvol5": (five, "invvol"), "equal5": (five, "equal"), "erc5": (five, "erc"), "invvol4_no_btc": (["momentum", "lowrisk", "trend", "xtrend"], "invvol"), "invvol5_qqq_trend": (["momentum", "lowrisk", "qqq_trend", "xtrend", "btc"], "invvol"), "invvol5_trend_v7": (["momentum", "lowrisk", "trend_v7", "xtrend", "btc"], "invvol"), "three_sleeve_v7": (["momentum", "lowrisk", "trend_v7"], "equal"), "three_sleeve_v10": (["momentum", "lowrisk", "trend"], "equal"), } B, avg_w = {}, {} for name, (cols, scheme) in books.items(): W = month_weights(R.loc[:, cols], scheme) B[name] = blend(R.loc[:, cols], W, rf) avg_w[name] = W.loc[pd.Period(SPAN[0], "M"):].mean().round(3).to_dict() for name in ("invvol5", "three_sleeve_v7", "three_sleeve_v10"): for t in (0.20, 0.25, 0.30): B[f"{name}_vt{int(t*100)}"] = managed(B[name], rf, t, 2.0, "ewma20")[0] res = {"streams": {}, "books": {}} print("\n=== streams (1x)") for c in R.columns: res["streams"][c] = summary(R[c].dropna(), rf) print(line(c, res["streams"][c])) print("\n=== books") for name, r in B.items(): res["books"][name] = summary(r, rf) print(line(name, res["books"][name])) print("\naverage shares:") for k, v in avg_w.items(): print(f" {k:20s} {v}") m = (1 + R.loc[SPAN[0]:].fillna(0.0)).resample("ME").prod() - 1 corr = m[five + ["trend_v7", "qqq_trend", "spy"]].corr().round(2) print("\nmonthly correlation:\n", corr.to_string()) # verdict p = res["books"]["invvol5"] best_name = [max(five, key=lambda c: np.nan_to_num(res["streams"][c]["halves"][i], nan=-9)) for i in range(2)] best = [res["streams"][best_name[i]]["halves"][i] for i in range(2)] ts = res["books"]["three_sleeve_v7"]["halves"] checks = { "A1": all(p["halves"][i] >= best[i] for i in range(2)), "A2": all(p["halves"][i] >= ts[i] for i in range(2)), "A3": res["books"]["invvol5_vt25"]["mdd"] >= -0.35, } res["verdict"] = {"checks": checks, "best_single": dict(zip(["first", "second"], zip(best_name, best))), "decision": "ACCEPT" if all(checks.values()) else "REJECT"} res["avg_shares"] = avg_w res["monthly_corr"] = corr.to_dict() yr = pd.DataFrame({k: (1 + B[k].loc[SPAN[0]:]).groupby(B[k].loc[SPAN[0]:].index.year).prod() - 1 for k in ("invvol5", "invvol5_vt25", "three_sleeve_v7", "three_sleeve_v10")}) yr["trend"] = (1 + R["trend"].loc[SPAN[0]:]).groupby(R.loc[SPAN[0]:].index.year).prod() - 1 yr["spy"] = (1 + R["spy"].loc[SPAN[0]:].fillna(0.0)).groupby(R.loc[SPAN[0]:].index.year).prod() - 1 print("\nyearly returns (%):\n", (yr * 100).round(1).to_string()) res["yearly"] = {k: {int(y): float(v) for y, v in yr[k].items()} for k in yr} print(f"\nbest single stream Sharpe by half: {res['verdict']['best_single']}") print(f"invvol5 halves {p['halves']}, three_sleeve_v7 halves {ts}, invvol5_vt25 mdd " f"{res['books']['invvol5_vt25']['mdd']:.3f}") print(f"checks {checks} -> {res['verdict']['decision']}") with open(res_path, "w") as fh: json.dump(res, fh, indent=2) pd.DataFrame(B).to_pickle(os.path.join(OUT, "books.pkl")) from research.lab.trials import append_trials append_trials(os.path.join(RESULTS, "trials_us.jsonl"), [{"name": f"stream_portfolio:{k}", **{x: y for x, y in v.items()}} for k, v in res["books"].items()], {"study": "stream_portfolio", "role": "preregistered; both halves previously inspected"}) if __name__ == "__main__": main()