Commit Graph

3 Commits

Author SHA1 Message Date
cd0721eada fix(backtest): settle delisted US holdings (engine v3), PIT membership before 2011, registry causality test
- execution/trader: a held US stock with no print for 5 sessions that has left
  the index is sold at its last print (execution.SETTLE_AFTER_SESSIONS), in both
  the batch replay and the live session ledger. Acquired names used to stay at
  their last mark forever: lowrisk_top20 ended 2017-12..2026-10 with 11 of 26
  dead positions, and CAGR rises 1.7-5.5pp once they settle. ENGINE_VERSION
  cash-ledger-pit-v3; the API cache key follows.
- universe_history: current members whose pre-2011 addition the Wikipedia change
  log missed now start at the list page's "Date added" (80 of 94 current members
  added 2001-2010 were open-start, e.g. ISRG, MA, BKNG). Membership from 2016 on
  is unchanged; the research panel cache now keys on the membership intervals.
- research/lab/registered: pure-ETF strategies replay on the ETF panel (they ran
  as all-cash books or crashed); prepare_strategy refuses an ETF strategy that
  lacks its ETF universe.
- research/lab/factors: trend_hzz uses a month's regression betas only from the
  session its forward returns end (short months read 1-2 sessions ahead).
- tests: every registered strategy (US and CN) is truncation-invariant on the
  production signal path; batch and live ledgers settle a delisting identically.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 00:21:49 -07:00
92af4e5ac0 feat(research): Tiingo backfill of delisted names; honest PIT panels
Yahoo purges delisted members: the S&P research panel was missing 209
ever-members (AGN/CELG/ESRX/ATVI/TWTR/XLNX/...), the production store 89, and
the S&P 400/600 panels ~180 more. The gap inflated momentum-family factors
(overnight_mom holdout 16.9->14.8%/yr, rec_126 14.7->10.7 on the research
panel; factor_panel_check).

research/lab/fetch_tiingo.py pulls full EOD histories (free tier: 500 unique
symbols/month, 50/hour; paced, resume-safe, key never logged).
research/lab/tiingo_merge.py overlays the caches on the research S&P panel at
build time (>=50% PIT-coverage gate, median-overlap cross-check, stub-column
replacement, rename splices GPS<-GAP, DISCA<-WBD, WLTW<-WTW, ADS<-BFH,
CCE<-CCEP, ESV<-VAL, FBHS<-FBIN, HFC<-DINO) and can apply the same to the
production store (--production [--apply]; data/us*.csv, .bak-tiingo backups,
under the panel lock). load_us() rebuilds when the overlay changes
(_tiingo_sig); fetch_us_mid.load_bars falls back to converted CSVs.

Round 1 (2026-10-08): 165/209 S&P names and 183 mid/small PIT members
restored (2011-19 snapshots were 15-20% short of PIT members). prod_impact
A/B on registered replays: overnight_recovery_top10 23.8%/0.83/-38.3% vs
27.8%/0.88/-47.3% backfill-off; lowrisk_top20 6.1%/0.58 vs 8.6%/0.60 -
honest long returns with shallower drawdowns; trend_rider_v10 (ETF path)
unchanged at 55.0%/1.64. Official precompute refreshed (backup at
data/precomputed/backtest_results.json.pretiingo). midsmall_scan on the
corrected panel: no factor beats the EW benchmark (0.60 holdout Sharpe vs
0.51-0.52 best) - no mid/small sleeve.
2026-10-08 09:48:05 -07:00
0e378efa67 research: point-in-time factor research harness (research/lab)
Event-driven long-only simulator with next-open fills, CN tradability
(halts, limit-up/down, stamp duty), PIT S&P 500 / CSI 300 membership,
rebalance-phase averaging, factor IC scans, maximin stepwise composite
search across 5-year blocks, walk-forward LightGBM/ridge combiners,
overlays, and deflated Sharpe. Fetchers pull 20y US OHLCV from Yahoo and
CSI 300/500 ever-members (membership snapshot from baostock, bars from
Yahoo). Research data lives in data/research/ (ignored).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-25 19:11:59 +08:00