feat(research): Tiingo backfill of delisted names; honest PIT panels

Yahoo purges delisted members: the S&P research panel was missing 209
ever-members (AGN/CELG/ESRX/ATVI/TWTR/XLNX/...), the production store 89, and
the S&P 400/600 panels ~180 more. The gap inflated momentum-family factors
(overnight_mom holdout 16.9->14.8%/yr, rec_126 14.7->10.7 on the research
panel; factor_panel_check).

research/lab/fetch_tiingo.py pulls full EOD histories (free tier: 500 unique
symbols/month, 50/hour; paced, resume-safe, key never logged).
research/lab/tiingo_merge.py overlays the caches on the research S&P panel at
build time (>=50% PIT-coverage gate, median-overlap cross-check, stub-column
replacement, rename splices GPS<-GAP, DISCA<-WBD, WLTW<-WTW, ADS<-BFH,
CCE<-CCEP, ESV<-VAL, FBHS<-FBIN, HFC<-DINO) and can apply the same to the
production store (--production [--apply]; data/us*.csv, .bak-tiingo backups,
under the panel lock). load_us() rebuilds when the overlay changes
(_tiingo_sig); fetch_us_mid.load_bars falls back to converted CSVs.

Round 1 (2026-10-08): 165/209 S&P names and 183 mid/small PIT members
restored (2011-19 snapshots were 15-20% short of PIT members). prod_impact
A/B on registered replays: overnight_recovery_top10 23.8%/0.83/-38.3% vs
27.8%/0.88/-47.3% backfill-off; lowrisk_top20 6.1%/0.58 vs 8.6%/0.60 -
honest long returns with shallower drawdowns; trend_rider_v10 (ETF path)
unchanged at 55.0%/1.64. Official precompute refreshed (backup at
data/precomputed/backtest_results.json.pretiingo). midsmall_scan on the
corrected panel: no factor beats the EW benchmark (0.60 holdout Sharpe vs
0.51-0.52 best) - no mid/small sleeve.

CLAUDE.md

@@ -81,6 +81,30 @@ PYTHONPATH=. uv run python -m research.lab.sector_cycle                  # busin
 uv run python -m research.lab.fetch_wiki_attention && PYTHONPATH=. uv run --with numba python -m research.lab.attention   # Wikipedia-pageview attention
 PYTHONPATH=. uv run --with numba python -m research.lab.concentration     # industry caps on the momentum sleeve
 PYTHONPATH=. uv run python -m research.lab.universe_midcap              # overnight_recovery_top10 on S&P 500 ∪ S&P 400 (preregistered; FAIL 2026-10-08)
+uv run python -m research.lab.fetch_us_fundamentals --universe mid      # EDGAR PIT facts for S&P 400 ever-members (name-checked CIKs)
+PYTHONPATH=. uv run python -m research.lab.midcap_sleeve [--only h1..h4]  # separate S&P 400 sleeve vs IJH (preregistered; all REJECT 2026-10-08)
+PYTHONPATH=. uv run python -m research.lab.fetch_shortvolume              # FINRA REG SHO daily short volume (~8y archive, resume-safe)
+PYTHONPATH=. uv run python -m research.lab.short_volume [us|midsmall|all] # SVR (short-volume ratio) IC / quintile scan on PIT panels
+PYTHONPATH=. uv run python -m research.lab.fetch_fails                   # SEC fails-to-deliver archives (2018+, resume-safe)
+PYTHONPATH=. uv run python -m research.lab.fails [us|midsmall|all]       # FTD intensity scans (value/shares per ADV$, PIT +21d publication lag)
+PYTHONPATH=. uv run python -m research.lab.fetch_13d [--quarters N]            # EDGAR quarterly index: SC 13D/13G rows (activist/passive stakes)
+PYTHONPATH=. uv run python -m research.lab.fetch_13d_targets [--limit N]      # subject company of each SC 13D (index lists the filer)
+PYTHONPATH=. uv run python -m research.lab.activists [us|midsmall|all]        # 13D event study: market-adjusted CARs, momentum/SVR splits
+PYTHONPATH=. uv run python -m research.lab.activist_strategy [midsmall|us]   # 13D event sleeve (lab engine; dead in 2021-26 — research only)
+PYTHONPATH=. uv run python -m research.lab.fetch_13f                        # SEC DERA 13F structured data sets (2016q1-2023q4), per-CUSIP aggregates
+PYTHONPATH=. uv run python -m research.lab.inst_flow [us|midsmall]          # institutional q/q flow factor scan (IC t 2.0 but Q5-Q1 spread nil)
+PYTHONPATH=. uv run python -m research.lab.fetch_gdelt && PYTHONPATH=. uv run python -m research.lab.news_volume  # GDELT news volume (API rate-limited to ~1 query/10-15s; sample only)
+PYTHONPATH=. uv run python -m research.svr_overlay                        # SVR-veto overlay on overnight_recovery_top10 (lab)
+
+# Tiingo backfill (delisted names Yahoo purged; keys from TIINGO_API_KEY, never logged)
+PYTHONPATH=. uv run python -m research.lab.fetch_tiingo --file list.txt --min-interval 75  # EOD backfill (free tier: 500 symbols/month, 50/hour; cache = resume)
+PYTHONPATH=. uv run python -m research.lab.panel_gaps                    # price gaps vs PIT membership (both panels); writes data/research/tiingo_plan.json
+PYTHONPATH=. uv run python -m research.lab.tiingo_merge                  # report: overlay caches onto the research S&P panel (auto at panel build)
+PYTHONPATH=. uv run python -m research.lab.tiingo_merge --production [--apply]  # merge into data/us*.csv (backups .bak-tiingo; --apply writes)
+PYTHONPATH=. uv run python -m research.lab.tiingo_merge --yahoo-mid      # convert caches to yahoo_mid format (fetch_us_mid.load_bars falls back to it)
+PYTHONPATH=. uv run python -m research.factor_panel_check                # pre-vs-post overlay S&P factor comparison
+PYTHONPATH=. uv run python -m research.midsmall_scan --panel both        # mid/small factor scan: Yahoo-only vs Tiingo-extended panel
+PYTHONPATH=. uv run python -m research.prod_impact [STRATEGY ...]        # A/B registered replays: backfilled store vs .bak-tiingo snapshot
 
 # Setup
 uv sync                                  # Install/sync dependencies
@@ -126,7 +150,10 @@ uv run python -m unittest discover tests
 - `recovery_momentum.py` — Recovery (price/63d low) + 12-1mo momentum composite. Best US performer.
 - `robust_momentum.py` — Bar strategies from the 2026-09 PIT research (report: https://claude.ai/artifact/Mo9nrzs5SWj5FfyshERyWS): US `overnight_recovery` (overnight-return momentum + distance from 126d low), CN `lowrisk_momentum` (amplitude-cut momentum + low beta + price-volume corr + momentum/vol) and close-only `lowbeta_sharpe_mom`. Top-N with rank hysteresis, rebalance every 10 days, weights drift between rebalances. `low_risk` (US `lowrisk_top20`: idiosyncratic vol + low beta, top 20, monthly) is the defensive sleeve of the 2026-09-29 US research (report: https://claude.ai/artifact/VGhZvKkS9UBYZkLeqGjcNC).
 - `sleeve_portfolio.py` — Several strategies in one account with fixed capital shares. `us_three_sleeve` = 1/3 each of `overnight_recovery_top10`, `lowrisk_top20`, `trend_rider_v7_vt24` (a mixed stock+ETF strategy); sleeve shares drift with their returns and reset at a month start once any is >25% off. 2011-2026 research: Sharpe 1.21 vs 0.88-1.03 for the sleeves alone. Research finding behind it: choosing factors by their recent record (rotation, factor momentum, last year's winner) did not beat a fixed composite; construction knobs (N, weighting, dynamic N) barely matter. On synthetic 3x ETFs from 1990 (`research/lab/letf_history.py`), TrendRider V7's 2016-2026 record does not hold: 1990-99 Sharpe 0.2-0.3, 2003-07 negative, 1990-2026 about 0.7 with MaxDD -58% (vt24) to -69% (vt36).
+- `trend_rider_v9.py` — V7 pipeline (V3 regime + vol target + profit-take) plus an instrument trailing stop: while risk-on, a close 20% below the held leg's 30-day high (PIT, through t-1) switches to the risk-off basket immediately; re-entry still needs the normal regime confirmation. Registered as `trend_rider_v9` (VT34, the default) and `trend_rider_v9_vt24/_vt32/_vt36`. Engine window 2017-12→2026-10, $10k, $2/order: `trend_rider_v9` 52.3%/yr Sharpe 1.55 MaxDD -29.3%, `_vt36` 52.5%/1.53, `_vt32` 51.5%/1.57, `_vt24` 42.4%/1.58. 2010-2026 live-book simulation ($2/order): every tier improves on V7 (e.g. vt34: 34.4%/1.14 vs vt32 32.2%/1.10); 2010-2016 blocks are essentially unchanged.
+- `trend_rider_v10` (same file) — V9 plus a refined profit-take: clear at +28% (not 30%), restore band 8pp (re-enter sooner on dips), 10-day re-clear lockout after a restore (stops the 2022-style clear/restore churn on GLD/DBC). Registered as `trend_rider_v10` (VT34), `_vt32`, `_vt36`. Engine window to 2026-10-07: v10 55.9%/1.66/MaxDD -27.4%, v10_vt32 55.0%/1.68, v10_vt36 56.2%/1.63 — beats every other registered strategy on both CAGR and Sharpe (previous best: v9_vt36 52.5%, v9_vt24 1.58). 2010-2026 live-book simulation confirms robustness: the PT refinement lifts every 3-4y block's Sharpe (2010-13 0.83→0.92, 2016-21 1.74→1.89, 2021-26 1.38→1.43; full period 36.8%/1.22 vs 34.4%/1.14 for plain V9-trail). Research trail: `research/tr9_{lab,run,long,trials,audit}.py`; rejected knobs are logged in `data/research/results/trials_us.jsonl` (macro throttles on VIX term/credit, RSP breadth, instrument MA exits, fast re-entry, SOXL/TECL/TMF/SQQQ baskets).
 - `qqq_trend.py` — `QQQVolTrend`: QQQ exposure = 1{QQQ > 200-day average} × min(1.5, 30% / EWMA vol), held as TQQQ (E/3) + SHY. 1990-2026 on synthetic ETFs: 16.0% vs QQQ 14.9%, MaxDD -37% vs -83%; 2011-2026 on real ETFs it trails QQQ (17.5% vs 19.1%) with a smaller drawdown. `us_momentum_qqq_trend` (in `sleeve_portfolio.py`) holds it half and half with `overnight_recovery_top10`. Research behind it (`research/lab/qqq_scan.py`, `beat_qqq.py`): measured as alpha against QQQ with a 2006-16 design / 2017-26 holdout split, momentum top 10 is the only stock selection whose alpha held (+10%/yr, t 1.4); single factors, walk-forward ML, a mega-cap universe, SEC insider trades, 10-K/10-Q text similarity, earnings-announcement returns, industry momentum and a PIT S&P 400/600 universe (`panel_mid.py`) showed none. In the web engine (2016-09 to 2026-09, $2/order) `us_momentum_qqq_trend` earns 21.0% vs QQQ 20.9%, MaxDD -32%.
+- `svr_momentum.py` — RESEARCH ONLY (not registered): `overnight_recovery_top10` + a FINRA short-volume veto (drop top-70%-quantile SVR names). The 2026-10-07 window looked good (Sharpe 0.87→0.89, MaxDD -47%→-39%) but the quantile neighborhood killed it — 0.60/0.75/0.80 vetoes all score below baseline. A lone spike; registered and reverted the same day. The raw cross-sectional reading (SVR negative predictor, `research/lab/short_volume.py`) stands; the concentrated-portfolio exclusion rule does not.
 - `agent_review.py` — `overnight_recovery_agent_top10`: overnight_recovery's top 50, re-ranked each day by an LLM agent's scores (−2..+2 from filings, fundamentals, demand-chain logic, event risk; `final = quant percentile + 0.25 × score`), vetoes for five event types, top 10 on the base strategy's rebalance rows and hysteresis. Score files in `data/agent/us/` are stamped by `agent_review.py submit` and steer a session only if stamped before its 16:00 New York close; without one the strategy holds exactly what `overnight_recovery_top10` holds, so the two books' difference measures the agent. `AGENT_REVIEW_STRATEGIES` in trader.py keeps it US-only and out of `precompute_backtest.py` (it has no history before its first score file). Research behind the design (2026-09-30): picking 10 of the quant top 50 at random earns 13.4%/yr vs 26.2% for the top 10 (2006-2026), and an LLM cannot be backtested before its training cutoff, so the record starts 2026-09-29.
 - `factor_combo.py` — Configurable factor-combination strategies with daily/weekly/biweekly/monthly rebalancing. US champions: `rec_mfilt+deep_upvol`, `ma200+mom7m+rec126`, `rec_mfilt+ma200`, `mom7m+rec126`. CN champions: `up_cap+quality_mom`, `down_resil+qual_mom`, `rec63+mom_gap`, `up_cap+mom_gap` — these were mined on today's CSI 300 members; on PIT members they earn ~3-7% a year (2016-2026). All registered in trader.py as `fc_{signal}_{freq}` (e.g., `fc_rec_mfilt_deep_upvol_monthly`). 32 new strategies total.