fix(backtest): settle delisted US holdings (engine v3), PIT membership before 2011, registry causality test
- execution/trader: a held US stock with no print for 5 sessions that has left the index is sold at its last print (execution.SETTLE_AFTER_SESSIONS), in both the batch replay and the live session ledger. Acquired names used to stay at their last mark forever: lowrisk_top20 ended 2017-12..2026-10 with 11 of 26 dead positions, and CAGR rises 1.7-5.5pp once they settle. ENGINE_VERSION cash-ledger-pit-v3; the API cache key follows. - universe_history: current members whose pre-2011 addition the Wikipedia change log missed now start at the list page's "Date added" (80 of 94 current members added 2001-2010 were open-start, e.g. ISRG, MA, BKNG). Membership from 2016 on is unchanged; the research panel cache now keys on the membership intervals. - research/lab/registered: pure-ETF strategies replay on the ETF panel (they ran as all-cash books or crashed); prepare_strategy refuses an ETF strategy that lacks its ETF universe. - research/lab/factors: trend_hzz uses a month's regression betas only from the session its forward returns end (short months read 1-2 sessions ahead). - tests: every registered strategy (US and CN) is truncation-invariant on the production signal path; batch and live ledgers settle a delisting identically. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -148,11 +148,11 @@ uv run python -m unittest discover tests
|
||||
|
||||
**Universe provider** (`universe.py`): Dynamically fetches index constituents (S&P 500 from Wikipedia; CSI 300 from China Securities Index's published constituent file, falling back to the latest `data/csi300_history.json` snapshot, then Wikipedia, whose table lagged the index by ~60 names in 2026) and converts to Yahoo Finance ticker format. Constituent lists are cached daily in `data/universe_*.json`. New markets are added by registering in `UNIVERSES` dict.
|
||||
|
||||
**Point-in-time membership** (`universe_history.py`): S&P 500 intervals rebuilt from Wikipedia's "Historical components of the S&P 500" change log into `data/sp500_history.json`; CSI 300 monthly snapshots in `data/csi300_history.json` (seeded from baostock 2008–2026, appended whenever the current list changes). `mask_prices_pit(prices, market)` masks non-member dates.
|
||||
**Point-in-time membership** (`universe_history.py`): S&P 500 intervals rebuilt from Wikipedia's "Historical components of the S&P 500" change log into `data/sp500_history.json`. The log is sparse before 2011, so current members' earlier additions come from the list page's "Date added"; companies removed before 2011 that the log never recorded stay missing, and the research panel holds only 347 / 367 / 438 / 453 / 484 priced members in mid-2004 / 2006 / 2010 / 2012 / 2016 (about 500 from 2020): US research results before ~2014 are survivor-biased. CSI 300 monthly snapshots in `data/csi300_history.json` (seeded from baostock 2008–2026, appended whenever the current list changes). `mask_prices_pit(prices, market)` masks non-member dates.
|
||||
|
||||
**Data manager** (`data_manager.py`): Persistent price storage in `data/{market}.csv` (close) and `data/{market}_{open,high,low,volume}.csv`. First run downloads 10 years of history; every run re-downloads the last 30 days. New tickers from index rebalances are backfilled automatically; tickers missing from a partial batch are retried alone before being marked unavailable (retried weekly). `update(market, tickers, with_open=True)` returns (close, open); `update(..., fields=[...])` returns `{field: panel}`.
|
||||
|
||||
**Backtest engine** (`main.py`, `backtest_runner.py`): all replay paths use `signals.session_targets` and the cash-funded order ledger in `execution.py`. Price strategies may use the session open and fill at its close; close-only inputs and BarStrategy inputs use prior completed bars. Historical membership restricts ranks and selection separately from price history. Missing quotes and zero volume cannot fill; the book checks available cash after fees, handles whole shares/lots, and applies conservative CN closing-limit restrictions and sell-side stamp tax. Registry replays retain at least 315 prior sessions and score all compared strategies on a common actual window. `--offline` on main/precompute reads a frozen local snapshot. Data coverage and adjusted-share/corporate-action limitations are exposed in API metadata.
|
||||
**Backtest engine** (`main.py`, `backtest_runner.py`): all replay paths use `signals.session_targets` and the cash-funded order ledger in `execution.py`. Price strategies may use the session open and fill at its close; close-only inputs and BarStrategy inputs use prior completed bars. Historical membership restricts ranks and selection separately from price history. Missing quotes and zero volume cannot fill, except that a held US stock with no print for 5 sessions that has left the index is settled at its last print (`execution.SETTLE_AFTER_SESSIONS`; acquisitions otherwise froze capital forever); the book checks available cash after fees, handles whole shares/lots, and applies conservative CN closing-limit restrictions and sell-side stamp tax. Registry replays retain at least 315 prior sessions and score all compared strategies on a common actual window. `--offline` on main/precompute reads a frozen local snapshot. Data coverage and adjusted-share/corporate-action limitations are exposed in API metadata.
|
||||
|
||||
`precompute_backtest.py` reruns the ledger for cash-injection scenarios; metrics use unit NAV after actual fees. The API and `web/src/shared/strategyTypes.ts` share this cash-flow contract. `ENGINE_VERSION` in `execution.py` identifies the execution protocol; API cache keys also include its version.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user