fix(backtest): settle delisted US holdings (engine v3), PIT membership before 2011, registry causality test

- execution/trader: a held US stock with no print for 5 sessions that has left
  the index is sold at its last print (execution.SETTLE_AFTER_SESSIONS), in both
  the batch replay and the live session ledger. Acquired names used to stay at
  their last mark forever: lowrisk_top20 ended 2017-12..2026-10 with 11 of 26
  dead positions, and CAGR rises 1.7-5.5pp once they settle. ENGINE_VERSION
  cash-ledger-pit-v3; the API cache key follows.
- universe_history: current members whose pre-2011 addition the Wikipedia change
  log missed now start at the list page's "Date added" (80 of 94 current members
  added 2001-2010 were open-start, e.g. ISRG, MA, BKNG). Membership from 2016 on
  is unchanged; the research panel cache now keys on the membership intervals.
- research/lab/registered: pure-ETF strategies replay on the ETF panel (they ran
  as all-cash books or crashed); prepare_strategy refuses an ETF strategy that
  lacks its ETF universe.
- research/lab/factors: trend_hzz uses a month's regression betas only from the
  session its forward returns end (short months read 1-2 sessions ahead).
- tests: every registered strategy (US and CN) is truncation-invariant on the
  production signal path; batch and live ledgers settle a delisting identically.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-10 00:21:49 -07:00
parent a9683b6d1f
commit cd0721eada
12 changed files with 279 additions and 41 deletions

View File

@@ -158,6 +158,8 @@ def build_sp500_history() -> dict[str, list[list[str | None]]]:
for t, end in open_end.items():
intervals.setdefault(t, []).append([None, end])
_close_unlogged_additions(intervals, current)
# Sort intervals per ticker oldest→newest
for t, ivs in intervals.items():
ivs.sort(key=lambda iv: (iv[0] or "0000-00-00"))
@@ -165,6 +167,30 @@ def build_sp500_history() -> dict[str, list[list[str | None]]]:
return intervals
# The change log is sparse before 2011 (0-2 logged additions a year in
# 2001-2006, 6-12 in 2007-2010, against 20-30 real changes), so a current
# member whose addition it missed would count as a member since before the
# log: 80 of the 94 current members added 2001-2010 did, e.g. ISRG, MA, BKNG.
# The list page's "Date added" starts those intervals. Later dates are left to
# the log, which is complete there, because "Date added" also records renames
# and re-listings (META 2013, CPAY 2018) as additions.
UNLOGGED_ADDITIONS_BEFORE = "2011-01-01"
def _close_unlogged_additions(intervals: dict, current: pd.DataFrame) -> None:
col = next((c for c in current.columns if "added" in str(c).lower()), None)
if col is None:
print("--- WARNING: no 'Date added' column; pre-2011 additions stay open-start ---")
return
added = pd.to_datetime(current[col], errors="coerce")
for symbol, day in zip(current["Symbol"], added):
if pd.isna(day) or day >= pd.Timestamp(UNLOGGED_ADDITIONS_BEFORE):
continue
for iv in intervals.get(_normalize_ticker(symbol), []):
if iv[0] is None and iv[1] is None:
iv[0] = day.strftime("%Y-%m-%d")
# ---------------------------------------------------------------------------
# Cache I/O
# ---------------------------------------------------------------------------