fix(backtest): settle delisted US holdings (engine v3), PIT membership before 2011, registry causality test
- execution/trader: a held US stock with no print for 5 sessions that has left the index is sold at its last print (execution.SETTLE_AFTER_SESSIONS), in both the batch replay and the live session ledger. Acquired names used to stay at their last mark forever: lowrisk_top20 ended 2017-12..2026-10 with 11 of 26 dead positions, and CAGR rises 1.7-5.5pp once they settle. ENGINE_VERSION cash-ledger-pit-v3; the API cache key follows. - universe_history: current members whose pre-2011 addition the Wikipedia change log missed now start at the list page's "Date added" (80 of 94 current members added 2001-2010 were open-start, e.g. ISRG, MA, BKNG). Membership from 2016 on is unchanged; the research panel cache now keys on the membership intervals. - research/lab/registered: pure-ETF strategies replay on the ETF panel (they ran as all-cash books or crashed); prepare_strategy refuses an ETF strategy that lacks its ETF universe. - research/lab/factors: trend_hzz uses a month's regression betas only from the session its forward returns end (short months read 1-2 sessions ahead). - tests: every registered strategy (US and CN) is truncation-invariant on the production signal path; batch and live ledgers settle a delisting identically. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -158,6 +158,8 @@ def build_sp500_history() -> dict[str, list[list[str | None]]]:
|
||||
for t, end in open_end.items():
|
||||
intervals.setdefault(t, []).append([None, end])
|
||||
|
||||
_close_unlogged_additions(intervals, current)
|
||||
|
||||
# Sort intervals per ticker oldest→newest
|
||||
for t, ivs in intervals.items():
|
||||
ivs.sort(key=lambda iv: (iv[0] or "0000-00-00"))
|
||||
@@ -165,6 +167,30 @@ def build_sp500_history() -> dict[str, list[list[str | None]]]:
|
||||
return intervals
|
||||
|
||||
|
||||
# The change log is sparse before 2011 (0-2 logged additions a year in
|
||||
# 2001-2006, 6-12 in 2007-2010, against 20-30 real changes), so a current
|
||||
# member whose addition it missed would count as a member since before the
|
||||
# log: 80 of the 94 current members added 2001-2010 did, e.g. ISRG, MA, BKNG.
|
||||
# The list page's "Date added" starts those intervals. Later dates are left to
|
||||
# the log, which is complete there, because "Date added" also records renames
|
||||
# and re-listings (META 2013, CPAY 2018) as additions.
|
||||
UNLOGGED_ADDITIONS_BEFORE = "2011-01-01"
|
||||
|
||||
|
||||
def _close_unlogged_additions(intervals: dict, current: pd.DataFrame) -> None:
|
||||
col = next((c for c in current.columns if "added" in str(c).lower()), None)
|
||||
if col is None:
|
||||
print("--- WARNING: no 'Date added' column; pre-2011 additions stay open-start ---")
|
||||
return
|
||||
added = pd.to_datetime(current[col], errors="coerce")
|
||||
for symbol, day in zip(current["Symbol"], added):
|
||||
if pd.isna(day) or day >= pd.Timestamp(UNLOGGED_ADDITIONS_BEFORE):
|
||||
continue
|
||||
for iv in intervals.get(_normalize_ticker(symbol), []):
|
||||
if iv[0] is None and iv[1] is None:
|
||||
iv[0] = day.strftime("%Y-%m-%d")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Cache I/O
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
Reference in New Issue
Block a user