gahow/quant-trading
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: robust-momentum strategies live; fix live universe, lots and UI backtests

Browse Source 
Strategies
- BarStrategy protocol (close/open/high/low/volume bars; row d uses bars
  through d-1) and RobustMomentumStrategy: overnight_recovery_top10 (US),
  lowrisk_momentum_top15 and lowbeta_sharpe_mom_top20 (CN), top-N with
  rank hysteresis, 10-day rebalance, weights drift between rebalances.
- trader: strategy_fields/strategy_bars/bar_session_weights; one signal
  path (_signals_for_sessions) replaces generate_target_weights.

Live fixes
- CSI 300 constituents come from China Securities Index's file; the
  Wikipedia table was 122 names off the real index.
- S&P 500 change log moved to "Historical components of the S&P 500".
- data_manager: tickers missing from a partial Yahoo batch are retried
  alone before being marked unavailable (CEG, GEV, SNDK ... were dropped).
  update() takes fields=[...] and stores high/low/volume panels.
- Whole-lot orders go to the nearest lot within cash (CN books held ~62%
  cash); held names are resized only beyond RESIZE_BAND (10%), so books no
  longer pay a ticket fee to trade daily drift.

Web backtests
- backtest_runner.py replaces the duplicated bridge/precompute code: PIT
  membership in both markets (data/csi300_history.json), stock universe
  limited to index members (ETFs and benchmarks in the shared panel were
  pickable), and the book kept like the live trader (simulate_live_book:
  signal timing, drift, order floor, resize band, fees on orders placed).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Gahow Wang
2026-09-25 19:12:02 +08:00
parent 0e378efa67
commit cc1366b2dc
16 changed files with 16027 additions and 606 deletions
Download Patch File Download Diff File Expand all files Collapse all files

26
CLAUDE.md
View File

@@ -9,7 +9,7 @@ quant-trading: a quantitative backtesting framework and live forward-testing tra
The repository has two halves that share one data directory and one `.env`:
- **Python engine** (repo root, `uv`): strategies, backtests, `trader.py`, research scripts.
- **Web UI and API** (`web/`, Node + React + Vite): the API spawns `uv run python bridge.py {list,backtest,live}` at the repo root and reads `data/precomputed/backtest_results.json`, written by `precompute_backtest.py`.
- **Web UI and API** (`web/`, Node + React + Vite): the API spawns `uv run python bridge.py {list,backtest,live}` at the repo root and reads `data/precomputed/backtest_results.json`, written by `precompute_backtest.py`. Both backtest paths run through `backtest_runner.py`.
`bridge.py`'s JSON output and `web/src/shared/strategyTypes.ts` are one contract: change both in the same commit.
@@ -40,6 +40,13 @@ uv run python trader.py simulate --integer-shares --fixed-fee 2.0 ... # Integ
# Precompute the strategy page's backtests (writes data/precomputed/)
uv run python precompute_backtest.py
# Factor research harness (PIT universe, next-open fills, phase averaging; data in data/research/)
uv run python -m research.lab.fetch_us # 20y OHLCV for every S&P 500 ever-member
uv run python -m research.lab.fetch_cn_yahoo # CSI 300/500 ever-members (membership from baostock snapshot)
PYTHONPATH=. uv run python -m research.lab.factor_scan --market us # or cn_hs300
PYTHONPATH=. uv run python -m research.lab.stepwise --market us --objective maximin
PYTHONPATH=. uv run python -m research.lab.evaluate --market us # candidates vs champions
# Setup
uv sync # Install/sync dependencies
cp .env.example .env # Then fill IPADS_API_KEY and QUANT_MARKET_PROXY
@@ -55,15 +62,17 @@ uv run python -m unittest discover tests
## Architecture
**Universe provider** (`universe.py`): Dynamically fetches index constituents (S&P 500 from Wikipedia, CSI 300 from Wikipedia) and converts to Yahoo Finance ticker format. Constituent lists are cached daily in `data/universe_*.json`. New markets are added by registering in `UNIVERSES` dict.
**Universe provider** (`universe.py`): Dynamically fetches index constituents (S&P 500 from Wikipedia; CSI 300 from China Securities Index's published constituent file, falling back to the latest `data/csi300_history.json` snapshot, then Wikipedia, whose table lagged the index by ~60 names in 2026) and converts to Yahoo Finance ticker format. Constituent lists are cached daily in `data/universe_*.json`. New markets are added by registering in `UNIVERSES` dict.
**Data manager** (`data_manager.py`): Persistent price storage in `data/{market}.csv` (close) and `data/{market}_open.csv` (open). First run downloads 10 years of history; subsequent runs append only new trading days (fast bulk fetch for gaps ≤7 days). New tickers from index rebalances are backfilled automatically. Tickers with >50% missing data are dropped. Pass `with_open=True` to also maintain open price files.
**Point-in-time membership** (`universe_history.py`): S&P 500 intervals rebuilt from Wikipedia's "Historical components of the S&P 500" change log into `data/sp500_history.json`; CSI 300 monthly snapshots in `data/csi300_history.json` (seeded from baostock 2008–2026, appended whenever the current list changes). `mask_prices_pit(prices, market)` masks non-member dates.
**Backtest engine** (`main.py`): Orchestrates data loading, strategy execution, and visualization. The `backtest()` function is vectorized — it takes a strategy and price DataFrame, applies transaction costs (proportional + optional fixed per-trade fee) via turnover, and returns an equity curve. Supports two execution modes: `close` (classic) and `open-close` (signal on open prices, execute at close).
**Data manager** (`data_manager.py`): Persistent price storage in `data/{market}.csv` (close) and `data/{market}_{open,high,low,volume}.csv`. First run downloads 10 years of history; every run re-downloads the last 30 days. New tickers from index rebalances are backfilled automatically; tickers missing from a partial batch are retried alone before being marked unavailable (retried weekly). `update(market, tickers, with_open=True)` returns (close, open); `update(..., fields=[...])` returns `{field: panel}`.
**Daily trader** (`trader.py`): Live/forward-testing system with persistent portfolio state in `data/trader_{market}_{strategy}.json`. The `auto` subcommand runs both signal generation and execution in a single invocation — designed for cron. The `simulate` subcommand replays a date range day-by-day with realistic portfolio tracking (fractional shares, cash, commissions). Available strategies: `recovery_mom_top10`, `recovery_mom_top20`, `momentum`, `momentum_quality`, `dual_momentum`, `inverse_vol`, `trend_following`, `buy_and_hold`, plus 32 factor-combo strategies (`fc_{signal}_{freq}` — see `strategies/factor_combo.py`).
**Backtest engine** (`main.py`): Orchestrates data loading, strategy execution, and visualization. `backtest()` has two modes. Close mode (no `open_data`, used by `main.py` and old research scripts) is vectorized and trades at the close the signal saw. Live mode (`open_data` given, used by the web UI through `backtest_runner.py`) keeps the book like `trader.py`: same signal timing (`strategy_positions`), holdings that drift, the $50/0.1% order floor, `trader.RESIZE_BAND`, fees on the orders placed (`simulate_live_book`).
**Strategy protocol** (`strategies/base.py`): All strategies inherit from `Strategy` ABC and implement `generate_signals(data) → DataFrame` where the returned DataFrame contains portfolio weights (rows = dates, columns = assets, values sum to ~1.0 per row). Each strategy is responsible for applying its own 1-day lag via `.shift(1)` to avoid lookahead bias — the backtest engine does not shift.
**Daily trader** (`trader.py`): Live/forward-testing system with persistent portfolio state in `data/trader_{market}_{strategy}.json`. The `auto` subcommand runs both signal generation and execution in a single invocation — designed for cron. The `simulate` subcommand replays a date range day-by-day with realistic portfolio tracking (fractional shares, cash, commissions). Orders: whole A-share lots are allocated nearest-lot within cash (`_allocate_lots`); a held name is resized only when ≥ `RESIZE_BAND` (10%) off target. Available strategies: `recovery_mom_top10`, `recovery_mom_top20`, `momentum`, `momentum_quality`, `dual_momentum`, `inverse_vol`, `trend_following`, `buy_and_hold`, 32 factor-combo strategies (`fc_{signal}_{freq}` — see `strategies/factor_combo.py`), and the robust-momentum bar strategies `overnight_recovery_top10`, `lowrisk_momentum_top15`, `lowbeta_sharpe_mom_top20`.
**Strategy protocol** (`strategies/base.py`): All strategies inherit from `Strategy` ABC and implement `generate_signals(data) → DataFrame` where the returned DataFrame contains portfolio weights (rows = dates, columns = assets, values sum to ~1.0 per row). Each strategy is responsible for applying its own 1-day lag via `.shift(1)` to avoid lookahead bias — the backtest engine does not shift. A `BarStrategy` declares `fields` (close/open/high/low/volume) and receives `{field: DataFrame}`; live it uses completed bars only (row d from bars through d-1, traded at the close of d), while close-price strategies read the session's open as their latest price.
**Strategies**:
- `buy_and_hold.py` — Equal-weight static allocation
@@ -76,7 +85,8 @@ uv run python -m unittest discover tests
- `momentum_quality.py` — Momentum + return consistency + low drawdown
- `adaptive_momentum.py` — Momentum weighted by inverse volatility
- `recovery_momentum.py` — Recovery (price/63d low) + 12-1mo momentum composite. Best US performer.
- `factor_combo.py` — Configurable factor-combination strategies with daily/weekly/biweekly/monthly rebalancing. US champions: `rec_mfilt+deep_upvol` (50.7% CAGR monthly), `ma200+mom7m+rec126`, `rec_mfilt+ma200`, `mom7m+rec126`. CN champions: `up_cap+quality_mom` (26.1% CAGR monthly), `down_resil+qual_mom`, `rec63+mom_gap`, `up_cap+mom_gap`. All registered in trader.py as `fc_{signal}_{freq}` (e.g., `fc_rec_mfilt_deep_upvol_monthly`). 32 new strategies total.
- `robust_momentum.py` — Bar strategies from the 2026-09 PIT research (report: https://claude.ai/artifact/Mo9nrzs5SWj5FfyshERyWS): US `overnight_recovery` (overnight-return momentum + distance from 126d low), CN `lowrisk_momentum` (amplitude-cut momentum + low beta + price-volume corr + momentum/vol) and close-only `lowbeta_sharpe_mom`. Top-N with rank hysteresis, rebalance every 10 days, weights drift between rebalances.
- `factor_combo.py` — Configurable factor-combination strategies with daily/weekly/biweekly/monthly rebalancing. US champions: `rec_mfilt+deep_upvol`, `ma200+mom7m+rec126`, `rec_mfilt+ma200`, `mom7m+rec126`. CN champions: `up_cap+quality_mom`, `down_resil+qual_mom`, `rec63+mom_gap`, `up_cap+mom_gap` — these were mined on today's CSI 300 members; on PIT members they earn ~3-7% a year (2016-2026). All registered in trader.py as `fc_{signal}_{freq}` (e.g., `fc_rec_mfilt_deep_upvol_monthly`). 32 new strategies total.
**Metrics** (`metrics.py`): Standalone functions for portfolio analytics (Sharpe, Sortino, Calmar, max drawdown, etc.). `summary()` prints a formatted report and returns a dict.
@@ -84,7 +94,7 @@ uv run python -m unittest discover tests
- Strategies output **weights**, not buy/sell signals. Weights are normalized so each row sums to 1 (or 0 during warm-up).
- Warm-up periods: each strategy zeros out its first N rows where indicators are undefined.
- `data/` contains persistent price CSVs, universe JSON caches, and trader state files, plus `data/precomputed/` (precomputed backtests) and `data/web/` (watchlist, per-stock cache, backtest result cache). Not checked into git (only `data/.gitkeep` and `data/sp500_history.json` are tracked).
- `data/` contains persistent price CSVs, universe JSON caches, and trader state files, plus `data/precomputed/` (precomputed backtests), `data/web/` (watchlist, per-stock cache, backtest result cache) and `data/research/` (research panels, ~2 GB). Not checked into git (only `data/.gitkeep`, `data/sp500_history.json` and `data/csi300_history.json` are tracked).
- Python 3.12+, managed with `uv`.
- Do not show matplotlib figures when running backtests; use `--no-plot`.