# CLAUDE.md This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. ## Project Overview quant-trading: a quantitative backtesting framework and live forward-testing trader for equity strategies across US (S&P 500) and China A-shares (CSI 300), plus a web UI to browse backtests and live books. Uses Yahoo Finance data with persistent local CSV storage. The repository has two halves that share one data directory and one `.env`: - **Python engine** (repo root, `uv`): strategies, backtests, `trader.py`, research scripts. - **Web UI and API** (`web/`, Node + React + Vite): the API spawns `uv run python bridge.py {list,backtest,live}` at the repo root and reads `data/precomputed/backtest_results.json`, written by `precompute_backtest.py`. Both backtest paths run through `backtest_runner.py`. `bridge.py`'s JSON output and `web/src/shared/strategyTypes.ts` are one contract: change both in the same commit. ## Commands ```bash # Backtesting uv run python main.py # Run US backtest (default) uv run python main.py --market cn # Run China A-share backtest uv run python main.py --capital 50000 # Custom starting capital uv run python main.py --top-n 20 # Override stock selection count uv run python main.py --years 5 # Backtest last N years only uv run python main.py --no-plot # Skip matplotlib charts uv run python main.py --fixed-fee 2.0 # Add $2 per-trade fixed fee uv run python main.py --execution open-close # Signal on open, execute at close # Daily trading simulator uv run python trader.py auto --market us --strategy recovery_mom_top10 # Single daily run (for cron) uv run python trader.py morning --market us --strategy recovery_mom_top10 # Morning: generate orders uv run python trader.py evening --market us --strategy recovery_mom_top10 # Evening: record execution uv run python trader.py monitor # Monitor ALL markets & strategies (for tmux) uv run python trader.py monitor --market us # Monitor US only uv run python trader.py status --market us --strategy recovery_mom_top10 # Portfolio status uv run python trader.py simulate --market us --strategy recovery_mom_top10 --start 2026-01-01 --end 2026-04-01 # Historical replay uv run python trader.py log --market us --strategy sim_recovery_mom_top10 --start 2026-03-01 # View daily log uv run python trader.py simulate --integer-shares --fixed-fee 2.0 ... # Integer shares mode uv run python trader.py compare --market us --book selected_2026_09 # A named live book # Precompute the strategy page's backtests (writes data/precomputed/) uv run python precompute_backtest.py # Factor research harness (PIT universe, next-open fills, phase averaging; data in data/research/) uv run python -m research.lab.fetch_us # 20y OHLCV for every S&P 500 ever-member uv run python -m research.lab.fetch_cn_yahoo # CSI 300/500 ever-members (membership from baostock snapshot) PYTHONPATH=. uv run python -m research.lab.factor_scan --market us # or cn_hs300 PYTHONPATH=. uv run python -m research.lab.stepwise --market us --objective maximin PYTHONPATH=. uv run python -m research.lab.evaluate --market us # candidates vs champions uv run python -m research.lab.fetch_us_fundamentals # SEC EDGAR point-in-time facts + Yahoo splits (US) PYTHONPATH=. uv run python -m research.lab.rotation --market us # walk-forward factor weighting schemes PYTHONPATH=. uv run python -m research.lab.persistence # does last year's best combo persist? PYTHONPATH=. uv run --with numba python -m research.lab.us_sweep --part overlays # construction / overlay sweeps PYTHONPATH=. uv run --with numba python -m research.lab.us_final # 3-sleeve US portfolio check PYTHONPATH=. uv run --with numba python -m research.lab.short_horizon # short-horizon / residual sleeves, concentration PYTHONPATH=. uv run python -m research.lab.levered_book # vol-managed leverage, beta-hedged alpha PYTHONPATH=. uv run python -m research.lab.fetch_long_history [--no-proxy] # indices from 1927/1985, ETFs, BTC, FRED yields PYTHONPATH=. uv run python -m research.lab.letf_history # synthetic 3x ETFs from 1990; TrendRider stress test PYTHONPATH=. uv run python -m research.lab.vol_trend # vol-targeted leveraged trend, walk-forward PYTHONPATH=. uv run python -m research.lab.diversifiers # multi-asset ETF trend, bitcoin trend PYTHONPATH=. uv run python -m research.lab.us_target50 # the book 2006-2026 and the leverage 50%/yr needs PYTHONPATH=. uv run --with numba python -m research.lab.qqq_scan # alpha vs QQQ of every sleeve / factor / mega-cap variant PYTHONPATH=. uv run --with numba python -m research.lab.beat_qqq # sleeves picked on 2006-16, judged 2017-26; odds of beating QQQ PYTHONPATH=. uv run --with numba python -m research.lab.momentum_fundamentals [--sensitivity] # financial-statement vetoes/confirmations on overnight_recovery_top10 uv run python -m research.lab.fetch_insider && PYTHONPATH=. uv run python -m research.lab.insider # SEC Form 4 insider factors PYTHONPATH=. uv run python -m research.lab.fetch_lazy_prices && PYTHONPATH=. uv run python -m research.lab.lazy_prices # 10-K/10-Q text similarity, 8-K earnings dates, SIC PYTHONPATH=. uv run --with numba python -m research.lab.earnings # earnings-announcement return (needs sec_8k_earnings.csv) PYTHONPATH=. uv run --with numba python -m research.lab.industry # industry momentum on SIC groups PYTHONPATH=. uv run python -m research.lab.fetch_us_mid holdings|prices|status # PIT S&P 400/600 from iShares holdings (research/lab/panel_mid.py) # Setup uv sync # Install/sync dependencies cp .env.example .env # Then fill IPADS_API_KEY and QUANT_MARKET_PROXY (cd web && npm ci) # Tests uv run python -m unittest discover tests (cd web && npm test && npm run build) # Web UI in development (API :8787, Vite :5173) (cd web && npm run dev) ``` ## Architecture **Universe provider** (`universe.py`): Dynamically fetches index constituents (S&P 500 from Wikipedia; CSI 300 from China Securities Index's published constituent file, falling back to the latest `data/csi300_history.json` snapshot, then Wikipedia, whose table lagged the index by ~60 names in 2026) and converts to Yahoo Finance ticker format. Constituent lists are cached daily in `data/universe_*.json`. New markets are added by registering in `UNIVERSES` dict. **Point-in-time membership** (`universe_history.py`): S&P 500 intervals rebuilt from Wikipedia's "Historical components of the S&P 500" change log into `data/sp500_history.json`; CSI 300 monthly snapshots in `data/csi300_history.json` (seeded from baostock 2008–2026, appended whenever the current list changes). `mask_prices_pit(prices, market)` masks non-member dates. **Data manager** (`data_manager.py`): Persistent price storage in `data/{market}.csv` (close) and `data/{market}_{open,high,low,volume}.csv`. First run downloads 10 years of history; every run re-downloads the last 30 days. New tickers from index rebalances are backfilled automatically; tickers missing from a partial batch are retried alone before being marked unavailable (retried weekly). `update(market, tickers, with_open=True)` returns (close, open); `update(..., fields=[...])` returns `{field: panel}`. **Backtest engine** (`main.py`): Orchestrates data loading, strategy execution, and visualization. `backtest()` has two modes. Close mode (no `open_data`, used by `main.py` and old research scripts) is vectorized and trades at the close the signal saw. Live mode (`open_data` given, used by the web UI through `backtest_runner.py`) keeps the book like `trader.py`: same signal timing (`strategy_positions`), holdings that drift, the $50/0.1% order floor, `trader.RESIZE_BAND`, fees on the orders placed (`simulate_live_book`). **Daily trader** (`trader.py`): Live/forward-testing system with persistent portfolio state in `data/trader_{market}_{strategy}.json`. The `auto` subcommand runs both signal generation and execution in a single invocation — designed for cron. The `simulate` subcommand replays a date range day-by-day with realistic portfolio tracking (fractional shares, cash, commissions). Orders: whole A-share lots are allocated nearest-lot within cash (`_allocate_lots`); a held name is resized only when ≥ `RESIZE_BAND` (10%) off target. Available strategies: `recovery_mom_top10`, `recovery_mom_top20`, `momentum`, `momentum_quality`, `dual_momentum`, `inverse_vol`, `trend_following`, `buy_and_hold`, 32 factor-combo strategies (`fc_{signal}_{freq}` — see `strategies/factor_combo.py`), the robust-momentum bar strategies `overnight_recovery_top10`, `lowrisk_momentum_top15`, `lowbeta_sharpe_mom_top20`, `lowrisk_top20`, and the US sleeve portfolios `us_three_sleeve` and `us_momentum_qqq_trend`. **Live books** (`trader.LIVE_BOOKS`): a book is a cohort of strategies, each with its own account, started on one date with one capital and a flat fee per order. The original book (every live strategy, `MONITOR_CAPITAL`, `MARKET_FEES`) keeps state in `data/trader_*.json`; a named book keeps it in `data/books//`. The monitor runs every book; `auto`/`status`/`log`/`compare`/`replay` and `bridge.py live` take `--book`, and `bridge.py books` lists them for the UI's book selector. `selected_2026_09` (29 strategies, US $10,000 / CN ¥100,000, $2 / ¥10 per order) starts on 2026-09-25 (US) and 2026-09-28 (CN). **Strategy protocol** (`strategies/base.py`): All strategies inherit from `Strategy` ABC and implement `generate_signals(data) → DataFrame` where the returned DataFrame contains portfolio weights (rows = dates, columns = assets, values sum to ~1.0 per row). Each strategy is responsible for applying its own 1-day lag via `.shift(1)` to avoid lookahead bias — the backtest engine does not shift. A `BarStrategy` declares `fields` (close/open/high/low/volume) and receives `{field: DataFrame}`; live it uses completed bars only (row d from bars through d-1, traded at the close of d), while close-price strategies read the session's open as their latest price. **Strategies**: - `buy_and_hold.py` — Equal-weight static allocation - `momentum.py` — Cross-sectional momentum (12-1 month factor), selects top_n winners - `inverse_vol.py` — Inverse-volatility (risk parity) weighting - `multi_factor.py` — Combines momentum + value factors with benchmark MA timing filter - `mean_reversion.py` — Short-term mean reversion - `trend_following.py` — MA crossover + momentum trend filter - `dual_momentum.py` — Absolute + relative momentum - `momentum_quality.py` — Momentum + return consistency + low drawdown - `adaptive_momentum.py` — Momentum weighted by inverse volatility - `recovery_momentum.py` — Recovery (price/63d low) + 12-1mo momentum composite. Best US performer. - `robust_momentum.py` — Bar strategies from the 2026-09 PIT research (report: https://claude.ai/artifact/Mo9nrzs5SWj5FfyshERyWS): US `overnight_recovery` (overnight-return momentum + distance from 126d low), CN `lowrisk_momentum` (amplitude-cut momentum + low beta + price-volume corr + momentum/vol) and close-only `lowbeta_sharpe_mom`. Top-N with rank hysteresis, rebalance every 10 days, weights drift between rebalances. `low_risk` (US `lowrisk_top20`: idiosyncratic vol + low beta, top 20, monthly) is the defensive sleeve of the 2026-09-29 US research (report: https://claude.ai/artifact/VGhZvKkS9UBYZkLeqGjcNC). - `sleeve_portfolio.py` — Several strategies in one account with fixed capital shares. `us_three_sleeve` = 1/3 each of `overnight_recovery_top10`, `lowrisk_top20`, `trend_rider_v7_vt24` (a mixed stock+ETF strategy); sleeve shares drift with their returns and reset at a month start once any is >25% off. 2011-2026 research: Sharpe 1.21 vs 0.88-1.03 for the sleeves alone. Research finding behind it: choosing factors by their recent record (rotation, factor momentum, last year's winner) did not beat a fixed composite; construction knobs (N, weighting, dynamic N) barely matter. On synthetic 3x ETFs from 1990 (`research/lab/letf_history.py`), TrendRider V7's 2016-2026 record does not hold: 1990-99 Sharpe 0.2-0.3, 2003-07 negative, 1990-2026 about 0.7 with MaxDD -58% (vt24) to -69% (vt36). - `qqq_trend.py` — `QQQVolTrend`: QQQ exposure = 1{QQQ > 200-day average} × min(1.5, 30% / EWMA vol), held as TQQQ (E/3) + SHY. 1990-2026 on synthetic ETFs: 16.0% vs QQQ 14.9%, MaxDD -37% vs -83%; 2011-2026 on real ETFs it trails QQQ (17.5% vs 19.1%) with a smaller drawdown. `us_momentum_qqq_trend` (in `sleeve_portfolio.py`) holds it half and half with `overnight_recovery_top10`. Research behind it (`research/lab/qqq_scan.py`, `beat_qqq.py`): measured as alpha against QQQ with a 2006-16 design / 2017-26 holdout split, momentum top 10 is the only stock selection whose alpha held (+10%/yr, t 1.4); single factors, walk-forward ML, a mega-cap universe, SEC insider trades, 10-K/10-Q text similarity, earnings-announcement returns, industry momentum and a PIT S&P 400/600 universe (`panel_mid.py`) showed none. In the web engine (2016-09 to 2026-09, $2/order) `us_momentum_qqq_trend` earns 21.0% vs QQQ 20.9%, MaxDD -32%. - `factor_combo.py` — Configurable factor-combination strategies with daily/weekly/biweekly/monthly rebalancing. US champions: `rec_mfilt+deep_upvol`, `ma200+mom7m+rec126`, `rec_mfilt+ma200`, `mom7m+rec126`. CN champions: `up_cap+quality_mom`, `down_resil+qual_mom`, `rec63+mom_gap`, `up_cap+mom_gap` — these were mined on today's CSI 300 members; on PIT members they earn ~3-7% a year (2016-2026). All registered in trader.py as `fc_{signal}_{freq}` (e.g., `fc_rec_mfilt_deep_upvol_monthly`). 32 new strategies total. **Metrics** (`metrics.py`): Standalone functions for portfolio analytics (Sharpe, Sortino, Calmar, max drawdown, etc.). `summary()` prints a formatted report and returns a dict. ## Key Conventions - Strategies output **weights**, not buy/sell signals. Weights are normalized so each row sums to 1 (or 0 during warm-up). - Warm-up periods: each strategy zeros out its first N rows where indicators are undefined. - `data/` contains persistent price CSVs, universe JSON caches, and trader state files, plus `data/precomputed/` (precomputed backtests), `data/web/` (watchlist, per-stock cache, backtest result cache) and `data/research/` (research panels, ~2 GB). Not checked into git (only `data/.gitkeep` is tracked). `data/sp500_history.json` is rebuilt from Wikipedia on first use; `data/csi300_history.json` is not rebuilt automatically (its 2008-2026 snapshots came from baostock), so copy it to a new machine or CN membership fails to load. - Python 3.12+, managed with `uv`. - Do not show matplotlib figures when running backtests; use `--no-plot`. ## Server Deployment Three `systemd --user` services, defined in `deploy/systemd/`, run from `/home/gahow/projects/quant-trading`: | Service | Script | Role | |---|---|---| | `quant-trading-monitor` | `deploy/start-monitor.sh` | `trader.py monitor`: morning/evening phases for US + CN, all live strategies | | `quant-trading-api` | `deploy/start-web-api.sh` | Web API on `127.0.0.1:8787` | | `quant-trading-vite` | `deploy/start-web-vite.sh` | Vite on `127.0.0.1:5174`, behind nginx on `:5443` | ```bash cp deploy/systemd/*.service ~/.config/systemd/user/ systemctl --user daemon-reload systemctl --user enable --now quant-trading-monitor quant-trading-api quant-trading-vite journalctl --user -u quant-trading-monitor -f ``` The monitor catches up on startup, so restarting it between phases is safe. If it starts mid-day (after open, before close), it falls back to `auto` mode for the evening phase. Compare live books after a while: ```bash uv run python trader.py compare # US strategies uv run python trader.py compare --market cn # CN strategies ``` ### Alternative: cron Run daily after market close. The `auto` command is idempotent — safe to re-run: ```bash # US market: 5pm ET weekdays 0 17 * * 1-5 cd /home/gahow/projects/quant-trading && uv run python trader.py auto --market us --strategy recovery_mom_top10 >> data/cron.log 2>&1 # CN market: 4pm CST weekdays 0 16 * * 1-5 cd /home/gahow/projects/quant-trading && uv run python trader.py auto --market cn --strategy inverse_vol >> data/cron.log 2>&1 ```