quant-trading/CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Project Overview

quant-trading: a quantitative backtesting framework and live forward-testing trader for equity strategies across US (S&P 500) and China A-shares (CSI 300), plus a web UI to browse backtests and live books. Uses Yahoo Finance data with persistent local CSV storage.

The repository has two halves that share one data directory and one .env:

• Python engine (repo root, uv): strategies, backtests, trader.py, research scripts.
• Web UI and API (web/, Node + React + Vite): the API spawns uv run python bridge.py {list,backtest,live} at the repo root and reads data/precomputed/backtest_results.json, written by precompute_backtest.py. Both backtest paths run through backtest_runner.py.

bridge.py's JSON output and web/src/shared/strategyTypes.ts are one contract: change both in the same commit.

Commands

# Backtesting
uv run python main.py                    # Run US backtest (default)
uv run python main.py --market cn        # Run China A-share backtest
uv run python main.py --capital 50000    # Custom starting capital
uv run python main.py --top-n 20         # Override stock selection count
uv run python main.py --years 5          # Backtest last N years only
uv run python main.py --no-plot          # Skip matplotlib charts
uv run python main.py --fixed-fee 2.0    # Add $2 per-trade fixed fee
uv run python main.py --execution open-close  # Signal on open, execute at close

# Daily trading simulator
uv run python trader.py auto --market us --strategy recovery_mom_top10   # Single daily run (for cron)
uv run python trader.py morning --market us --strategy recovery_mom_top10  # Morning: generate orders
uv run python trader.py evening --market us --strategy recovery_mom_top10  # Evening: record execution
uv run python trader.py monitor                                            # Monitor ALL markets & strategies (for tmux)
uv run python trader.py monitor --market us                                # Monitor US only
uv run python trader.py status  --market us --strategy recovery_mom_top10  # Portfolio status
uv run python trader.py simulate --market us --strategy recovery_mom_top10 --start 2026-01-01 --end 2026-04-01  # Historical replay
uv run python trader.py log --market us --strategy sim_recovery_mom_top10 --start 2026-03-01  # View daily log
uv run python trader.py simulate --integer-shares --fixed-fee 2.0 ...    # Integer shares mode
uv run python trader.py compare --market us --book selected_2026_09      # A named live book

# Precompute the strategy page's backtests (writes data/precomputed/)
uv run python precompute_backtest.py

# Factor research harness (PIT universe, next-open fills, phase averaging; data in data/research/)
uv run python -m research.lab.fetch_us                 # 20y OHLCV for every S&P 500 ever-member
uv run python -m research.lab.fetch_cn_yahoo           # CSI 300/500 ever-members (membership from baostock snapshot)
PYTHONPATH=. uv run python -m research.lab.factor_scan --market us       # or cn_hs300
PYTHONPATH=. uv run python -m research.lab.stepwise --market us --objective maximin
PYTHONPATH=. uv run python -m research.lab.evaluate --market us          # candidates vs champions
uv run python -m research.lab.fetch_us_fundamentals   # SEC EDGAR point-in-time facts + Yahoo splits (US)
PYTHONPATH=. uv run python -m research.lab.rotation --market us          # walk-forward factor weighting schemes
PYTHONPATH=. uv run python -m research.lab.persistence                   # does last year's best combo persist?
PYTHONPATH=. uv run --with numba python -m research.lab.us_sweep --part overlays   # construction / overlay sweeps
PYTHONPATH=. uv run --with numba python -m research.lab.us_final         # 3-sleeve US portfolio check
PYTHONPATH=. uv run --with numba python -m research.lab.short_horizon    # short-horizon / residual sleeves, concentration
PYTHONPATH=. uv run python -m research.lab.levered_book                  # vol-managed leverage, beta-hedged alpha
PYTHONPATH=. uv run python -m research.lab.fetch_long_history [--no-proxy]  # indices from 1927/1985, ETFs, BTC, FRED yields
PYTHONPATH=. uv run python -m research.lab.letf_history                  # synthetic 3x ETFs from 1990; TrendRider stress test
PYTHONPATH=. uv run python -m research.lab.vol_trend                     # vol-targeted leveraged trend, walk-forward
PYTHONPATH=. uv run python -m research.lab.diversifiers                  # multi-asset ETF trend, bitcoin trend
PYTHONPATH=. uv run python -m research.lab.us_target50                   # the book 2006-2026 and the leverage 50%/yr needs
PYTHONPATH=. uv run --with numba python -m research.lab.qqq_scan         # alpha vs QQQ of every sleeve / factor / mega-cap variant
PYTHONPATH=. uv run --with numba python -m research.lab.beat_qqq         # sleeves picked on 2006-16, judged 2017-26; odds of beating QQQ
PYTHONPATH=. uv run --with numba python -m research.lab.momentum_fundamentals [--sensitivity]  # financial-statement vetoes/confirmations on overnight_recovery_top10
uv run python -m research.lab.fetch_insider && PYTHONPATH=. uv run python -m research.lab.insider        # SEC Form 4 insider factors
PYTHONPATH=. uv run python -m research.lab.fetch_lazy_prices && PYTHONPATH=. uv run python -m research.lab.lazy_prices  # 10-K/10-Q text similarity, 8-K earnings dates, SIC
PYTHONPATH=. uv run --with numba python -m research.lab.earnings         # earnings-announcement return (needs sec_8k_earnings.csv)
PYTHONPATH=. uv run --with numba python -m research.lab.industry         # industry momentum on SIC groups
PYTHONPATH=. uv run python -m research.lab.fetch_us_mid holdings|prices|status   # PIT S&P 400/600 from iShares holdings (research/lab/panel_mid.py)

# Setup
uv sync                                  # Install/sync dependencies
cp .env.example .env                     # Then fill IPADS_API_KEY and QUANT_MARKET_PROXY
(cd web && npm ci)

# Tests
uv run python -m unittest discover tests
(cd web && npm test && npm run build)

# Web UI in development (API :8787, Vite :5173)
(cd web && npm run dev)

Architecture

Universe provider (universe.py): Dynamically fetches index constituents (S&P 500 from Wikipedia; CSI 300 from China Securities Index's published constituent file, falling back to the latest data/csi300_history.json snapshot, then Wikipedia, whose table lagged the index by ~60 names in 2026) and converts to Yahoo Finance ticker format. Constituent lists are cached daily in data/universe_*.json. New markets are added by registering in UNIVERSES dict.

Point-in-time membership (universe_history.py): S&P 500 intervals rebuilt from Wikipedia's "Historical components of the S&P 500" change log into data/sp500_history.json; CSI 300 monthly snapshots in data/csi300_history.json (seeded from baostock 2008–2026, appended whenever the current list changes). mask_prices_pit(prices, market) masks non-member dates.

Data manager (data_manager.py): Persistent price storage in data/{market}.csv (close) and data/{market}_{open,high,low,volume}.csv. First run downloads 10 years of history; every run re-downloads the last 30 days. New tickers from index rebalances are backfilled automatically; tickers missing from a partial batch are retried alone before being marked unavailable (retried weekly). update(market, tickers, with_open=True) returns (close, open); update(..., fields=[...]) returns {field: panel}.

Backtest engine (main.py): Orchestrates data loading, strategy execution, and visualization. backtest() has two modes. Close mode (no open_data, used by main.py and old research scripts) is vectorized and trades at the close the signal saw. Live mode (open_data given, used by the web UI through backtest_runner.py) keeps the book like trader.py: same signal timing (strategy_positions), holdings that drift, the $50/0.1% order floor, trader.RESIZE_BAND, fees on the orders placed (simulate_live_book).

Daily trader (trader.py): Live/forward-testing system with persistent portfolio state in data/trader_{market}_{strategy}.json. The auto subcommand runs both signal generation and execution in a single invocation — designed for cron. The simulate subcommand replays a date range day-by-day with realistic portfolio tracking (fractional shares, cash, commissions). Orders: whole A-share lots are allocated nearest-lot within cash (_allocate_lots); a held name is resized only when ≥ RESIZE_BAND (10%) off target. Available strategies: recovery_mom_top10, recovery_mom_top20, momentum, momentum_quality, dual_momentum, inverse_vol, trend_following, buy_and_hold, 32 factor-combo strategies (fc_{signal}_{freq} — see strategies/factor_combo.py), the robust-momentum bar strategies overnight_recovery_top10, lowrisk_momentum_top15, lowbeta_sharpe_mom_top20, lowrisk_top20, and the US sleeve portfolios us_three_sleeve and us_momentum_qqq_trend.

Live books (trader.LIVE_BOOKS): a book is a cohort of strategies, each with its own account, started on one date with one capital and a flat fee per order. The original book (every live strategy, MONITOR_CAPITAL, MARKET_FEES) keeps state in data/trader_*.json; a named book keeps it in data/books/<book>/. The monitor runs every book; auto/status/log/compare/replay and bridge.py live take --book, and bridge.py books lists them for the UI's book selector. selected_2026_09 (29 strategies, US $10,000 / CN ¥100,000, $2 / ¥10 per order) starts on 2026-09-25 (US) and 2026-09-28 (CN).

Strategy protocol (strategies/base.py): All strategies inherit from Strategy ABC and implement generate_signals(data) → DataFrame where the returned DataFrame contains portfolio weights (rows = dates, columns = assets, values sum to ~1.0 per row). Each strategy is responsible for applying its own 1-day lag via .shift(1) to avoid lookahead bias — the backtest engine does not shift. A BarStrategy declares fields (close/open/high/low/volume) and receives {field: DataFrame}; live it uses completed bars only (row d from bars through d-1, traded at the close of d), while close-price strategies read the session's open as their latest price.

Strategies:

• buy_and_hold.py — Equal-weight static allocation
• momentum.py — Cross-sectional momentum (12-1 month factor), selects top_n winners
• inverse_vol.py — Inverse-volatility (risk parity) weighting
• multi_factor.py — Combines momentum + value factors with benchmark MA timing filter
• mean_reversion.py — Short-term mean reversion
• trend_following.py — MA crossover + momentum trend filter
• dual_momentum.py — Absolute + relative momentum
• momentum_quality.py — Momentum + return consistency + low drawdown
• adaptive_momentum.py — Momentum weighted by inverse volatility
• recovery_momentum.py — Recovery (price/63d low) + 12-1mo momentum composite. Best US performer.
• robust_momentum.py — Bar strategies from the 2026-09 PIT research (report: https://claude.ai/artifact/Mo9nrzs5SWj5FfyshERyWS): US overnight_recovery (overnight-return momentum + distance from 126d low), CN lowrisk_momentum (amplitude-cut momentum + low beta + price-volume corr + momentum/vol) and close-only lowbeta_sharpe_mom. Top-N with rank hysteresis, rebalance every 10 days, weights drift between rebalances. low_risk (US lowrisk_top20: idiosyncratic vol + low beta, top 20, monthly) is the defensive sleeve of the 2026-09-29 US research (report: https://claude.ai/artifact/VGhZvKkS9UBYZkLeqGjcNC).
• sleeve_portfolio.py — Several strategies in one account with fixed capital shares. us_three_sleeve = 1/3 each of overnight_recovery_top10, lowrisk_top20, trend_rider_v7_vt24 (a mixed stock+ETF strategy); sleeve shares drift with their returns and reset at a month start once any is >25% off. 2011-2026 research: Sharpe 1.21 vs 0.88-1.03 for the sleeves alone. Research finding behind it: choosing factors by their recent record (rotation, factor momentum, last year's winner) did not beat a fixed composite; construction knobs (N, weighting, dynamic N) barely matter. On synthetic 3x ETFs from 1990 (research/lab/letf_history.py), TrendRider V7's 2016-2026 record does not hold: 1990-99 Sharpe 0.2-0.3, 2003-07 negative, 1990-2026 about 0.7 with MaxDD -58% (vt24) to -69% (vt36).
• qqq_trend.py — QQQVolTrend: QQQ exposure = 1{QQQ > 200-day average} × min(1.5, 30% / EWMA vol), held as TQQQ (E/3) + SHY. 1990-2026 on synthetic ETFs: 16.0% vs QQQ 14.9%, MaxDD -37% vs -83%; 2011-2026 on real ETFs it trails QQQ (17.5% vs 19.1%) with a smaller drawdown. us_momentum_qqq_trend (in sleeve_portfolio.py) holds it half and half with overnight_recovery_top10. Research behind it (research/lab/qqq_scan.py, beat_qqq.py): measured as alpha against QQQ with a 2006-16 design / 2017-26 holdout split, momentum top 10 is the only stock selection whose alpha held (+10%/yr, t 1.4); single factors, walk-forward ML, a mega-cap universe, SEC insider trades, 10-K/10-Q text similarity, earnings-announcement returns, industry momentum and a PIT S&P 400/600 universe (panel_mid.py) showed none. In the web engine (2016-09 to 2026-09, $2/order) us_momentum_qqq_trend earns 21.0% vs QQQ 20.9%, MaxDD -32%.
• factor_combo.py — Configurable factor-combination strategies with daily/weekly/biweekly/monthly rebalancing. US champions: rec_mfilt+deep_upvol, ma200+mom7m+rec126, rec_mfilt+ma200, mom7m+rec126. CN champions: up_cap+quality_mom, down_resil+qual_mom, rec63+mom_gap, up_cap+mom_gap — these were mined on today's CSI 300 members; on PIT members they earn ~3-7% a year (2016-2026). All registered in trader.py as fc_{signal}_{freq} (e.g., fc_rec_mfilt_deep_upvol_monthly). 32 new strategies total.

Metrics (metrics.py): Standalone functions for portfolio analytics (Sharpe, Sortino, Calmar, max drawdown, etc.). summary() prints a formatted report and returns a dict.

Key Conventions

• Strategies output weights, not buy/sell signals. Weights are normalized so each row sums to 1 (or 0 during warm-up).
• Warm-up periods: each strategy zeros out its first N rows where indicators are undefined.
• data/ contains persistent price CSVs, universe JSON caches, and trader state files, plus data/precomputed/ (precomputed backtests), data/web/ (watchlist, per-stock cache, backtest result cache) and data/research/ (research panels, ~2 GB). Not checked into git (only data/.gitkeep and data/csi300_history.json are tracked; data/sp500_history.json is rebuilt from Wikipedia on first use).
• Python 3.12+, managed with uv.
• Do not show matplotlib figures when running backtests; use --no-plot.

Server Deployment

Three systemd --user services, defined in deploy/systemd/, run from /home/gahow/projects/quant-trading:

Service	Script	Role
quant-trading-monitor	deploy/start-monitor.sh	trader.py monitor: morning/evening phases for US + CN, all live strategies
quant-trading-api	deploy/start-web-api.sh	Web API on 127.0.0.1:8787
quant-trading-vite	deploy/start-web-vite.sh	Vite on 127.0.0.1:5174, behind nginx on :5443

cp deploy/systemd/*.service ~/.config/systemd/user/
systemctl --user daemon-reload
systemctl --user enable --now quant-trading-monitor quant-trading-api quant-trading-vite
journalctl --user -u quant-trading-monitor -f

The monitor catches up on startup, so restarting it between phases is safe. If it starts mid-day (after open, before close), it falls back to auto mode for the evening phase.

Compare live books after a while:

uv run python trader.py compare              # US strategies
uv run python trader.py compare --market cn  # CN strategies

Alternative: cron

Run daily after market close. The auto command is idempotent — safe to re-run:

# US market: 5pm ET weekdays
0 17 * * 1-5  cd /home/gahow/projects/quant-trading && uv run python trader.py auto --market us --strategy recovery_mom_top10 >> data/cron.log 2>&1

# CN market: 4pm CST weekdays
0 16 * * 1-5  cd /home/gahow/projects/quant-trading && uv run python trader.py auto --market cn --strategy inverse_vol >> data/cron.log 2>&1