gahow/quant-trading
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

live: record each session's target and its frictionless return; compare shows Ideal and Gap

Browse Source 
A $10k book in whole shares holds SNDK at 17.8% for a 10% target, so the
live record mixes the strategy with rounding and fee noise (2026-09-25..10-12:
overnight_recovery_top10 -9.0% live vs -7.65% at its own weights). Each
daily_log entry now stores the target held from that close and ideal_return,
the previous target's return to this close with fractional shares, no fees,
no resize band and no idle cash. compare prints Ideal and Gap (live minus
ideal) over the recorded days. The recorded returns compound to the
frictionless ledger (new contract test); an in-memory rerun of 10-09/10-12 on
live states reproduces the recorded equity to the cent.

systemd: SuccessExitStatus=143 on the monitor and overnight-paper units, so a
stop or restart no longer logs 'Failed' and the journal shows only crashes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Gahow Wang
2026-10-12 18:49:28 -07:00
parent a605d68ca8
commit 31a684fe70
5 changed files with 80 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
CLAUDE.md
View File

@@ -166,7 +166,7 @@ uv run python -m unittest discover tests
`precompute_backtest.py` reruns the ledger for cash-injection scenarios; metrics use unit NAV after actual fees. The API and `web/src/shared/strategyTypes.ts` share this cash-flow contract. `ENGINE_VERSION` in `execution.py` identifies the execution protocol; API cache keys also include its version.
**Daily trader** (`trader.py`): Live/forward-testing system with persistent portfolio state in `data/trader_{market}_{strategy}.json`. The `auto` subcommand runs both signal generation and execution in a single invocation — designed for cron. The `simulate` subcommand replays a date range day-by-day with realistic portfolio tracking (fractional shares, cash, commissions). Orders: whole A-share lots are allocated nearest-lot within cash (`_allocate_lots`); a held name is resized only when ≥ `RESIZE_BAND` (10%) off target. Available strategies: `recovery_mom_top10`, `recovery_mom_top20`, `momentum`, `momentum_quality`, `dual_momentum`, `inverse_vol`, `trend_following`, `buy_and_hold`, 32 factor-combo strategies (`fc_{signal}_{freq}` — see `strategies/factor_combo.py`), the robust-momentum bar strategies `overnight_recovery_top10`, `lowrisk_momentum_top15`, `lowbeta_sharpe_mom_top20`, `lowrisk_top20`, the US sleeve portfolios `us_three_sleeve` and `us_momentum_qqq_trend`, and the agent-reviewed `overnight_recovery_agent_top10` (US only, forward only).
**Daily trader** (`trader.py`): Live/forward-testing system with persistent portfolio state in `data/trader_{market}_{strategy}.json`. The `auto` subcommand runs both signal generation and execution in a single invocation — designed for cron. The `simulate` subcommand replays a date range day-by-day with realistic portfolio tracking (fractional shares, cash, commissions). Orders: whole A-share lots are allocated nearest-lot within cash (`_allocate_lots`); a held name is resized only when ≥ `RESIZE_BAND` (10%) off target. Each `daily_log` entry also records the session `target` and `ideal_return` (what the previous target earned to this close with fractional shares, no fees, no resize band and no idle cash; recorded from 2026-10-13); `compare` prints `Ideal` and `Gap` (live minus ideal) over those days, separating the strategy from $10k whole-share and fee noise. Available strategies: `recovery_mom_top10`, `recovery_mom_top20`, `momentum`, `momentum_quality`, `dual_momentum`, `inverse_vol`, `trend_following`, `buy_and_hold`, 32 factor-combo strategies (`fc_{signal}_{freq}` — see `strategies/factor_combo.py`), the robust-momentum bar strategies `overnight_recovery_top10`, `lowrisk_momentum_top15`, `lowbeta_sharpe_mom_top20`, `lowrisk_top20`, the US sleeve portfolios `us_three_sleeve` and `us_momentum_qqq_trend`, and the agent-reviewed `overnight_recovery_agent_top10` (US only, forward only).
**Live books** (`trader.LIVE_BOOKS`): a book is a cohort of strategies, each with its own account, started on one date with one capital and a flat fee per order. The original book (every live strategy, `MONITOR_CAPITAL`, `MARKET_FEES`) keeps state in `data/trader_*.json`; a named book keeps it in `data/books/<book>/`. The monitor runs every book; `auto`/`status`/`log`/`compare`/`replay` and `bridge.py live` take `--book`, and `bridge.py books` lists them for the UI's book selector. `selected_2026_09` (30 strategies, US $10,000 / CN ¥100,000, $2 / ¥10 per order) starts on 2026-09-25 (US) and 2026-09-28 (CN); `overnight_recovery_agent_top10` joined it on 2026-09-30 and starts with the book.

3
deploy/systemd/quant-trading-monitor.service
View File

@@ -7,6 +7,9 @@ Type=simple
WorkingDirectory=/home/gahow/projects/quant-trading
ExecStart=/home/gahow/projects/quant-trading/deploy/start-monitor.sh
Restart=always
# SIGTERM from a stop/restart ends the Python process with 143; count it as a
# clean stop so 'Failed' in the journal means a real crash.
SuccessExitStatus=143
RestartSec=10
Environment=PYTHONUNBUFFERED=1

3
deploy/systemd/quant-trading-overnight-paper.service
View File

@@ -7,6 +7,9 @@ Type=simple
WorkingDirectory=/home/gahow/projects/quant-trading
ExecStart=/home/gahow/projects/quant-trading/deploy/start-overnight-paper.sh
Restart=always
# SIGTERM from a stop/restart ends the Python process with 143; count it as a
# clean stop so 'Failed' in the journal means a real crash.
SuccessExitStatus=143
RestartSec=30
Environment=PYTHONUNBUFFERED=1

20
tests/test_execution_contract.py
View File

@@ -52,6 +52,26 @@ class ExecutionContracts(unittest.TestCase):
self.assertAlmostEqual(state["daily_equity"][str(day.date())], batch["equity"].loc[day], places=2)
self.assertEqual(state["trade_log"], batch["trade_log"])
def test_recorded_ideal_return_compounds_to_the_frictionless_ledger(self):
dates = pd.bdate_range("2026-01-05", periods=6)
prices = pd.DataFrame({"A": [100, 110, 104, 70, 90, 95], "B": [20, 21, 22, 22, 23, 21.5]}, index=dates)
targets = pd.DataFrame([[.5, .5], [.5, .5], [0, .9], [0, 1], [.6, .4], [.6, .4]],
index=dates, columns=prices.columns)
ideal = simulate_book(targets, prices, 10000, transaction_cost=0, fixed_fee=0, integer_shares=False,
resize_band=0, min_trade_value=0)
state = trader.init_state("us", "test", 10000)
for day in dates: # live-like: whole shares, fees, the 10% band
trader._run_session(state, "us", day, "B", prices, prices, {"tx_cost": .001, "fixed_fee": 2.},
True, targets.loc[day])
log = state["daily_log"]
self.assertNotIn("ideal_return", log[0])
self.assertEqual(log[2]["target"], {"B": 0.9})
growth = np.cumprod([1 + e["ideal_return"] for e in log[1:]])
np.testing.assert_allclose(growth, ideal["equity"].iloc[1:] / ideal["equity"].iloc[0], rtol=1e-6)
window = trader._ideal_window(state)
self.assertEqual(window["start"], str(dates[0].date()))
self.assertAlmostEqual(window["ideal"], growth[-1] - 1)
def test_a_delisted_stock_is_settled_at_its_last_print_in_both_ledgers(self):
dates = pd.bdate_range("2026-01-05", periods=9)
# A is acquired: last print on day 2, out of the index from day 4.

59
trader.py
View File

@@ -734,11 +734,15 @@ def bar_session_weights(strategy: BarStrategy, bars: dict, days: pd.DatetimeInde
def record_daily_snapshot(state: dict, date_str: str, prices: dict,
trades: list[dict], prev_equity: float) -> None:
trades: list[dict], prev_equity: float,
target: dict | None = None, ideal_return: float | None = None) -> None:
"""Append a daily snapshot to state['daily_log'].
Each entry captures the full picture: date, holdings with prices/values,
cash, total equity, daily return, and today's operations.
cash, total equity, daily return, and today's operations. ``target`` is the
weight the strategy asked to hold from this close, and ``ideal_return`` what
the previous entry's target earned to this close without whole shares, fees,
the resize band or idle cash (see ``_ideal_return``).
"""
holdings = state["holdings"]
cash = state["cash"]
@@ -778,6 +782,10 @@ def record_daily_snapshot(state: dict, date_str: str, prices: dict,
"holdings": positions,
"operations": operations,
}
if target is not None:
entry["target"] = {t: round(float(w), 6) for t, w in sorted(target.items())}
if ideal_return is not None:
entry["ideal_return"] = round(float(ideal_return), 8)
if "daily_log" not in state:
state["daily_log"] = []
@@ -797,6 +805,33 @@ def get_prices_for_date(tickers: list[str], date_idx, price_df: pd.DataFrame) ->
return {t: float(row[t]) for t in tickers if t in row.index and pd.notna(row[t])}
def _ideal_return(prev_entry: dict | None, day: pd.Timestamp, close_data: pd.DataFrame) -> float | None:
"""Return of ``prev_entry``'s target weights from its close to ``day``'s
close: the strategy's own weights held frictionlessly (fractional shares,
no fees, no resize band; the unallocated part earns nothing). A name
without a price on either date contributes 0. None when the previous entry
recorded no target (state written before targets were recorded)."""
if not prev_entry or "target" not in prev_entry or prev_entry["date"] >= str(day.date()):
return None
w = prev_entry["target"]
if not w:
return 0.0
p0 = get_prices_for_date(list(w), pd.Timestamp(prev_entry["date"]), close_data)
p1 = get_prices_for_date(list(w), day, close_data)
return float(sum(x * (p1[t] / p0[t] - 1) for t, x in w.items() if t in p0 and t in p1 and p0[t] > 0))
def _ideal_window(state: dict) -> dict | None:
"""Live vs ideal over the days whose ``ideal_return`` is recorded."""
log = state.get("daily_log") or []
k = next((i for i, e in enumerate(log) if e.get("ideal_return") is not None), None)
if k is None or k == 0:
return None
ideal = float(np.prod([1 + e.get("ideal_return", 0.0) for e in log[k:]]) - 1)
live = log[-1]["equity"] / log[k - 1]["equity"] - 1
return {"start": log[k - 1]["date"], "live": live, "ideal": ideal, "gap": live - ideal}
def _with_last_known_prices(state: dict, prices: dict) -> dict:
"""Fill prices of held tickers missing from the panel with their last recorded price.
@@ -923,7 +958,9 @@ def _run_session(state: dict, market: str, day: pd.Timestamp, benchmark: str,
previous = [d for d in sorted(state["daily_equity"]) if d < day_str]
prev_eq = state["daily_equity"][previous[-1]] if previous else state["initial_capital"]
state["daily_equity"][day_str] = round(post_value, 2)
record_daily_snapshot(state, day_str, close_prices, trades, prev_eq)
prev_entry = state["daily_log"][-1] if state.get("daily_log") else None
record_daily_snapshot(state, day_str, close_prices, trades, prev_eq, target=target_weights,
ideal_return=_ideal_return(prev_entry, day, close_data))
state["last_morning"] = day_str
state["last_evening"] = day_str
state["pending_trades"] = None
@@ -1317,6 +1354,7 @@ def cmd_compare(args):
"n_days": n_days,
"n_positions": len(state.get("holdings", {})),
"cash": state.get("cash", 0),
"ideal": _ideal_window(state),
})
# Sort by total return descending
@@ -1324,13 +1362,22 @@ def cmd_compare(args):
# Print ranking
print(f"\n {'#':<3} {'Strategy':<25} {'Return':>9} {'Ann.Ret':>9} "
f"{'Sharpe':>8} {'MaxDD':>8} {'Trades':>7} {'Final':>14}")
print(f" {'─'*87}")
f"{'Sharpe':>8} {'MaxDD':>8} {'Trades':>7} {'Final':>14} {'Ideal':>8} {'Gap':>7}")
print(f" {'─'*104}")
for i, r in enumerate(results, 1):
iw = r["ideal"]
ideal = f"{iw['ideal'] * 100:>+7.1f}% {iw['gap'] * 100:>+6.1f}pp" if iw else f"{'—':>8} {'—':>7}"
print(f" {i:<3} {r['name']:<25} {r['total_ret']:>+8.1f}% {r['ann_ret']:>+8.1f}% "
f"{r['sharpe']:>8.2f} {r['max_dd']:>7.1f}% {r['n_trades']:>7} "
f"${r['final']:>12,.2f}")
f"${r['final']:>12,.2f} {ideal}")
starts = sorted({r["ideal"]["start"] for r in results if r["ideal"]})
if starts:
print(f" Ideal: the strategy's own target weights held without whole shares, fees, the resize band "
f"or idle cash, from the close of {starts[0]}" + (f" (latest start {starts[-1]})" if len(starts) > 1 else "")
+ ". Gap: live return minus ideal over the same days.")
else:
print(" Ideal: not recorded yet (state files record session targets from the next session on).")
# Equity curve comparison (last 20 dates)
recent_dates = all_dates[-20:]