gahow/ibkr-quant
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Restart the Gateway when its historical data farm stops serving

Browse Source 
On 2026-10-12 the Gateway answered account requests but every bar request
timed out from 09:50 to 10:24 ET: make gateway reported OK, so the Gateway was
never restarted, and six retries 5 minutes apart gave up after 35 minutes. The
data came back only with the Gateway's daily restart.

ibkr-quant gateway --history also requires SPY daily bars within 30 s; the
trade job's ExecStartPre passes it, so ensure-gateway.sh restarts a Gateway
whose data farm is down. The trade job now retries every 10 minutes until
about 15:30 ET (34 starts in 6 h) instead of giving up after 35 minutes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Gahow Wang
2026-10-12 23:27:16 -07:00
parent 8238651af6
commit e55c4ffbdb
5 changed files with 53 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files

20
src/ibkr_quant/cli.py
View File

@@ -93,11 +93,22 @@ async def check(cfg: Config) -> None:
print("Delayed data is already requested; check market data permissions and availability in IB Gateway/TWS.")
async def probe_gateway(cfg: Config) -> None:
async def probe_gateway(cfg: Config, history: bool = False) -> None:
# Broker's connect fails unless the Gateway answers positions, orders and
# account updates, which needs a live session with IBKR's servers.
async with Broker(dataclasses.replace(cfg.ib, client_id=cfg.ib.health_client_id)) as broker:
print(f"IB Gateway OK: account {broker.account}, server version {broker.ib.client.serverVersion()}")
status = f"account {broker.account}, server version {broker.ib.client.serverVersion()}"
if history:
# The historical data farm can fail on its own: on 2026-10-12 account requests
# worked while every bar request timed out for 35 minutes, until the Gateway's
# daily restart. Without this check `make gateway` never restarted it.
contracts, _ = await broker.qualify_stocks(["SPY"])
bars = (await broker.daily_bars(contracts, "5 D", timeout=30))[0] if contracts else None
if bars is None or bars.empty:
raise SystemExit(f"IB Gateway answers account requests ({status}) but returned no SPY bars "
"within 30 s; its historical data farm is not serving")
status += f", historical data OK (SPY through {bars['date'].max()})"
print(f"IB Gateway OK: {status}")
async def update_bars(cfg: Config) -> None:
@@ -279,7 +290,8 @@ def main() -> None:
commands = parser.add_subparsers(dest="command", required=True)
commands.add_parser("universe", help="download S&P 500 constituents")
commands.add_parser("check", help="connect and show account, positions, open orders, SPY price")
commands.add_parser("gateway", help="exit non-zero unless IB Gateway answers account requests")
gateway_parser = commands.add_parser("gateway", help="exit non-zero unless IB Gateway answers account requests")
gateway_parser.add_argument("--history", action="store_true", help="also require SPY daily bars from the historical data farm")
commands.add_parser("bars", help="download adjusted daily bars for the universe")
rebalance_parser = commands.add_parser("rebalance", help="plan orders (dry run unless --submit)")
rebalance_parser.add_argument("--submit", action="store_true")
@@ -299,7 +311,7 @@ def main() -> None:
case "check":
asyncio.run(check(cfg))
case "gateway":
asyncio.run(probe_gateway(cfg))
asyncio.run(probe_gateway(cfg, args.history))
case "bars":
asyncio.run(update_bars(cfg))
case "rebalance":