gahow/quant-trading
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(data): store raw price panels and refresh a trailing window on every update

Browse Source 
The append-only updater only fetched when last_date+1 < local date, never
revisited appended rows and forward-filled before saving. First-seen values
(partial intraday bars, late or missing prints) were frozen permanently: vs
Yahoo, 22% (US), 58% (CN) and 27% (ETF) of recent days had >0.5% daily-return
errors, CN never received same-day data, and HK tickers created US-holiday rows.

Panels are now stored raw (NaN = no print) and forward-filled on read; every
update re-downloads the last 30 days for the requested tickers, rescales older
history when Yahoo's split/dividend adjustment moves, drops rows printed by a
minority of tickers, and writes atomically under a per-market file lock.
Tickers Yahoo has no data for are retried weekly instead of every run.
rebuild() repairs panels written by the old updater.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Gahow Wang
2026-09-25 01:16:10 +08:00
parent 0d983edfc0
commit 838749d8b2
2 changed files with 248 additions and 169 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
.gitignore vendored
View File

@@ -44,3 +44,8 @@ docs/superpowers/
# OS
.DS_Store
Thumbs.db
# Price panel locks, repair backups and Yahoo availability markers
data/.*.lock
data/backup_*/
data/*_unavailable.json