Hi,
I've throughly checked and I could confirm that there is an issue with resampling when RollOver data is used. I'm using three datas,
data0 bt.feeds.IBData
for SPY Index
data1 bt.feeds.RollOver
for SPY OPT Calls
data2 bt.feeds.RollOver
SPY OPT Puts
For simplicity I added just one strike to each RollOver
I tested with and without RollOver for data1 and data2 and below are the outputs in each case. I'm printing in ibdata.py
in def _load_rtbar
so you can see date2num
timestamps and match them in each case.
Issues I observed are
-
Data is processed in batches for each data and not all three datas for every rtbar like in case of second output without RollOver. I checked IB logs and can see that data is sent for all three datas every 5 seconds in both cases. In second output you can see prints for data0, data1 and data2 for every rtbar (5seconds) In case of first ouput with RollOver you can see them in batches
-
Data is being reported a minute later sometime. Check line after 737699.800636574 in both cases. In case of without Rollover Data for 2020-10-02T00:43:00 is being reported correctly with Close as 337.02. Whereas in case of Rollover after the same date2num 737699.800636574 data is being reported for a minute old 2020-10-02T00:42:00 with close as 336.91
Any help with this bt.feeds.RollOver
resampling is much appreciated.
@backtrader @vladisld Any inputs please
With RollOver Data
2020-10-02T00:41:00, ************************* Data0: Close 337.10 *************************
737699.7993055555 Time: 2020-10-02 00:41:00, Open: 337.09, High: 337.09, Low: 337.07, Close 337.08
737699.7993634259 Time: 2020-10-02 00:41:05, Open: 337.07, High: 337.07, Low: 337.05, Close 337.06
737699.7994212963 Time: 2020-10-02 00:41:10, Open: 337.06, High: 337.08, Low: 337.03, Close 337.03
737699.7994791666 Time: 2020-10-02 00:41:15, Open: 337.02, High: 337.02, Low: 337.0, Close 337.01
737699.799537037 Time: 2020-10-02 00:41:20, Open: 337.01, High: 337.01, Low: 336.97, Close 336.99
737699.7995949074 Time: 2020-10-02 00:41:25, Open: 336.98, High: 336.99, Low: 336.97, Close 336.99
737699.7996527777 Time: 2020-10-02 00:41:30, Open: 337.0, High: 337.01, Low: 336.99, Close 336.99
737699.7997106481 Time: 2020-10-02 00:41:35, Open: 337.0, High: 337.0, Low: 336.94, Close 336.96
737699.7997685185 Time: 2020-10-02 00:41:40, Open: 336.94, High: 336.97, Low: 336.94, Close 336.96
737699.799826389 Time: 2020-10-02 00:41:45, Open: 336.95, High: 336.96, Low: 336.85, Close 336.85
737699.7998842592 Time: 2020-10-02 00:41:50, Open: 336.86, High: 336.92, Low: 336.86, Close 336.89
737699.7999421296 Time: 2020-10-02 00:41:55, Open: 336.89, High: 336.91, Low: 336.88, Close 336.91
737699.8000578703 Time: 2020-10-02 00:42:05, Open: 2.1, High: 2.1, Low: 2.09, Close 2.09
737699.8001157407 Time: 2020-10-02 00:42:10, Open: 2.12, High: 2.12, Low: 2.12, Close 2.12
737699.8001736111 Time: 2020-10-02 00:42:15, Open: 2.12, High: 2.12, Low: 2.12, Close 2.12
737699.8002314814 Time: 2020-10-02 00:42:20, Open: 2.12, High: 2.12, Low: 2.12, Close 2.12
737699.8002893518 Time: 2020-10-02 00:42:25, Open: 2.13, High: 2.13, Low: 2.13, Close 2.13
737699.8003472222 Time: 2020-10-02 00:42:30, Open: 2.12, High: 2.12, Low: 2.12, Close 2.12
737699.8004050925 Time: 2020-10-02 00:42:35, Open: 2.12, High: 2.12, Low: 2.12, Close 2.12
737699.800462963 Time: 2020-10-02 00:42:40, Open: 2.13, High: 2.13, Low: 2.13, Close 2.13
737699.8005208333 Time: 2020-10-02 00:42:45, Open: 2.13, High: 2.13, Low: 2.13, Close 2.13
737699.8005787038 Time: 2020-10-02 00:42:50, Open: 2.13, High: 2.13, Low: 2.13, Close 2.13
737699.800636574 Time: 2020-10-02 00:42:55, Open: 2.15, High: 2.15, Low: 2.15, Close 2.15
737699.8006944444 Time: 2020-10-02 00:43:00, Open: 2.15, High: 2.15, Low: 2.15, Close 2.15
737699.8000578703 Time: 2020-10-02 00:42:05, Open: 1.77, High: 1.77, Low: 1.77, Close 1.77
737699.8001157407 Time: 2020-10-02 00:42:10, Open: 1.75, High: 1.75, Low: 1.75, Close 1.75
737699.8001736111 Time: 2020-10-02 00:42:15, Open: 1.75, High: 1.75, Low: 1.75, Close 1.75
737699.8002314814 Time: 2020-10-02 00:42:20, Open: 1.75, High: 1.75, Low: 1.75, Close 1.75
737699.8002893518 Time: 2020-10-02 00:42:25, Open: 1.75, High: 1.75, Low: 1.75, Close 1.75
737699.8003472222 Time: 2020-10-02 00:42:30, Open: 1.73, High: 1.74, Low: 1.73, Close 1.74
737699.8004050925 Time: 2020-10-02 00:42:35, Open: 1.74, High: 1.74, Low: 1.74, Close 1.74
737699.800462963 Time: 2020-10-02 00:42:40, Open: 1.74, High: 1.74, Low: 1.74, Close 1.74
737699.8005208333 Time: 2020-10-02 00:42:45, Open: 1.74, High: 1.74, Low: 1.74, Close 1.74
737699.8005787038 Time: 2020-10-02 00:42:50, Open: 1.71, High: 1.71, Low: 1.71, Close 1.71
737699.800636574 Time: 2020-10-02 00:42:55, Open: 1.71, High: 1.71, Low: 1.7, Close 1.7
737699.8006944444 Time: 2020-10-02 00:43:00, Open: 1.7, High: 1.7, Low: 1.69, Close 1.69
2020-10-02T00:42:00, ************************* Data0: Close 336.91 ************************
737699.8 Time: 2020-10-02 00:42:00, Open: 336.91, High: 336.92, Low: 336.9, Close 336.91
737699.8000578703 Time: 2020-10-02 00:42:05, Open: 336.91, High: 336.92, Low: 336.9, Close 336.91
737699.8001157407 Time: 2020-10-02 00:42:10, Open: 336.9, High: 336.99, Low: 336.88, Close 336.98
737699.8001736111 Time: 2020-10-02 00:42:15, Open: 336.98, High: 336.99, Low: 336.95, Close 336.97
737699.8002314814 Time: 2020-10-02 00:42:20, Open: 336.97, High: 336.98, Low: 336.96, Close 336.98
737699.8002893518 Time: 2020-10-02 00:42:25, Open: 336.98, High: 337.0, Low: 336.97, Close 336.99
737699.8003472222 Time: 2020-10-02 00:42:30, Open: 336.98, High: 336.99, Low: 336.95, Close 336.95
737699.8004050925 Time: 2020-10-02 00:42:35, Open: 336.94, High: 336.97, Low: 336.94, Close 336.96
737699.800462963 Time: 2020-10-02 00:42:40, Open: 336.96, High: 336.98, Low: 336.95, Close 336.96
737699.8005208333 Time: 2020-10-02 00:42:45, Open: 336.98, High: 336.99, Low: 336.97, Close 336.98
737699.8005787038 Time: 2020-10-02 00:42:50, Open: 336.99, High: 337.03, Low: 336.99, Close 337.03
737699.800636574 Time: 2020-10-02 00:42:55, Open: 337.03, High: 337.03, Low: 337.01, Close 337.02
737699.8006944444 Time: 2020-10-02 00:43:00, Open: 337.03, High: 337.07, Low: 337.03, Close 337.07
737699.8007523149 Time: 2020-10-02 00:43:05, Open: 2.17, High: 2.17, Low: 2.17, Close 2.17
737699.8008101851 Time: 2020-10-02 00:43:10, Open: 2.18, High: 2.18, Low: 2.18, Close 2.18
737699.8008680556 Time: 2020-10-02 00:43:15, Open: 2.18, High: 2.18, Low: 2.18, Close 2.18
Without RollOver Data
2020-10-02T00:41:00, ************************* Data0: Close 337.10 *************************
737699.7993055555 Time: 2020-10-02 00:41:00, Open: 337.09, High: 337.09, Low: 337.07, Close 337.08
737699.7993055555 Time: 2020-10-02 00:41:00, Open: 2.19, High: 2.19, Low: 2.19, Close 2.19
737699.7993055555 Time: 2020-10-02 00:41:00, Open: 1.69, High: 1.69, Low: 1.69, Close 1.69
737699.7993634259 Time: 2020-10-02 00:41:05, Open: 337.07, High: 337.07, Low: 337.05, Close 337.06
737699.7993634259 Time: 2020-10-02 00:41:05, Open: 2.17, High: 2.17, Low: 2.17, Close 2.17
737699.7993634259 Time: 2020-10-02 00:41:05, Open: 1.68, High: 1.68, Low: 1.68, Close 1.68
737699.7994212963 Time: 2020-10-02 00:41:10, Open: 337.06, High: 337.08, Low: 337.03, Close 337.03
737699.7994212963 Time: 2020-10-02 00:41:10, Open: 2.15, High: 2.17, Low: 2.15, Close 2.15
737699.7994212963 Time: 2020-10-02 00:41:10, Open: 1.68, High: 1.68, Low: 1.68, Close 1.68
737699.7994791666 Time: 2020-10-02 00:41:15, Open: 337.02, High: 337.02, Low: 337.0, Close 337.01
737699.7994791666 Time: 2020-10-02 00:41:15, Open: 2.15, High: 2.15, Low: 2.15, Close 2.15
737699.7994791666 Time: 2020-10-02 00:41:15, Open: 1.68, High: 1.68, Low: 1.68, Close 1.68
737699.799537037 Time: 2020-10-02 00:41:20, Open: 337.01, High: 337.01, Low: 336.97, Close 336.99
737699.799537037 Time: 2020-10-02 00:41:20, Open: 2.11, High: 2.11, Low: 2.11, Close 2.11
737699.799537037 Time: 2020-10-02 00:41:20, Open: 1.72, High: 1.72, Low: 1.72, Close 1.72
737699.7995949074 Time: 2020-10-02 00:41:25, Open: 336.98, High: 336.99, Low: 336.97, Close 336.99
737699.7995949074 Time: 2020-10-02 00:41:25, Open: 2.12, High: 2.12, Low: 2.12, Close 2.12
737699.7995949074 Time: 2020-10-02 00:41:25, Open: 1.72, High: 1.72, Low: 1.72, Close 1.72
737699.7996527777 Time: 2020-10-02 00:41:30, Open: 337.0, High: 337.01, Low: 336.99, Close 336.99
737699.7996527777 Time: 2020-10-02 00:41:30, Open: 2.13, High: 2.13, Low: 2.13, Close 2.13
737699.7996527777 Time: 2020-10-02 00:41:30, Open: 1.72, High: 1.72, Low: 1.72, Close 1.72
737699.7997106481 Time: 2020-10-02 00:41:35, Open: 337.0, High: 337.0, Low: 336.94, Close 336.96
737699.7997106481 Time: 2020-10-02 00:41:35, Open: 2.13, High: 2.13, Low: 2.13, Close 2.13
737699.7997106481 Time: 2020-10-02 00:41:35, Open: 1.72, High: 1.72, Low: 1.72, Close 1.72
737699.7997685185 Time: 2020-10-02 00:41:40, Open: 336.94, High: 336.97, Low: 336.94, Close 336.96
737699.7997685185 Time: 2020-10-02 00:41:40, Open: 2.12, High: 2.12, Low: 2.12, Close 2.12
737699.7997685185 Time: 2020-10-02 00:41:40, Open: 1.72, High: 1.72, Low: 1.72, Close 1.72
737699.799826389 Time: 2020-10-02 00:41:45, Open: 336.95, High: 336.96, Low: 336.85, Close 336.85
737699.799826389 Time: 2020-10-02 00:41:45, Open: 2.1, High: 2.1, Low: 2.1, Close 2.1
737699.799826389 Time: 2020-10-02 00:41:45, Open: 1.72, High: 1.72, Low: 1.72, Close 1.72
737699.7998842592 Time: 2020-10-02 00:41:50, Open: 336.86, High: 336.92, Low: 336.86, Close 336.89
737699.7998842592 Time: 2020-10-02 00:41:50, Open: 2.07, High: 2.09, Low: 2.07, Close 2.09
737699.7998842592 Time: 2020-10-02 00:41:50, Open: 1.76, High: 1.76, Low: 1.76, Close 1.76
737699.7999421296 Time: 2020-10-02 00:41:55, Open: 336.89, High: 336.91, Low: 336.88, Close 336.91
737699.7999421296 Time: 2020-10-02 00:41:55, Open: 2.08, High: 2.08, Low: 2.08, Close 2.08
737699.7999421296 Time: 2020-10-02 00:41:55, Open: 1.76, High: 1.76, Low: 1.76, Close 1.76
2020-10-02T00:42:00, ************************* Data0: Close 336.91 ************************
737699.8 Time: 2020-10-02 00:42:00, Open: 336.91, High: 336.92, Low: 336.9, Close 336.91
737699.8 Time: 2020-10-02 00:42:00, Open: 2.08, High: 2.08, Low: 2.08, Close 2.08
737699.8 Time: 2020-10-02 00:42:00, Open: 1.76, High: 1.76, Low: 1.76, Close 1.76
737699.8000578703 Time: 2020-10-02 00:42:05, Open: 336.91, High: 336.92, Low: 336.9, Close 336.91
737699.8000578703 Time: 2020-10-02 00:42:05, Open: 2.1, High: 2.1, Low: 2.09, Close 2.09
737699.8000578703 Time: 2020-10-02 00:42:05, Open: 1.77, High: 1.77, Low: 1.77, Close 1.77
737699.8001157407 Time: 2020-10-02 00:42:10, Open: 336.9, High: 336.99, Low: 336.88, Close 336.98
737699.8001157407 Time: 2020-10-02 00:42:10, Open: 2.12, High: 2.12, Low: 2.12, Close 2.12
737699.8001157407 Time: 2020-10-02 00:42:10, Open: 1.75, High: 1.75, Low: 1.75, Close 1.75
737699.8001736111 Time: 2020-10-02 00:42:15, Open: 336.98, High: 336.99, Low: 336.95, Close 336.97
737699.8001736111 Time: 2020-10-02 00:42:15, Open: 2.12, High: 2.12, Low: 2.12, Close 2.12
737699.8001736111 Time: 2020-10-02 00:42:15, Open: 1.75, High: 1.75, Low: 1.75, Close 1.75
737699.8002314814 Time: 2020-10-02 00:42:20, Open: 336.97, High: 336.98, Low: 336.96, Close 336.98
737699.8002314814 Time: 2020-10-02 00:42:20, Open: 2.12, High: 2.12, Low: 2.12, Close 2.12
737699.8002314814 Time: 2020-10-02 00:42:20, Open: 1.75, High: 1.75, Low: 1.75, Close 1.75
737699.8002893518 Time: 2020-10-02 00:42:25, Open: 336.98, High: 337.0, Low: 336.97, Close 336.99
737699.8002893518 Time: 2020-10-02 00:42:25, Open: 2.13, High: 2.13, Low: 2.13, Close 2.13
737699.8002893518 Time: 2020-10-02 00:42:25, Open: 1.75, High: 1.75, Low: 1.75, Close 1.75
737699.8003472222 Time: 2020-10-02 00:42:30, Open: 336.98, High: 336.99, Low: 336.95, Close 336.95
737699.8003472222 Time: 2020-10-02 00:42:30, Open: 2.12, High: 2.12, Low: 2.12, Close 2.12
737699.8003472222 Time: 2020-10-02 00:42:30, Open: 1.73, High: 1.74, Low: 1.73, Close 1.74
737699.8004050925 Time: 2020-10-02 00:42:35, Open: 336.94, High: 336.97, Low: 336.94, Close 336.96
737699.8004050925 Time: 2020-10-02 00:42:35, Open: 2.12, High: 2.12, Low: 2.12, Close 2.12
737699.8004050925 Time: 2020-10-02 00:42:35, Open: 1.74, High: 1.74, Low: 1.74, Close 1.74
737699.800462963 Time: 2020-10-02 00:42:40, Open: 336.96, High: 336.98, Low: 336.95, Close 336.96
737699.800462963 Time: 2020-10-02 00:42:40, Open: 2.13, High: 2.13, Low: 2.13, Close 2.13
737699.800462963 Time: 2020-10-02 00:42:40, Open: 1.74, High: 1.74, Low: 1.74, Close 1.74
737699.8005208333 Time: 2020-10-02 00:42:45, Open: 336.98, High: 336.99, Low: 336.97, Close 336.98
737699.8005208333 Time: 2020-10-02 00:42:45, Open: 2.13, High: 2.13, Low: 2.13, Close 2.13
737699.8005208333 Time: 2020-10-02 00:42:45, Open: 1.74, High: 1.74, Low: 1.74, Close 1.74
737699.8005787038 Time: 2020-10-02 00:42:50, Open: 336.99, High: 337.03, Low: 336.99, Close 337.03
737699.8005787038 Time: 2020-10-02 00:42:50, Open: 2.13, High: 2.13, Low: 2.13, Close 2.13
737699.8005787038 Time: 2020-10-02 00:42:50, Open: 1.71, High: 1.71, Low: 1.71, Close 1.71
737699.800636574 Time: 2020-10-02 00:42:55, Open: 337.03, High: 337.03, Low: 337.01, Close 337.02
737699.800636574 Time: 2020-10-02 00:42:55, Open: 2.15, High: 2.15, Low: 2.15, Close 2.15
737699.800636574 Time: 2020-10-02 00:42:55, Open: 1.71, High: 1.71, Low: 1.7, Close 1.7
2020-10-02T00:43:00, ************************* Data0: Close 337.02 *************************
737699.8006944444 Time: 2020-10-02 00:43:00, Open: 337.03, High: 337.07, Low: 337.03, Close 337.07
737699.8006944444 Time: 2020-10-02 00:43:00, Open: 2.15, High: 2.15, Low: 2.15, Close 2.15
737699.8006944444 Time: 2020-10-02 00:43:00, Open: 1.7, High: 1.7, Low: 1.69, Close 1.69
737699.8007523149 Time: 2020-10-02 00:43:05, Open: 337.06, High: 337.09, Low: 337.06, Close 337.09
737699.8007523149 Time: 2020-10-02 00:43:05, Open: 2.17, High: 2.17, Low: 2.17, Close 2.17
737699.8007523149 Time: 2020-10-02 00:43:05, Open: 1.69, High: 1.69, Low: 1.69, Close 1.69
737699.8008101851 Time: 2020-10-02 00:43:10, Open: 337.1, High: 337.1, Low: 337.07, Close 337.1
737699.8008101851 Time: 2020-10-02 00:43:10, Open: 2.18, High: 2.18, Low: 2.18, Close 2.18
737699.8008101851 Time: 2020-10-02 00:43:10, Open: 1.69, High: 1.69, Low: 1.69, Close 1.69
737699.8008680556 Time: 2020-10-02 00:43:15, Open: 337.1, High: 337.1, Low: 337.03, Close 337.06
737699.8008680556 Time: 2020-10-02 00:43:15, Open: 2.18, High: 2.18, Low: 2.18, Close 2.18
737699.8008680556 Time: 2020-10-02 00:43:15, Open: 1.69, High: 1.69, Low: 1.69, Close 1.69

