Next() not triggering on ibkr data feed
-
So I used this code to fetch data from ibkr and save it to a csv which I use later for testing and training. So I must have changed something and I don't know what but the next() in my strategy no longer triggers on the incoming data when it used too. I have created datasets with this class before, I must have changed something that this doesn't trigger next anymore. I read the documentation multiple times and googled but I am puzzled what I have changed that next() doesn't trigger.
Code below:from __future__ import (absolute_import, division, print_function, unicode_literals) import backtrader as bt import datetime import csv class St(bt.Strategy): def __init__(self): csvfile = open('ES_Continuous_08-20-2020~08-21-2020', 'w') writer = csv.writer(csvfile, delimiter=',', quotechar='|', quoting=csv.QUOTE_MINIMAL) self.writer = writer def logdata(self): txt = [] txt.append('{}'.format( self.data.datetime.datetime(0).isoformat()) ) txt.append('{:.2f}'.format(self.data.open[0])) txt.append('{:.2f}'.format(self.data.high[0])) txt.append('{:.2f}'.format(self.data.low[0])) txt.append('{:.2f}'.format(self.data.close[0])) txt.append('{:.2f}'.format(self.data.volume[0])) print(txt) self.writer.writerow(txt) def next(self): print("next called") self.logdata() def start(self): print('Historical Data gathering is about to start') def stop(self): print('Historical Data gathering is finished') def run(from_date, to_date): cerebro = bt.Cerebro(stdstats=False) ibstore = bt.stores.IBStore(host='127.0.0.1', port=7496, clientId=35, _debug=True) data = ibstore.getdata(dataname='ES-CONTFUT-GLOBEX-USD', fromdate=from_date, todate=to_date, timeframe=bt.TimeFrame.Seconds) cerebro.adddata(data) cerebro.addstrategy(St) cerebro.run() cerebro.plot() if __name__ == '__main__': from_date = datetime.datetime(2020,8,20) to_date = datetime.datetime(2020,8,21) print("ES_Continuous_" + from_date.strftime('%m-%d-%Y') + "~" + to_date.strftime('%m-%d-%Y')) run(from_date, to_date)
I know that data is flowing in because when I turn debug mode on I get data coming in
Example:<historicalData reqId=16777217, date=1597883367, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=0, count=0, WAP=3361.75, hasGaps=False> <historicalData reqId=16777217, date=1597883368, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=0, count=0, WAP=3361.75, hasGaps=False> <historicalData reqId=16777217, date=1597883369, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=0, count=0, WAP=3361.75, hasGaps=False> <historicalData reqId=16777217, date=1597883370, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=0, count=0, WAP=3361.75, hasGaps=False> <historicalData reqId=16777217, date=1597883371, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=0, count=0, WAP=3361.75, hasGaps=False> <historicalData reqId=16777217, date=1597883372, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=0, count=0, WAP=3361.75, hasGaps=False> <historicalData reqId=16777217, date=1597883373, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=0, count=0, WAP=3361.75, hasGaps=False> <historicalData reqId=16777217, date=1597883374, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=0, count=0, WAP=3361.75, hasGaps=False> <historicalData reqId=16777217, date=1597883375, open=3362.0, high=3362.0, low=3361.75, close=3361.75, volume=3, count=2, WAP=3361.825, hasGaps=False> <historicalData reqId=16777217, date=1597883376, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=0, count=0, WAP=3361.75, hasGaps=False> <historicalData reqId=16777217, date=1597883377, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=0, count=0, WAP=3361.75, hasGaps=False> <historicalData reqId=16777217, date=1597883378, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=0, count=0, WAP=3361.75, hasGaps=False> <historicalData reqId=16777217, date=1597883379, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=0, count=0, WAP=3361.75, hasGaps=False> <historicalData reqId=16777217, date=1597883380, open=3362.0, high=3362.0, low=3362.0, close=3362.0, volume=1, count=1, WAP=3362.0, hasGaps=False> <historicalData reqId=16777217, date=1597883381, open=3362.0, high=3362.0, low=3362.0, close=3362.0, volume=0, count=0, WAP=3362.0, hasGaps=False> <historicalData reqId=16777217, date=1597883382, open=3362.0, high=3362.0, low=3362.0, close=3362.0, volume=0, count=0, WAP=3362.0, hasGaps=False> <historicalData reqId=16777217, date=1597883383, open=3362.0, high=3362.0, low=3362.0, close=3362.0, volume=0, count=0, WAP=3362.0, hasGaps=False> <historicalData reqId=16777217, date=1597883384, open=3362.0, high=3362.0, low=3362.0, close=3362.0, volume=0, count=0, WAP=3362.0, hasGaps=False> <historicalData reqId=16777217, date=1597883385, open=3362.0, high=3362.0, low=3362.0, close=3362.0, volume=1, count=1, WAP=3362.0, hasGaps=False> <historicalData reqId=16777217, date=1597883386, open=3362.0, high=3362.0, low=3362.0, close=3362.0, volume=0, count=0, WAP=3362.0, hasGaps=False> <historicalData reqId=16777217, date=1597883387, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=1, count=1, WAP=3361.75, hasGaps=False> <historicalData reqId=16777217, date=1597883388, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=0, count=0, WAP=3361.75, hasGaps=False> <historicalData reqId=16777217, date=1597883389, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=0, count=0, WAP=3361.75, hasGaps=False> <historicalData reqId=16777217, date=1597883390, open=3362.0, high=3362.0, low=3361.75, close=3361.75, volume=23, count=5, WAP=3361.75, hasGaps=False> <historicalData reqId=16777217, date=1597883391, open=3361.5, high=3361.5, low=3361.5, close=3361.5, volume=39, count=8, WAP=3361.5, hasGaps=False> <historicalData reqId=16777217, date=1597883392, open=3361.5, high=3361.5, low=3361.5, close=3361.5, volume=8, count=6, WAP=3361.5, hasGaps=False> <historicalData reqId=16777217, date=1597883393, open=3361.5, high=3361.5, low=3361.5, close=3361.5, volume=0, count=0, WAP=3361.5, hasGaps=False> <historicalData reqId=16777217, date=1597883394, open=3361.5, high=3361.5, low=3361.5, close=3361.5, volume=0, count=0, WAP=3361.5, hasGaps=False> <historicalData reqId=16777217, date=1597883395, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=20, count=1, WAP=3361.75, hasGaps=False> <historicalData reqId=16777217, date=1597883396, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=0, count=0, WAP=3361.75, hasGaps=False> <historicalData reqId=16777217, date=1597883397, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=0, count=0, WAP=3361.75, hasGaps=False> <historicalData reqId=16777217, date=1597883398, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=20, count=1, WAP=3361.75, hasGaps=False> <historicalData reqId=16777217, date=1597883399, open=3361.5, high=3361.5, low=3361.5, close=3361.5, volume=1, count=1, WAP=3361.5, hasGaps=False> <historicalData reqId=16777217, date=finished-20200819 19:00:00-20200819 19:30:00, open=-1, high=-1, low=-1, close=-1, volume=-1, count=-1, WAP=-1, hasGaps=False>
Any ideas on how to get next to trigger? Verify print statements are never hit and csv is not written too.
-
try to initialize the Cerebro engine with
preload=False
:cerebro = bt.Cerebro(stdstats=False, live=False, preload=False)
please report if it solved the problem. It seems that IBData will not report the end of historical data load ending properly in case the preload was requested - need to investigate further.
-
@vladisld Could you please explain what preload=False do, and when should we set it to True and False when we are using the interactive broker?
-
@shspersonalwork I know the following about preload, however I want to know the impact of this setting on Live and Offline mode, and how did you debug the problem to this parameter setting?
"preload -> Whether to preload the different
data feeds
passed to cerebro for the Strategies"