For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

Next() not triggering on ibkr data feed



  • So I used this code to fetch data from ibkr and save it to a csv which I use later for testing and training. So I must have changed something and I don't know what but the next() in my strategy no longer triggers on the incoming data when it used too. I have created datasets with this class before, I must have changed something that this doesn't trigger next anymore. I read the documentation multiple times and googled but I am puzzled what I have changed that next() doesn't trigger.
    Code below:

    from __future__ import (absolute_import, division, print_function,
                            unicode_literals)
    import backtrader as bt
    import datetime
    import csv
    
    
    class St(bt.Strategy):
        def __init__(self):
            csvfile = open('ES_Continuous_08-20-2020~08-21-2020', 'w')
            writer = csv.writer(csvfile, delimiter=',', quotechar='|', quoting=csv.QUOTE_MINIMAL)
            self.writer = writer
    
        def logdata(self):
            txt = []
    
            txt.append('{}'.format(
                self.data.datetime.datetime(0).isoformat())
            )
            txt.append('{:.2f}'.format(self.data.open[0]))
            txt.append('{:.2f}'.format(self.data.high[0]))
            txt.append('{:.2f}'.format(self.data.low[0]))
            txt.append('{:.2f}'.format(self.data.close[0]))
            txt.append('{:.2f}'.format(self.data.volume[0]))
            print(txt)
            self.writer.writerow(txt)
    
        def next(self):
            print("next called")
            self.logdata()
    
        def start(self):
            print('Historical Data gathering is about to start')
    
        def stop(self):
            print('Historical Data gathering is finished')
    
    
    def run(from_date, to_date):
        cerebro = bt.Cerebro(stdstats=False)
        ibstore = bt.stores.IBStore(host='127.0.0.1', port=7496, clientId=35, _debug=True)
        data = ibstore.getdata(dataname='ES-CONTFUT-GLOBEX-USD', fromdate=from_date, todate=to_date, timeframe=bt.TimeFrame.Seconds)
        cerebro.adddata(data)
        cerebro.addstrategy(St)
        cerebro.run()
        cerebro.plot()
    
    if __name__ == '__main__':
        from_date = datetime.datetime(2020,8,20)
        to_date = datetime.datetime(2020,8,21)
        print("ES_Continuous_" + from_date.strftime('%m-%d-%Y') + "~" + to_date.strftime('%m-%d-%Y'))
        run(from_date, to_date)
    

    I know that data is flowing in because when I turn debug mode on I get data coming in
    Example:

    <historicalData reqId=16777217, date=1597883367, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=0, count=0, WAP=3361.75, hasGaps=False>
    <historicalData reqId=16777217, date=1597883368, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=0, count=0, WAP=3361.75, hasGaps=False>
    <historicalData reqId=16777217, date=1597883369, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=0, count=0, WAP=3361.75, hasGaps=False>
    <historicalData reqId=16777217, date=1597883370, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=0, count=0, WAP=3361.75, hasGaps=False>
    <historicalData reqId=16777217, date=1597883371, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=0, count=0, WAP=3361.75, hasGaps=False>
    <historicalData reqId=16777217, date=1597883372, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=0, count=0, WAP=3361.75, hasGaps=False>
    <historicalData reqId=16777217, date=1597883373, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=0, count=0, WAP=3361.75, hasGaps=False>
    <historicalData reqId=16777217, date=1597883374, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=0, count=0, WAP=3361.75, hasGaps=False>
    <historicalData reqId=16777217, date=1597883375, open=3362.0, high=3362.0, low=3361.75, close=3361.75, volume=3, count=2, WAP=3361.825, hasGaps=False>
    <historicalData reqId=16777217, date=1597883376, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=0, count=0, WAP=3361.75, hasGaps=False>
    <historicalData reqId=16777217, date=1597883377, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=0, count=0, WAP=3361.75, hasGaps=False>
    <historicalData reqId=16777217, date=1597883378, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=0, count=0, WAP=3361.75, hasGaps=False>
    <historicalData reqId=16777217, date=1597883379, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=0, count=0, WAP=3361.75, hasGaps=False>
    <historicalData reqId=16777217, date=1597883380, open=3362.0, high=3362.0, low=3362.0, close=3362.0, volume=1, count=1, WAP=3362.0, hasGaps=False>
    <historicalData reqId=16777217, date=1597883381, open=3362.0, high=3362.0, low=3362.0, close=3362.0, volume=0, count=0, WAP=3362.0, hasGaps=False>
    <historicalData reqId=16777217, date=1597883382, open=3362.0, high=3362.0, low=3362.0, close=3362.0, volume=0, count=0, WAP=3362.0, hasGaps=False>
    <historicalData reqId=16777217, date=1597883383, open=3362.0, high=3362.0, low=3362.0, close=3362.0, volume=0, count=0, WAP=3362.0, hasGaps=False>
    <historicalData reqId=16777217, date=1597883384, open=3362.0, high=3362.0, low=3362.0, close=3362.0, volume=0, count=0, WAP=3362.0, hasGaps=False>
    <historicalData reqId=16777217, date=1597883385, open=3362.0, high=3362.0, low=3362.0, close=3362.0, volume=1, count=1, WAP=3362.0, hasGaps=False>
    <historicalData reqId=16777217, date=1597883386, open=3362.0, high=3362.0, low=3362.0, close=3362.0, volume=0, count=0, WAP=3362.0, hasGaps=False>
    <historicalData reqId=16777217, date=1597883387, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=1, count=1, WAP=3361.75, hasGaps=False>
    <historicalData reqId=16777217, date=1597883388, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=0, count=0, WAP=3361.75, hasGaps=False>
    <historicalData reqId=16777217, date=1597883389, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=0, count=0, WAP=3361.75, hasGaps=False>
    <historicalData reqId=16777217, date=1597883390, open=3362.0, high=3362.0, low=3361.75, close=3361.75, volume=23, count=5, WAP=3361.75, hasGaps=False>
    <historicalData reqId=16777217, date=1597883391, open=3361.5, high=3361.5, low=3361.5, close=3361.5, volume=39, count=8, WAP=3361.5, hasGaps=False>
    <historicalData reqId=16777217, date=1597883392, open=3361.5, high=3361.5, low=3361.5, close=3361.5, volume=8, count=6, WAP=3361.5, hasGaps=False>
    <historicalData reqId=16777217, date=1597883393, open=3361.5, high=3361.5, low=3361.5, close=3361.5, volume=0, count=0, WAP=3361.5, hasGaps=False>
    <historicalData reqId=16777217, date=1597883394, open=3361.5, high=3361.5, low=3361.5, close=3361.5, volume=0, count=0, WAP=3361.5, hasGaps=False>
    <historicalData reqId=16777217, date=1597883395, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=20, count=1, WAP=3361.75, hasGaps=False>
    <historicalData reqId=16777217, date=1597883396, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=0, count=0, WAP=3361.75, hasGaps=False>
    <historicalData reqId=16777217, date=1597883397, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=0, count=0, WAP=3361.75, hasGaps=False>
    <historicalData reqId=16777217, date=1597883398, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=20, count=1, WAP=3361.75, hasGaps=False>
    <historicalData reqId=16777217, date=1597883399, open=3361.5, high=3361.5, low=3361.5, close=3361.5, volume=1, count=1, WAP=3361.5, hasGaps=False>
    <historicalData reqId=16777217, date=finished-20200819  19:00:00-20200819  19:30:00, open=-1, high=-1, low=-1, close=-1, volume=-1, count=-1, WAP=-1, hasGaps=False>
    
    

    Any ideas on how to get next to trigger? Verify print statements are never hit and csv is not written too.



  • try to initialize the Cerebro engine with preload=False:

    cerebro = bt.Cerebro(stdstats=False, live=False, preload=False)
    

    please report if it solved the problem. It seems that IBData will not report the end of historical data load ending properly in case the preload was requested - need to investigate further.



  • @vladisld Could you please explain what preload=False do, and when should we set it to True and False when we are using the interactive broker?



  • @shspersonalwork I know the following about preload, however I want to know the impact of this setting on Live and Offline mode, and how did you debug the problem to this parameter setting?

    "preload -> Whether to preload the different data feeds passed to cerebro for the Strategies"


Log in to reply
 

});