3. Next() not triggering on ibkr data feed
For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

Next() not triggering on ibkr data feed


  • So I used this code to fetch data from ibkr and save it to a csv which I use later for testing and training. So I must have changed something and I don't know what but the next() in my strategy no longer triggers on the incoming data when it used too. I have created datasets with this class before, I must have changed something that this doesn't trigger next anymore. I read the documentation multiple times and googled but I am puzzled what I have changed that next() doesn't trigger.
    Code below:

    from __future__ import (absolute_import, division, print_function,
                        unicode_literals)
import backtrader as bt
import datetime
import csv


class St(bt.Strategy):
    def __init__(self):
        csvfile = open('ES_Continuous_08-20-2020~08-21-2020', 'w')
        writer = csv.writer(csvfile, delimiter=',', quotechar='|', quoting=csv.QUOTE_MINIMAL)
        self.writer = writer

    def logdata(self):
        txt = []

        txt.append('{}'.format(
            self.data.datetime.datetime(0).isoformat())
        )
        txt.append('{:.2f}'.format(self.data.open[0]))
        txt.append('{:.2f}'.format(self.data.high[0]))
        txt.append('{:.2f}'.format(self.data.low[0]))
        txt.append('{:.2f}'.format(self.data.close[0]))
        txt.append('{:.2f}'.format(self.data.volume[0]))
        print(txt)
        self.writer.writerow(txt)

    def next(self):
        print("next called")
        self.logdata()

    def start(self):
        print('Historical Data gathering is about to start')

    def stop(self):
        print('Historical Data gathering is finished')


def run(from_date, to_date):
    cerebro = bt.Cerebro(stdstats=False)
    ibstore = bt.stores.IBStore(host='127.0.0.1', port=7496, clientId=35, _debug=True)
    data = ibstore.getdata(dataname='ES-CONTFUT-GLOBEX-USD', fromdate=from_date, todate=to_date, timeframe=bt.TimeFrame.Seconds)
    cerebro.adddata(data)
    cerebro.addstrategy(St)
    cerebro.run()
    cerebro.plot()

if __name__ == '__main__':
    from_date = datetime.datetime(2020,8,20)
    to_date = datetime.datetime(2020,8,21)
    print("ES_Continuous_" + from_date.strftime('%m-%d-%Y') + "~" + to_date.strftime('%m-%d-%Y'))
    run(from_date, to_date)

    I know that data is flowing in because when I turn debug mode on I get data coming in
    Example:

    <historicalData reqId=16777217, date=1597883367, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=0, count=0, WAP=3361.75, hasGaps=False>
<historicalData reqId=16777217, date=1597883368, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=0, count=0, WAP=3361.75, hasGaps=False>
<historicalData reqId=16777217, date=1597883369, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=0, count=0, WAP=3361.75, hasGaps=False>
<historicalData reqId=16777217, date=1597883370, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=0, count=0, WAP=3361.75, hasGaps=False>
<historicalData reqId=16777217, date=1597883371, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=0, count=0, WAP=3361.75, hasGaps=False>
<historicalData reqId=16777217, date=1597883372, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=0, count=0, WAP=3361.75, hasGaps=False>
<historicalData reqId=16777217, date=1597883373, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=0, count=0, WAP=3361.75, hasGaps=False>
<historicalData reqId=16777217, date=1597883374, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=0, count=0, WAP=3361.75, hasGaps=False>
<historicalData reqId=16777217, date=1597883375, open=3362.0, high=3362.0, low=3361.75, close=3361.75, volume=3, count=2, WAP=3361.825, hasGaps=False>
<historicalData reqId=16777217, date=1597883376, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=0, count=0, WAP=3361.75, hasGaps=False>
<historicalData reqId=16777217, date=1597883377, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=0, count=0, WAP=3361.75, hasGaps=False>
<historicalData reqId=16777217, date=1597883378, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=0, count=0, WAP=3361.75, hasGaps=False>
<historicalData reqId=16777217, date=1597883379, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=0, count=0, WAP=3361.75, hasGaps=False>
<historicalData reqId=16777217, date=1597883380, open=3362.0, high=3362.0, low=3362.0, close=3362.0, volume=1, count=1, WAP=3362.0, hasGaps=False>
<historicalData reqId=16777217, date=1597883381, open=3362.0, high=3362.0, low=3362.0, close=3362.0, volume=0, count=0, WAP=3362.0, hasGaps=False>
<historicalData reqId=16777217, date=1597883382, open=3362.0, high=3362.0, low=3362.0, close=3362.0, volume=0, count=0, WAP=3362.0, hasGaps=False>
<historicalData reqId=16777217, date=1597883383, open=3362.0, high=3362.0, low=3362.0, close=3362.0, volume=0, count=0, WAP=3362.0, hasGaps=False>
<historicalData reqId=16777217, date=1597883384, open=3362.0, high=3362.0, low=3362.0, close=3362.0, volume=0, count=0, WAP=3362.0, hasGaps=False>
<historicalData reqId=16777217, date=1597883385, open=3362.0, high=3362.0, low=3362.0, close=3362.0, volume=1, count=1, WAP=3362.0, hasGaps=False>
<historicalData reqId=16777217, date=1597883386, open=3362.0, high=3362.0, low=3362.0, close=3362.0, volume=0, count=0, WAP=3362.0, hasGaps=False>
<historicalData reqId=16777217, date=1597883387, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=1, count=1, WAP=3361.75, hasGaps=False>
<historicalData reqId=16777217, date=1597883388, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=0, count=0, WAP=3361.75, hasGaps=False>
<historicalData reqId=16777217, date=1597883389, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=0, count=0, WAP=3361.75, hasGaps=False>
<historicalData reqId=16777217, date=1597883390, open=3362.0, high=3362.0, low=3361.75, close=3361.75, volume=23, count=5, WAP=3361.75, hasGaps=False>
<historicalData reqId=16777217, date=1597883391, open=3361.5, high=3361.5, low=3361.5, close=3361.5, volume=39, count=8, WAP=3361.5, hasGaps=False>
<historicalData reqId=16777217, date=1597883392, open=3361.5, high=3361.5, low=3361.5, close=3361.5, volume=8, count=6, WAP=3361.5, hasGaps=False>
<historicalData reqId=16777217, date=1597883393, open=3361.5, high=3361.5, low=3361.5, close=3361.5, volume=0, count=0, WAP=3361.5, hasGaps=False>
<historicalData reqId=16777217, date=1597883394, open=3361.5, high=3361.5, low=3361.5, close=3361.5, volume=0, count=0, WAP=3361.5, hasGaps=False>
<historicalData reqId=16777217, date=1597883395, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=20, count=1, WAP=3361.75, hasGaps=False>
<historicalData reqId=16777217, date=1597883396, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=0, count=0, WAP=3361.75, hasGaps=False>
<historicalData reqId=16777217, date=1597883397, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=0, count=0, WAP=3361.75, hasGaps=False>
<historicalData reqId=16777217, date=1597883398, open=3361.75, high=3361.75, low=3361.75, close=3361.75, volume=20, count=1, WAP=3361.75, hasGaps=False>
<historicalData reqId=16777217, date=1597883399, open=3361.5, high=3361.5, low=3361.5, close=3361.5, volume=1, count=1, WAP=3361.5, hasGaps=False>
<historicalData reqId=16777217, date=finished-20200819  19:00:00-20200819  19:30:00, open=-1, high=-1, low=-1, close=-1, volume=-1, count=-1, WAP=-1, hasGaps=False>

    Any ideas on how to get next to trigger? Verify print statements are never hit and csv is not written too.

    0

  • try to initialize the Cerebro engine with preload=False:

    cerebro = bt.Cerebro(stdstats=False, live=False, preload=False)

    please report if it solved the problem. It seems that IBData will not report the end of historical data load ending properly in case the preload was requested - need to investigate further.

    0
    S
     1 Reply
  • S

    @vladisld Could you please explain what preload=False do, and when should we set it to True and False when we are using the interactive broker?

    0
    S
     1 Reply
  • S

    @shspersonalwork I know the following about preload, however I want to know the impact of this setting on Live and Offline mode, and how did you debug the problem to this parameter setting?

    "preload -> Whether to preload the different data feeds passed to cerebro for the Strategies"

    0
Log in to reply
 
});