Hi,

When live trading on IB I wish to handle orders rejected by the type of error code received from IB, the problem is that the error code is received in notify_store, when I only want to address errors for rejected orders when I encounter them them in notify_order (notify_store receives many other messages that I don't wish to address).

The only way I see of doing this is by logging notify_store messages as they come in, then getting the last message received for the order id when I get order.Rejected for it in notify_order.
Is there an easier way of doing this? Is there a way of getting the error code from notify_order that I missed?

Thank you.

@vladisld said in Running multiple symbols data feeds in live IB:

The next notification will be received only when each data will receive at least a single bar

You were right, it did present a problem, thanks for the heads up.

What happens is that with the default qcheck at 0.5 'next' waits 0.5 a second before releasing the data he currently have if all the datas didn't update, then when it all the datas arrive the queue is released at once. The weird thing is that when I set qcheck = 0 I still got some delay that increased over time. So I set qcheck = 0.001 which does synchronizes the datas when they are all available, which seems to be the best solution.

When using prenext I still get the 0.5 data delay until all data data start arriving, so there's probably a parameter I'm missing.

For anyone wondering in the future, the corrected code is:

from __future__ import (absolute_import, division, print_function,
                        unicode_literals)

import backtrader as bt
import backtrader.stores.ibstore as ibstore
import datetime
import pytz


class TestStrategy(bt.Strategy):

    def _print_current_bar_price(self, data):
        txt = list()
        txt.append(data._name)
        txt.append('%04d' % len(data))
        txt.append('%s' % datetime.datetime.now().time())
        txt.append('%s' % data.datetime.datetime(0))
        txt.append('{}'.format(data.open[0]))
        txt.append('{}'.format(data.high[0]))
        txt.append('{}'.format(data.low[0]))
        txt.append('{}'.format(data.close[0]))
        txt.append('{}'.format(data.volume[0]))
        print(', '.join(txt))

    def __init__(self):
        self._count = [0] * len(self.datas)

    def notify_data(self, data, status, *args, **kwargs):
        print('*' * 5, data._name, ' data is ', data._getstatusname(status), *args)

    def notify_store(self, msg, *args, **kwargs):
        print('*' * 5, 'STORE NOTIF:', msg)

    def prenext(self):
        # call next() even when data is not available for all tickers
        self.next()

    def next(self):
        # run on the symbols
        for i, d in enumerate(self.datas):
            if len(d) > self._count[i]:
                self._count[i] = len(d)
                self._print_current_bar_price(d)


if __name__ == '__main__':
    symbols = ["AAPL", "BA", "BBBY", "NVDA"]
    port = 7497

    # Create a cerebro entity
    cerebro = bt.Cerebro()

    # Add a strategy
    cerebro.addstrategy(TestStrategy)

    store = ibstore.IBStore(port=port)
    for s in symbols:
        data = store.getdata(dataname=s + '-STK-SMART-USD', tz=pytz.timezone('US/Eastern'), timeframe=bt.TimeFrame.Seconds, compression=1, qcheck=0.001, backfill_start=False)
        cerebro.adddata(data, name=s)
    cerebro.broker = store.getbroker()  # connect IB broker

    # Run over everything
    cerebro.run()

@vladisld Thank you again for your great help. I applied your fix and it indeed solved the problem.
The other delay I've encountered is simply because it looks like Backtrader waits until it receives new data from all the data feeds before starting to run.

Backfill doesn't give more delay (besides the time it takes to load the historical data), what's weird is that it seems like Backtrader still waits until it receives new data from all the data feeds, and only then starts to load the backfill data.

@vladisld Thank you so much for your reply, I tried the fix and it indeed seems to solve the problem.

I did notice that once you try to use more then three data feeds it can take a long time (minutes) for Backtrader to start running. notify_data sends message that Delayed data is used, but it takes minutes until the data actually start to arrive, or it never arrives at all.

TWS Time at connection:20200224 12:27:20 EST
ReqData: reqMktData for BA is issued.
ReqData: reqMktData for BA is issued.
ReqData: reqMktData for BBBY is issued.
ReqData: reqMktData for BA is issued.
ReqData: reqMktData for BBBY is issued.
ReqData: reqMktData for CAMP is issued.
ReqData: reqMktData for BA is issued.
ReqData: reqMktData for BBBY is issued.
ReqData: reqMktData for CAMP is issued.
ReqData: reqMktData for DL is issued.
***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm connection is OK:usfarm.nj>
***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm connection is OK:cashfarm>
***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm connection is OK:usfarm>
***** STORE NOTIF: <error id=-1, errorCode=2106, errorMsg=HMDS data farm connection is OK:ushmds>
***** STORE NOTIF: <error id=-1, errorCode=2158, errorMsg=Sec-def data farm connection is OK:secdefnj>
***** BA  data is  DELAYED
***** BBBY  data is  DELAYED
***** CAMP  data is  DELAYED
***** DL  data is  DELAYED

Any ideas what could be causing it?

Thanks again!

Hi,
I'm trying to run a strategy for live IB trading that runs on several symbols 1 second bars.
The issue I'm encountering is that sometimes when I try to run I get the following error:

Traceback (most recent call last):
  File "C:\Work\Python\Backtrader\venv\lib\site-packages\IPython\core\interactiveshell.py", line 2722, in safe_execfile
    self.compile if shell_futures else None)
  File "C:\Work\Python\Backtrader\venv\lib\site-packages\IPython\utils\py3compat.py", line 168, in execfile
    exec(compiler(f.read(), fname, 'exec'), glob, loc)
  File "C:\Work\Python\Backtrader\load_issue_script.py", line 55, in <module>
    cerebro.run()
  File "C:\Work\Python\Backtrader\venv\lib\site-packages\backtrader\cerebro.py", line 1127, in run
    runstrat = self.runstrategies(iterstrat)
  File "C:\Work\Python\Backtrader\venv\lib\site-packages\backtrader\cerebro.py", line 1298, in runstrategies
    self._runnext(runstrats)
  File "C:\Work\Python\Backtrader\venv\lib\site-packages\backtrader\cerebro.py", line 1630, in _runnext
    strat._next()
  File "C:\Work\Python\Backtrader\venv\lib\site-packages\backtrader\strategy.py", line 347, in _next
    super(Strategy, self)._next()
  File "C:\Work\Python\Backtrader\venv\lib\site-packages\backtrader\lineiterator.py", line 260, in _next
    clock_len = self._clk_update()
  File "C:\Work\Python\Backtrader\venv\lib\site-packages\backtrader\strategy.py", line 327, in _clk_update
    newdlens = [len(d) for d in self.datas]
  File "C:\Work\Python\Backtrader\venv\lib\site-packages\backtrader\strategy.py", line 327, in <listcomp>
    newdlens = [len(d) for d in self.datas]
  File "C:\Work\Python\Backtrader\venv\lib\site-packages\backtrader\lineseries.py", line 464, in __len__
    return len(self.lines)
  File "C:\Work\Python\Backtrader\venv\lib\site-packages\backtrader\lineseries.py", line 220, in __len__
    return len(self.lines[0])
ValueError: __len__() should return >= 0

It seems that the code doesn't always wait for all the data feeds to have data before trying to run on them. Sometimes it works without a problem but a lot of time it crashes, especially the more data feeds you add.

Can someone please tell me why this is happening? I wrote a very simple script that replicates the issue:

from __future__ import (absolute_import, division, print_function,
                        unicode_literals)

import backtrader as bt
import backtrader.stores.ibstore as ibstore
import pytz


class TestStrategy(bt.Strategy):

    def _print_current_bar_price(self, data):
        txt = list()
        txt.append(data._name)
        txt.append('%04d' % len(data))
        txt.append('%s' % data.datetime.datetime(0))
        txt.append('{}'.format(data.open[0]))
        txt.append('{}'.format(data.high[0]))
        txt.append('{}'.format(data.low[0]))
        txt.append('{}'.format(data.close[0]))
        txt.append('{}'.format(data.volume[0]))
        print(', '.join(txt))

    def notify_data(self, data, status, *args, **kwargs):
        print('*' * 5, data._name, ' data is ', data._getstatusname(status), *args)

    def notify_store(self, msg, *args, **kwargs):
        print('*' * 5, 'STORE NOTIF:', msg)

    def next(self):
        # run on the symbols
        for i, d in enumerate(self.datas):
            self._print_current_bar_price(d)


if __name__ == '__main__':
    symbols = ["AAPL", "BBBY", "BA", "NVDA"]
    port = 7497

    # Create a cerebro entity
    cerebro = bt.Cerebro()

    # Add a strategy
    cerebro.addstrategy(TestStrategy)

    store = ibstore.IBStore(port=port)
    for s in symbols:
        data = store.getdata(dataname=s + '-STK-SMART-USD', tz=pytz.timezone('US/Eastern'), timeframe=bt.TimeFrame.Seconds, compression=1)
        cerebro.adddata(data, name=s)
    cerebro.broker = store.getbroker()  # connect IB broker

    # Run over everything
    cerebro.run()

The full output for it is:

TWS Time at connection:20200221 11:39:43 EST
***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm connection is OK:usfarm.nj>
***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm connection is OK:cashfarm>
***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm connection is OK:usfarm>
***** STORE NOTIF: <error id=-1, errorCode=2106, errorMsg=HMDS data farm connection is OK:euhmds>
***** STORE NOTIF: <error id=-1, errorCode=2106, errorMsg=HMDS data farm connection is OK:fundfarm>
***** STORE NOTIF: <error id=-1, errorCode=2106, errorMsg=HMDS data farm connection is OK:ushmds>
***** STORE NOTIF: <error id=-1, errorCode=2158, errorMsg=Sec-def data farm connection is OK:secdefnj>
***** AAPL  data is  DELAYED
***** BBBY  data is  DELAYED
***** BA  data is  DELAYED
***** NVDA  data is  DELAYED
Traceback (most recent call last):
  File "C:\Work\Python\Backtrader\venv\lib\site-packages\IPython\core\interactiveshell.py", line 2722, in safe_execfile
    self.compile if shell_futures else None)
  File "C:\Work\Python\Backtrader\venv\lib\site-packages\IPython\utils\py3compat.py", line 168, in execfile
    exec(compiler(f.read(), fname, 'exec'), glob, loc)
  File "C:\Work\Python\Backtrader\load_issue_script.py", line 52, in <module>
    cerebro.run()
  File "C:\Work\Python\Backtrader\venv\lib\site-packages\backtrader\cerebro.py", line 1127, in run
    runstrat = self.runstrategies(iterstrat)
  File "C:\Work\Python\Backtrader\venv\lib\site-packages\backtrader\cerebro.py", line 1298, in runstrategies
    self._runnext(runstrats)
  File "C:\Work\Python\Backtrader\venv\lib\site-packages\backtrader\cerebro.py", line 1630, in _runnext
    strat._next()
  File "C:\Work\Python\Backtrader\venv\lib\site-packages\backtrader\strategy.py", line 347, in _next
    super(Strategy, self)._next()
  File "C:\Work\Python\Backtrader\venv\lib\site-packages\backtrader\lineiterator.py", line 260, in _next
    clock_len = self._clk_update()
  File "C:\Work\Python\Backtrader\venv\lib\site-packages\backtrader\strategy.py", line 327, in _clk_update
    newdlens = [len(d) for d in self.datas]
  File "C:\Work\Python\Backtrader\venv\lib\site-packages\backtrader\strategy.py", line 327, in <listcomp>
    newdlens = [len(d) for d in self.datas]
  File "C:\Work\Python\Backtrader\venv\lib\site-packages\backtrader\lineseries.py", line 464, in __len__
    return len(self.lines)
  File "C:\Work\Python\Backtrader\venv\lib\site-packages\backtrader\lineseries.py", line 220, in __len__
    return len(self.lines[0])
ValueError: __len__() should return >= 0

Thank you