For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

Starting with Interactive Brokers (IB) - no data feed



  • Hi
    I am following the blog in Medium:
    https://medium.com/@danjrod/interactive-brokers-in-python-with-backtrader-23dea376b2fc
    So far I can get connection notification from TWS (ie the message: "TWS Time at connection...." but there is no data received. There are no run time errors or other messages.
    I am using the code (without modification) in the blog section "Getting a data stream".
    I am using a demo (paper trading) account on IB
    Thanks



  • I think I might have found the problem
    The relevant code in the blog (which was written in May of last year) is

        data = store.getdata(dataname='TWTR',
                             timeframe=bt.TimeFrame.Ticks) 
    

    However I think that IB may now require more info in the dataname string than just the ticker (eg also exchange information)


  • administrators

    The complete documentation for working with Interactive Brokers: https://www.backtrader.com/docu/live/ib/ib/

    As pointed out there, some tickers require extra information (a lot more tokens in the string) and some require less. If TWTR only required one token initially and now you say it may require more (you provide no data as to what the actual problem is) it may simply be because it now trades in more exchanges (i.e.: internationally) and probably under different currencies.

    The only thing you can do is query the contracts and see how many come back under the ticker TWTR.



  • @backtrader thanks for you reply. I will do as you suggest, and do some more investigation using TWS directly to find out what the problem is.



  • I am at a loss! I have managed to get data from IB for forex pairs but I have failed with stocks, and futures. This applies for both backfill and realtime data

    I have used the example contracts in
    https://interactivebrokers.github.io/tws-api/basic_contracts.html
    as a guide
    I am using the demo account on IB
    I have TWS running with no errors
    Any suggestions would be welcome as I am running out of ideas!

    Here is my code for GBP.USD which works:

    def run(args=None):
    
        cerebro = bt.Cerebro(stdstats=False)
        cerebro.addstrategy(St)
        store = bt.stores.IBStore(port=7497)
        data = store.getdata(dataname='GBP.USD',
                             sectype='CASH',
                             exchange='IDEALPRO',
                             fromdate=datetime.datetime(2019, 9, 11),
                             todate=datetime.datetime(2019, 9, 12),
                             timeframe=bt.TimeFrame.Minutes,
                             compression=5)
        cerebro.adddata(data)
        cerebro.run()
    
    if __name__ == '__main__':
        run()
    
    C:\Users\Owner\Miniconda3\envs\projectv36_2\python.exe C:/Users/Owner/TestTA-LIB/test_IB.py
    Server Version: 76
    TWS Time at connection:20190924 21:18:34 BST
    ***** DATA NOTIF: DELAYED
    1,2019-09-11T21:15:00,1.23210,1.23269,1.23210,1.23269,-1.00000
    2,2019-09-11T21:20:00,1.23269,1.23315,1.23260,1.23273,-1.00000
    .
    .
    .
    .
    33,2019-09-11T23:55:00,1.23341,1.23345,1.23330,1.23342,-1.00000
    34,2019-09-12T00:00:00,1.23342,1.23342,1.23286,1.23307,-1.00000
    
    Process finished with exit code 0
    

    and here is my code for AAPL which doesn't work

        data = store.getdata(dataname='AAPL',
                             sectype='STK',
                             exchange='SMART',
                             currency='USD',
                             fromdate=datetime.datetime(2019, 9, 11),
                             todate=datetime.datetime(2019, 9, 12),
                             timeframe=bt.TimeFrame.Minutes,
                             compression=5)
       
    
    C:\Users\Owner\Miniconda3\envs\projectv36_2\python.exe C:/Users/Owner/TestTA-LIB/test_IB.py
    Server Version: 76
    TWS Time at connection:20190924 20:41:11 BST
    ***** DATA NOTIF: DELAYED
    
    

    nothing appears even after waiting 30 mins with the program running!


  • administrators

    @rluc99 said in Starting with Interactive Brokers (IB) - no data feed:

                             fromdate=datetime.datetime(2019, 9, 11),
                             todate=datetime.datetime(2019, 9, 12),
    

    You seem to be willing to do a historical download, but there is no historical parameter.



  • @backtrader Thanks. I will give historical=True a try.
    I thought it wasn't needed if a todate was specified (otherwise why does the first case work?)


  • administrators

    Because Forex trades 24 hours a day.



  • @backtrader I see. So in that case I presume I will only get historical stock data from IB when the market is open (and with historical=True) even though it is available from TWS at any time.



  • Still no luck accessing data. Is there some way of debugging the communication between IB and backtrader?

    Code:

        data = store.getdata(dataname='AAPL',
                             secType='STK',
                             exchange='SMART',
                             currency='USD',
                             historical=True,
                             timeframe=bt.TimeFrame.Minutes,
                             compression=5)
    

    output:

    C:\Users\Owner\Miniconda3\envs\projectv36_2\python.exe C:/Users/Owner/TestTA-LIB/test_IB.py
    Server Version: 76
    TWS Time at connection:20190926 19:15:55 British Summer Time
    
    Process finished with exit code 0
    

    code

        data = store.getdata(dataname='AAPL',
                             secType='STK',
                             exchange='SMART',
                             currency='USD',
                             historical=False,
                             timeframe=bt.TimeFrame.Minutes,
                             compression=5)
    

    output:

    C:\Users\Owner\Miniconda3\envs\projectv36_2\python.exe C:/Users/Owner/TestTA-LIB/test_IB.py
    Server Version: 76
    TWS Time at connection:20190926 19:17:03 British Summer Time
    ***** DATA NOTIF: DELAYED
    
    
    


  • First let's get some historical data. Try doing it like this. Set up your stock key word arguments as follows:

    stockkwargs = dict(
           timeframe=bt.TimeFrame.Minutes,
           rtbar=False,  # use RealTime 5 seconds bars
           historical=True,  # only historical download
           qcheck=0.5,  # timeout in seconds (float) to check for events
           fromdate=datetime(2019, 9, 24),  # get data from..
           todate=datetime(2019, 9, 25),  # get data from..
           latethrough=False,  # let late samples through
           tradename=None  # use a different asset as order target
       )
    

    Then get your data using:

    data0 = store.getdata(dataname="AAPL-STK-SMART-USD", **stockkwargs)
    

    Then add your data using:

    cerebro.resampledata(data0, timeframe=bt.TimeFrame.Minutes, compression=5)
    

    If you print from next() in your strategy, or call a log function from there, you should get some results. Make sure TWS is open and connected.

    This whole code works for historical. Once you have that working, switch to live and try again when the market is open.

    #!/usr/bin/env python
    # -*- coding: utf-8; py-indent-offset:4 -*-e
    
    from __future__ import absolute_import, division, print_function, unicode_literals
    import backtrader as bt
    import datetime
    
    class St(bt.Strategy):
    
        def logdata(self):
                txt = []
                txt.append("{}".format(self.data._name))
                txt.append("{}".format(len(self)))
                txt.append("{}".format(self.data.datetime.date(0)))
                txt.append("{:.2f}".format(self.data.open[0]))
                txt.append("{:.2f}".format(self.data.high[0]))
                txt.append("{:.2f}".format(self.data.low[0]))
                txt.append("{:.2f}".format(self.data.close[0]))
                txt.append("{:.2f}".format(self.data.volume[0]))
                print(", ".join(txt))
    
    
        def next(self):
            self.logdata()
    
    
    def run():
    
        cerebro = bt.Cerebro(stdstats=False)
    
        store = bt.stores.IBStore(host="127.0.0.1", port=7497, clientId= ###insert_your_number_here### )
        cerebro.broker = store.getbroker()
    
        stockkwargs = dict(
            timeframe=bt.TimeFrame.Minutes,
            rtbar=False,  # use RealTime 5 seconds bars
            historical=True,  # only historical download
            qcheck=0.5,  # timeout in seconds (float) to check for events
            fromdate=datetime.datetime(2019, 9, 24),  # get data from..
            todate=datetime.datetime(2019, 9, 25),  # get data from..
            latethrough=False,  # let late samples through
            tradename=None  # use a different asset as order target
        )
        data0 = store.getdata(dataname="AAPL-STK-SMART-USD", **stockkwargs)
        cerebro.resampledata(data0, timeframe=bt.TimeFrame.Minutes, compression=5)
    
        cerebro.addstrategy(St)
        cerebro.run()
    
    
    
    
    if __name__ == "__main__":
    
        run()
    
    

    The output starts like this...

    TWS Time at connection:20190926 17:50:20 EST
    AAPL-STK-SMART-USD, 1, 2019-09-24, 219.90, 219.90, 219.80, 219.80, 12.00
    AAPL-STK-SMART-USD, 2, 2019-09-24, 219.80, 219.80, 219.80, 219.80, 0.00
    AAPL-STK-SMART-USD, 3, 2019-09-24, 219.90, 219.90, 219.90, 219.90, 1.00
    AAPL-STK-SMART-USD, 4, 2019-09-24, 219.90, 219.90, 219.90, 219.90, 0.00
    AAPL-STK-SMART-USD, 5, 2019-09-24, 219.90, 219.90, 219.90, 219.90, 0.00
    AAPL-STK-SMART-USD, 6, 2019-09-24, 219.96, 220.30, 219.96, 220.30, 12.00
    AAPL-STK-SMART-USD, 7, 2019-09-24, 220.30, 220.67, 220.30, 220.39, 35.00
    AAPL-STK-SMART-USD, 8, 2019-09-24, 220.21, 220.21, 220.16, 220.16, 16.00
    AAPL-STK-SMART-USD, 9, 2019-09-24, 220.05, 220.06, 220.03, 220.06, 4.00
    


  • @run-out . Thanks so much for taking the time to put together your comprehensive response!

    I copied your code exactly, apart from the clientId= parameter which I did not include since I am using the IB demo account without a client ID. See below:
    Capture.JPG

    Unfortunately, I get the same result as I was getting before. IE I can receive forex bars but not stocks.
    In other words this line:

     data0 = store.getdata(dataname="AAPL-STK-SMART-USD", **stockkwargs)
    

    produces:

    C:\Users\Owner\Miniconda3\envs\projectv36_2\python.exe C:/Users/Owner/TestTA-LIB/test_IB_connection.py
    Server Version: 76
    TWS Time at connection:20190927 16:41:10 British Summer Time
    
    Process finished with exit code 0
    

    Whereas replacing it with this line:

        data0 = store.getdata(dataname="GBP.USD-CASH-IDEALPRO", **stockkwargs)
    

    produces:

    C:\Users\Owner\Miniconda3\envs\projectv36_2\python.exe C:/Users/Owner/TestTA-LIB/test_IB_connection.py
    Server Version: 76
    TWS Time at connection:20190927 16:56:40 British Summer Time
    GBP.USD-CASH-IDEALPRO, 1, 2019-09-24, 1.25, 1.25, 1.25, 1.25, -1.00
    GBP.USD-CASH-IDEALPRO, 2, 2019-09-24, 1.25, 1.25, 1.25, 1.25, -1.00
    .
    .
    .
    GBP.USD-CASH-IDEALPRO, 32, 2019-09-24, 1.25, 1.25, 1.25, 1.25, -1.00
    GBP.USD-CASH-IDEALPRO, 33, 2019-09-24, 1.25, 1.25, 1.25, 1.25, -1.00
    
    Process finished with exit code 0
    

    TWS is open and connected and the market is open. AAPL bars are showing up on the chart (after a refresh). See below:
    Capture.JPG

    I am using the latest TWS version:
    Capture.JPG

    I feel it should be something very obvious but I have no idea what!



  • Sorry I haven't used the demo account. You might want to search https://groups.io/g/twsapi .



  • @run-out Thanks - I'll give it a try



  • I have discovered the problem. It is not related to Backtrader.

    I did some digging around on the IB website. It seems that the IB demo account does not allow historical bars through the API for anything other than FOREX. The same applies to paper trading or live accounts unless there is a subscription to the relevant market data feed. If there is a subscription in place then the historical bars are available at any time, including week-ends.
    (As an aside, historical bars are constructed by TWS regardless of whether there is a subscription in place or not.)
    Delayed data is available during market hours for accounts without market data subscription, but to obtain this data a flag has to be set in the API. I don't think Backtrader allows the user to set this flag but I have not investigated this in detail.

    Most of the above information comes from the following two links:
    http://interactivebrokers.github.io/tws-api/market_data.html
    http://interactivebrokers.github.io/tws-api/historical_data.html



  • @rluc99 Thanks for the update!



  • One comment that I see.

    I believe that we can remove the parameter on timeframe in stockkwargs so that it becomes as follows since we will be resampling the data anyway later on

    stockkwargs = dict(
    rtbar=False, # use RealTime 5 seconds bars
    historical=True, # only historical download
    qcheck=0.5, # timeout in seconds (float) to check for events
    fromdate=datetime.datetime(2019, 9, 24), # get data from..
    todate=datetime.datetime(2019, 9, 25), # get data from..
    latethrough=False, # let late samples through
    tradename=None # use a different asset as order target


Log in to reply
 

});