Navigation

    Backtrader Community

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

    Starting with Interactive Brokers (IB) - no data feed

    General Code/Help
    7
    21
    4405
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • R
      rluc99 last edited by

      Hi
      I am following the blog in Medium:
      https://medium.com/@danjrod/interactive-brokers-in-python-with-backtrader-23dea376b2fc
      So far I can get connection notification from TWS (ie the message: "TWS Time at connection...." but there is no data received. There are no run time errors or other messages.
      I am using the code (without modification) in the blog section "Getting a data stream".
      I am using a demo (paper trading) account on IB
      Thanks

      R 1 Reply Last reply Reply Quote 0
      • R
        rluc99 @rluc99 last edited by

        I think I might have found the problem
        The relevant code in the blog (which was written in May of last year) is

            data = store.getdata(dataname='TWTR',
                                 timeframe=bt.TimeFrame.Ticks) 
        

        However I think that IB may now require more info in the dataname string than just the ticker (eg also exchange information)

        1 Reply Last reply Reply Quote 0
        • B
          backtrader administrators last edited by

          The complete documentation for working with Interactive Brokers: https://www.backtrader.com/docu/live/ib/ib/

          As pointed out there, some tickers require extra information (a lot more tokens in the string) and some require less. If TWTR only required one token initially and now you say it may require more (you provide no data as to what the actual problem is) it may simply be because it now trades in more exchanges (i.e.: internationally) and probably under different currencies.

          The only thing you can do is query the contracts and see how many come back under the ticker TWTR.

          R 1 Reply Last reply Reply Quote 1
          • R
            rluc99 @backtrader last edited by

            @backtrader thanks for you reply. I will do as you suggest, and do some more investigation using TWS directly to find out what the problem is.

            R 1 Reply Last reply Reply Quote 0
            • R
              rluc99 @rluc99 last edited by

              I am at a loss! I have managed to get data from IB for forex pairs but I have failed with stocks, and futures. This applies for both backfill and realtime data

              I have used the example contracts in
              https://interactivebrokers.github.io/tws-api/basic_contracts.html
              as a guide
              I am using the demo account on IB
              I have TWS running with no errors
              Any suggestions would be welcome as I am running out of ideas!

              Here is my code for GBP.USD which works:

              def run(args=None):
              
                  cerebro = bt.Cerebro(stdstats=False)
                  cerebro.addstrategy(St)
                  store = bt.stores.IBStore(port=7497)
                  data = store.getdata(dataname='GBP.USD',
                                       sectype='CASH',
                                       exchange='IDEALPRO',
                                       fromdate=datetime.datetime(2019, 9, 11),
                                       todate=datetime.datetime(2019, 9, 12),
                                       timeframe=bt.TimeFrame.Minutes,
                                       compression=5)
                  cerebro.adddata(data)
                  cerebro.run()
              
              if __name__ == '__main__':
                  run()
              
              C:\Users\Owner\Miniconda3\envs\projectv36_2\python.exe C:/Users/Owner/TestTA-LIB/test_IB.py
              Server Version: 76
              TWS Time at connection:20190924 21:18:34 BST
              ***** DATA NOTIF: DELAYED
              1,2019-09-11T21:15:00,1.23210,1.23269,1.23210,1.23269,-1.00000
              2,2019-09-11T21:20:00,1.23269,1.23315,1.23260,1.23273,-1.00000
              .
              .
              .
              .
              33,2019-09-11T23:55:00,1.23341,1.23345,1.23330,1.23342,-1.00000
              34,2019-09-12T00:00:00,1.23342,1.23342,1.23286,1.23307,-1.00000
              
              Process finished with exit code 0
              

              and here is my code for AAPL which doesn't work

                  data = store.getdata(dataname='AAPL',
                                       sectype='STK',
                                       exchange='SMART',
                                       currency='USD',
                                       fromdate=datetime.datetime(2019, 9, 11),
                                       todate=datetime.datetime(2019, 9, 12),
                                       timeframe=bt.TimeFrame.Minutes,
                                       compression=5)
                 
              
              C:\Users\Owner\Miniconda3\envs\projectv36_2\python.exe C:/Users/Owner/TestTA-LIB/test_IB.py
              Server Version: 76
              TWS Time at connection:20190924 20:41:11 BST
              ***** DATA NOTIF: DELAYED
              
              

              nothing appears even after waiting 30 mins with the program running!

              1 Reply Last reply Reply Quote 0
              • B
                backtrader administrators last edited by

                @rluc99 said in Starting with Interactive Brokers (IB) - no data feed:

                                         fromdate=datetime.datetime(2019, 9, 11),
                                         todate=datetime.datetime(2019, 9, 12),
                

                You seem to be willing to do a historical download, but there is no historical parameter.

                • https://www.backtrader.com/docu/live/ib/ib/
                R 1 Reply Last reply Reply Quote 0
                • R
                  rluc99 @backtrader last edited by

                  @backtrader Thanks. I will give historical=True a try.
                  I thought it wasn't needed if a todate was specified (otherwise why does the first case work?)

                  1 Reply Last reply Reply Quote 0
                  • B
                    backtrader administrators last edited by

                    Because Forex trades 24 hours a day.

                    R 1 Reply Last reply Reply Quote 0
                    • R
                      rluc99 @backtrader last edited by

                      @backtrader I see. So in that case I presume I will only get historical stock data from IB when the market is open (and with historical=True) even though it is available from TWS at any time.

                      R 1 Reply Last reply Reply Quote 0
                      • R
                        rluc99 @rluc99 last edited by

                        Still no luck accessing data. Is there some way of debugging the communication between IB and backtrader?

                        Code:

                            data = store.getdata(dataname='AAPL',
                                                 secType='STK',
                                                 exchange='SMART',
                                                 currency='USD',
                                                 historical=True,
                                                 timeframe=bt.TimeFrame.Minutes,
                                                 compression=5)
                        

                        output:

                        C:\Users\Owner\Miniconda3\envs\projectv36_2\python.exe C:/Users/Owner/TestTA-LIB/test_IB.py
                        Server Version: 76
                        TWS Time at connection:20190926 19:15:55 British Summer Time
                        
                        Process finished with exit code 0
                        

                        code

                            data = store.getdata(dataname='AAPL',
                                                 secType='STK',
                                                 exchange='SMART',
                                                 currency='USD',
                                                 historical=False,
                                                 timeframe=bt.TimeFrame.Minutes,
                                                 compression=5)
                        

                        output:

                        C:\Users\Owner\Miniconda3\envs\projectv36_2\python.exe C:/Users/Owner/TestTA-LIB/test_IB.py
                        Server Version: 76
                        TWS Time at connection:20190926 19:17:03 British Summer Time
                        ***** DATA NOTIF: DELAYED
                        
                        
                        
                        1 Reply Last reply Reply Quote 0
                        • run-out
                          run-out last edited by

                          First let's get some historical data. Try doing it like this. Set up your stock key word arguments as follows:

                          stockkwargs = dict(
                                 timeframe=bt.TimeFrame.Minutes,
                                 rtbar=False,  # use RealTime 5 seconds bars
                                 historical=True,  # only historical download
                                 qcheck=0.5,  # timeout in seconds (float) to check for events
                                 fromdate=datetime(2019, 9, 24),  # get data from..
                                 todate=datetime(2019, 9, 25),  # get data from..
                                 latethrough=False,  # let late samples through
                                 tradename=None  # use a different asset as order target
                             )
                          

                          Then get your data using:

                          data0 = store.getdata(dataname="AAPL-STK-SMART-USD", **stockkwargs)
                          

                          Then add your data using:

                          cerebro.resampledata(data0, timeframe=bt.TimeFrame.Minutes, compression=5)
                          

                          If you print from next() in your strategy, or call a log function from there, you should get some results. Make sure TWS is open and connected.

                          This whole code works for historical. Once you have that working, switch to live and try again when the market is open.

                          #!/usr/bin/env python
                          # -*- coding: utf-8; py-indent-offset:4 -*-e
                          
                          from __future__ import absolute_import, division, print_function, unicode_literals
                          import backtrader as bt
                          import datetime
                          
                          class St(bt.Strategy):
                          
                              def logdata(self):
                                      txt = []
                                      txt.append("{}".format(self.data._name))
                                      txt.append("{}".format(len(self)))
                                      txt.append("{}".format(self.data.datetime.date(0)))
                                      txt.append("{:.2f}".format(self.data.open[0]))
                                      txt.append("{:.2f}".format(self.data.high[0]))
                                      txt.append("{:.2f}".format(self.data.low[0]))
                                      txt.append("{:.2f}".format(self.data.close[0]))
                                      txt.append("{:.2f}".format(self.data.volume[0]))
                                      print(", ".join(txt))
                          
                          
                              def next(self):
                                  self.logdata()
                          
                          
                          def run():
                          
                              cerebro = bt.Cerebro(stdstats=False)
                          
                              store = bt.stores.IBStore(host="127.0.0.1", port=7497, clientId= ###insert_your_number_here### )
                              cerebro.broker = store.getbroker()
                          
                              stockkwargs = dict(
                                  timeframe=bt.TimeFrame.Minutes,
                                  rtbar=False,  # use RealTime 5 seconds bars
                                  historical=True,  # only historical download
                                  qcheck=0.5,  # timeout in seconds (float) to check for events
                                  fromdate=datetime.datetime(2019, 9, 24),  # get data from..
                                  todate=datetime.datetime(2019, 9, 25),  # get data from..
                                  latethrough=False,  # let late samples through
                                  tradename=None  # use a different asset as order target
                              )
                              data0 = store.getdata(dataname="AAPL-STK-SMART-USD", **stockkwargs)
                              cerebro.resampledata(data0, timeframe=bt.TimeFrame.Minutes, compression=5)
                          
                              cerebro.addstrategy(St)
                              cerebro.run()
                          
                          
                          
                          
                          if __name__ == "__main__":
                          
                              run()
                          
                          

                          The output starts like this...

                          TWS Time at connection:20190926 17:50:20 EST
                          AAPL-STK-SMART-USD, 1, 2019-09-24, 219.90, 219.90, 219.80, 219.80, 12.00
                          AAPL-STK-SMART-USD, 2, 2019-09-24, 219.80, 219.80, 219.80, 219.80, 0.00
                          AAPL-STK-SMART-USD, 3, 2019-09-24, 219.90, 219.90, 219.90, 219.90, 1.00
                          AAPL-STK-SMART-USD, 4, 2019-09-24, 219.90, 219.90, 219.90, 219.90, 0.00
                          AAPL-STK-SMART-USD, 5, 2019-09-24, 219.90, 219.90, 219.90, 219.90, 0.00
                          AAPL-STK-SMART-USD, 6, 2019-09-24, 219.96, 220.30, 219.96, 220.30, 12.00
                          AAPL-STK-SMART-USD, 7, 2019-09-24, 220.30, 220.67, 220.30, 220.39, 35.00
                          AAPL-STK-SMART-USD, 8, 2019-09-24, 220.21, 220.21, 220.16, 220.16, 16.00
                          AAPL-STK-SMART-USD, 9, 2019-09-24, 220.05, 220.06, 220.03, 220.06, 4.00
                          

                          RunBacktest.com

                          R 1 Reply Last reply Reply Quote 1
                          • R
                            rluc99 @run-out last edited by

                            @run-out . Thanks so much for taking the time to put together your comprehensive response!

                            I copied your code exactly, apart from the clientId= parameter which I did not include since I am using the IB demo account without a client ID. See below:
                            Capture.JPG

                            Unfortunately, I get the same result as I was getting before. IE I can receive forex bars but not stocks.
                            In other words this line:

                             data0 = store.getdata(dataname="AAPL-STK-SMART-USD", **stockkwargs)
                            

                            produces:

                            C:\Users\Owner\Miniconda3\envs\projectv36_2\python.exe C:/Users/Owner/TestTA-LIB/test_IB_connection.py
                            Server Version: 76
                            TWS Time at connection:20190927 16:41:10 British Summer Time
                            
                            Process finished with exit code 0
                            

                            Whereas replacing it with this line:

                                data0 = store.getdata(dataname="GBP.USD-CASH-IDEALPRO", **stockkwargs)
                            

                            produces:

                            C:\Users\Owner\Miniconda3\envs\projectv36_2\python.exe C:/Users/Owner/TestTA-LIB/test_IB_connection.py
                            Server Version: 76
                            TWS Time at connection:20190927 16:56:40 British Summer Time
                            GBP.USD-CASH-IDEALPRO, 1, 2019-09-24, 1.25, 1.25, 1.25, 1.25, -1.00
                            GBP.USD-CASH-IDEALPRO, 2, 2019-09-24, 1.25, 1.25, 1.25, 1.25, -1.00
                            .
                            .
                            .
                            GBP.USD-CASH-IDEALPRO, 32, 2019-09-24, 1.25, 1.25, 1.25, 1.25, -1.00
                            GBP.USD-CASH-IDEALPRO, 33, 2019-09-24, 1.25, 1.25, 1.25, 1.25, -1.00
                            
                            Process finished with exit code 0
                            

                            TWS is open and connected and the market is open. AAPL bars are showing up on the chart (after a refresh). See below:
                            Capture.JPG

                            I am using the latest TWS version:
                            Capture.JPG

                            I feel it should be something very obvious but I have no idea what!

                            1 Reply Last reply Reply Quote 0
                            • run-out
                              run-out last edited by

                              Sorry I haven't used the demo account. You might want to search https://groups.io/g/twsapi .

                              RunBacktest.com

                              R 1 Reply Last reply Reply Quote 0
                              • R
                                rluc99 @run-out last edited by

                                @run-out Thanks - I'll give it a try

                                R 1 Reply Last reply Reply Quote 0
                                • R
                                  rluc99 @rluc99 last edited by

                                  I have discovered the problem. It is not related to Backtrader.

                                  I did some digging around on the IB website. It seems that the IB demo account does not allow historical bars through the API for anything other than FOREX. The same applies to paper trading or live accounts unless there is a subscription to the relevant market data feed. If there is a subscription in place then the historical bars are available at any time, including week-ends.
                                  (As an aside, historical bars are constructed by TWS regardless of whether there is a subscription in place or not.)
                                  Delayed data is available during market hours for accounts without market data subscription, but to obtain this data a flag has to be set in the API. I don't think Backtrader allows the user to set this flag but I have not investigated this in detail.

                                  Most of the above information comes from the following two links:
                                  http://interactivebrokers.github.io/tws-api/market_data.html
                                  http://interactivebrokers.github.io/tws-api/historical_data.html

                                  run-out SPECULARI FUND 2 Replies Last reply Reply Quote 2
                                  • run-out
                                    run-out @rluc99 last edited by

                                    @rluc99 Thanks for the update!

                                    RunBacktest.com

                                    1 Reply Last reply Reply Quote 0
                                    • Squirrel Singing
                                      Squirrel Singing last edited by

                                      One comment that I see.

                                      I believe that we can remove the parameter on timeframe in stockkwargs so that it becomes as follows since we will be resampling the data anyway later on

                                      stockkwargs = dict(
                                      rtbar=False, # use RealTime 5 seconds bars
                                      historical=True, # only historical download
                                      qcheck=0.5, # timeout in seconds (float) to check for events
                                      fromdate=datetime.datetime(2019, 9, 24), # get data from..
                                      todate=datetime.datetime(2019, 9, 25), # get data from..
                                      latethrough=False, # let late samples through
                                      tradename=None # use a different asset as order target

                                      Squirrel Singing

                                      1 Reply Last reply Reply Quote 0
                                      • J
                                        joe trs last edited by

                                        @rluc99 said in Starting with Interactive Brokers (IB) - no data feed:

                                        ***** DATA NOTIF: DELAYED

                                        My hats off to the great platform and community.

                                        Like @rluc99, I can successfully get live forex data on interactive brokers TWS but everything else (indices, commodities) gets stuck at:

                                        Server Version: 76
                                        TWS Time at connection:20200811 13:45:59 British Summer Time
                                        ***** DATA NOTIF: DELAYED

                                        I’m specifically trying to trade gold. I’ve subscribed to the all the London Metal Exchange data and historical=True data works fine. Am I missing something when it comes to live gold data?

                                        I tried rtbar=True but that gives DATA NOTIF UNKNOWN 420 (Pacing Violation)

                                        import backtrader as bt
                                        import datetime
                                        
                                        class St(bt.Strategy):
                                            def logdata(self):
                                                txt = []
                                                txt.append('{}'.format(len(self)))
                                                   
                                                txt.append('{}'.format(
                                                    self.data.datetime.datetime(0).isoformat())
                                                )
                                                txt.append('{:.2f}'.format(self.data.open[0]))
                                                txt.append('{:.2f}'.format(self.data.high[0]))
                                                txt.append('{:.2f}'.format(self.data.low[0]))
                                                txt.append('{:.2f}'.format(self.data.close[0]))
                                                txt.append('{:.2f}'.format(self.data.volume[0]))
                                                print(','.join(txt))
                                        
                                            data_live = False
                                        
                                            def notify_data(self, data, status, *args, **kwargs):
                                                print('*' * 5, 'DATA NOTIF:', data._getstatusname(status), *args)
                                                if status == data.LIVE:
                                                    self.data_live = True
                                        
                                            def notify_order(self, order):
                                                if order.status == order.Completed:
                                                    buysell = 'BUY ' if order.isbuy() else 'SELL'
                                                    txt = '{} {}@{}'.format(buysell, order.executed.size,
                                                                            order.executed.price)
                                                    print(txt)
                                                
                                            bought = 0
                                            sold = 0
                                        
                                            def next(self):
                                                self.logdata()
                                                if not self.data_live:
                                                    return    
                                            
                                                if not self.bought:
                                                    self.bought = len(self)  # keep entry bar
                                                    self.buy()
                                                elif not self.sold:
                                                    if len(self) == (self.bought + 3):
                                                        self.sell()
                                                      
                                        def run(args=None):
                                        
                                            cerebro = bt.Cerebro(stdstats=False)
                                        
                                            store = bt.stores.IBStore(host="127.0.0.1", port=7496, clientId=35)
                                            cerebro.broker = store.getbroker()
                                        
                                            stockkwargs = dict(
                                                timeframe=bt.TimeFrame.Ticks,
                                                what='BID',
                                                rtbar=False,  
                                                historical=True,  
                                                qcheck=0.5,  
                                                latethrough=False,
                                                tradename=None
                                            )
                                            
                                            data0 = store.getdata(dataname='XAUUSD-CMDTY-SMART-USD', **stockkwargs)
                                            cerebro.resampledata(data0, timeframe=bt.TimeFrame.Minutes, compression=3)
                                            
                                            cerebro.addstrategy(St)    
                                            cerebro.run()
                                            
                                        if __name__ == '__main__':
                                            run()
                                        

                                        Can anyone shine a light on this?

                                        T 1 Reply Last reply Reply Quote 0
                                        • T
                                          tejatap824 @joe trs last edited by

                                          @joe-trs Are you trading on a live, funded account?

                                          J 1 Reply Last reply Reply Quote 0
                                          • J
                                            joe trs @tejatap824 last edited by

                                            @tejatap824 Account is funded but trying to get paper trades going first. Looking to trade live once live data starts coming through

                                            1 Reply Last reply Reply Quote 0
                                            • 1
                                            • 2
                                            • 1 / 2
                                            • First post
                                              Last post
                                            Copyright © 2016, 2017, 2018, 2019, 2020, 2021 NodeBB Forums | Contributors