Backtrader Community

    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

    IBtest.py works with IB demo account but not with paper trading account.

    General Discussion
    7
    10
    4496
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • R
      rastegarr last edited by

      When I run ibtest.py with an IB demo account I'm able to receive minute data for both stocks and forex with no problem. However, when I connect to a paper trading account that I have received from IB, I am not able to receive the stock minute data. I have contacted IB and they have given me the right to receive the stock data.

      Is this due to setting the account information in the code? Is the default value for the account in ibtest.py is IB standard demo account?

      Thanks for this awesome package.

      R 1 Reply Last reply Reply Quote 0
      • R
        rastegarr @rastegarr last edited by backtrader

        @rastegarr

        Here is the message I get with 5 tickers:

        Server Version: 76
         TWS Time at connection:20161217 17:57:55 CST
        
        Strategy Created
        
        Timezone from ContractDetails: EST5EDT
         Datetime, Open, High, Low, Close, Volume, OpenInterest
         Timezone from ContractDetails: EST5EDT
         Datetime, Open, High, Low, Close, Volume, OpenInterest
         Timezone from ContractDetails: EST5EDT
         Datetime, Open, High, Low, Close, Volume, OpenInterest
         Timezone from ContractDetails: EST5EDT
         Datetime, Open, High, Low, Close, Volume, OpenInterest
         Timezone from ContractDetails: EST5EDT
         Datetime, Open, High, Low, Close, Volume, OpenInterest
         ***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm conne
         ction is OK:hfarm>
         ***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm conne
         ction is OK:eufarm>
         ***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm conne
         ction is OK:jfarm>
         ***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm conne
         ction is OK:usfuture>
         ***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm conne
         ction is OK:cashfarm>
         ***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm conne
         ction is OK:usfarm.us>
         ***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm conne
         ction is OK:usfarm>
         ***** STORE NOTIF: <error id=-1, errorCode=2106, errorMsg=HMDS data farm connect
         ion is OK:ushmds.us>
         ***** STORE NOTIF: <error id=-1, errorCode=2106, errorMsg=HMDS data farm connect
         ion is OK:ilhmds>
         ***** STORE NOTIF: <error id=-1, errorCode=2106, errorMsg=HMDS data farm connect
         ion is OK:euhmds>
         ***** STORE NOTIF: <error id=-1, errorCode=2106, errorMsg=HMDS data farm connect
         ion is OK:fundfarm>
         ***** STORE NOTIF: <error id=-1, errorCode=2106, errorMsg=HMDS data farm connect
         ion is OK:ushmds>
        
        1 Reply Last reply Reply Quote 0
        • B
          backtrader administrators last edited by

          You probaby have a modified version of the original ibtest.py, because the sample works a most with 2 tickers.

          Without knowing that and how the sample is run, the best educated guess is that the same as inthe following post happens (IB offers no backfilling for ticks)

          • https://community.backtrader.com/topic/10/ib-example-connection-delayed-on-forex-data-222
          R 1 Reply Last reply Reply Quote 0
          • R
            rastegarr @backtrader last edited by backtrader

            @backtrader

            Here is the code minus the arguments parsing function. As I said I can receive the data with IB demo account not with my paper trading. I even tested the original ibtest.py and have the same issue:

            class TestStrategy(bt.Strategy):
            
                params = dict(
                    # Standard MACD Parameters
                    macd1 = 12,
                    macd2 = 26,
                    macdsig = 9,
                    atrperiod = 14,  # ATR Period (standard)
                    atrdist = 3.0,   # ATR distance for stop price
                    smaperiod = 28,  # SMA Period (pretty standard)
                    dirperiod = 10,  # Lookback period to consider SMA trend direction
                    exectype=bt.Order.Market,
                    max_investment_per_trade = 500000,
                    max_loss_per_trade = 2000
                )
            
                def __init__(self):
                    self.counter = 0
                    self.data_live = True
                    self.orders = dict((datax._dataname, None) for datax in self.datas) 
                    self.macd_s = [None] * len(self.datas)
                    self.mcross_s = [None] * len(self.datas)
                    self.atr_s = [None] * len(self.datas)
                    self.sma_s = [None] * len(self.datas)
                    self.smadir_s = [None] * len(self.datas)
                    for indx in range(0, len(self.datas)):
                        datax = self.datas[indx]
                        self.macd_s[indx] = bt.indicators.MACD(datax,
                                                   period_me1=self.p.macd1,
                                                   period_me2=self.p.macd2,
                                                   period_signal=self.p.macdsig)
                        # Cross of macd.macd and macd.signal
                        self.mcross_s[indx] = bt.indicators.CrossOver(self.macd_s[indx].macd, self.macd_s[indx].signal)
                        # To set the stop price
                        self.atr_s[indx] = bt.indicators.ATR(datax, period=self.p.atrperiod)
                        # Control market trend
                        self.sma_s[indx] = bt.indicators.SMA(datax, period=self.p.smaperiod)
                        self.smadir_s[indx] = (self.sma_s[indx] - self.sma_s[indx](-self.p.dirperiod))
                    print('--------------------------------------------------')
                    print('Strategy Created')
                    print('--------------------------------------------------')
            
                def notify_data(self, data, status, *args, **kwargs):
                    print('*' * 5, 'DATA NOTIF:', data._getstatusname(status), data._dataname)
                    self.data_live =  ((status == data.LIVE) and self.data_live)
            
                def notify_store(self, msg, *args, **kwargs):
                    print('*' * 5, 'STORE NOTIF:', msg)
            
                def notify_order(self, order):
                    if order.status in [order.Completed, order.Cancelled, order.Rejected]:
                        self.orders[order.data._dataname] = None
                    print('-' * 50, 'ORDER BEGIN', datetime.datetime.now())
                    print(order)
                    print('-' * 50, 'ORDER END')
                    
                def notify_trade(self, trade):
                    print('-' * 50, 'TRADE BEGIN', datetime.datetime.now())
                    print(trade)
                    print('-' * 50, 'TRADE END')
                def prenext(self):
                    self.next(frompre=True)
                def next(self, frompre=False):
                    '''
                    for data in self.datas:
                        if self.data.datetime.time() < datetime.time(10, 0):
                           # don't operate until the market has been running 30 minutes
                            return  #
                    '''
                    #if one symbol is not alive, then don't trade 
                    if not self.data_live:
                        return
             
                    self.counter = self.counter + 1
                    print(self.counter)
                    for indx in range(0, len(self.datas)):
                        datax = self.datas[indx]
                        if len(datax):
                            txt = list()
                            txt.append(datax.p.dataname)
                            txt.append('%04d' % len(datax))
                            dtfmt = '%Y-%m-%dT%H:%M:%S.%f'
                            txt.append('{}'.format(datax.datetime[0]))
                            txt.append('%s' % datax.datetime.datetime(0).strftime(dtfmt))
                            txt.append('{}'.format(datax.open[0]))
                            txt.append('{}'.format(datax.high[0]))
                            txt.append('{}'.format(datax.low[0]))
                            txt.append('{}'.format(datax.close[0]))
                            txt.append('{}'.format(datax.volume[0]))
                            txt.append('{}'.format(datax.openinterest[0]))
                            print(', '.join(txt))
                            #if self.order:
                            #    return  # pending order execution
                            #The signals we will be using Intraday 1-minute graph MACD 12,29,9 and SIMPLE 28 DAY MOVING AVERGE (S28)  
                            positionx = self.getposition(datax)
                            
                            if datetime.time(15, 0) > datax.datetime.time() > datetime.time(10, 0):
                                #When we enter a short:                
                                #   MACD IS GREATER THAN .3               
                                #   MACD is = or smaller than zero. 
                                # We enter a long trade   
                                #  MACD IS SMALLER THAT -.5 (NEGATIVE .5)   
                                #   stock price is at or above the S28D (SIMPLE 28D MOVING AVERAGE) 
                                if self.macd_s[indx].macd[-1] > 0.3 or -.5 < self.macd_s[indx].macd[-1] < 0.0:
                                    if positionx.size == 0:  # not in the market
                                        self.order_s[datax.p.dataname] = self.sell(datax, size = floor(self.p.max_investment_per_trade / datax.close[-1]))
                                        print("Submit a %s-size short order for %s" %(floor(self.p.max_investment_per_trade / datax.close[-1]), datax._dataname))
                                    else:  
                                        print("%s already in market" %datax._dataname)   
                                elif self.macd_s[indx].macd[-1] < -0.5 or self.sma_s[indx][-1] < datax.close[-1]:
                                    if positionx.size == 0:  # not in the market
                                        self.order_s[datax.p.dataname] = self.buy(datax, size = floor(self.p.max_investment_per_trade / datax.close[0]))
                                        print("Submit a %s-size long order for %s" %(floor(self.p.max_investment_per_trade / datax.close[-1]), datax._dataname))
                                    else:  # in the market
                                        print("%s already in the market" %datax._dataname)
                                # When we liquidate a trade               
                                #  maximum loss is 2,000 (two thousand dollars)           
                                #  we liquidate a trade if the clock gets to 3.35 NY TIME
                                elif positionx.size < 0 and (datax.close[-1] - positionx.price) * positionx.size > self.p.max_loss_per_trade: # short? 
                                    self.order_s[datax.p.dataname] = self.close(datax) 
                                    print("Submit a closing order for %s positions." %(datax._dataname))
                                elif positionx.size > 0 and (positionx.price -  datax.close[-1]) * positionx.size > self.p.max_loss_per_trade: # short? 
                                    self.order_s[datax.p.dataname] = self.close(datax) 
                                    print("Submit a closing order for %s positions." %(datax._dataname))
                            elif datetime.time(15, 0) < datax.datetime.time():
                                if positionx.size:  
                                    self.order_s[datax.p.dataname] = self.close(datax) 
                                    print("Submit a closing order for %s positions." %(datax._dataname))
                                        
                def start(self):
                   for datax in self.datas:
                       if datax.contractdetails is not None:
                            print('Timezone from ContractDetails: {}'.format(datax.contractdetails.m_timeZoneId))
                       header = ['Datetime', 'Open', 'High', 'Low', 'Close', 'Volume', 'OpenInterest']
                       print(', '.join(header))
            
            
            def runstrategy():
            
                args = parse_args()
                #symbols = ['EUR.USD-CASH-IDEALPRO', 'EUR.CAD-CASH-IDEALPRO', 'EUR.NZD-CASH-IDEALPRO', 'NZD.USD-CASH-IDEALPRO', 'USD.NOK-CASH-IDEALPRO']
                #symbols = ['AAPL']
                symbols = ['AAPL-STK-SMART-USD', 'OXY-STK-SMART-USD', 'NOV-STK-SMART-USD',
                           'CRC-STK-SMART-USD', 'XOM-STK-SMART-USD', 'SLB-STK-SMART-USD']
                # Create a cerebro
            
                cerebro = bt.Cerebro()
                storekwargs = dict(
                    host=args.host, port=args.port,
                    clientId=args.clientId, timeoffset=not args.no_timeoffset,
                    reconnect=args.reconnect, timeout=args.timeout,
                    notifyall=args.notifyall, _debug=args.debug
                )
                if args.usestore:
                    ibstore = bt.stores.IBStore(**storekwargs)
                if args.broker:
                    if args.usestore:
                        broker = ibstore.getbroker()
                    else:
                        broker = bt.brokers.IBBroker(**storekwargs)
                    cerebro.setbroker(broker)
                timeframe = bt.TimeFrame.TFrame(args.timeframe)
                # Manage data1 parameters
                
                if args.resample or args.replay:
                    datatf = bt.TimeFrame.Ticks
                    #datatf = bt.TimeFrame.Minutes
                    datacomp = 1
                else:
                    datatf = timeframe
                    datacomp = args.compression
                fromdate = None
                if args.fromdate:
                    dtformat = '%Y-%m-%d' + ('T%H:%M:%S' * ('T' in args.fromdate))
                    fromdate = datetime.datetime.strptime(args.fromdate, dtformat)
                IBDataFactory = ibstore.getdata if args.usestore else bt.feeds.IBData
                datakwargs = dict(
                    timeframe=datatf, compression=datacomp,
                    historical=args.historical, fromdate=fromdate,
                    rtbar=args.rtbar,
                    qcheck=args.qcheck,
                    what=args.what,
                    backfill_start=not args.no_backfill_start,
                    backfill=not args.no_backfill,
                    latethrough=args.latethrough,
                    tz=args.timezone
                )
                if not args.usestore and not args.broker:   # neither store nor broker
                    datakwargs.update(storekwargs)  # pass the store args over the data
                rekwargs = dict(
                                timeframe=timeframe, compression=args.compression,
                                bar2edge=not args.no_bar2edge,
                                adjbartime=not args.no_adjbartime,
                                rightedge=not args.no_rightedge,
                                takelate=not args.no_takelate
                )
                for symbol in symbols:
                    datax = IBDataFactory(dataname=symbol, **datakwargs)
                    if args.replay:
                        cerebro.replaydata(dataname=datax, **rekwargs)
                    elif args.resample:
                        cerebro.resampledata(dataname=datax, **rekwargs)
                    else:
                        cerebro.adddata(datax)
                    
                valid = None
                if args.valid is not None:
                    valid = datetime.timedelta(seconds=args.valid)
            
                cerebro.addstrategy(TestStrategy, exectype=bt.Order.ExecType(args.exectype))
            
                cerebro.run(exactbars=args.exactbars)
                if args.plot and args.exactbars < 1:  # plot if possible
                    cerebro.plot()
            
            if __name__ == '__main__':
                runstrategy()
            
            1 Reply Last reply Reply Quote 0
            • B
              backtrader administrators last edited by

              If you have tried with the original ibtest.py, the most important part from the message is not having your code but as pointed out above:

              - *how is it being run*?
              

              In the link from above: https://community.backtrader.com/topic/10/ib-example-connection-delayed-on-forex-data-222/4 the problem was:

              • The default resolution if run with no parameters is TimeFrame.Ticks and Interactive Brokers offers no backfilling for such resolution. Which offers 2 alternatives:
              1. Running with an Interactive Brokers supported timeframe:
              ./ibtest.py --data0 EUR.USD-CASH-IDEALPRO --resample --timeframe Seconds --compression 1
              

              or

              1. Telling the sample to avoid backfilling if the resolution is not supported:
              ./ibtest.py --data0 EUR.USD-CASH-IDEALPRO --no-backfill_start --no-backfill
              

              None of those has a specific ticker, so you may want to quote how you are actually running the sample ibtest.py

              Being that a paper trading account there are several things to consider according to the documentation from IB which is sometimes conflicting (probably due to some old pages from the portal still being reachable), but being the most important apparently:

              • The data can only be used from one account: either the live account or the paper trading account

              Apparently some other restrictions are in place, but what is actually true or not may be better answered by Interactive Brokers itself.

              In any case the sample was tested against a demo account and a real trading account. The code here doesn't know anything about data permissions from Interactive Brokers. It asks the API to deliver data. If the data comes it will be pumped to your code, if it doesn't ...

              R 1 Reply Last reply Reply Quote 0
              • R
                rastegarr @backtrader last edited by backtrader

                @backtrader

                I ran it like this:

                ./ibtest.py --data0 EUR.USD-CASH-IDEALPRO --resample --timeframe Minutes --compression 1
                

                I have even tried

                ./ibtest.py --data0 EUR.USD-CASH-IDEALPRO --no-backfill_start --no-backfill
                

                None of them work with paper trading account. It works with demo account though!

                1 Reply Last reply Reply Quote 0
                • B
                  backtrader administrators last edited by backtrader

                  In that case it looks like a market data permission issue. The platform doesn't know if the account is a demo, paper trading or real. The connection happens to a TCP/IP destination.

                  You may try the samples in the package Ibpy to double check.

                  ibtest can also be run with --debug to print all TWS messages but it won't likely help

                  R 1 Reply Last reply Reply Quote 0
                  • R
                    rastegarr @backtrader last edited by

                    @backtrader

                    I run this IbPy example and it seems I can receive the tick data from my paper trading account using IbPy library. So that tells me it shouldn't be a data subscription issue:

                    from ib.ext.Contract import Contract
                    from ib.opt import ibConnection, message
                    from time import sleep
                    
                    # print all messages from TWS
                    def watcher(msg):
                        print (msg)
                    
                    # show Bid and Ask quotes
                    def my_BidAsk(msg):
                        if msg.field == 1:
                            print ('%s:%s: bid: %s' % (contractTuple[0],
                                       contractTuple[6], msg.price))
                        elif msg.field == 2:
                            print ('%s:%s: ask: %s' % (contractTuple[0], contractTuple[6], msg.price))
                    
                    def makeStkContract(contractTuple):
                        newContract = Contract()
                        newContract.m_symbol = contractTuple[0]
                        newContract.m_secType = contractTuple[1]
                        newContract.m_exchange = contractTuple[2]
                        newContract.m_currency = contractTuple[3]
                        newContract.m_expiry = contractTuple[4]
                        newContract.m_strike = contractTuple[5]
                        newContract.m_right = contractTuple[6]
                        print ('Contract Values:%s,%s,%s,%s,%s,%s,%s:' % contractTuple)
                        return newContract
                    
                    if __name__ == '__main__':
                        con = ibConnection(port = 4001, clientId = 123)
                        con.registerAll(watcher)
                        showBidAskOnly = False  # set False to see the raw messages
                        if showBidAskOnly:
                            con.unregister(watcher, message.tickSize, message.tickPrice,
                                           message.tickString, message.tickOptionComputation)
                            con.register(my_BidAsk, message.tickPrice)
                        con.connect()
                        sleep(1)
                        tickId = 59
                        
                        # Note: Option quotes will give an error if they aren't shown in TWS
                        contractTuple = ('AAPL', 'STK', 'SMART', 'USD', '', 0.0, '')
                        #contractTuple = ('QQQQ', 'OPT', 'SMART', 'USD', '20070921', 47.0, 'CALL')
                        #contractTuple = ('ES', 'FUT', 'GLOBEX', 'USD', '200709', 0.0, '')
                        #contractTuple = ('ES', 'FOP', 'GLOBEX', 'USD', '20070920', 1460.0, 'CALL')
                        #contractTuple = ('EUR', 'CASH', 'IDEALPRO', 'USD', '', 0.0, '')
                        for i in range(20):
                            stkContract = makeStkContract(contractTuple)
                            print ('* * * * REQUESTING MARKET DATA * * * *')
                            con.reqMktData(tickId, stkContract, '', False)
                            sleep(1)
                            print ('* * * * CANCELING MARKET DATA * * * *')
                            con.cancelMktData(tickId)
                        sleep(1)
                        con.disconnect()
                        sleep(1)
                    

                    Something that I observed is that if move makeStkContract and cancelMktData, out of loop I get bunch of errors (code 322).

                    1 Reply Last reply Reply Quote 0
                    • B
                      backtrader administrators last edited by backtrader

                      There isn't enough information to even attempt anything. For example: the only provided output is from the modified source, without having the actual command line execution

                      As such one can only try to point the low hanging fruits. One would be:

                      • Are you pointing to the right port? (usually 7496 for production and 7497 for paper trading)

                      The default port used by the sample and the IB ecosystem is 7496.

                      You quote above the following command:

                      ./ibtest.py --data0 EUR.USD-CASH-IDEALPRO --resample --timeframe Minutes --compression 1
                      

                      and this is theoretically working with the demo. But the demo also runs (default, can be of course be changed) on port 7497.

                      Cash Products like EUR.USD-CASH-IDEALPRO are allowed (afaik) to all IB customers (as long as your account is funded I think). Whether demo, paper trading or real account, that product is available for everyone. And the code (which relies on IBPy for interfacing to TWS) doesn't know what it is connecting to. The account type is controlled by the TWS (or IbGateway) instance you connect to.

                      1 Reply Last reply Reply Quote 0
                      • Aleem SEO
                        Aleem SEO last edited by

                        This post is deleted!
                        1 Reply Last reply Reply Quote 0
                        • 1 / 1
                        • First post
                          Last post
                        Copyright © 2016, 2017, 2018, 2019, 2020, 2021 NodeBB Forums | Contributors