For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

IBtest.py works with IB demo account but not with paper trading account.



  • When I run ibtest.py with an IB demo account I'm able to receive minute data for both stocks and forex with no problem. However, when I connect to a paper trading account that I have received from IB, I am not able to receive the stock minute data. I have contacted IB and they have given me the right to receive the stock data.

    Is this due to setting the account information in the code? Is the default value for the account in ibtest.py is IB standard demo account?

    Thanks for this awesome package.



  • @rastegarr

    Here is the message I get with 5 tickers:

    Server Version: 76
     TWS Time at connection:20161217 17:57:55 CST
    
    Strategy Created
    
    Timezone from ContractDetails: EST5EDT
     Datetime, Open, High, Low, Close, Volume, OpenInterest
     Timezone from ContractDetails: EST5EDT
     Datetime, Open, High, Low, Close, Volume, OpenInterest
     Timezone from ContractDetails: EST5EDT
     Datetime, Open, High, Low, Close, Volume, OpenInterest
     Timezone from ContractDetails: EST5EDT
     Datetime, Open, High, Low, Close, Volume, OpenInterest
     Timezone from ContractDetails: EST5EDT
     Datetime, Open, High, Low, Close, Volume, OpenInterest
     ***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm conne
     ction is OK:hfarm>
     ***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm conne
     ction is OK:eufarm>
     ***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm conne
     ction is OK:jfarm>
     ***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm conne
     ction is OK:usfuture>
     ***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm conne
     ction is OK:cashfarm>
     ***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm conne
     ction is OK:usfarm.us>
     ***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm conne
     ction is OK:usfarm>
     ***** STORE NOTIF: <error id=-1, errorCode=2106, errorMsg=HMDS data farm connect
     ion is OK:ushmds.us>
     ***** STORE NOTIF: <error id=-1, errorCode=2106, errorMsg=HMDS data farm connect
     ion is OK:ilhmds>
     ***** STORE NOTIF: <error id=-1, errorCode=2106, errorMsg=HMDS data farm connect
     ion is OK:euhmds>
     ***** STORE NOTIF: <error id=-1, errorCode=2106, errorMsg=HMDS data farm connect
     ion is OK:fundfarm>
     ***** STORE NOTIF: <error id=-1, errorCode=2106, errorMsg=HMDS data farm connect
     ion is OK:ushmds>
    

  • administrators

    You probaby have a modified version of the original ibtest.py, because the sample works a most with 2 tickers.

    Without knowing that and how the sample is run, the best educated guess is that the same as inthe following post happens (IB offers no backfilling for ticks)



  • @backtrader

    Here is the code minus the arguments parsing function. As I said I can receive the data with IB demo account not with my paper trading. I even tested the original ibtest.py and have the same issue:

    class TestStrategy(bt.Strategy):
    
        params = dict(
            # Standard MACD Parameters
            macd1 = 12,
            macd2 = 26,
            macdsig = 9,
            atrperiod = 14,  # ATR Period (standard)
            atrdist = 3.0,   # ATR distance for stop price
            smaperiod = 28,  # SMA Period (pretty standard)
            dirperiod = 10,  # Lookback period to consider SMA trend direction
            exectype=bt.Order.Market,
            max_investment_per_trade = 500000,
            max_loss_per_trade = 2000
        )
    
        def __init__(self):
            self.counter = 0
            self.data_live = True
            self.orders = dict((datax._dataname, None) for datax in self.datas) 
            self.macd_s = [None] * len(self.datas)
            self.mcross_s = [None] * len(self.datas)
            self.atr_s = [None] * len(self.datas)
            self.sma_s = [None] * len(self.datas)
            self.smadir_s = [None] * len(self.datas)
            for indx in range(0, len(self.datas)):
                datax = self.datas[indx]
                self.macd_s[indx] = bt.indicators.MACD(datax,
                                           period_me1=self.p.macd1,
                                           period_me2=self.p.macd2,
                                           period_signal=self.p.macdsig)
                # Cross of macd.macd and macd.signal
                self.mcross_s[indx] = bt.indicators.CrossOver(self.macd_s[indx].macd, self.macd_s[indx].signal)
                # To set the stop price
                self.atr_s[indx] = bt.indicators.ATR(datax, period=self.p.atrperiod)
                # Control market trend
                self.sma_s[indx] = bt.indicators.SMA(datax, period=self.p.smaperiod)
                self.smadir_s[indx] = (self.sma_s[indx] - self.sma_s[indx](-self.p.dirperiod))
            print('--------------------------------------------------')
            print('Strategy Created')
            print('--------------------------------------------------')
    
        def notify_data(self, data, status, *args, **kwargs):
            print('*' * 5, 'DATA NOTIF:', data._getstatusname(status), data._dataname)
            self.data_live =  ((status == data.LIVE) and self.data_live)
    
        def notify_store(self, msg, *args, **kwargs):
            print('*' * 5, 'STORE NOTIF:', msg)
    
        def notify_order(self, order):
            if order.status in [order.Completed, order.Cancelled, order.Rejected]:
                self.orders[order.data._dataname] = None
            print('-' * 50, 'ORDER BEGIN', datetime.datetime.now())
            print(order)
            print('-' * 50, 'ORDER END')
            
        def notify_trade(self, trade):
            print('-' * 50, 'TRADE BEGIN', datetime.datetime.now())
            print(trade)
            print('-' * 50, 'TRADE END')
        def prenext(self):
            self.next(frompre=True)
        def next(self, frompre=False):
            '''
            for data in self.datas:
                if self.data.datetime.time() < datetime.time(10, 0):
                   # don't operate until the market has been running 30 minutes
                    return  #
            '''
            #if one symbol is not alive, then don't trade 
            if not self.data_live:
                return
     
            self.counter = self.counter + 1
            print(self.counter)
            for indx in range(0, len(self.datas)):
                datax = self.datas[indx]
                if len(datax):
                    txt = list()
                    txt.append(datax.p.dataname)
                    txt.append('%04d' % len(datax))
                    dtfmt = '%Y-%m-%dT%H:%M:%S.%f'
                    txt.append('{}'.format(datax.datetime[0]))
                    txt.append('%s' % datax.datetime.datetime(0).strftime(dtfmt))
                    txt.append('{}'.format(datax.open[0]))
                    txt.append('{}'.format(datax.high[0]))
                    txt.append('{}'.format(datax.low[0]))
                    txt.append('{}'.format(datax.close[0]))
                    txt.append('{}'.format(datax.volume[0]))
                    txt.append('{}'.format(datax.openinterest[0]))
                    print(', '.join(txt))
                    #if self.order:
                    #    return  # pending order execution
                    #The signals we will be using Intraday 1-minute graph MACD 12,29,9 and SIMPLE 28 DAY MOVING AVERGE (S28)  
                    positionx = self.getposition(datax)
                    
                    if datetime.time(15, 0) > datax.datetime.time() > datetime.time(10, 0):
                        #When we enter a short:                
                        #   MACD IS GREATER THAN .3               
                        #   MACD is = or smaller than zero. 
                        # We enter a long trade   
                        #  MACD IS SMALLER THAT -.5 (NEGATIVE .5)   
                        #   stock price is at or above the S28D (SIMPLE 28D MOVING AVERAGE) 
                        if self.macd_s[indx].macd[-1] > 0.3 or -.5 < self.macd_s[indx].macd[-1] < 0.0:
                            if positionx.size == 0:  # not in the market
                                self.order_s[datax.p.dataname] = self.sell(datax, size = floor(self.p.max_investment_per_trade / datax.close[-1]))
                                print("Submit a %s-size short order for %s" %(floor(self.p.max_investment_per_trade / datax.close[-1]), datax._dataname))
                            else:  
                                print("%s already in market" %datax._dataname)   
                        elif self.macd_s[indx].macd[-1] < -0.5 or self.sma_s[indx][-1] < datax.close[-1]:
                            if positionx.size == 0:  # not in the market
                                self.order_s[datax.p.dataname] = self.buy(datax, size = floor(self.p.max_investment_per_trade / datax.close[0]))
                                print("Submit a %s-size long order for %s" %(floor(self.p.max_investment_per_trade / datax.close[-1]), datax._dataname))
                            else:  # in the market
                                print("%s already in the market" %datax._dataname)
                        # When we liquidate a trade               
                        #  maximum loss is 2,000 (two thousand dollars)           
                        #  we liquidate a trade if the clock gets to 3.35 NY TIME
                        elif positionx.size < 0 and (datax.close[-1] - positionx.price) * positionx.size > self.p.max_loss_per_trade: # short? 
                            self.order_s[datax.p.dataname] = self.close(datax) 
                            print("Submit a closing order for %s positions." %(datax._dataname))
                        elif positionx.size > 0 and (positionx.price -  datax.close[-1]) * positionx.size > self.p.max_loss_per_trade: # short? 
                            self.order_s[datax.p.dataname] = self.close(datax) 
                            print("Submit a closing order for %s positions." %(datax._dataname))
                    elif datetime.time(15, 0) < datax.datetime.time():
                        if positionx.size:  
                            self.order_s[datax.p.dataname] = self.close(datax) 
                            print("Submit a closing order for %s positions." %(datax._dataname))
                                
        def start(self):
           for datax in self.datas:
               if datax.contractdetails is not None:
                    print('Timezone from ContractDetails: {}'.format(datax.contractdetails.m_timeZoneId))
               header = ['Datetime', 'Open', 'High', 'Low', 'Close', 'Volume', 'OpenInterest']
               print(', '.join(header))
    
    
    def runstrategy():
    
        args = parse_args()
        #symbols = ['EUR.USD-CASH-IDEALPRO', 'EUR.CAD-CASH-IDEALPRO', 'EUR.NZD-CASH-IDEALPRO', 'NZD.USD-CASH-IDEALPRO', 'USD.NOK-CASH-IDEALPRO']
        #symbols = ['AAPL']
        symbols = ['AAPL-STK-SMART-USD', 'OXY-STK-SMART-USD', 'NOV-STK-SMART-USD',
                   'CRC-STK-SMART-USD', 'XOM-STK-SMART-USD', 'SLB-STK-SMART-USD']
        # Create a cerebro
    
        cerebro = bt.Cerebro()
        storekwargs = dict(
            host=args.host, port=args.port,
            clientId=args.clientId, timeoffset=not args.no_timeoffset,
            reconnect=args.reconnect, timeout=args.timeout,
            notifyall=args.notifyall, _debug=args.debug
        )
        if args.usestore:
            ibstore = bt.stores.IBStore(**storekwargs)
        if args.broker:
            if args.usestore:
                broker = ibstore.getbroker()
            else:
                broker = bt.brokers.IBBroker(**storekwargs)
            cerebro.setbroker(broker)
        timeframe = bt.TimeFrame.TFrame(args.timeframe)
        # Manage data1 parameters
        
        if args.resample or args.replay:
            datatf = bt.TimeFrame.Ticks
            #datatf = bt.TimeFrame.Minutes
            datacomp = 1
        else:
            datatf = timeframe
            datacomp = args.compression
        fromdate = None
        if args.fromdate:
            dtformat = '%Y-%m-%d' + ('T%H:%M:%S' * ('T' in args.fromdate))
            fromdate = datetime.datetime.strptime(args.fromdate, dtformat)
        IBDataFactory = ibstore.getdata if args.usestore else bt.feeds.IBData
        datakwargs = dict(
            timeframe=datatf, compression=datacomp,
            historical=args.historical, fromdate=fromdate,
            rtbar=args.rtbar,
            qcheck=args.qcheck,
            what=args.what,
            backfill_start=not args.no_backfill_start,
            backfill=not args.no_backfill,
            latethrough=args.latethrough,
            tz=args.timezone
        )
        if not args.usestore and not args.broker:   # neither store nor broker
            datakwargs.update(storekwargs)  # pass the store args over the data
        rekwargs = dict(
                        timeframe=timeframe, compression=args.compression,
                        bar2edge=not args.no_bar2edge,
                        adjbartime=not args.no_adjbartime,
                        rightedge=not args.no_rightedge,
                        takelate=not args.no_takelate
        )
        for symbol in symbols:
            datax = IBDataFactory(dataname=symbol, **datakwargs)
            if args.replay:
                cerebro.replaydata(dataname=datax, **rekwargs)
            elif args.resample:
                cerebro.resampledata(dataname=datax, **rekwargs)
            else:
                cerebro.adddata(datax)
            
        valid = None
        if args.valid is not None:
            valid = datetime.timedelta(seconds=args.valid)
    
        cerebro.addstrategy(TestStrategy, exectype=bt.Order.ExecType(args.exectype))
    
        cerebro.run(exactbars=args.exactbars)
        if args.plot and args.exactbars < 1:  # plot if possible
            cerebro.plot()
    
    if __name__ == '__main__':
        runstrategy()
    

  • administrators

    If you have tried with the original ibtest.py, the most important part from the message is not having your code but as pointed out above:

    - *how is it being run*?
    

    In the link from above: https://community.backtrader.com/topic/10/ib-example-connection-delayed-on-forex-data-222/4 the problem was:

    • The default resolution if run with no parameters is TimeFrame.Ticks and Interactive Brokers offers no backfilling for such resolution. Which offers 2 alternatives:
    1. Running with an Interactive Brokers supported timeframe:
    ./ibtest.py --data0 EUR.USD-CASH-IDEALPRO --resample --timeframe Seconds --compression 1
    

    or

    1. Telling the sample to avoid backfilling if the resolution is not supported:
    ./ibtest.py --data0 EUR.USD-CASH-IDEALPRO --no-backfill_start --no-backfill
    

    None of those has a specific ticker, so you may want to quote how you are actually running the sample ibtest.py

    Being that a paper trading account there are several things to consider according to the documentation from IB which is sometimes conflicting (probably due to some old pages from the portal still being reachable), but being the most important apparently:

    • The data can only be used from one account: either the live account or the paper trading account

    Apparently some other restrictions are in place, but what is actually true or not may be better answered by Interactive Brokers itself.

    In any case the sample was tested against a demo account and a real trading account. The code here doesn't know anything about data permissions from Interactive Brokers. It asks the API to deliver data. If the data comes it will be pumped to your code, if it doesn't ...



  • @backtrader

    I ran it like this:

    ./ibtest.py --data0 EUR.USD-CASH-IDEALPRO --resample --timeframe Minutes --compression 1
    

    I have even tried

    ./ibtest.py --data0 EUR.USD-CASH-IDEALPRO --no-backfill_start --no-backfill
    

    None of them work with paper trading account. It works with demo account though!


  • administrators

    In that case it looks like a market data permission issue. The platform doesn't know if the account is a demo, paper trading or real. The connection happens to a TCP/IP destination.

    You may try the samples in the package Ibpy to double check.

    ibtest can also be run with --debug to print all TWS messages but it won't likely help



  • @backtrader

    I run this IbPy example and it seems I can receive the tick data from my paper trading account using IbPy library. So that tells me it shouldn't be a data subscription issue:

    from ib.ext.Contract import Contract
    from ib.opt import ibConnection, message
    from time import sleep
    
    # print all messages from TWS
    def watcher(msg):
        print (msg)
    
    # show Bid and Ask quotes
    def my_BidAsk(msg):
        if msg.field == 1:
            print ('%s:%s: bid: %s' % (contractTuple[0],
                       contractTuple[6], msg.price))
        elif msg.field == 2:
            print ('%s:%s: ask: %s' % (contractTuple[0], contractTuple[6], msg.price))
    
    def makeStkContract(contractTuple):
        newContract = Contract()
        newContract.m_symbol = contractTuple[0]
        newContract.m_secType = contractTuple[1]
        newContract.m_exchange = contractTuple[2]
        newContract.m_currency = contractTuple[3]
        newContract.m_expiry = contractTuple[4]
        newContract.m_strike = contractTuple[5]
        newContract.m_right = contractTuple[6]
        print ('Contract Values:%s,%s,%s,%s,%s,%s,%s:' % contractTuple)
        return newContract
    
    if __name__ == '__main__':
        con = ibConnection(port = 4001, clientId = 123)
        con.registerAll(watcher)
        showBidAskOnly = False  # set False to see the raw messages
        if showBidAskOnly:
            con.unregister(watcher, message.tickSize, message.tickPrice,
                           message.tickString, message.tickOptionComputation)
            con.register(my_BidAsk, message.tickPrice)
        con.connect()
        sleep(1)
        tickId = 59
        
        # Note: Option quotes will give an error if they aren't shown in TWS
        contractTuple = ('AAPL', 'STK', 'SMART', 'USD', '', 0.0, '')
        #contractTuple = ('QQQQ', 'OPT', 'SMART', 'USD', '20070921', 47.0, 'CALL')
        #contractTuple = ('ES', 'FUT', 'GLOBEX', 'USD', '200709', 0.0, '')
        #contractTuple = ('ES', 'FOP', 'GLOBEX', 'USD', '20070920', 1460.0, 'CALL')
        #contractTuple = ('EUR', 'CASH', 'IDEALPRO', 'USD', '', 0.0, '')
        for i in range(20):
            stkContract = makeStkContract(contractTuple)
            print ('* * * * REQUESTING MARKET DATA * * * *')
            con.reqMktData(tickId, stkContract, '', False)
            sleep(1)
            print ('* * * * CANCELING MARKET DATA * * * *')
            con.cancelMktData(tickId)
        sleep(1)
        con.disconnect()
        sleep(1)
    

    Something that I observed is that if move makeStkContract and cancelMktData, out of loop I get bunch of errors (code 322).


  • administrators

    There isn't enough information to even attempt anything. For example: the only provided output is from the modified source, without having the actual command line execution

    As such one can only try to point the low hanging fruits. One would be:

    • Are you pointing to the right port? (usually 7496 for production and 7497 for paper trading)

    The default port used by the sample and the IB ecosystem is 7496.

    You quote above the following command:

    ./ibtest.py --data0 EUR.USD-CASH-IDEALPRO --resample --timeframe Minutes --compression 1
    

    and this is theoretically working with the demo. But the demo also runs (default, can be of course be changed) on port 7497.

    Cash Products like EUR.USD-CASH-IDEALPRO are allowed (afaik) to all IB customers (as long as your account is funded I think). Whether demo, paper trading or real account, that product is available for everyone. And the code (which relies on IBPy for interfacing to TWS) doesn't know what it is connecting to. The account type is controlled by the TWS (or IbGateway) instance you connect to.