IBtest.py works with IB demo account but not with paper trading account.

[rastegarr]

When I run ibtest.py with an IB demo account I'm able to receive minute data for both stocks and forex with no problem. However, when I connect to a paper trading account that I have received from IB, I am not able to receive the stock minute data. I have contacted IB and they have given me the right to receive the stock data.

Is this due to setting the account information in the code? Is the default value for the account in ibtest.py is IB standard demo account?

Thanks for this awesome package.

[rastegarr]

@rastegarr

Here is the message I get with 5 tickers:

Server Version: 76
 TWS Time at connection:20161217 17:57:55 CST

Strategy Created

Timezone from ContractDetails: EST5EDT
 Datetime, Open, High, Low, Close, Volume, OpenInterest
 Timezone from ContractDetails: EST5EDT
 Datetime, Open, High, Low, Close, Volume, OpenInterest
 Timezone from ContractDetails: EST5EDT
 Datetime, Open, High, Low, Close, Volume, OpenInterest
 Timezone from ContractDetails: EST5EDT
 Datetime, Open, High, Low, Close, Volume, OpenInterest
 Timezone from ContractDetails: EST5EDT
 Datetime, Open, High, Low, Close, Volume, OpenInterest
 ***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm conne
 ction is OK:hfarm>
 ***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm conne
 ction is OK:eufarm>
 ***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm conne
 ction is OK:jfarm>
 ***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm conne
 ction is OK:usfuture>
 ***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm conne
 ction is OK:cashfarm>
 ***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm conne
 ction is OK:usfarm.us>
 ***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm conne
 ction is OK:usfarm>
 ***** STORE NOTIF: <error id=-1, errorCode=2106, errorMsg=HMDS data farm connect
 ion is OK:ushmds.us>
 ***** STORE NOTIF: <error id=-1, errorCode=2106, errorMsg=HMDS data farm connect
 ion is OK:ilhmds>
 ***** STORE NOTIF: <error id=-1, errorCode=2106, errorMsg=HMDS data farm connect
 ion is OK:euhmds>
 ***** STORE NOTIF: <error id=-1, errorCode=2106, errorMsg=HMDS data farm connect
 ion is OK:fundfarm>
 ***** STORE NOTIF: <error id=-1, errorCode=2106, errorMsg=HMDS data farm connect
 ion is OK:ushmds>

[backtrader]

You probaby have a modified version of the original ibtest.py, because the sample works a most with 2 tickers.

Without knowing that and how the sample is run, the best educated guess is that the same as inthe following post happens (IB offers no backfilling for ticks)

• https://community.backtrader.com/topic/10/ib-example-connection-delayed-on-forex-data-222

[rastegarr]

@backtrader

Here is the code minus the arguments parsing function. As I said I can receive the data with IB demo account not with my paper trading. I even tested the original ibtest.py and have the same issue:

class TestStrategy(bt.Strategy):

    params = dict(
        # Standard MACD Parameters
        macd1 = 12,
        macd2 = 26,
        macdsig = 9,
        atrperiod = 14,  # ATR Period (standard)
        atrdist = 3.0,   # ATR distance for stop price
        smaperiod = 28,  # SMA Period (pretty standard)
        dirperiod = 10,  # Lookback period to consider SMA trend direction
        exectype=bt.Order.Market,
        max_investment_per_trade = 500000,
        max_loss_per_trade = 2000
    )

    def __init__(self):
        self.counter = 0
        self.data_live = True
        self.orders = dict((datax._dataname, None) for datax in self.datas) 
        self.macd_s = [None] * len(self.datas)
        self.mcross_s = [None] * len(self.datas)
        self.atr_s = [None] * len(self.datas)
        self.sma_s = [None] * len(self.datas)
        self.smadir_s = [None] * len(self.datas)
        for indx in range(0, len(self.datas)):
            datax = self.datas[indx]
            self.macd_s[indx] = bt.indicators.MACD(datax,
                                       period_me1=self.p.macd1,
                                       period_me2=self.p.macd2,
                                       period_signal=self.p.macdsig)
            # Cross of macd.macd and macd.signal
            self.mcross_s[indx] = bt.indicators.CrossOver(self.macd_s[indx].macd, self.macd_s[indx].signal)
            # To set the stop price
            self.atr_s[indx] = bt.indicators.ATR(datax, period=self.p.atrperiod)
            # Control market trend
            self.sma_s[indx] = bt.indicators.SMA(datax, period=self.p.smaperiod)
            self.smadir_s[indx] = (self.sma_s[indx] - self.sma_s[indx](-self.p.dirperiod))
        print('--------------------------------------------------')
        print('Strategy Created')
        print('--------------------------------------------------')

    def notify_data(self, data, status, *args, **kwargs):
        print('*' * 5, 'DATA NOTIF:', data._getstatusname(status), data._dataname)
        self.data_live =  ((status == data.LIVE) and self.data_live)

    def notify_store(self, msg, *args, **kwargs):
        print('*' * 5, 'STORE NOTIF:', msg)

    def notify_order(self, order):
        if order.status in [order.Completed, order.Cancelled, order.Rejected]:
            self.orders[order.data._dataname] = None
        print('-' * 50, 'ORDER BEGIN', datetime.datetime.now())
        print(order)
        print('-' * 50, 'ORDER END')
        
    def notify_trade(self, trade):
        print('-' * 50, 'TRADE BEGIN', datetime.datetime.now())
        print(trade)
        print('-' * 50, 'TRADE END')
    def prenext(self):
        self.next(frompre=True)
    def next(self, frompre=False):
        '''
        for data in self.datas:
            if self.data.datetime.time() < datetime.time(10, 0):
               # don't operate until the market has been running 30 minutes
                return  #
        '''
        #if one symbol is not alive, then don't trade 
        if not self.data_live:
            return
 
        self.counter = self.counter + 1
        print(self.counter)
        for indx in range(0, len(self.datas)):
            datax = self.datas[indx]
            if len(datax):
                txt = list()
                txt.append(datax.p.dataname)
                txt.append('%04d' % len(datax))
                dtfmt = '%Y-%m-%dT%H:%M:%S.%f'
                txt.append('{}'.format(datax.datetime[0]))
                txt.append('%s' % datax.datetime.datetime(0).strftime(dtfmt))
                txt.append('{}'.format(datax.open[0]))
                txt.append('{}'.format(datax.high[0]))
                txt.append('{}'.format(datax.low[0]))
                txt.append('{}'.format(datax.close[0]))
                txt.append('{}'.format(datax.volume[0]))
                txt.append('{}'.format(datax.openinterest[0]))
                print(', '.join(txt))
                #if self.order:
                #    return  # pending order execution
                #The signals we will be using Intraday 1-minute graph MACD 12,29,9 and SIMPLE 28 DAY MOVING AVERGE (S28)  
                positionx = self.getposition(datax)
                
                if datetime.time(15, 0) > datax.datetime.time() > datetime.time(10, 0):
                    #When we enter a short:                
                    #   MACD IS GREATER THAN .3               
                    #   MACD is = or smaller than zero. 
                    # We enter a long trade   
                    #  MACD IS SMALLER THAT -.5 (NEGATIVE .5)   
                    #   stock price is at or above the S28D (SIMPLE 28D MOVING AVERAGE) 
                    if self.macd_s[indx].macd[-1] > 0.3 or -.5 < self.macd_s[indx].macd[-1] < 0.0:
                        if positionx.size == 0:  # not in the market
                            self.order_s[datax.p.dataname] = self.sell(datax, size = floor(self.p.max_investment_per_trade / datax.close[-1]))
                            print("Submit a %s-size short order for %s" %(floor(self.p.max_investment_per_trade / datax.close[-1]), datax._dataname))
                        else:  
                            print("%s already in market" %datax._dataname)   
                    elif self.macd_s[indx].macd[-1] < -0.5 or self.sma_s[indx][-1] < datax.close[-1]:
                        if positionx.size == 0:  # not in the market
                            self.order_s[datax.p.dataname] = self.buy(datax, size = floor(self.p.max_investment_per_trade / datax.close[0]))
                            print("Submit a %s-size long order for %s" %(floor(self.p.max_investment_per_trade / datax.close[-1]), datax._dataname))
                        else:  # in the market
                            print("%s already in the market" %datax._dataname)
                    # When we liquidate a trade               
                    #  maximum loss is 2,000 (two thousand dollars)           
                    #  we liquidate a trade if the clock gets to 3.35 NY TIME
                    elif positionx.size < 0 and (datax.close[-1] - positionx.price) * positionx.size > self.p.max_loss_per_trade: # short? 
                        self.order_s[datax.p.dataname] = self.close(datax) 
                        print("Submit a closing order for %s positions." %(datax._dataname))
                    elif positionx.size > 0 and (positionx.price -  datax.close[-1]) * positionx.size > self.p.max_loss_per_trade: # short? 
                        self.order_s[datax.p.dataname] = self.close(datax) 
                        print("Submit a closing order for %s positions." %(datax._dataname))
                elif datetime.time(15, 0) < datax.datetime.time():
                    if positionx.size:  
                        self.order_s[datax.p.dataname] = self.close(datax) 
                        print("Submit a closing order for %s positions." %(datax._dataname))
                            
    def start(self):
       for datax in self.datas:
           if datax.contractdetails is not None:
                print('Timezone from ContractDetails: {}'.format(datax.contractdetails.m_timeZoneId))
           header = ['Datetime', 'Open', 'High', 'Low', 'Close', 'Volume', 'OpenInterest']
           print(', '.join(header))


def runstrategy():

    args = parse_args()
    #symbols = ['EUR.USD-CASH-IDEALPRO', 'EUR.CAD-CASH-IDEALPRO', 'EUR.NZD-CASH-IDEALPRO', 'NZD.USD-CASH-IDEALPRO', 'USD.NOK-CASH-IDEALPRO']
    #symbols = ['AAPL']
    symbols = ['AAPL-STK-SMART-USD', 'OXY-STK-SMART-USD', 'NOV-STK-SMART-USD',
               'CRC-STK-SMART-USD', 'XOM-STK-SMART-USD', 'SLB-STK-SMART-USD']
    # Create a cerebro

    cerebro = bt.Cerebro()
    storekwargs = dict(
        host=args.host, port=args.port,
        clientId=args.clientId, timeoffset=not args.no_timeoffset,
        reconnect=args.reconnect, timeout=args.timeout,
        notifyall=args.notifyall, _debug=args.debug
    )
    if args.usestore:
        ibstore = bt.stores.IBStore(**storekwargs)
    if args.broker:
        if args.usestore:
            broker = ibstore.getbroker()
        else:
            broker = bt.brokers.IBBroker(**storekwargs)
        cerebro.setbroker(broker)
    timeframe = bt.TimeFrame.TFrame(args.timeframe)
    # Manage data1 parameters
    
    if args.resample or args.replay:
        datatf = bt.TimeFrame.Ticks
        #datatf = bt.TimeFrame.Minutes
        datacomp = 1
    else:
        datatf = timeframe
        datacomp = args.compression
    fromdate = None
    if args.fromdate:
        dtformat = '%Y-%m-%d' + ('T%H:%M:%S' * ('T' in args.fromdate))
        fromdate = datetime.datetime.strptime(args.fromdate, dtformat)
    IBDataFactory = ibstore.getdata if args.usestore else bt.feeds.IBData
    datakwargs = dict(
        timeframe=datatf, compression=datacomp,
        historical=args.historical, fromdate=fromdate,
        rtbar=args.rtbar,
        qcheck=args.qcheck,
        what=args.what,
        backfill_start=not args.no_backfill_start,
        backfill=not args.no_backfill,
        latethrough=args.latethrough,
        tz=args.timezone
    )
    if not args.usestore and not args.broker:   # neither store nor broker
        datakwargs.update(storekwargs)  # pass the store args over the data
    rekwargs = dict(
                    timeframe=timeframe, compression=args.compression,
                    bar2edge=not args.no_bar2edge,
                    adjbartime=not args.no_adjbartime,
                    rightedge=not args.no_rightedge,
                    takelate=not args.no_takelate
    )
    for symbol in symbols:
        datax = IBDataFactory(dataname=symbol, **datakwargs)
        if args.replay:
            cerebro.replaydata(dataname=datax, **rekwargs)
        elif args.resample:
            cerebro.resampledata(dataname=datax, **rekwargs)
        else:
            cerebro.adddata(datax)
        
    valid = None
    if args.valid is not None:
        valid = datetime.timedelta(seconds=args.valid)

    cerebro.addstrategy(TestStrategy, exectype=bt.Order.ExecType(args.exectype))

    cerebro.run(exactbars=args.exactbars)
    if args.plot and args.exactbars < 1:  # plot if possible
        cerebro.plot()

if __name__ == '__main__':
    runstrategy()

[backtrader]

If you have tried with the original ibtest.py, the most important part from the message is not having your code but as pointed out above:

- *how is it being run*?

In the link from above: https://community.backtrader.com/topic/10/ib-example-connection-delayed-on-forex-data-222/4 the problem was:

• The default resolution if run with no parameters is TimeFrame.Ticks and Interactive Brokers offers no backfilling for such resolution. Which offers 2 alternatives:

1. Running with an Interactive Brokers supported timeframe:

./ibtest.py --data0 EUR.USD-CASH-IDEALPRO --resample --timeframe Seconds --compression 1

or

2. Telling the sample to avoid backfilling if the resolution is not supported:

./ibtest.py --data0 EUR.USD-CASH-IDEALPRO --no-backfill_start --no-backfill

None of those has a specific ticker, so you may want to quote how you are actually running the sample ibtest.py

Being that a paper trading account there are several things to consider according to the documentation from IB which is sometimes conflicting (probably due to some old pages from the portal still being reachable), but being the most important apparently:

• The data can only be used from one account: either the live account or the paper trading account

Apparently some other restrictions are in place, but what is actually true or not may be better answered by Interactive Brokers itself.

In any case the sample was tested against a demo account and a real trading account. The code here doesn't know anything about data permissions from Interactive Brokers. It asks the API to deliver data. If the data comes it will be pumped to your code, if it doesn't ...

[rastegarr]

@backtrader

I ran it like this:

./ibtest.py --data0 EUR.USD-CASH-IDEALPRO --resample --timeframe Minutes --compression 1

I have even tried

./ibtest.py --data0 EUR.USD-CASH-IDEALPRO --no-backfill_start --no-backfill

None of them work with paper trading account. It works with demo account though!

[backtrader]

In that case it looks like a market data permission issue. The platform doesn't know if the account is a demo, paper trading or real. The connection happens to a TCP/IP destination.

You may try the samples in the package Ibpy to double check.

ibtest can also be run with --debug to print all TWS messages but it won't likely help

[rastegarr]

@backtrader

I run this IbPy example and it seems I can receive the tick data from my paper trading account using IbPy library. So that tells me it shouldn't be a data subscription issue:

from ib.ext.Contract import Contract
from ib.opt import ibConnection, message
from time import sleep

# print all messages from TWS
def watcher(msg):
    print (msg)

# show Bid and Ask quotes
def my_BidAsk(msg):
    if msg.field == 1:
        print ('%s:%s: bid: %s' % (contractTuple[0],
                   contractTuple[6], msg.price))
    elif msg.field == 2:
        print ('%s:%s: ask: %s' % (contractTuple[0], contractTuple[6], msg.price))

def makeStkContract(contractTuple):
    newContract = Contract()
    newContract.m_symbol = contractTuple[0]
    newContract.m_secType = contractTuple[1]
    newContract.m_exchange = contractTuple[2]
    newContract.m_currency = contractTuple[3]
    newContract.m_expiry = contractTuple[4]
    newContract.m_strike = contractTuple[5]
    newContract.m_right = contractTuple[6]
    print ('Contract Values:%s,%s,%s,%s,%s,%s,%s:' % contractTuple)
    return newContract

if __name__ == '__main__':
    con = ibConnection(port = 4001, clientId = 123)
    con.registerAll(watcher)
    showBidAskOnly = False  # set False to see the raw messages
    if showBidAskOnly:
        con.unregister(watcher, message.tickSize, message.tickPrice,
                       message.tickString, message.tickOptionComputation)
        con.register(my_BidAsk, message.tickPrice)
    con.connect()
    sleep(1)
    tickId = 59
    
    # Note: Option quotes will give an error if they aren't shown in TWS
    contractTuple = ('AAPL', 'STK', 'SMART', 'USD', '', 0.0, '')
    #contractTuple = ('QQQQ', 'OPT', 'SMART', 'USD', '20070921', 47.0, 'CALL')
    #contractTuple = ('ES', 'FUT', 'GLOBEX', 'USD', '200709', 0.0, '')
    #contractTuple = ('ES', 'FOP', 'GLOBEX', 'USD', '20070920', 1460.0, 'CALL')
    #contractTuple = ('EUR', 'CASH', 'IDEALPRO', 'USD', '', 0.0, '')
    for i in range(20):
        stkContract = makeStkContract(contractTuple)
        print ('* * * * REQUESTING MARKET DATA * * * *')
        con.reqMktData(tickId, stkContract, '', False)
        sleep(1)
        print ('* * * * CANCELING MARKET DATA * * * *')
        con.cancelMktData(tickId)
    sleep(1)
    con.disconnect()
    sleep(1)

Something that I observed is that if move makeStkContract and cancelMktData, out of loop I get bunch of errors (code 322).

[backtrader]

There isn't enough information to even attempt anything. For example: the only provided output is from the modified source, without having the actual command line execution

As such one can only try to point the low hanging fruits. One would be:

• Are you pointing to the right port? (usually 7496 for production and 7497 for paper trading)

The default port used by the sample and the IB ecosystem is 7496.

You quote above the following command:

./ibtest.py --data0 EUR.USD-CASH-IDEALPRO --resample --timeframe Minutes --compression 1

and this is theoretically working with the demo. But the demo also runs (default, can be of course be changed) on port 7497.

Cash Products like EUR.USD-CASH-IDEALPRO are allowed (afaik) to all IB customers (as long as your account is funded I think). Whether demo, paper trading or real account, that product is available for everyone. And the code (which relies on IBPy for interfacing to TWS) doesn't know what it is connecting to. The account type is controlled by the TWS (or IbGateway) instance you connect to.

[Aleem SEO]

This post is deleted!