Multiple feeds first order is rejected when live with IB

hbf

I implemented a live version of the 'Multi Example' using IB.

I had to add the following to the notify_order function due receiving a ValueError (Something to do with indexing an order that didn't exist in the if not order.alive(): statement):

        if order.status == order.Completed:
            return

It appears that whenever an order is submitted to the broker, the order is initially rejected before being accepted and completed.

962,2018-12-11T23:52:30,GBP.AUD-CASH-IDEALPRO,1.73331,1.73343,1.73330,1.73342,0.00000
2018-12-11T23:52:30 GBP.AUD-CASH-IDEALPRO Position 0
2018-12-11T23:52:30 GBP.AUD-CASH-IDEALPRO Buy 3
2018-12-11T23:53:00 EUR.USD-CASH-IDEALPRO Order 1 Status Rejected
-- No longer alive main Ref
2018-12-11T23:53:00 GBP.USD-CASH-IDEALPRO Order 2 Status Rejected
-- No longer alive main Ref
2018-12-11T23:53:00 GBP.AUD-CASH-IDEALPRO Order 3 Status Rejected
-- No longer alive main Ref
2018-12-11T23:53:00 EUR.USD-CASH-IDEALPRO Order 1 Status Accepted
2018-12-11T23:53:00 GBP.USD-CASH-IDEALPRO Order 2 Status Accepted
2018-12-11T23:53:00 GBP.AUD-CASH-IDEALPRO Order 3 Status Accepted
2018-12-11T23:53:00 EUR.USD-CASH-IDEALPRO Order 1 Status Completed
2018-12-11T23:53:00 GBP.USD-CASH-IDEALPRO Order 2 Status Completed
2018-12-11T23:53:00 GBP.AUD-CASH-IDEALPRO Order 3 Status Completed

Although this issue is not affecting performance yet, I was wondering if anyone had an insight as to what was causing it?

I assume it is the code modification I added above; however, without that modification I am unsure how to manage multiple orders.

from __future__ import (absolute_import, division, print_function,
                        unicode_literals)


import argparse
import datetime
import math

import backtrader as bt

class TestStrat(bt.Strategy):

    def notify_order(self, order):
        if order.status == order.Submitted:
            return

        dt, dn = self.data.datetime.datetime(0).isoformat(), order.data._name
        print('{} {} Order {} Status {}'.format(
            dt, dn, order.ref, order.getstatusname())
        )

        if order.status == order.Completed:
            return

        whichord = ['main', 'stop', 'limit', 'close']
        if not order.alive():  # not alive - nullify
            dorders = self.o[order.data]
            idx = dorders.index(order)
            dorders[idx] = None
            print('-- No longer alive {} Ref'.format(whichord[idx]))
            if all(x is None for x in dorders):
                dorders[:] = []  # empty list - New orders allowed

    def notify_trade(self, trade):
        dt, dn = self.data.datetime.datetime(0).isoformat(), trade.data._name
        if trade.isclosed:
            print('{} {} Gross Profit {} Net Profit {}' \
            .format(dt, dn, round(trade.pnl,2), round(trade.pnlcomm,2)))

    def __init__(self):
        self.o = dict()  # orders per data (main, stop, limit, manual-close)
        self.holding = dict()  # holding periods per data

    def logdata(self, d):
        txt = []
        txt.append('{}'.format(len(self)))
        txt.append('{}'.format(d.datetime.datetime(0).isoformat()))
        txt.append('{}'.format(d._name))
        txt.append('{:.5f}'.format(d.open[0]))
        txt.append('{:.5f}'.format(d.high[0]))
        txt.append('{:.5f}'.format(d.low[0]))
        txt.append('{:.5f}'.format(d.close[0]))
        txt.append('{:.5f}'.format(d.volume[0]))
        print(','.join(txt))

    data_live = False

    def notify_data(self, data, status, *args, **kwargs):
        print('*' * 5, data._name ,'DATA NOTIF:', data._getstatusname(status),
              *args)
        if status == data.LIVE:
            self.data_live = True

    def next(self):
        for i, d in enumerate(self.datas):

            self.logdata(d)

            if not self.data_live:
                return

            dt, dn = self.data.datetime.datetime(0).isoformat(), d._name
            pos = self.getposition(d).size
            print('{} {} Position {}'.format(dt, dn, pos))

            if not pos and not self.o.get(d, None):  # no market / no orders

                if (d.close[0] < d.close[-1]):
                    self.o[d] = [self.buy(data=d)]
                    print('{} {} Buy {}'.format(dt, dn, self.o[d][0].ref))

                elif (d.close[0] > d.close[-1]):
                    self.o[d] = [self.buy(data=d)]
                    print('{} {} Buy {}'.format(dt, dn, self.o[d][0].ref))

                self.holding[d] = 0

            elif pos:

                self.holding[d] += 1

                if self.holding[d] >= 2:
                    o = self.close(data=d)
                    self.o[d].append(o) # manual order to list of orders
                    print('{} {} Manual Close {}'.format(dt, dn, o.ref))


def runstrat(args=None):

    args = parse_args(args)

    cerebro = bt.Cerebro()

    # Data feed kwargs
    kwargs = dict()

    # Store
    store = bt.stores.IBStore(port=7497, host='127.0.0.1')

    # Data feeds
    securities = ['EUR.USD-CASH-IDEALPRO','GBP.USD-CASH-IDEALPRO',
                  'GBP.AUD-CASH-IDEALPRO']

    for security in securities:
        data = store.getdata(dataname=security, timeframe=bt.TimeFrame.Ticks)
        cerebro.resampledata(data, timeframe=args.timeframe,
                             compression=args.compression, name=security)

    # Broker
    cerebro.broker = store.getbroker()

    # Strategy
    cerebro.addstrategy(TestStrat, **eval('dict(' + args.strat + ')'))
    # cerebro.addstrategy(TestStrat, **eval('dict(' + args.strat + ')'))

    # Execute
    cerebro.run(**eval('dict(' + args.cerebro + ')'))


def parse_args(pargs=None):
    parser = argparse.ArgumentParser(
        formatter_class=argparse.ArgumentDefaultsHelpFormatter,
        description=(
            'Multiple Values'
        )
    )

    parser.add_argument('--cerebro', required=False, default='',
                        metavar='kwargs', help='kwargs in key=value format')

    parser.add_argument('--broker', required=False, default='',
                        metavar='kwargs', help='kwargs in key=value format')

    parser.add_argument('--sizer', required=False, default='',
                        metavar='kwargs', help='kwargs in key=value format')

    parser.add_argument('--strat', required=False, default='',
                        metavar='kwargs', help='kwargs in key=value format')

    parser.add_argument('--timeframe', required=False,
                        default=bt.TimeFrame.Seconds, action='store',
                        help='datafeed timeframe parameter')

    parser.add_argument('--compression', required=False,
                        default=30, action='store',
                        help='datafeed compression parameter')

    return parser.parse_args(pargs)


if __name__ == '__main__':
    runstrat()

Thank you for your help.

backtrader

You should activate the _debug parameter of the IBStore. See: Docs - Live Trading - Interactive Brokers

The most likely explanation is that an error code is being returned, which the code has no provisions for. The default behavior in such a case is to assume the order has been rejected.

The error codes

        # 100-199 Order/Data/Historical related
        # 200-203 tickerId and Order Related
        # 300-399 A mix of things: orders, connectivity, tickers, misc errors
        # 400-449 Seem order related again
        # 500-531 Connectivity/Communication Errors
        # 10000-100027 Mix of special orders/routing
        # 1100-1102 TWS connectivy to the outside
        # 1300- Socket dropped in client-TWS communication
        # 2100-2110 Informative about Data Farm status (id=-1)

The documentation from IB is lacking and the error codes were probably added on a "Needed by Institutional Customer X" basis, with some of those codes later gaining ground into the realm of regular customers but with no documentation as to what, how, when and why.

hbf

@backtrader Thank you for that! Feeling stupid for not realising there was a _debug parameter.

It appears that it was an issue with the order size being below the 20000 IdealPro minimum, resulting in a 399 error code. After amending the order size to meet the minimum criteria, the code which was previously added can be removed:

        if order.status == order.Completed:
            return