Bracket order rejection and failure to transmit Interactive Brokers

hbf

Hi,

I am attempting to implement some relatively simple strategies within Interactive Brokers. At the moment I am using a demo account.

There are two things happening:

Market orders are sent to IB; however, they are not accepted as I need to manually click Transmit on the TWS. Only very randomly is the market order accepted automatically by IB without having to click Transmit.
The two corresponding stop and limit orders orders for the Bracket are not submitted when problem 1) occurs. When the market order is actually accepted, then the two corresponding orders are created.

I am struggling to understand why orders would be submitted but not transmitted and believe that this would help find a solution to my problem.

I appreciate your help. Thank you.

EDIT:
Potentially a very simple thing which I failed to realise - market orders are being sent but the price moves up/down before being accepted?

from __future__ import (absolute_import, division, print_function,
                        unicode_literals)

import backtrader as bt

class St(bt.Strategy):

    def __init__(self):
        self.ema5high = bt.indicators.EMA(self.data.high,period=5)
        self.ema5low = bt.indicators.EMA(self.data.low,period=5)

    def logdata(self):
        txt = []
        txt.append('{}'.format(len(self)))
        txt.append('{}'.format(self.data.datetime.datetime(0).isoformat()))
        txt.append('{:.5f}'.format(self.data.open[0]))
        txt.append('{:.5f}'.format(self.data.high[0]))
        txt.append('{:.5f}'.format(self.data.low[0]))
        txt.append('{:.5f}'.format(self.data.close[0]))
        txt.append('{:.5f}'.format(self.data.volume[0]))
        print(','.join(txt))

    data_live = False

    def notify_data(self, data, status, *args, **kwargs):
        print('*' * 5, 'DATA NOTIF:', data._getstatusname(status), *args)
        if status == data.LIVE:
            self.data_live = True

    def notify_order(self, order):
        date = self.data.datetime.datetime(0).isoformat()
        if order.status == order.Accepted:
            print('-'*32,' NOTIFY ORDER ','-'*32)
            print('Order Accepted')
            print('{}, Status {}: Ref: {}, Size: {}, Price: {}' \
            .format(date, order.status, order.ref, order.size,
                    'NA' if not order.price else round(order.price,5)))

        if order.status == order.Completed:
            print('-'*32,' NOTIFY ORDER ','-'*32)
            print('Order Completed')
            print('{}, Status {}: Ref: {}, Size: {}, Price: {}' \
            .format(date, order.status, order.ref, order.size,
                    'NA' if not order.price else round(order.price,5)))
            print('Created: {} Price: {} Size: {}' \
            .format(bt.num2date(order.created.dt),
                                order.created.price,order.created.size))
            print('-'*80)

        if order.status == order.Canceled:
            print('-'*32,' NOTIFY ORDER ','-'*32)
            print('Order Canceled')
            print('{}, Status {}: Ref: {}, Size: {}, Price: {}' \
            .format(date, order.status, order.ref, order.size,
                    'NA' if not order.price else round(order.price,5)))

        if order.status == order.Rejected:
            print('-'*32,' NOTIFY ORDER ','-'*32)
            print('WARNING! Order Rejected')
            print('{}, Status {}: Ref: {}, Size: {}, Price: {}' \
            .format(date, order.status, order.ref, order.size,
                    'NA' if not order.price else round(order.price,5)))
            print('-'*80)

    def notify_trade(self, trade):
        date = self.data.datetime.datetime()
        if trade.isclosed:
            print('-'*32,' NOTIFY TRADE ','-'*32)
            print('{}, Close Price: {}, Profit, Gross {}, Net {}' \
            .format(date, trade.price, round(trade.pnl,2),
                    round(trade.pnlcomm,2)))
            print('-'*80)

    def next(self):
        self.logdata()
        if not self.data_live:
            return

        #open = self.data.open[0]
        close = self.data.close[0]

        if not self.position:
            # look to enter
            if close < self.ema5low[0]:
                # we have closed below the ema5low and are entering
                # a long position via a bracket order
                long_tp = close + 0.0006
                long_stop = close - 0.0008
                self.buy_bracket(size=25000, limitprice=long_tp,
                                 stopprice=long_stop,
                                 exectype=bt.Order.Market)

            elif close > self.ema5high[0]:
                short_tp = close - 0.0006
                short_stop = close + 0.0008
                # note that the limit price is now the stop and stop price
                # is take profit
                self.sell_bracket(size=25000, stopprice=short_tp,
                                  limitprice=short_stop,
                                  exectype=bt.Order.Market)

def run(args=None):
    cerebro = bt.Cerebro(stdstats=False)
    store = bt.stores.IBStore(port=7497, host='127.0.0.1')

    data = store.getdata(dataname='EUR.USD-CASH-IDEALPRO',
                         timeframe=bt.TimeFrame.Ticks)
    cerebro.resampledata(data, timeframe=bt.TimeFrame.Seconds, compression=30)

    cerebro.broker = store.getbroker()

    cerebro.addstrategy(St)
    cerebro.run()


if __name__ == '__main__':
    run()

backtrader

@hbf said in Bracket order rejection and failure to transmit Interactive Brokers:

Market orders are sent to IB; however, they are not accepted as I need to manually click Transmit on the TWS. Only very randomly is the market order accepted automatically by IB without having to click Transmit.

The title say they are being rejected, but you say here that they are being put on-hold. Which one is it?

Is it for buy, is it for sell, is it for both?

Why not changing the size to 1 to see if you are hitting a limit?

@hbf said in Bracket order rejection and failure to transmit Interactive Brokers:

The two corresponding stop and limit orders orders for the Bracket are not submitted when problem 1) occurs. When the market order is actually accepted, then the two corresponding orders are created.

If the orders conforming the bracket were there before the main (the bracketed) order happens to be there ... it would be a disaster in most cases.

@hbf said in Bracket order rejection and failure to transmit Interactive Brokers:

            elif close > self.ema5high[0]:
                short_tp = close - 0.0006
                short_stop = close + 0.0008
                # note that the limit price is now the stop and stop price
                # is take profit
                self.sell_bracket(size=25000, stopprice=short_tp,
                                  limitprice=short_stop,
                                  exectype=bt.Order.Market)

This is wrong. The stop price of a sell order can obviously not be under the expected matching price.

@hbf said in Bracket order rejection and failure to transmit Interactive Brokers:

Potentially a very simple thing which I failed to realise - market orders are being sent but the price moves up/down before being accepted?

You probably don't mean what you said. A Market order is matched when it enter the market. The price doesn't move up/down ... it takes the available price (if the volume allows it).

What you mean is that the price moves from the close price from which you derive your limit and stop prices. Yes it can move. This could actually put your limit or stop orders at the wrong side of the bracket. Whether this is a blocker for TWS ...

hbf

Thank you for your thorough response.

@backtrader said in Bracket order rejection and failure to transmit Interactive Brokers:

@hbf said in Bracket order rejection and failure to transmit Interactive Brokers:

Market orders are sent to IB; however, they are not accepted as I need to manually click Transmit on the TWS. Only very randomly is the market order accepted automatically by IB without having to click Transmit.

The title say they are being rejected, but you say here that they are being put on-hold. Which one is it?

Is it for buy, is it for sell, is it for both?

Sorry for the ambiguity. The orders are being put on-hold. It is for both buy and sell orders.

Why not changing the size to 1 to see if you are hitting a limit?

I have amended the size to 1, orders are still put on-hold.

@hbf said in Bracket order rejection and failure to transmit Interactive Brokers:

The two corresponding stop and limit orders orders for the Bracket are not submitted when problem 1) occurs. When the market order is actually accepted, then the two corresponding orders are created.

If the orders conforming the bracket were there before the main (the bracketed) order happens to be there ... it would be a disaster in most cases.

Of course!

@hbf said in Bracket order rejection and failure to transmit Interactive Brokers:

            elif close > self.ema5high[0]:
                short_tp = close - 0.0006
                short_stop = close + 0.0008
                # note that the limit price is now the stop and stop price
                # is take profit
                self.sell_bracket(size=25000, stopprice=short_tp,
                                  limitprice=short_stop,
                                  exectype=bt.Order.Market)

This is wrong. The stop price of a sell order can obviously not be under the expected matching price.

Whoops. Thank you for pointing this out.

@hbf said in Bracket order rejection and failure to transmit Interactive Brokers:

Potentially a very simple thing which I failed to realise - market orders are being sent but the price moves up/down before being accepted?

You probably don't mean what you said. A Market order is matched when it enter the market. The price doesn't move up/down ... it takes the available price (if the volume allows it).

What you mean is that the price moves from the close price from which you derive your limit and stop prices. Yes it can move. This could actually put your limit or stop orders at the wrong side of the bracket. Whether this is a blocker for TWS ...

That is correct, again, my bad.

Thank you again, I really appreciate your time and your platform.

Updated code:

from __future__ import (absolute_import, division, print_function,
                        unicode_literals)

import backtrader as bt

class St(bt.Strategy):

    def __init__(self):
        self.ema5high = bt.indicators.EMA(self.data.high,period=5)
        self.ema5low = bt.indicators.EMA(self.data.low,period=5)

    def logdata(self):
        txt = []
        txt.append('{}'.format(len(self)))
        txt.append('{}'.format(self.data.datetime.datetime(0).isoformat()))
        txt.append('{:.5f}'.format(self.data.open[0]))
        txt.append('{:.5f}'.format(self.data.high[0]))
        txt.append('{:.5f}'.format(self.data.low[0]))
        txt.append('{:.5f}'.format(self.data.close[0]))
        txt.append('{:.5f}'.format(self.data.volume[0]))
        print(','.join(txt))

    data_live = False

    def notify_data(self, data, status, *args, **kwargs):
        print('*' * 5, 'DATA NOTIF:', data._getstatusname(status), *args)
        if status == data.LIVE:
            self.data_live = True

    def notify_order(self, order):
        date = self.data.datetime.datetime(0).isoformat()
        if order.status == order.Accepted:
            print('-'*32,' NOTIFY ORDER ','-'*32)
            print('Order Accepted')
            print('{}, Status {}: Ref: {}, Size: {}, Price: {}' \
            .format(date, order.status, order.ref, order.size,
                    'NA' if not order.price else round(order.price,5)))

        if order.status == order.Completed:
            print('-'*32,' NOTIFY ORDER ','-'*32)
            print('Order Completed')
            print('{}, Status {}: Ref: {}, Size: {}, Price: {}' \
            .format(date, order.status, order.ref, order.size,
                    'NA' if not order.price else round(order.price,5)))
            print('Created: {} Price: {} Size: {}' \
            .format(bt.num2date(order.created.dt),
                                order.created.price,order.created.size))
            print('-'*80)

        if order.status == order.Canceled:
            print('-'*32,' NOTIFY ORDER ','-'*32)
            print('Order Canceled')
            print('{}, Status {}: Ref: {}, Size: {}, Price: {}' \
            .format(date, order.status, order.ref, order.size,
                    'NA' if not order.price else round(order.price,5)))

        if order.status == order.Rejected:
            print('-'*32,' NOTIFY ORDER ','-'*32)
            print('WARNING! Order Rejected')
            print('{}, Status {}: Ref: {}, Size: {}, Price: {}' \
            .format(date, order.status, order.ref, order.size,
                    'NA' if not order.price else round(order.price,5)))
            print('-'*80)

    def notify_trade(self, trade):
        date = self.data.datetime.datetime()
        if trade.isclosed:
            print('-'*32,' NOTIFY TRADE ','-'*32)
            print('{}, Close Price: {}, Profit, Gross {}, Net {}' \
            .format(date, trade.price, round(trade.pnl,2),
                    round(trade.pnlcomm,2)))
            print('-'*80)

    def next(self):
        self.logdata()
        if not self.data_live:
            return

        close = self.data.close[0]

        if not self.position:
            # look to enter
            if close < self.ema5low[0]:
                # we have closed below the ema5low and are entering
                # a long position via a bracket order
                long_tp = close + 0.0006
                long_stop = close - 0.0008
                self.buy_bracket(size=1, limitprice=long_tp,
                                 stopprice=long_stop,
                                 exectype=bt.Order.Market)

            elif close > self.ema5high[0]:
                short_tp = close - 0.0006
                short_stop = close + 0.0008
                self.sell_bracket(size=1, stopprice=short_stop,
                                  limitprice=short_tp,
                                  exectype=bt.Order.Market)

def run(args=None):
    cerebro = bt.Cerebro(stdstats=False)
    store = bt.stores.IBStore(port=7497, host='127.0.0.1')

    data = store.getdata(dataname='EUR.USD-CASH-IDEALPRO',
                         timeframe=bt.TimeFrame.Ticks)
    cerebro.resampledata(data, timeframe=bt.TimeFrame.Seconds, compression=30)

    cerebro.broker = store.getbroker()

    cerebro.addstrategy(St)
    cerebro.run()


if __name__ == '__main__':
    run()

Orders remain on-hold.

hbf

@backtrader, sorry to bug you. The problem remains.

from __future__ import (absolute_import, division, print_function,
                        unicode_literals)

import backtrader as bt

class St(bt.Strategy):

    def __init__(self):
        self.ema5high = bt.indicators.EMA(self.data.high,period=5)
        self.ema5low = bt.indicators.EMA(self.data.low,period=5)

    def logdata(self):
        txt = []
        txt.append('{}'.format(len(self)))
        txt.append('{}'.format(self.data.datetime.datetime(0).isoformat()))
        txt.append('{:.5f}'.format(self.data.open[0]))
        txt.append('{:.5f}'.format(self.data.high[0]))
        txt.append('{:.5f}'.format(self.data.low[0]))
        txt.append('{:.5f}'.format(self.data.close[0]))
        txt.append('{:.5f}'.format(self.data.volume[0]))
        print(','.join(txt))

    data_live = False

    def notify_data(self, data, status, *args, **kwargs):
        print('*' * 5, 'DATA NOTIF:', data._getstatusname(status), *args)
        if status == data.LIVE:
            self.data_live = True

    def notify_order(self, order):
        date = self.data.datetime.datetime(0).isoformat()
        if order.status == order.Accepted:
            print('-'*32,' NOTIFY ORDER ','-'*32)
            print('Order Accepted')
            print('{}, Status {}: Ref: {}, Size: {}, Price: {}' \
            .format(date, order.status, order.ref, order.size,
                    'NA' if not order.price else round(order.price,5)))

        if order.status == order.Completed:
            print('-'*32,' NOTIFY ORDER ','-'*32)
            print('Order Completed')
            print('{}, Status {}: Ref: {}, Size: {}, Price: {}' \
            .format(date, order.status, order.ref, order.size,
                    'NA' if not order.price else round(order.price,5)))
            print('Created: {} Price: {} Size: {}' \
            .format(bt.num2date(order.created.dt),
                                order.created.price,order.created.size))
            print('-'*80)

        if order.status == order.Canceled:
            print('-'*32,' NOTIFY ORDER ','-'*32)
            print('Order Canceled')
            print('{}, Status {}: Ref: {}, Size: {}, Price: {}' \
            .format(date, order.status, order.ref, order.size,
                    'NA' if not order.price else round(order.price,5)))

        if order.status == order.Rejected:
            print('-'*32,' NOTIFY ORDER ','-'*32)
            print('WARNING! Order Rejected')
            print('{}, Status {}: Ref: {}, Size: {}, Price: {}' \
            .format(date, order.status, order.ref, order.size,
                    'NA' if not order.price else round(order.price,5)))
            print('-'*80)

    def notify_trade(self, trade):
        date = self.data.datetime.datetime()
        if trade.isclosed:
            print('-'*32,' NOTIFY TRADE ','-'*32)
            print('{}, Close Price: {}, Profit, Gross {}, Net {}' \
            .format(date, trade.price, round(trade.pnl,2),
                    round(trade.pnlcomm,2)))
            print('-'*80)

    def next(self):
        self.logdata()
        if not self.data_live:
            return

        close = self.data.close[0]

        if not self.position:
            # look to enter
            if close < self.ema5low[0]:
                # we have closed below the ema5low and are entering
                # a long position via a bracket order
                long_tp = close + 3
                long_stop = close - 5
                self.buy_bracket(size=1, limitprice=long_tp,
                                 stopprice=long_stop,
                                 exectype=bt.Order.Market)

            elif close > self.ema5high[0]:
                short_tp = close - 3
                short_stop = close + 5
                self.sell_bracket(size=1, stopprice=short_stop,
                                  limitprice=short_tp,
                                  exectype=bt.Order.Market)

def run(args=None):
    cerebro = bt.Cerebro(stdstats=False)
    store = bt.stores.IBStore(port=7496, host='127.0.0.1')

    data = store.getdata(dataname='ES-201812-GLOBEX',
                         timeframe=bt.TimeFrame.Ticks)
    cerebro.resampledata(data, timeframe=bt.TimeFrame.Minutes, compression=1)

    cerebro.broker = store.getbroker()

    cerebro.addstrategy(St)
    cerebro.run()


if __name__ == '__main__':
    run()

Output after IB connection (cut delayed data to save characters), running on a simulated account:

Server Version: 76
TWS Time at connection:20181115 17:06:55 AEST
***** DATA NOTIF: DELAYED
5,2018-11-14T17:04:00,2700.25000,2701.50000,2700.25000,2701.25000,198.00000
...
427,2018-11-15T00:06:00,2700.50000,2700.75000,2700.00000,2700.00000,31.00000
***** DATA NOTIF: LIVE
428,2018-11-15T00:07:00,2700.00000,2700.00000,2700.00000,2700.00000,2.00000
429,2018-11-15T00:08:00,2700.25000,2700.50000,2700.00000,2700.25000,42.00000
430,2018-11-15T00:09:00,2700.25000,2700.50000,2700.25000,2700.50000,73.00000
--------------------------------  NOTIFY ORDER  --------------------------------
WARNING! Order Rejected
2018-11-15T00:10:00, Status 8: Ref: 1, Size: -1, Price: NA
--------------------------------------------------------------------------------
431,2018-11-15T00:10:00,2700.50000,2701.25000,2700.50000,2701.00000,235.00000

The orders do not transmit.

backtrader

https://www.interactivebrokers.com/en/index.php?f=583

This is the documentation from Interactive Brokers for Bracket Orders. The main order is presented as a Limit order. It may well be that when you set the main order to Market, TWS has a protection mode preventing automated transmission. The reason: "A Market order could actually execute with a price which lies outside of the bracket limits". And this would render the bracket useless.

Try switching to Limit.

Or a long shot: the API is in Read-Only mode.

backtrader

Following with the reasoning from above ... your use is probably best served by this scenario:

Issue a Market order
Upon confirmation of execution you have the actual execution price and the actual market price (close)
You can then issue two OCO orders which will bracket your existing positions. You calculate the stop-loss and take-profit prices from the execution price or from the market price

See: Docs - OCO Orders

hbf

@backtrader

Thank you again for your response. Adjusting the order to limit appears to have fixed the problem.

In regards to implementation of your scenario above, would you issue the two OCO orders in next? How would you get the execution price if such is the case? Or would you have to wait until the next bar?

backtrader

You can issue the new orders from notify_order or later from next (you either keep a reference to the execution price or look at the current price of the position)

hbf

@backtrader Fantastic, thank you!