3. limit order execute-fill has different behavior than market order in backtest
For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

limit order execute-fill has different behavior than market order in backtest

  • A

    The limit order fill behavior is different than the market order fill behavior. With a market order, the fill won't happen until the next printed bar. But for limit orders, the fill happens in the current bar.

    I don't understand why these are different. And is there a way I can get the limit orders to behave like the market orders, in terms of waiting until next printed bar to get filled?

    Here is a 1Mb data sample NQ data I use in this example.

    from datetime import timedelta, datetime as Dt
from types import SimpleNamespace as Sns
import backtrader as bt
import pandas as pd
import os

inst = Sns (symbol     = 'NQ'
            ,stratName = 'lmt_test'
            ,startDate = Dt(2019, 4, 16 )
            ,endDate   = Dt(2019, 4, 17 )
            ,showlogs  = True
            ,datafile  = os.path.expanduser('~/')+'dev/datas/'+'NQ_test.txt'
            )

# Build dataframes
rawdatacolnames = ['ddate','ttime','open','high','low','close','volume']
df1 = pd.read_csv(inst.datafile, names=rawdatacolnames)
df1['dtime'] = df1.ddate+' '+df1.ttime
df1.dtime = pd.to_datetime(df1.dtime, format = '%m/%d/%Y %H:%M')
df1 = df1[(df1.dtime >= inst.startDate) & (df1.dtime <= inst.endDate)]
df1.index = df1.dtime
df1 = df1[['open','high','low','close']]

ohlc_dict = {'open':'first', 'high':'max', 'low':'min', 'close': 'last'}
df15 = df1.resample('15Min').agg(ohlc_dict)

# ------------------------- Lmt Test Strategy ---------------------------
class lmt_test(bt.Strategy): 
    params = dict(inst = dict())
    # ----------------------------------------------------------------------------------#
    def log(self, txt, f_name='', dt=None): 
        dt = dt or self.data.datetime.datetime()
        print(f'{dt.strftime("%y/%m/%d-%H:%M:%S")}:: {f_name:<12}:: {txt}')
    # ----------------------------------------------------------------------------------#
    def barlog(self, d,                     f_name='BARLOG'): 
        self.log(f'lend0:{len(d)}, lend1:{len(self.data1)} - '+
                f'o:{d.open[0]:.2f} h:{d. high[0]:.2f} '+
                f'l:{d. low[0]:.2f} c:{d.close[0]:.2f}', f_name)
    # ----------------------------------------------------------------------------------#
    def notify_order(self, order,           f_name='NOTIFY ORDER'): 
        self.log(f'Order {order.info.orderTag}-{order.ref} '+
                 f'@{order.plen}: {order.getstatusname()}', f_name)
        if order.status in [order.Completed]:
            orderprintstr = f'{["SELL","BUY "][order.isbuy()]} EXECUTED: {order.info.orderTag}: '+\
                            f'${order.executed.price:.2f}'
            self.log(orderprintstr,f_name,)
            self.log('New position: '+str(self.getposition(self.data1).size),f_name)
    # ----------------------------------------------------------------------------------#
    def notify_trade(self, trade,           f_name='NOTIFY TRADE'): 
        if trade.isclosed:
            self.log(f'PNL: {trade.pnl:.2f}',f_name)
    # ----------------------------------------------------------------------------------#
    def __init__(self):
        if not self.p.inst.showlogs:
            self.log    = lambda x: None
            self.barlog = lambda x: None
    # ----------------------------------------------------------------------------------#
    def next(self,                          f_name='NEXT_BB'): 
        self.barlog(self.data0)

        if (len(self.data0)-5)%45 == 0:
            self.placeOrder(len(self.data1)%2, 2)
    # ----------------------------------------------------------------------------------#
    def placeOrder(self, odr, osz,          f_name='PLACE ORDER'): 
        self.log(f'{["Buy","Sell"][odr]} {osz} at {self.data0.close[0]}, '+
                  'Current position = '+str(self.getposition(self.data1).size),f_name)

        # Cancel open orders
        openOrders = self.broker.get_orders_open()
        if openOrders: self.cancel(openOrders[0])

        # Create new orders
        lmtOrderArgs = dict(data=self.data1, size=1, exectype=bt.Order.Limit, orderTag='Lmt')
        if odr: 
            self.sell(data=self.data1, orderTag='Mkt')
            self.sell(**lmtOrderArgs, price = self.data0.close[0])
        else:
            self.buy (data=self.data1, orderTag='Mkt')
            self.buy (**lmtOrderArgs, price = self.data0.close[0])

# ------------------------- Run Cerebro Strategy ---------------------------
print(Dt.now().strftime('%m-%d %H:%M:%S::'),
        'Running:',inst.symbol,'from:',str(inst.startDate)[:-9],'to:',str(inst.endDate)[:-9])

cerebro = bt.Cerebro(runonce=False, stdstats=False)

data0 = bt.feeds.PandasData(dataname = df1)
cerebro.adddata(data0, 'm1')

data1 = bt.feeds.PandasData(dataname = df15)
cerebro.adddata(data1, 'm15')

cerebro.addstrategy(eval(inst.stratName),inst=inst)

thestrat = cerebro.run(tradehistory=False)[0]

print('Final Portfolio Value: {:.2f}' .format(cerebro.broker.getvalue()))
print(Dt.now().strftime('%m-%d %H:%M:%S::'),f'Symbol {inst.symbol} Backtest Complete.')

    and here is an excerpt from the logs:

    19/04/16-00:43:00:: BARLOG      :: lend0:44, lend1:3 - o:7662.75 h:7662.75 l:7662.50 c:7662.75
19/04/16-00:44:00:: BARLOG      :: lend0:45, lend1:3 - o:7662.75 h:7662.75 l:7662.50 c:7662.50
19/04/16-00:45:00:: BARLOG      :: lend0:46, lend1:4 - o:7662.50 h:7663.00 l:7662.50 c:7662.50
19/04/16-00:46:00:: BARLOG      :: lend0:47, lend1:4 - o:7662.50 h:7662.50 l:7662.00 c:7662.00
19/04/16-00:47:00:: BARLOG      :: lend0:48, lend1:4 - o:7662.00 h:7662.00 l:7661.75 c:7661.75
19/04/16-00:48:00:: BARLOG      :: lend0:49, lend1:4 - o:7661.75 h:7662.00 l:7661.50 c:7661.75
19/04/16-00:49:00:: BARLOG      :: lend0:50, lend1:4 - o:7661.75 h:7662.50 l:7661.75 c:7662.25
19/04/16-00:49:00:: PLACE ORDER :: Buy 2 at 7662.25, Current position = -2
19/04/16-00:50:00:: NOTIFY ORDER:: Order Mkt-3 @4: Submitted
19/04/16-00:50:00:: NOTIFY ORDER:: Order Lmt-4 @4: Submitted
19/04/16-00:50:00:: NOTIFY ORDER:: Order Mkt-3 @4: Accepted
19/04/16-00:50:00:: NOTIFY ORDER:: Order Lmt-4 @4: Accepted
19/04/16-00:50:00:: NOTIFY ORDER:: Order Lmt-4 @4: Completed
19/04/16-00:50:00:: NOTIFY ORDER:: BUY  EXECUTED: Lmt: $7662.25
19/04/16-00:50:00:: NOTIFY ORDER:: New position: -1
19/04/16-00:50:00:: BARLOG      :: lend0:51, lend1:4 - o:7662.25 h:7662.25 l:7662.00 c:7662.00
19/04/16-00:51:00:: BARLOG      :: lend0:52, lend1:4 - o:7662.00 h:7662.50 l:7662.00 c:7662.50
19/04/16-00:52:00:: BARLOG      :: lend0:53, lend1:4 - o:7662.50 h:7663.00 l:7662.50 c:7662.75
19/04/16-00:53:00:: BARLOG      :: lend0:54, lend1:4 - o:7662.75 h:7663.50 l:7662.75 c:7663.50
19/04/16-00:54:00:: BARLOG      :: lend0:55, lend1:4 - o:7663.50 h:7663.75 l:7663.00 c:7663.00
19/04/16-00:55:00:: BARLOG      :: lend0:56, lend1:4 - o:7663.00 h:7663.00 l:7662.50 c:7662.75
19/04/16-00:56:00:: BARLOG      :: lend0:57, lend1:4 - o:7663.00 h:7663.00 l:7663.00 c:7663.00
19/04/16-00:57:00:: BARLOG      :: lend0:58, lend1:4 - o:7663.00 h:7663.50 l:7663.00 c:7663.00
19/04/16-00:58:00:: BARLOG      :: lend0:59, lend1:4 - o:7663.00 h:7663.25 l:7663.00 c:7663.25
19/04/16-00:59:00:: BARLOG      :: lend0:60, lend1:4 - o:7663.25 h:7663.50 l:7663.00 c:7663.25
19/04/16-01:00:00:: NOTIFY ORDER:: Order Mkt-3 @4: Completed
19/04/16-01:00:00:: NOTIFY ORDER:: BUY  EXECUTED: Mkt: $7663.25
19/04/16-01:00:00:: NOTIFY ORDER:: New position: 0
19/04/16-01:00:00:: NOTIFY TRADE:: PNL: -1.50
19/04/16-01:00:00:: BARLOG      :: lend0:61, lend1:5 - o:7663.25 h:7663.50 l:7662.00 c:7662.25
19/04/16-01:01:00:: BARLOG      :: lend0:62, lend1:5 - o:7662.25 h:7662.75 l:7662.25 c:7662.25
19/04/16-01:02:00:: BARLOG      :: lend0:63, lend1:5 - o:7662.25 h:7662.25 l:7661.75 c:7661.75
19/04/16-01:03:00:: BARLOG      :: lend0:64, lend1:5 - o:7661.75 h:7661.75 l:7661.25 c:7661.50
19/04/16-01:04:00:: BARLOG      :: lend0:65, lend1:5 - o:7661.50 h:7662.25 l:7661.50 c:7662.00

    As you can see from the log the order placement happens at 00:49(HH:MM) or len(self.data0)=50. Then, at 00:50 (len(self.data0)=51) both market order and limit order are submitted and accepted. But here's what I can't figure out:

    Note that for both the limit and the market order I have the parameter data = self.data1

    The market order (which is first in the queue) waits until the new 15m bar-print to execute. This occurs just before 01:00 (len(self.data0)=61 & len(self.data1) increases from 4 to 5). That is the behavior I expect. But the limit order executes, just before the call of next at 00:50 (len(self.data0)=51). It's not waiting for a new bar to print, it's executing inside the current self.data1 bar.

    Why is this order execution behavior different in this way? Is there a way to make the limit order behave as the market order does?

    1
  • A

    @backtrader any insight on this?

    This would also apply if there were two different products.

    For example if I had 1-min NQ data and 5-sec ES data, and I put on a market and limit order at 9:00a in NQ. The limit order would fill based on the 9:00a bar (when 9:00:05 next() is called), but the market order would fill based on the 9:01 bar at 9:01:00 when the next new bar prints.

    0
  • B
    administrators

    Waiting to find time to look into it.

    0
Log in to reply
 
});