limit order execute-fill has different behavior than market order in backtest
-
The limit order fill behavior is different than the market order fill behavior. With a market order, the fill won't happen until the next printed bar. But for limit orders, the fill happens in the current bar.
I don't understand why these are different. And is there a way I can get the limit orders to behave like the market orders, in terms of waiting until next printed bar to get filled?
Here is a 1Mb data sample NQ data I use in this example.
from datetime import timedelta, datetime as Dt from types import SimpleNamespace as Sns import backtrader as bt import pandas as pd import os inst = Sns (symbol = 'NQ' ,stratName = 'lmt_test' ,startDate = Dt(2019, 4, 16 ) ,endDate = Dt(2019, 4, 17 ) ,showlogs = True ,datafile = os.path.expanduser('~/')+'dev/datas/'+'NQ_test.txt' ) # Build dataframes rawdatacolnames = ['ddate','ttime','open','high','low','close','volume'] df1 = pd.read_csv(inst.datafile, names=rawdatacolnames) df1['dtime'] = df1.ddate+' '+df1.ttime df1.dtime = pd.to_datetime(df1.dtime, format = '%m/%d/%Y %H:%M') df1 = df1[(df1.dtime >= inst.startDate) & (df1.dtime <= inst.endDate)] df1.index = df1.dtime df1 = df1[['open','high','low','close']] ohlc_dict = {'open':'first', 'high':'max', 'low':'min', 'close': 'last'} df15 = df1.resample('15Min').agg(ohlc_dict) # ------------------------- Lmt Test Strategy --------------------------- class lmt_test(bt.Strategy): params = dict(inst = dict()) # ----------------------------------------------------------------------------------# def log(self, txt, f_name='', dt=None): dt = dt or self.data.datetime.datetime() print(f'{dt.strftime("%y/%m/%d-%H:%M:%S")}:: {f_name:<12}:: {txt}') # ----------------------------------------------------------------------------------# def barlog(self, d, f_name='BARLOG'): self.log(f'lend0:{len(d)}, lend1:{len(self.data1)} - '+ f'o:{d.open[0]:.2f} h:{d. high[0]:.2f} '+ f'l:{d. low[0]:.2f} c:{d.close[0]:.2f}', f_name) # ----------------------------------------------------------------------------------# def notify_order(self, order, f_name='NOTIFY ORDER'): self.log(f'Order {order.info.orderTag}-{order.ref} '+ f'@{order.plen}: {order.getstatusname()}', f_name) if order.status in [order.Completed]: orderprintstr = f'{["SELL","BUY "][order.isbuy()]} EXECUTED: {order.info.orderTag}: '+\ f'${order.executed.price:.2f}' self.log(orderprintstr,f_name,) self.log('New position: '+str(self.getposition(self.data1).size),f_name) # ----------------------------------------------------------------------------------# def notify_trade(self, trade, f_name='NOTIFY TRADE'): if trade.isclosed: self.log(f'PNL: {trade.pnl:.2f}',f_name) # ----------------------------------------------------------------------------------# def __init__(self): if not self.p.inst.showlogs: self.log = lambda x: None self.barlog = lambda x: None # ----------------------------------------------------------------------------------# def next(self, f_name='NEXT_BB'): self.barlog(self.data0) if (len(self.data0)-5)%45 == 0: self.placeOrder(len(self.data1)%2, 2) # ----------------------------------------------------------------------------------# def placeOrder(self, odr, osz, f_name='PLACE ORDER'): self.log(f'{["Buy","Sell"][odr]} {osz} at {self.data0.close[0]}, '+ 'Current position = '+str(self.getposition(self.data1).size),f_name) # Cancel open orders openOrders = self.broker.get_orders_open() if openOrders: self.cancel(openOrders[0]) # Create new orders lmtOrderArgs = dict(data=self.data1, size=1, exectype=bt.Order.Limit, orderTag='Lmt') if odr: self.sell(data=self.data1, orderTag='Mkt') self.sell(**lmtOrderArgs, price = self.data0.close[0]) else: self.buy (data=self.data1, orderTag='Mkt') self.buy (**lmtOrderArgs, price = self.data0.close[0]) # ------------------------- Run Cerebro Strategy --------------------------- print(Dt.now().strftime('%m-%d %H:%M:%S::'), 'Running:',inst.symbol,'from:',str(inst.startDate)[:-9],'to:',str(inst.endDate)[:-9]) cerebro = bt.Cerebro(runonce=False, stdstats=False) data0 = bt.feeds.PandasData(dataname = df1) cerebro.adddata(data0, 'm1') data1 = bt.feeds.PandasData(dataname = df15) cerebro.adddata(data1, 'm15') cerebro.addstrategy(eval(inst.stratName),inst=inst) thestrat = cerebro.run(tradehistory=False)[0] print('Final Portfolio Value: {:.2f}' .format(cerebro.broker.getvalue())) print(Dt.now().strftime('%m-%d %H:%M:%S::'),f'Symbol {inst.symbol} Backtest Complete.')and here is an excerpt from the logs:
19/04/16-00:43:00:: BARLOG :: lend0:44, lend1:3 - o:7662.75 h:7662.75 l:7662.50 c:7662.75 19/04/16-00:44:00:: BARLOG :: lend0:45, lend1:3 - o:7662.75 h:7662.75 l:7662.50 c:7662.50 19/04/16-00:45:00:: BARLOG :: lend0:46, lend1:4 - o:7662.50 h:7663.00 l:7662.50 c:7662.50 19/04/16-00:46:00:: BARLOG :: lend0:47, lend1:4 - o:7662.50 h:7662.50 l:7662.00 c:7662.00 19/04/16-00:47:00:: BARLOG :: lend0:48, lend1:4 - o:7662.00 h:7662.00 l:7661.75 c:7661.75 19/04/16-00:48:00:: BARLOG :: lend0:49, lend1:4 - o:7661.75 h:7662.00 l:7661.50 c:7661.75 19/04/16-00:49:00:: BARLOG :: lend0:50, lend1:4 - o:7661.75 h:7662.50 l:7661.75 c:7662.25 19/04/16-00:49:00:: PLACE ORDER :: Buy 2 at 7662.25, Current position = -2 19/04/16-00:50:00:: NOTIFY ORDER:: Order Mkt-3 @4: Submitted 19/04/16-00:50:00:: NOTIFY ORDER:: Order Lmt-4 @4: Submitted 19/04/16-00:50:00:: NOTIFY ORDER:: Order Mkt-3 @4: Accepted 19/04/16-00:50:00:: NOTIFY ORDER:: Order Lmt-4 @4: Accepted 19/04/16-00:50:00:: NOTIFY ORDER:: Order Lmt-4 @4: Completed 19/04/16-00:50:00:: NOTIFY ORDER:: BUY EXECUTED: Lmt: $7662.25 19/04/16-00:50:00:: NOTIFY ORDER:: New position: -1 19/04/16-00:50:00:: BARLOG :: lend0:51, lend1:4 - o:7662.25 h:7662.25 l:7662.00 c:7662.00 19/04/16-00:51:00:: BARLOG :: lend0:52, lend1:4 - o:7662.00 h:7662.50 l:7662.00 c:7662.50 19/04/16-00:52:00:: BARLOG :: lend0:53, lend1:4 - o:7662.50 h:7663.00 l:7662.50 c:7662.75 19/04/16-00:53:00:: BARLOG :: lend0:54, lend1:4 - o:7662.75 h:7663.50 l:7662.75 c:7663.50 19/04/16-00:54:00:: BARLOG :: lend0:55, lend1:4 - o:7663.50 h:7663.75 l:7663.00 c:7663.00 19/04/16-00:55:00:: BARLOG :: lend0:56, lend1:4 - o:7663.00 h:7663.00 l:7662.50 c:7662.75 19/04/16-00:56:00:: BARLOG :: lend0:57, lend1:4 - o:7663.00 h:7663.00 l:7663.00 c:7663.00 19/04/16-00:57:00:: BARLOG :: lend0:58, lend1:4 - o:7663.00 h:7663.50 l:7663.00 c:7663.00 19/04/16-00:58:00:: BARLOG :: lend0:59, lend1:4 - o:7663.00 h:7663.25 l:7663.00 c:7663.25 19/04/16-00:59:00:: BARLOG :: lend0:60, lend1:4 - o:7663.25 h:7663.50 l:7663.00 c:7663.25 19/04/16-01:00:00:: NOTIFY ORDER:: Order Mkt-3 @4: Completed 19/04/16-01:00:00:: NOTIFY ORDER:: BUY EXECUTED: Mkt: $7663.25 19/04/16-01:00:00:: NOTIFY ORDER:: New position: 0 19/04/16-01:00:00:: NOTIFY TRADE:: PNL: -1.50 19/04/16-01:00:00:: BARLOG :: lend0:61, lend1:5 - o:7663.25 h:7663.50 l:7662.00 c:7662.25 19/04/16-01:01:00:: BARLOG :: lend0:62, lend1:5 - o:7662.25 h:7662.75 l:7662.25 c:7662.25 19/04/16-01:02:00:: BARLOG :: lend0:63, lend1:5 - o:7662.25 h:7662.25 l:7661.75 c:7661.75 19/04/16-01:03:00:: BARLOG :: lend0:64, lend1:5 - o:7661.75 h:7661.75 l:7661.25 c:7661.50 19/04/16-01:04:00:: BARLOG :: lend0:65, lend1:5 - o:7661.50 h:7662.25 l:7661.50 c:7662.00As you can see from the log the order placement happens at
00:49(HH:MM) orlen(self.data0)=50. Then, at00:50(len(self.data0)=51) both market order and limit order are submitted and accepted. But here's what I can't figure out:Note that for both the limit and the market order I have the parameter
data = self.data1The market order (which is first in the queue) waits until the new 15m bar-print to execute. This occurs just before
01:00(len(self.data0)=61&len(self.data1)increases from 4 to 5). That is the behavior I expect. But the limit order executes, just before the call ofnextat00:50(len(self.data0)=51). It's not waiting for a new bar to print, it's executing inside the currentself.data1bar.Why is this order execution behavior different in this way? Is there a way to make the limit order behave as the market order does?
-
@backtrader any insight on this?
This would also apply if there were two different products.
For example if I had 1-min NQ data and 5-sec ES data, and I put on a market and limit order at 9:00a in NQ. The limit order would fill based on the 9:00a bar (when 9:00:05 next() is called), but the market order would fill based on the 9:01 bar at 9:01:00 when the next new bar prints.
-
Waiting to find time to look into it.