Navigation

    Backtrader Community

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

    limit order execute-fill has different behavior than market order in backtest

    General Code/Help
    2
    3
    114
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • A
      algoguy235 last edited by algoguy235

      The limit order fill behavior is different than the market order fill behavior. With a market order, the fill won't happen until the next printed bar. But for limit orders, the fill happens in the current bar.

      I don't understand why these are different. And is there a way I can get the limit orders to behave like the market orders, in terms of waiting until next printed bar to get filled?

      Here is a 1Mb data sample NQ data I use in this example.

      from datetime import timedelta, datetime as Dt
      from types import SimpleNamespace as Sns
      import backtrader as bt
      import pandas as pd
      import os
      
      inst = Sns (symbol     = 'NQ'
                  ,stratName = 'lmt_test'
                  ,startDate = Dt(2019, 4, 16 )
                  ,endDate   = Dt(2019, 4, 17 )
                  ,showlogs  = True
                  ,datafile  = os.path.expanduser('~/')+'dev/datas/'+'NQ_test.txt'
                  )
      
      # Build dataframes
      rawdatacolnames = ['ddate','ttime','open','high','low','close','volume']
      df1 = pd.read_csv(inst.datafile, names=rawdatacolnames)
      df1['dtime'] = df1.ddate+' '+df1.ttime
      df1.dtime = pd.to_datetime(df1.dtime, format = '%m/%d/%Y %H:%M')
      df1 = df1[(df1.dtime >= inst.startDate) & (df1.dtime <= inst.endDate)]
      df1.index = df1.dtime
      df1 = df1[['open','high','low','close']]
      
      ohlc_dict = {'open':'first', 'high':'max', 'low':'min', 'close': 'last'}
      df15 = df1.resample('15Min').agg(ohlc_dict)
      
      # ------------------------- Lmt Test Strategy ---------------------------
      class lmt_test(bt.Strategy): 
          params = dict(inst = dict())
          # ----------------------------------------------------------------------------------#
          def log(self, txt, f_name='', dt=None): 
              dt = dt or self.data.datetime.datetime()
              print(f'{dt.strftime("%y/%m/%d-%H:%M:%S")}:: {f_name:<12}:: {txt}')
          # ----------------------------------------------------------------------------------#
          def barlog(self, d,                     f_name='BARLOG'): 
              self.log(f'lend0:{len(d)}, lend1:{len(self.data1)} - '+
                      f'o:{d.open[0]:.2f} h:{d. high[0]:.2f} '+
                      f'l:{d. low[0]:.2f} c:{d.close[0]:.2f}', f_name)
          # ----------------------------------------------------------------------------------#
          def notify_order(self, order,           f_name='NOTIFY ORDER'): 
              self.log(f'Order {order.info.orderTag}-{order.ref} '+
                       f'@{order.plen}: {order.getstatusname()}', f_name)
              if order.status in [order.Completed]:
                  orderprintstr = f'{["SELL","BUY "][order.isbuy()]} EXECUTED: {order.info.orderTag}: '+\
                                  f'${order.executed.price:.2f}'
                  self.log(orderprintstr,f_name,)
                  self.log('New position: '+str(self.getposition(self.data1).size),f_name)
          # ----------------------------------------------------------------------------------#
          def notify_trade(self, trade,           f_name='NOTIFY TRADE'): 
              if trade.isclosed:
                  self.log(f'PNL: {trade.pnl:.2f}',f_name)
          # ----------------------------------------------------------------------------------#
          def __init__(self):
              if not self.p.inst.showlogs:
                  self.log    = lambda x: None
                  self.barlog = lambda x: None
          # ----------------------------------------------------------------------------------#
          def next(self,                          f_name='NEXT_BB'): 
              self.barlog(self.data0)
      
              if (len(self.data0)-5)%45 == 0:
                  self.placeOrder(len(self.data1)%2, 2)
          # ----------------------------------------------------------------------------------#
          def placeOrder(self, odr, osz,          f_name='PLACE ORDER'): 
              self.log(f'{["Buy","Sell"][odr]} {osz} at {self.data0.close[0]}, '+
                        'Current position = '+str(self.getposition(self.data1).size),f_name)
      
              # Cancel open orders
              openOrders = self.broker.get_orders_open()
              if openOrders: self.cancel(openOrders[0])
      
              # Create new orders
              lmtOrderArgs = dict(data=self.data1, size=1, exectype=bt.Order.Limit, orderTag='Lmt')
              if odr: 
                  self.sell(data=self.data1, orderTag='Mkt')
                  self.sell(**lmtOrderArgs, price = self.data0.close[0])
              else:
                  self.buy (data=self.data1, orderTag='Mkt')
                  self.buy (**lmtOrderArgs, price = self.data0.close[0])
      
      # ------------------------- Run Cerebro Strategy ---------------------------
      print(Dt.now().strftime('%m-%d %H:%M:%S::'),
              'Running:',inst.symbol,'from:',str(inst.startDate)[:-9],'to:',str(inst.endDate)[:-9])
      
      cerebro = bt.Cerebro(runonce=False, stdstats=False)
      
      data0 = bt.feeds.PandasData(dataname = df1)
      cerebro.adddata(data0, 'm1')
      
      data1 = bt.feeds.PandasData(dataname = df15)
      cerebro.adddata(data1, 'm15')
      
      cerebro.addstrategy(eval(inst.stratName),inst=inst)
      
      thestrat = cerebro.run(tradehistory=False)[0]
      
      print('Final Portfolio Value: {:.2f}' .format(cerebro.broker.getvalue()))
      print(Dt.now().strftime('%m-%d %H:%M:%S::'),f'Symbol {inst.symbol} Backtest Complete.')
      

      and here is an excerpt from the logs:

      19/04/16-00:43:00:: BARLOG      :: lend0:44, lend1:3 - o:7662.75 h:7662.75 l:7662.50 c:7662.75
      19/04/16-00:44:00:: BARLOG      :: lend0:45, lend1:3 - o:7662.75 h:7662.75 l:7662.50 c:7662.50
      19/04/16-00:45:00:: BARLOG      :: lend0:46, lend1:4 - o:7662.50 h:7663.00 l:7662.50 c:7662.50
      19/04/16-00:46:00:: BARLOG      :: lend0:47, lend1:4 - o:7662.50 h:7662.50 l:7662.00 c:7662.00
      19/04/16-00:47:00:: BARLOG      :: lend0:48, lend1:4 - o:7662.00 h:7662.00 l:7661.75 c:7661.75
      19/04/16-00:48:00:: BARLOG      :: lend0:49, lend1:4 - o:7661.75 h:7662.00 l:7661.50 c:7661.75
      19/04/16-00:49:00:: BARLOG      :: lend0:50, lend1:4 - o:7661.75 h:7662.50 l:7661.75 c:7662.25
      19/04/16-00:49:00:: PLACE ORDER :: Buy 2 at 7662.25, Current position = -2
      19/04/16-00:50:00:: NOTIFY ORDER:: Order Mkt-3 @4: Submitted
      19/04/16-00:50:00:: NOTIFY ORDER:: Order Lmt-4 @4: Submitted
      19/04/16-00:50:00:: NOTIFY ORDER:: Order Mkt-3 @4: Accepted
      19/04/16-00:50:00:: NOTIFY ORDER:: Order Lmt-4 @4: Accepted
      19/04/16-00:50:00:: NOTIFY ORDER:: Order Lmt-4 @4: Completed
      19/04/16-00:50:00:: NOTIFY ORDER:: BUY  EXECUTED: Lmt: $7662.25
      19/04/16-00:50:00:: NOTIFY ORDER:: New position: -1
      19/04/16-00:50:00:: BARLOG      :: lend0:51, lend1:4 - o:7662.25 h:7662.25 l:7662.00 c:7662.00
      19/04/16-00:51:00:: BARLOG      :: lend0:52, lend1:4 - o:7662.00 h:7662.50 l:7662.00 c:7662.50
      19/04/16-00:52:00:: BARLOG      :: lend0:53, lend1:4 - o:7662.50 h:7663.00 l:7662.50 c:7662.75
      19/04/16-00:53:00:: BARLOG      :: lend0:54, lend1:4 - o:7662.75 h:7663.50 l:7662.75 c:7663.50
      19/04/16-00:54:00:: BARLOG      :: lend0:55, lend1:4 - o:7663.50 h:7663.75 l:7663.00 c:7663.00
      19/04/16-00:55:00:: BARLOG      :: lend0:56, lend1:4 - o:7663.00 h:7663.00 l:7662.50 c:7662.75
      19/04/16-00:56:00:: BARLOG      :: lend0:57, lend1:4 - o:7663.00 h:7663.00 l:7663.00 c:7663.00
      19/04/16-00:57:00:: BARLOG      :: lend0:58, lend1:4 - o:7663.00 h:7663.50 l:7663.00 c:7663.00
      19/04/16-00:58:00:: BARLOG      :: lend0:59, lend1:4 - o:7663.00 h:7663.25 l:7663.00 c:7663.25
      19/04/16-00:59:00:: BARLOG      :: lend0:60, lend1:4 - o:7663.25 h:7663.50 l:7663.00 c:7663.25
      19/04/16-01:00:00:: NOTIFY ORDER:: Order Mkt-3 @4: Completed
      19/04/16-01:00:00:: NOTIFY ORDER:: BUY  EXECUTED: Mkt: $7663.25
      19/04/16-01:00:00:: NOTIFY ORDER:: New position: 0
      19/04/16-01:00:00:: NOTIFY TRADE:: PNL: -1.50
      19/04/16-01:00:00:: BARLOG      :: lend0:61, lend1:5 - o:7663.25 h:7663.50 l:7662.00 c:7662.25
      19/04/16-01:01:00:: BARLOG      :: lend0:62, lend1:5 - o:7662.25 h:7662.75 l:7662.25 c:7662.25
      19/04/16-01:02:00:: BARLOG      :: lend0:63, lend1:5 - o:7662.25 h:7662.25 l:7661.75 c:7661.75
      19/04/16-01:03:00:: BARLOG      :: lend0:64, lend1:5 - o:7661.75 h:7661.75 l:7661.25 c:7661.50
      19/04/16-01:04:00:: BARLOG      :: lend0:65, lend1:5 - o:7661.50 h:7662.25 l:7661.50 c:7662.00
      

      As you can see from the log the order placement happens at 00:49(HH:MM) or len(self.data0)=50. Then, at 00:50 (len(self.data0)=51) both market order and limit order are submitted and accepted. But here's what I can't figure out:

      Note that for both the limit and the market order I have the parameter data = self.data1

      The market order (which is first in the queue) waits until the new 15m bar-print to execute. This occurs just before 01:00 (len(self.data0)=61 & len(self.data1) increases from 4 to 5). That is the behavior I expect. But the limit order executes, just before the call of next at 00:50 (len(self.data0)=51). It's not waiting for a new bar to print, it's executing inside the current self.data1 bar.

      Why is this order execution behavior different in this way? Is there a way to make the limit order behave as the market order does?

      1 Reply Last reply Reply Quote 1
      • A
        algoguy235 last edited by algoguy235

        @backtrader any insight on this?

        This would also apply if there were two different products.

        For example if I had 1-min NQ data and 5-sec ES data, and I put on a market and limit order at 9:00a in NQ. The limit order would fill based on the 9:00a bar (when 9:00:05 next() is called), but the market order would fill based on the 9:01 bar at 9:01:00 when the next new bar prints.

        1 Reply Last reply Reply Quote 0
        • B
          backtrader administrators last edited by

          Waiting to find time to look into it.

          1 Reply Last reply Reply Quote 0
          • 1 / 1
          • First post
            Last post
          Copyright © 2016, 2017, 2018 NodeBB Forums | Contributors
          $(document).ready(function () { app.coldLoad(); }); }