Stop order getting filled on same minute it's entered without cheat

algoguy235

When my strategy's trigger gets hit, I create a market order and a trailing stop.

For example at 10:00a I enter a long market order and a short trailing stop - let's say the close price at 10a is 2700 my trailstop amount is 15. If the 10a candle has a low of 2685 or lower, the trailing stop will trigger ON the 10a candle. Despite that the market order doesn't fill until 10:01a (since I'm using 1m data). One of the strange artifacts is that the position actually goes to -1, and then back to 0 when the market order fills.

Is this normal behavior? Shouldn't it be that the stop order doesn't get an opportunity to fill until 10:01a, just like the market order.

I don't have any cheat on open or cheat on close. Has anyone seen this before?

backtrader

@algoguy235 said in Stop order getting filled on same minute it's entered without cheat:

Is this normal behavior?

We don't know. No data, no code, no logs, no bullet points.

The most likely explanation:

@algoguy235 said in Stop order getting filled on same minute it's entered without cheat:

One of the strange artifacts is that the position actually goes to -1, and then back to 0 when the market order fills.

Your stop is obviously getting filled, you are shorting the market and the closing the position with your market order. Which means that your logic to enter the market is flawed.

algoguy235

I've narrowed down the problem: My input data are 1m candles. In addition, I have an upsampled "dummy" data-feed - it simply replay's the candle 3 times per minute - that I use for a special indicator. In the code below there is data_1m and data_20s. I want all the logic including the trailing stop to operate from the 1m data in the strat code is self.data1. However, the trailing stop seems to be getting filled based on self.data0 as you can see from the logs, even though I specified data=self.data1 when I created the order.

import backtrader as bt
import pandas as pd
from datetime import timedelta, datetime as Dt
from types import SimpleNamespace as Sns

inst = Sns( filename      = 'datas/NQ_test.txt'
           ,startDate     = Dt(2019, 5, 1)
           ,endDate       = Dt(2019, 5, 6)
           ,showlogs      = True
           ,trailstop_int = 10
           )

# ************ Test Strat ************
class test(bt.Strategy): 
    params = dict(showlogs = True, ts_int = 10)
    # ----------------------------------------------------------------------------------#
    def Log(self, txt, f_name = '', dt = None): 
        if self.p.showlogs:
            dt = dt or self.data.datetime.datetime()
            print(f'{dt.strftime("%y/%m/%d-%H:%M:%S")}:: {f_name:<12}:: {txt}')
    # ----------------------------------------------------------------------------------#
    def candle_log(self,                    f_name='CAND_LOG'): 
        self.Log(f'o:{self.data1.open[0]:.2f} h:{self.data1. high[0]:.2f} '+
                 f'l:{self.data1. low[0]:.2f} c:{self.data1.close[0]:.2f}', f_name)     
    # ----------------------------------------------------------------------------------#
    def __init__(self): 
        self.new_1m_print = self.data1.datetime() != self.data1.datetime()(-1)
    # ----------------------------------------------------------------------------------#
    def next(self,                          f_name='NEXT'): 
        # self.candle_log()
        if self.new_1m_print:
            if   (((self.data1.close[0] - self.data1.low[0]) > self.p.ts_int) and 
                            (self.getposition(self.data1).size <= 0)):
                self.positionB = self.getposition(self.data1).size
                odr=0 # Order Direction is BUY
                osz = abs(self.positionB) + 1
                self.placeOrder(odr, osz)
            
            elif (((self.data1.high[0] - self.data1.open[0]) > self.p.ts_int) and
                            (self.getposition(self.data1).size >= 0)):
                self.positionB = self.getposition(self.data1).size
                odr=1 # Order Direction is SELL
                osz = abs(self.positionB) + 1
                self.placeOrder(odr, osz)
    # ----------------------------------------------------------------------------------#
    def placeOrder(self, odr, osz,          f_name = 'PLACE ORDER'):
        self.Log(f'{["Buy","Sell"][odr]} {osz}',f_name)

        # Cancel open orders
        openOrders = self.broker.get_orders_open()
        if openOrders: 
            for o in openOrders:
                self.cancel(o)

        # Create market orders
        orderArgs = dict(data = self.data1, orderTag = 'Entry', size = osz)
        if odr: 
            self.sell(**orderArgs)
        else:   
            self.buy (**orderArgs)
            
        # Create trailing orders
        orderArgs = dict(data = self.data1, orderTag = 'TrailStop', 
                         trailamount = self.p.ts_int + 1, exectype = bt.Order.StopTrail)
        if odr: 
            self.buy (**orderArgs)
        else: 
            self.sell(**orderArgs)
    # ----------------------------------------------------------------------------------#
    def notify_order(self, order,           f_name='NOTIFY ORDER'): 
        # self.Log(f'Order {order.ref} Status: {order.getstatusname()}', f_name)
        if order.status in [order.Completed]:
            if order.isbuy():
                self.Log(f'BUY  EXECUTED {order.info.orderTag}, '+
                         f'Price: {order.executed.price:.2f}', f_name,)
            else:  # is Sell
                self.Log(f'SELL EXECUTED {order.info.orderTag}, '+
                         f'Price: {order.executed.price:.2f}', f_name,)
            self.Log('New position: '+str(self.getposition(self.data1).size),f_name)
    # ----------------------------------------------------------------------------------#
    def notify_trade(self, trade,           f_name='NOTIFY TRADE'): 
        if trade.isclosed:
            self.Log(f'PNL: {trade.pnl:.2f}', f_name)

# ************  RUN  ************
def upsample(): 
    rawdatacolnames = ['ddate','ttime','open','high','low','close','volume']
    dfraw = pd.read_csv(inst.filename, names=rawdatacolnames)
    dfraw['dtime'] = dfraw.ddate+' '+dfraw.ttime
    dfraw.dtime = pd.to_datetime(dfraw.dtime,format='%m/%d/%Y %H:%M')
    dfraw = dfraw[(dfraw.dtime >= inst.startDate) & (dfraw.dtime <= inst.endDate)]
    dfraw.set_index('dtime',drop=True,inplace=True)
    dfraw = dfraw[['open','high','low','close','volume']]
    df_upsamp = dfraw.resample('20s').ffill()
    return dfraw, df_upsamp

dfraw1m, df_upsamp = upsample()

cerebro = bt.Cerebro(runonce=False)

data_20s = bt.feeds.PandasData(dataname = df_upsamp, 
                               timeframe = bt.TimeFrame.Seconds, 
                               compression = 20)
cerebro.adddata(data_20s)

data_1m = bt.feeds.PandasData(dataname = dfraw1m, 
                              timeframe = bt.TimeFrame.Minutes)
cerebro.adddata(data_1m)

cerebro.addstrategy(test,showlogs = inst.showlogs,ts_int = inst.trailstop_int)

cerebro.run()

And here is a sample of the logs that shows the problem:

19/05/01-07:00:00:: PLACE ORDER :: Buy 1
19/05/01-07:01:00:: NOTIFY ORDER:: BUY  EXECUTED Entry, Price: 7835.75
19/05/01-07:01:00:: NOTIFY ORDER:: New position: 1
19/05/01-07:25:00:: NOTIFY ORDER:: SELL EXECUTED TrailStop, Price: 7842.00
19/05/01-07:25:00:: NOTIFY ORDER:: New position: 0
19/05/01-07:25:00:: NOTIFY TRADE:: PNL: 6.25
19/05/01-10:00:00:: PLACE ORDER :: Sell 1
19/05/01-10:01:00:: NOTIFY ORDER:: SELL EXECUTED Entry, Price: 7842.25
19/05/01-10:01:00:: NOTIFY ORDER:: New position: -1
19/05/01-10:56:00:: NOTIFY ORDER:: BUY  EXECUTED TrailStop, Price: 7844.00
19/05/01-10:56:00:: NOTIFY ORDER:: New position: 0
19/05/01-10:56:00:: NOTIFY TRADE:: PNL: -1.75
19/05/01-11:04:00:: PLACE ORDER :: Buy 1
19/05/01-11:04:20:: NOTIFY ORDER:: SELL EXECUTED TrailStop, Price: 7849.25
19/05/01-11:04:20:: NOTIFY ORDER:: New position: -1
19/05/01-11:05:00:: NOTIFY ORDER:: BUY  EXECUTED Entry, Price: 7860.75
19/05/01-11:05:00:: NOTIFY ORDER:: New position: 0
19/05/01-11:05:00:: NOTIFY TRADE:: PNL: -11.50
19/05/01-11:50:00:: PLACE ORDER :: Buy 1
19/05/01-11:50:20:: NOTIFY ORDER:: SELL EXECUTED TrailStop, Price: 7818.75
19/05/01-11:50:20:: NOTIFY ORDER:: New position: -1
19/05/01-11:51:00:: NOTIFY ORDER:: BUY  EXECUTED Entry, Price: 7830.00
19/05/01-11:51:00:: NOTIFY ORDER:: New position: 0
19/05/01-11:51:00:: NOTIFY TRADE:: PNL: -11.25
19/05/02-06:32:00:: PLACE ORDER :: Sell 1
19/05/02-06:33:00:: NOTIFY ORDER:: SELL EXECUTED Entry, Price: 7769.00
19/05/02-06:33:00:: NOTIFY ORDER:: New position: -1
19/05/02-06:35:00:: NOTIFY ORDER:: BUY  EXECUTED TrailStop, Price: 7769.50
19/05/02-06:35:00:: NOTIFY ORDER:: New position: 0
19/05/02-06:35:00:: NOTIFY TRADE:: PNL: -0.50

So as you can see e.g. in the 3rd trade and the 4th trade, the trailing stop is getting filled in the 20 second mark in-between minutes. I would like to figure out how to prevent that, so I'm only getting filled on the 1m data that is specified.

One interesting aspect is that it looks like the Market order respects data= parameter, but not the StopTrail.

For completeness, here is the 1Mb NQ data I use in this example.

Developing Coder

Try using parent parameter, it will tag the child order for scheduling when the parent order has been executed. Something like this might work:

        # Create market orders
        orderArgs = dict(data=self.data1, orderTag='Entry', size=osz)
        if odr:
            entry_order = self.sell(**orderArgs)
        else:
            entry_order = self.buy(**orderArgs)

        # Create trailing orders
        orderArgs = dict(data=self.data1, orderTag='TrailStop',
                         trailamount=self.p.ts_int + 1, exectype=bt.Order.StopTrail,
                         parent=entry_order)
        if odr:
            self.buy(**orderArgs)
        else:
            self.sell(**orderArgs)

More info: https://www.backtrader.com/blog/posts/2018-02-01-stop-trading/stop-trading.html

p.s. I'm not sure it's a good idea to up-sample your data. In effect, your are giving the system an option to cheat by duplicating candles. For example, if you see a candle with new dimensions then you can predict the next two candles with perfect accuracy.

algoguy235

Yass. Thank you @Developing-Coder this solves my issue. To get it working properly I had to add transmit=False to market order, after that it worked as desired. Thank you!