Custom Dynamic trailstop indicator stops after updating tradeopen parameter

[stevenm100]

Hi,
I had originally written this for a multidata strategy, and it didnt work as expected, so i wrote a single data strategy and it still didnt work, so wondering if anyone else has run into this...

I have wanted an ATR trailing stop for a while, and can make it work using the strategy 'next', but wanted a custom indicator because then I can make it print the line on the plot. I notice that nobody has shared a custom trailingATR indicator anywhere, so had a go myself.

I added some print() statements to help me debug:

class TrailStop(bt.Indicator):
    lines = ('trailatr','trailstop',)
    params = ( ('tradeopen',False),('atr_period', 10),('trail_mult', 4),)
    plotinfo = dict(subplot=False)
    plotlines = dict(trailstop=dict(color='red', ls='--',_plotskip=False,),  
                        trailatr=dict(color='red', ls='-', _plotskip=True),
                    )         

    def init(self):
        self.l.trailatr = bt.ind.AverageTrueRange(period=self.p.atr_period)#self.p.atr_period)

    def next(self):
        print('from inside indicator next, tradeopen param = {}'.format(self.p.tradeopen)) # unable to access next when changing the param????
        if self.p.tradeopen == True:
            self.lines.trailstop[0] = max(self.data.close[0], self.lines.trailstop[-1])
            print('inside indicator if statement')
            print(self.l.trailstop[0])
        else:
            self.lines.trailstop[0] = 0
            print('in indicator else statement')
            print(self.l.trailstop[0])

I instatiate the indicator in the strategy 'init':

self.trailstop = TrailStop()
self.l.trailatr = self.trailstop.trailstop

note that the max statement is just a dummy to help me understand what was going on, the value is zero (i know from printing that too). The story continues....
When I change the indicators parameter from the strategy 'next' like this:

self.trailstop.p.tradeopen = True

The printing stops and i get nothing from the indicator 'next' or from the strategy 'next' but the strategy runs through the data, reaches 'stop' and plots the chart at the end.

In particular, I am puzzled why the indicator seems not to be accessing the IF statement or the ELSE statement after the parameter update.

A snippet of the many lines of output looks like this:

from inside indicator next, tradeopen param = False
in indicator else statement
from inside indicator next, tradeopen param = False
in indicator else statement
Trailstop parameter set to True (inside strategy next statement)
15:15:00, NIO, BUY EXECUTED, Price: 43.12, Size: 383.00, Cost: 16514.96

and that is where the printing stops.

Any ideas what is going on, or has anyone else seen this? This is my first go at a dynamic indicator, and if i cant get it to work i'll have to live without seeing the trailstop on the chart.
Failing that, has anyone got a trailingATR custom indicator to work?

[run-out]

@stevenm100 Please include your entire code.

[stevenm100]

@run-out

sorry for not posting the whole thing.....Ive cut it down to something easier to understand and follow.

Ive added some debugging print statements to follow along.

The code attempts to use the custom indicator (copying the idea in the dynamic highest example, setting the param to True/False, 0 otherwise), and also calculating the stop price in the strategy next to show that it works outside of the custom indicator.
note that if you copy/paste it directly after one week, you'll need to update the dates in the yfinance statement as it only goes back 7 days.

I learned that the indicator runs through all of the data before the strategy does (printing len shows 388 from next in the indicator before the strategy next begins printing), and so the title is probably now incorrect

but the thing I was trying to achieve still remains...how do I set the indicator parameter to True from the strategy? it doesnt seem to work for me as it does in the Dynamic Indicator example.

I understand that I cant see the self.l.trailstop line on the chart because it is zero throughout (according to strategy print, it is nan according to indicator print), and so outside of the plotted area. I had wanted to see the trailstop appear in the visible range during the period that the param is set to True.

Any thoughts or suggestions on why the custom indicator isnt working?

representative output:

***INDICATOR NEXT self.l.trailatr[0] = nan, tradeopen param = False, at len: 385***
***INDICATOR NEXT self.l.trailatr[0] = nan, tradeopen param = False, at len: 386***
***INDICATOR NEXT self.l.trailatr[0] = nan, tradeopen param = False, at len: 387***
***INDICATOR NEXT self.l.trailatr[0] = nan, tradeopen param = False, at len: 388***
***STRAT NEXT self.TrailStop.trailstop[0] = 0.0, at len: 22***
******BUYING, SET PARAM TRUE*****
stop price = 142.62
***STRAT NEXT self.TrailStop.trailstop[0] = 0.0, at len: 23***
stop price updated = 142.78
***STRAT NEXT self.TrailStop.trailstop[0] = 0.0, at len: 24***
stop price updated = 142.89
***STRAT NEXT self.TrailStop.trailstop[0] = 0.0, at len: 25***
***STRAT NEXT self.TrailStop.trailstop[0] = 0.0, at len: 26***

import backtrader.feeds as btfeed
import pandas as pd
import backtrader as bt
from pandas_datareader import data as pdr
import yfinance as yf

yf.pdr_override()

class TrailStop(bt.Indicator):
    lines = ('trailatr','trailstop',)
    params = ( ('tradeopen',False),('atr_period', 10),('trail_mult', 4),)
    plotinfo = dict(subplot=False)
    plotlines = dict(trailstop=dict(color='blue', ls='--',_plotskip=False,),  
                        trailatr=dict(color='black', ls='-', _plotskip=False),
                    )         

    def init(self):
        self.l.trailatr = bt.indicators.AverageTrueRange(period=self.p.atr_period)

    def next(self):
        print('***INDICATOR NEXT self.l.trailatr[0] = {}, tradeopen param = {}, at len: {}***'.format(self.l.trailatr[0], self.p.tradeopen, len(self.l.trailatr)))
        
        if self.p.tradeopen == True: # using "if True:" this gets accessed and result is self.l.trailstop[0] = 1. suggests uim using the wrong way to access param?
            self.l.trailstop[0] = max(self.dataclose[0] - self.p.trail_mult * self.atr[0], self.l.trailstop[-1])
            #print('inside indicator if statement')

        else:
            self.l.trailstop[0] = min(0,1)
            #print('in indicator else statement')

 
# Create a Stratey
class TestStrategy(bt.Strategy):
    params = (
        ('fast_ma',20),
        ('trail_mult', 4),
    )

    def log(self, txt, dt=None): 
        ''' Logging function fot this strategy'''
        dt = dt or self.datas[0].datetime.date(0)
        #print('%s, %s' % (dt.isoformat(), txt)) # #-out to turn logging off

    def __init__(self):
        # Keep a reference to the "close" line in the data[0] dataseries
        self.dataclose = self.datas[0].close

        self.atr = bt.indicators.AverageTrueRange(self.datas[0])    # using for manually calculating exit in strategy next
        self.stopprice = 0                                          # for manually working out the stop in strategy next

        self.TrailStop = TrailStop(self.datas[0])                   # instantiate the TrailStop Class

        # To keep track of pending orders and buy price/commission
        self.order = None
        self.buyprice = None
        self.buycomm = None

        # Add a MovingAverageSimple indicator
        self.sma = bt.indicators.ExponentialMovingAverage(self.datas[0].close, period=self.p.fast_ma)

        self.buysig = bt.indicators.AllN(self.datas[0].low > self.sma, period=3)
        

    def notify_order(self, order):
        if order.status in [order.Submitted, order.Accepted]:
            # Buy/Sell order submitted/accepted to/by broker - Nothing to do
            return

        # Check if an order has been completed
        # Attention: broker could reject order if not enough cash
        if order.status in [order.Completed]:
            if order.isbuy():
                self.log(
                    'BUY EXECUTED, Price: %.2f, Cost: %.2f, Comm %.2f' %
                    (order.executed.price,
                     order.executed.value,
                     order.executed.comm))

                self.buyprice = order.executed.price
                self.buycomm = order.executed.comm
            else:  # Sell
                self.log('SELL EXECUTED, Price: %.2f, Cost: %.2f, Comm %.2f' %
                         (order.executed.price,
                          order.executed.value,
                          order.executed.comm))

            # Keep track of which bar execution took place
            self.bar_executed = len(self)

        elif order.status in [order.Canceled, order.Margin, order.Rejected]:
            self.log('Order Canceled/Margin/Rejected')

        # Write down: no pending order
        self.order = None

    def notify_trade(self, trade):
        if not trade.isclosed:
            return

        self.log('OPERATION PROFIT, GROSS %.2f, NET %.2f' %
                 (trade.pnl, trade.pnlcomm))

    def next(self):
        # Simply log the closing price of the series from the reference
        self.log('Close, %.2f' % self.dataclose[0])

        print('***STRAT NEXT self.TrailStop.trailstop[0] = {}, at len: {}***'.format(self.TrailStop.trailstop[0], len(self.datas[0])))

        # Check if an order is pending ... if yes, we cannot send a 2nd one
        if self.order:
            return

        # Check if we are in the market
        if not self.position:

            # Not yet ... we MIGHT BUY if ...
            if self.buysig:
                # BUY, BUY, BUY!!! (with default parameters)
                self.log('BUY CREATE, %.2f' % self.dataclose[0])

                # Keep track of the created order to avoid a 2nd order
                self.order = self.buy()
                self.TrailStop.p.tradeopen = True
                print('******BUYING, SET PARAM TRUE*****')
                self.stopprice = self.data.close[0] - self.p.trail_mult * self.atr[0]
                print('stop price = {:.2f}'.format(self.stopprice))

        elif self.data.close[0] <  self.stopprice :
            self.close()
            self.stopprice = 0
            self.TrailStop.p.tradeopen = False
            print('******SELLING, SET PARAM FALSE*****')


        if self.stopprice < self.dataclose[0] - self.p.trail_mult * self.atr[0]:    # if old price < new price
            self.stopprice = self.dataclose[0] - self.p.trail_mult * self.atr[0]    # assign new price
            print('stop price updated = {:.2f}'.format(self.stopprice))


starting_balance=50000


if __name__ == '__main__':
    
    ticker_id       =   'AAPL'
    
    dataframe = pdr.get_data_yahoo(ticker_id, period='1d', interval='1m', start='2021-07-20',end='2021-07-21',prepost=False)

    dataframe = dataframe[dataframe.index.strftime('%H:%M') < '20:00']
    data = bt.feeds.PandasData(dataname=dataframe)

    # Create a cerebro entity, add strategy
    cerebro = bt.Cerebro()
    cerebro.addstrategy(TestStrategy)

    # Add the Data Feed to Cerebro, set desired cash, set desired commission
    cerebro.adddata(data)
    cerebro.broker.setcash(starting_balance)
    cerebro.broker.setcommission(commission=0.0)

    # Add a FixedSize sizer according to the stake
    cerebro.addsizer(bt.sizers.PercentSizer, percents=10)

    # Run over everything
    cerebro.run()
    cerebro.plot(style='candlestick')

[run-out]

@stevenm100 In order to match the dynamic indicator article, you will need to make a number of changes.

1. Change your TrailStop init to include dunders:

def __init__(self):

2. Indicator next: Modify your atr and data close to the following:

self.l.trailstop[0] = max(
    self.datas[0].close[0] - self.p.trail_mult * self.l.trailatr[0],
    self.l.trailstop[-1],
)

If you wish to add in the parameter manually, add tradeopen to your params in strategy and you can then feed this into the indicator TradeStop.

class TestStrategy(bt.Strategy):
    params = (
        ("fast_ma", 20),
        ("trail_mult", 4),
        ("tradeopen", True),
    )
...
self.TrailStop = TrailStop(
    self.datas[0], tradeopen=self.p.tradeopen
    )

To dynamically update, you can add the trade open boolean in notify_trade:

def notify_trade(self, trade):
    self.TrailStop.p.tradeopen = trade.isopen
 

Finally, I believe you need to set runonce=False when launching cerebro, This will evaluate the indicator with the strategy.

cerebro.run(runonce=False)

This seems to generate some results like you are looking for I believe.

***INDICATOR NEXT self.l.trailatr[0] = 0.10698710357525403, tradeopen param = True, at len: 214 tailstop = 146.34227850992914***
***STRAT NEXT self.TrailStop.trailstop[0] = 146.34227850992914, at len: 214***
inside indicator if statement
***INDICATOR NEXT self.l.trailatr[0] = 0.10328759976069737, tradeopen param = True, at len: 215 tailstop = 146.34227850992914***
***STRAT NEXT self.TrailStop.trailstop[0] = 146.34227850992914, at len: 215***
inside indicator if statement
***INDICATOR NEXT self.l.trailatr[0] = 0.10495835150337765, tradeopen param = True, at len: 216 tailstop = 146.34227850992914***
***STRAT NEXT self.TrailStop.trailstop[0] = 146.34227850992914, at len: 216***
******SELLING, SET PARAM FALSE*****
stop price updated = 145.93
in indicator else statement
***INDICATOR NEXT self.l.trailatr[0] = 0.10146324877491489, tradeopen param = False, at len: 217 tailstop = 0.0***
***STRAT NEXT self.TrailStop.trailstop[0] = 0.0, at len: 217***
stop price updated = 145.94
in indicator else statement
***INDICATOR NEXT self.l.trailatr[0] = 0.09918740729586091, tradeopen param = False, at len: 218 tailstop = 0.0***
***STRAT NEXT self.TrailStop.trailstop[0] = 0.0, at len: 218***
stop price updated = 145.95

The total code is here:

import backtrader.feeds as btfeed
import pandas as pd
import backtrader as bt
from pandas_datareader import data as pdr
import yfinance as yf

yf.pdr_override()


class TrailStop(bt.Indicator):
    lines = (
        "trailatr",
        "trailstop",
    )
    params = (
        ("tradeopen", False),
        ("atr_period", 10),
        ("trail_mult", 4),
    )
    plotinfo = dict(subplot=False)
    plotlines = dict(
        trailstop=dict(
            color="blue",
            ls="--",
            _plotskip=False,
        ),
        trailatr=dict(color="black", ls="-", _plotskip=False),
    )

    def __init__(self):
        self.l.trailatr = bt.indicators.AverageTrueRange(period=self.p.atr_period)

    def next(self):

        # self.p.tradeopen = True
        if (
            self.p.tradeopen
        ):  # using "if True:" this gets accessed and result is self.l.trailstop[0] = 1. suggests uim using the wrong way to access param?
            self.l.trailstop[0] = max(
                self.datas[0].close[0] - self.p.trail_mult * self.l.trailatr[0],
                self.l.trailstop[-1],
            )
            print("inside indicator if statement")

        else:
            self.l.trailstop[0] = min(0, 1)
            print("in indicator else statement")

        print(
            "***INDICATOR NEXT self.l.trailatr[0] = {}, tradeopen param = {}, at len: {} tailstop = {}***".format(
                self.l.trailatr[0],
                self.p.tradeopen,
                len(self.l.trailatr),
                self.l.trailstop[0],
            )
        )


# Create a Stratey
class TestStrategy(bt.Strategy):
    params = (
        ("fast_ma", 20),
        ("trail_mult", 4),
        ("tradeopen", True),
    )

    def log(self, txt, dt=None):
        """ Logging function fot this strategy"""
        dt = dt or self.datas[0].datetime.date(0)
        # print('%s, %s' % (dt.isoformat(), txt)) # #-out to turn logging off

    def __init__(self):
        # Keep a reference to the "close" line in the data[0] dataseries
        self.dataclose = self.datas[0].close
        atestst = self.p.tradeopen
        self.atr = bt.indicators.AverageTrueRange(
            self.datas[0]
        )  # using for manually calculating exit in strategy next
        self.stopprice = 0  # for manually working out the stop in strategy next

        self.TrailStop = TrailStop(
            self.datas[0], tradeopen=self.p.tradeopen
        )  # instantiate the
        # TrailStop Class

        # To keep track of pending orders and buy price/commission
        self.order = None
        self.buyprice = None
        self.buycomm = None

        # Add a MovingAverageSimple indicator
        self.sma = bt.indicators.ExponentialMovingAverage(
            self.datas[0].close, period=self.p.fast_ma
        )

        self.buysig = bt.indicators.AllN(self.datas[0].low > self.sma, period=3)

    def notify_order(self, order):
        if order.status in [order.Submitted, order.Accepted]:
            # Buy/Sell order submitted/accepted to/by broker - Nothing to do
            return

        # Check if an order has been completed
        # Attention: broker could reject order if not enough cash
        if order.status in [order.Completed]:
            if order.isbuy():
                self.log(
                    "BUY EXECUTED, Price: %.2f, Cost: %.2f, Comm %.2f"
                    % (order.executed.price, order.executed.value, order.executed.comm)
                )

                self.buyprice = order.executed.price
                self.buycomm = order.executed.comm
            else:  # Sell
                self.log(
                    "SELL EXECUTED, Price: %.2f, Cost: %.2f, Comm %.2f"
                    % (order.executed.price, order.executed.value, order.executed.comm)
                )

            # Keep track of which bar execution took place
            self.bar_executed = len(self)

        elif order.status in [order.Canceled, order.Margin, order.Rejected]:
            self.log("Order Canceled/Margin/Rejected")

        # Write down: no pending order
        self.order = None

    def notify_trade(self, trade):
        self.TrailStop.p.tradeopen = trade.isopen
        if not trade.isclosed:
            return

        self.log("OPERATION PROFIT, GROSS %.2f, NET %.2f" % (trade.pnl, trade.pnlcomm))

    def next(self):
        # Simply log the closing price of the series from the reference
        self.log("Close, %.2f" % self.dataclose[0])

        print(
            "***STRAT NEXT self.TrailStop.trailstop[0] = {}, at len: {}***".format(
                self.TrailStop.trailstop[0], len(self.datas[0])
            )
        )

        # Check if an order is pending ... if yes, we cannot send a 2nd one
        if self.order:
            return

        # Check if we are in the market
        if not self.position:

            # Not yet ... we MIGHT BUY if ...
            if self.buysig:
                # BUY, BUY, BUY!!! (with default parameters)
                self.log("BUY CREATE, %.2f" % self.dataclose[0])

                # Keep track of the created order to avoid a 2nd order
                self.order = self.buy()
                self.TrailStop.p.tradeopen = True
                print("******BUYING, SET PARAM TRUE*****")
                self.stopprice = self.data.close[0] - self.p.trail_mult * self.atr[0]
                print("stop price = {:.2f}".format(self.stopprice))

        elif self.data.close[0] < self.stopprice:
            self.close()
            self.stopprice = 0
            self.TrailStop.p.tradeopen = False
            print("******SELLING, SET PARAM FALSE*****")

        if (
            self.stopprice < self.dataclose[0] - self.p.trail_mult * self.atr[0]
        ):  # if old price < new price
            self.stopprice = (
                self.dataclose[0] - self.p.trail_mult * self.atr[0]
            )  # assign new price
            print("stop price updated = {:.2f}".format(self.stopprice))


starting_balance = 50000

if __name__ == "__main__":
    ticker_id = "AAPL"

    dataframe = pdr.get_data_yahoo(
        ticker_id,
        period="1d",
        interval="1m",
        start="2021-07-20",
        end="2021-07-21",
        prepost=False,
    )

    dataframe = dataframe[dataframe.index.strftime("%H:%M") < "20:00"]
    data = bt.feeds.PandasData(dataname=dataframe)

    # Create a cerebro entity, add strategy
    cerebro = bt.Cerebro()
    cerebro.addstrategy(TestStrategy)

    # Add the Data Feed to Cerebro, set desired cash, set desired commission
    cerebro.adddata(data)
    cerebro.broker.setcash(starting_balance)
    cerebro.broker.setcommission(commission=0.0)

    # Add a FixedSize sizer according to the stake
    cerebro.addsizer(bt.sizers.PercentSizer, percents=10)

    # Run over everything
    cerebro.run(runonce=False)
    cerebro.plot(style="candlestick")

[stevenm100]

@run-out
Yes, that works. Thank you very much for your input, very much appreciated.

[run-out]

@stevenm100 Have you tried calling the dictionary from notify_trade?

def notify_trade(self, trade):
    # self.TrailStop.p.tradeopen = trade.isopen
    self.tradestate[trade.data]['tradeopen']     =   True

[run-out]

@stevenm100
I notice now that this structure seems a little odd.

for i, d in enumerate(self.datas):
            self.inds[d] = dict()
            # Trailstop
            self.TrailStop                  =   TrailStop(d, tradeopen=self.tradestate[d]['tradeopen'])    #need to check if this param is unique to each data or is global?
            self.inds[d]['trailstop']       =   self.TrailStop.trailstop
            self.inds[d]['dyntrailstop']    =   self.TrailStop.dyntrailstop
            self.inds[d]['sma']             =   bt.indicators.MovingAverageSimple(d.close, period=30)

I would be inclined to drop the two lines (trailstop and dyntrailstop) and just have the indicator part of the inds dictionary. The indicator needs to be dynamically updated but you are not saving the indicator itself, just the lines.

for i, d in enumerate(self.datas):
            self.inds[d] = dict()
            # Trailstop
            self.inds[d]['trailstop'] = TrailStop(d, tradeopen=self.tradestate[d]['tradeopen'])    
            self.inds[d]['sma'] = bt.indicators.MovingAverageSimple(d.close, period=30)

Then add the trailstop and dyntrailstop lines in next as example:

elif pos and d.close[0] < self.inds[d]['trailstop'].dyntrailstop[0]:

Don't forget to add [0] in next to indicate the current bar. It's more explicit.

Now you should be able to set the parameter directly on the indicator instead of relying on global setting.

The parameter is attached to the indicator in strategy. We now have an indicator for each symbol. If you wish to change the indicator parameter directly try:

def notify_trade(self, trade):
    # self.TrailStop.p.tradeopen = trade.isopen
    self.ind[trade.data].p.tradeopen = True

[stevenm100]

@run-out
Thanks for the detailed reply. Agree there was some clumsiness about the setup, but I had mirrored what I had in a single data example which was working. I have streamlined it now though per your suggestion.
Anyway, after some banging of my head on the desk, I found what was wrong with my multi-data version and it was simple.....the magical runonce=False, which i hadnt copied over from my single-data example.

# Run over everything
cerebro.run(runonce=False, maxcpus=1) # runonce = False is required

now it all works as expected. Thanks for helping me though it. Hopefully the Dynamic Trailstop indicator is useful to other folks too now that its shared. Cheers!