For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

DivideZeroError on some indicators



  • First off, there is a related post on DivideZeroError here but it doesn't seem to answer the issue that I am experiencing.

    I have quite a large number of indicators in my strategy but some of them keep throwing the same DivideZeroError. As far as I can tell I'm using the indicators correctly, for example BollingerBands are fairly widely used.

    BollingerBands as far as I can tell doesn't have any division in it but every time I enable it I get this same error.

    ~/miniconda3/envs/pythontrading/lib/python3.5/site-packages/backtrader/linebuffer.py in next(self)
        743     def next(self):
        744         if self.bline:
    --> 745             self[0] = self.operation(self.a[0], self.b[0])
        746         elif not self.r:
        747             if not self.btime:
    
    ZeroDivisionError: float division by zero
    

    I have tried putting a try/Except block around both the init and the next functions and neither catch the error so it seems like its happening somewhere else but only gets triggered when I enable these specific indicators.

    I am running a minute timescale datafeed which is then resampled to 5M, 30M, 60M 4H.


  • administrators

    We don't need to see where you trigger the error inside the library. We need to see the code with which you trigger the error (and associated data)



  • @backtrader

    Here is the associated code. I'm not sure exactly where the error is being triggered. It looks to me to be in the next() function of the strategy which triggers an operation on an indicator, however the indicators that seem to be causing the problem are not accessed in the Strategy.

    # Inside Strategy
    
    def __init__(self):
    
      # These indicators don't cause any problem
      self.trend = btind.SMA(self.data3, period=10); self.trend.csv =self.p.l
      self.trendslope = self.trend - self.trend(-1); self.trendslope.csv = self.p.l
    
      self.r1 = btind.RSI(self.data0, plotname='rsi_1M'); self.r1.csv = self.p.l
      self.r5 = btind.RSI(self.data1, plotname='rsi_5M'); self.r5.csv = self.p.l
      self.r30 = btind.RSI(self.data2, plotname='rsi_30M'); self.r30.csv = self.p.l
      self.r60 = btind.RSI(self.data3, plotname='rsi_1H'); self.r60.csv = self.p.l
      self.r240 = btind.RSI(self.data4, plotname='rsi_4H'); self.r240.csv = self.p.l
            
      self.s1 = StochasticRSI(self.r1, plotname='sto_1M'); self.s1.csv = self.p.l
      self.s5 = StochasticRSI(self.r5, plotname='sto_5M'); self.s5.csv = self.p.l
      self.s30 = StochasticRSI(self.r30, plotname='sto_30M'); self.s30.csv = self.p.l
      self.s60 = StochasticRSI(self.r60, plotname='sto_1H'); self.s60.csv = self.p.l
      self.s240 = StochasticRSI(self.r240, plotname='sto_4H'); self.s240.csv = self.p.l
            
      btind.ADX(plot=True).csv = self.p.l
      btind.ADXR(plot=False).csv = self.p.l
      btind.ATR(plot=True).csv = self.p.l
      btind.AbsPriceOsc(plot=False).csv = self.p.l
      btind.AccDeOsc(plot=True).csv = self.p.l
    
      #####
      # Enabling any of the following indicators causes the DivideZeroError, but 
      # there are other indicators not shown here that also cause the problem
      btind.AdaptiveMovingAverageOsc(plot=False).csv = self.p.l
      btind.AdaptiveMovingAverage(plot=False).csv = self.p.l
      btind.BBands(plot=False).csv = self.p.l
      ######
    
    def next(self):
      if self.orefs:
           return
    
      if not self.position:
        
        # Long Entry
        if self.s240.D[0] < 20 and self.trendslope > 0:
    
          if ((self.s60.D[0] > self.s60.K[0]) and 
            (self.s30.D[0] > self.s30.K[0]) and 
            (self.s5.D[0] > self.s5.K[0])):
    
            if (self.s1.K[0] < 20) and (self.s1.D[0] > self.s1.K[0]):
    
              os = self.buy_bracket(
                exectype=bt.Order.Market, 
                stopprice=self.data0.close[0] * (1 - self.p.sl), 
                limitprice=self.data0.close[0] * (1 + self.p.tp))
    
              self.orefs = [o.ref for o in os]
    
    def getwriterheaders(self):
      self.indobscsv = [self]
                
      indobs = itertools.chain(
        self.getindicators_lines(), self.getobservers())
        self.indobscsv.extend(filter(lambda x: x.csv, indobs))
                
      headers = list()
                
      # prepare the indicators/observers data headers
      for iocsv in self.indobscsv:
        name = iocsv.plotinfo.plotname or iocsv.__class__.__name__
        linealiases = iocsv.getlinealiases()
                
        def joinnames(alias):
          if alias:
            join = name + '_' + alias
            return join
          else:
            return alias
                                    
        aliases = tuple(map(joinnames, linealiases))
        headers.extend(aliases)
                
        return headers
                    
    def getwritervalues(self):
      values = list()
                
      for iocsv in self.indobscsv:
        name = iocsv.plotinfo.plotname or iocsv.__class__.__name__
        lio = len(iocsv)
        if lio:
          values.extend(map(lambda l: l[0], iocsv.lines.itersize()))
        else:
          values.extend([''] * iocsv.lines.size())
                
        return values
    
    
    # Running the Strategy
    cerebro = bt.Cerebro()
    cerebro.addstrategy(St, tp=0.005, sl=0.007, printlog=True, l=True)
    cerebro.addwriter(bt.WriterFile, csv=True, csv_counter=False, out='log.csv')
    
    # Data is a Pandas DataFrame of 1min OHLCV
    data = bt.feeds.PandasData(dataname=candles_test, openinterest=None, timeframe=bt.TimeFrame.Minutes)
    cerebro.adddata(data)
    cerebro.resampledata(data, timeframe=bt.TimeFrame.Minutes, compression=5, boundoff=1)
    cerebro.resampledata(data, timeframe=bt.TimeFrame.Minutes, compression=30, boundoff=1)
    cerebro.resampledata(data, timeframe=bt.TimeFrame.Minutes, compression=60, boundoff=1)
    cerebro.resampledata(data, timeframe=bt.TimeFrame.Minutes, compression=240, boundoff=1)
    cerebro.broker.setcash(20000.0)
    cerebro.addsizer(bt.sizers.PercentSizer, percents=98)
    cerebro.broker.setcommission(commission=0.00125)
    cerebro.run(runonce=False)
    


  • @backtrader

    Here is a log of the close price of the data just before the error occurs. You can see that there is a period where the price doesn't change for 19 consecutive minutes.

    Only the RSI and Stochastic Indicators implement SafeDiv and these indicators don't seem to cause the issue to occur. Its other indicators like BollingerBands that trigger this error when enabled.

    2018-05-10 03:25:00, running 9286.70
    2018-05-10 03:26:00, running 9291.65
    2018-05-10 03:27:00, running 9291.65
    2018-05-10 03:28:00, running 9291.70
    2018-05-10 03:29:00, running 9291.70
    2018-05-10 03:30:00, running 9291.70
    2018-05-10 03:31:00, running 9291.70
    2018-05-10 03:32:00, running 9291.70
    2018-05-10 03:33:00, running 9291.70
    2018-05-10 03:34:00, running 9291.70
    2018-05-10 03:35:00, running 9291.70
    2018-05-10 03:36:00, running 9291.70
    2018-05-10 03:37:00, running 9291.70
    2018-05-10 03:38:00, running 9291.70
    2018-05-10 03:39:00, running 9291.70
    2018-05-10 03:40:00, running 9291.70
    2018-05-10 03:41:00, running 9291.70
    2018-05-10 03:42:00, running 9291.70
    2018-05-10 03:43:00, running 9291.70
    2018-05-10 03:44:00, running 9291.70
    2018-05-10 03:45:00, running 9291.70
    2018-05-10 03:46:00, running 9291.70
    

    Edit:
    One of the indicators that is causing a problem is the BollingerBandsPct. Looking into the code for the indicator it seems that if the close price doesn't change for more than the period of the indicator (default period=20) then the line.top and line.bottom will both be equal to the close price as there is no stddev for a series of values that are all the same (stddev=0).

    If both line.top and line.bottom are equal to each other then in the Percent calculation the denominator will be top - bottom = 0.

    class BollingerBands(Indicator):
        '''
        Defined by John Bollinger in the 80s. It measures volatility by defining
        upper and lower bands at distance x standard deviations
    
        Formula:
          - midband = SimpleMovingAverage(close, period)
          - topband = midband + devfactor * StandardDeviation(data, period)
          - botband = midband - devfactor * StandardDeviation(data, period)
    
        See:
          - http://en.wikipedia.org/wiki/Bollinger_Bands
        '''
        alias = ('BBands',)
    
        lines = ('mid', 'top', 'bot',)
        params = (('period', 20), ('devfactor', 2.0), ('movav', MovAv.Simple),)
    
        plotinfo = dict(subplot=False)
        plotlines = dict(
            mid=dict(ls='--'),
            top=dict(_samecolor=True),
            bot=dict(_samecolor=True),
        )
    
        def _plotlabel(self):
            plabels = [self.p.period, self.p.devfactor]
            plabels += [self.p.movav] * self.p.notdefault('movav')
            return plabels
    
        def __init__(self):
            self.lines.mid = ma = self.p.movav(self.data, period=self.p.period)
    
            #########################################
            # if the last 20 closing prices are all the same then StdDev will evaluate 
            # to zero
            #########################################
            stddev = self.p.devfactor * StdDev(self.data, ma, period=self.p.period,
                                               movav=self.p.movav)
    
            #########################################
            # both self.lines.top and self.lines.bottom will be equal to the ma
            # which is now equal to the close price
            #########################################
            self.lines.top = ma + stddev
            self.lines.bot = ma - stddev
    
            super(BollingerBands, self).__init__()
    
    
    class BollingerBandsPct(BollingerBands):
        '''
        Extends the Bollinger Bands with a Percentage line
        '''
        lines = ('pctb',)
        plotlines = dict(pctb=dict(_name='%B'))  # display the line as %B on chart
    
        def __init__(self):
            super(BollingerBandsPct, self).__init__()
    
            #########################################
            # The denominator here will evaluate to zero as self.l.top
            # and self.l.bot are the same.
            #########################################
            self.l.pctb = (self.data - self.l.bot) / (self.l.top - self.l.bot)
    

    Edit2:
    I duplicated the BollingerBandsPct in my own code and replaced the percent calculation with bt.DivByZero and it resolved the problem for this indicator. Unfortunately there are many others similar to this.

    Is there a particular reason why only the Stochastic and the Rsi indicators use the safeDiv function and not the other indicators?

    class MyBollingerBandsPct(btind.BollingerBands):
        '''
        Extends the Bollinger Bands with a Percentage line
        '''
        lines = ('pctb',)
        plotlines = dict(pctb=dict(_name='%B'))  # display the line as %B on chart
    
        def __init__(self):
            super(MyBollingerBandsPct, self).__init__()
            self.l.pctb = bt.DivByZero((self.data - self.l.bot), (self.l.top - self.l.bot), zero=0)
    

  • administrators

    @sfkiwi said in DivideZeroError on some indicators:

    it seems that if the close price doesn't change for more than the period of the indicator (default period=20) then the line.top and line.bottom will both be equal to the close price as there is no stddev for a series of values that are all the same (stddev=0).

    Indeed. Hence the reason that you show what you do and with what you do it.



  • @backtrader

    Ok but, unlikely as it is, the price stayed stationary for more than 20mins so shouldn't the indicators still be protected from a divide by zero error?

    I am adding this protection to my fork of the latest code in the repo although having some difficulty getting the latest code in the repo to run without error. Once I get it to work I can submit a PR unless there is a reason you don't want this protection on the indicators?


  • administrators

    @sfkiwi said in DivideZeroError on some indicators:

    Ok but, unlikely as it is, the price stayed stationary for more than 20mins so shouldn't the indicators still be protected from a divide by zero error?

    Nobody has claimed nothing about the price or the protection. Only that you said something was broken without posting any code or data. And when the code and data were posted, the error was somewhere else. Hence the need to post something sensible.

    class BollingerBandsPct(BollingerBands):
        '''
        Extends the Bollinger Bands with a Percentage line
        '''
        lines = ('pctb',)
        params = (('safediv', False), ('safezero', 0.0))
        plotlines = dict(pctb=dict(_name='%B'))  # display the line as %B on chart
    
        def __init__(self):
            super(BollingerBandsPct, self).__init__()
            if not self.p.safediv:
                self.l.pctb = (self.data - self.l.bot) / (self.l.top - self.l.bot)
            else:
                self.l.pctb = DivByZero(self.data - self.l.bot, self.l.top - self.l.bot, self.p.safezero)
    
    class BollingerBandsPct_SafeDiv(BollingerBandsPct):
        params = (('safediv', True),)
    

    That's something you can directly use in your code. There is no need for a fork to do that.