DivideZeroError on some indicators
-
First off, there is a related post on DivideZeroError here but it doesn't seem to answer the issue that I am experiencing.
I have quite a large number of indicators in my strategy but some of them keep throwing the same DivideZeroError. As far as I can tell I'm using the indicators correctly, for example BollingerBands are fairly widely used.
BollingerBands as far as I can tell doesn't have any division in it but every time I enable it I get this same error.
~/miniconda3/envs/pythontrading/lib/python3.5/site-packages/backtrader/linebuffer.py in next(self) 743 def next(self): 744 if self.bline: --> 745 self[0] = self.operation(self.a[0], self.b[0]) 746 elif not self.r: 747 if not self.btime: ZeroDivisionError: float division by zeroI have tried putting a try/Except block around both the init and the next functions and neither catch the error so it seems like its happening somewhere else but only gets triggered when I enable these specific indicators.
I am running a minute timescale datafeed which is then resampled to 5M, 30M, 60M 4H.
-
We don't need to see where you trigger the error inside the library. We need to see the code with which you trigger the error (and associated data)
-
Here is the associated code. I'm not sure exactly where the error is being triggered. It looks to me to be in the next() function of the strategy which triggers an operation on an indicator, however the indicators that seem to be causing the problem are not accessed in the Strategy.
# Inside Strategy def __init__(self): # These indicators don't cause any problem self.trend = btind.SMA(self.data3, period=10); self.trend.csv =self.p.l self.trendslope = self.trend - self.trend(-1); self.trendslope.csv = self.p.l self.r1 = btind.RSI(self.data0, plotname='rsi_1M'); self.r1.csv = self.p.l self.r5 = btind.RSI(self.data1, plotname='rsi_5M'); self.r5.csv = self.p.l self.r30 = btind.RSI(self.data2, plotname='rsi_30M'); self.r30.csv = self.p.l self.r60 = btind.RSI(self.data3, plotname='rsi_1H'); self.r60.csv = self.p.l self.r240 = btind.RSI(self.data4, plotname='rsi_4H'); self.r240.csv = self.p.l self.s1 = StochasticRSI(self.r1, plotname='sto_1M'); self.s1.csv = self.p.l self.s5 = StochasticRSI(self.r5, plotname='sto_5M'); self.s5.csv = self.p.l self.s30 = StochasticRSI(self.r30, plotname='sto_30M'); self.s30.csv = self.p.l self.s60 = StochasticRSI(self.r60, plotname='sto_1H'); self.s60.csv = self.p.l self.s240 = StochasticRSI(self.r240, plotname='sto_4H'); self.s240.csv = self.p.l btind.ADX(plot=True).csv = self.p.l btind.ADXR(plot=False).csv = self.p.l btind.ATR(plot=True).csv = self.p.l btind.AbsPriceOsc(plot=False).csv = self.p.l btind.AccDeOsc(plot=True).csv = self.p.l ##### # Enabling any of the following indicators causes the DivideZeroError, but # there are other indicators not shown here that also cause the problem btind.AdaptiveMovingAverageOsc(plot=False).csv = self.p.l btind.AdaptiveMovingAverage(plot=False).csv = self.p.l btind.BBands(plot=False).csv = self.p.l ###### def next(self): if self.orefs: return if not self.position: # Long Entry if self.s240.D[0] < 20 and self.trendslope > 0: if ((self.s60.D[0] > self.s60.K[0]) and (self.s30.D[0] > self.s30.K[0]) and (self.s5.D[0] > self.s5.K[0])): if (self.s1.K[0] < 20) and (self.s1.D[0] > self.s1.K[0]): os = self.buy_bracket( exectype=bt.Order.Market, stopprice=self.data0.close[0] * (1 - self.p.sl), limitprice=self.data0.close[0] * (1 + self.p.tp)) self.orefs = [o.ref for o in os] def getwriterheaders(self): self.indobscsv = [self] indobs = itertools.chain( self.getindicators_lines(), self.getobservers()) self.indobscsv.extend(filter(lambda x: x.csv, indobs)) headers = list() # prepare the indicators/observers data headers for iocsv in self.indobscsv: name = iocsv.plotinfo.plotname or iocsv.__class__.__name__ linealiases = iocsv.getlinealiases() def joinnames(alias): if alias: join = name + '_' + alias return join else: return alias aliases = tuple(map(joinnames, linealiases)) headers.extend(aliases) return headers def getwritervalues(self): values = list() for iocsv in self.indobscsv: name = iocsv.plotinfo.plotname or iocsv.__class__.__name__ lio = len(iocsv) if lio: values.extend(map(lambda l: l[0], iocsv.lines.itersize())) else: values.extend([''] * iocsv.lines.size()) return values# Running the Strategy cerebro = bt.Cerebro() cerebro.addstrategy(St, tp=0.005, sl=0.007, printlog=True, l=True) cerebro.addwriter(bt.WriterFile, csv=True, csv_counter=False, out='log.csv') # Data is a Pandas DataFrame of 1min OHLCV data = bt.feeds.PandasData(dataname=candles_test, openinterest=None, timeframe=bt.TimeFrame.Minutes) cerebro.adddata(data) cerebro.resampledata(data, timeframe=bt.TimeFrame.Minutes, compression=5, boundoff=1) cerebro.resampledata(data, timeframe=bt.TimeFrame.Minutes, compression=30, boundoff=1) cerebro.resampledata(data, timeframe=bt.TimeFrame.Minutes, compression=60, boundoff=1) cerebro.resampledata(data, timeframe=bt.TimeFrame.Minutes, compression=240, boundoff=1) cerebro.broker.setcash(20000.0) cerebro.addsizer(bt.sizers.PercentSizer, percents=98) cerebro.broker.setcommission(commission=0.00125) cerebro.run(runonce=False)
-
Here is a log of the close price of the data just before the error occurs. You can see that there is a period where the price doesn't change for 19 consecutive minutes.
Only the RSI and Stochastic Indicators implement SafeDiv and these indicators don't seem to cause the issue to occur. Its other indicators like BollingerBands that trigger this error when enabled.
2018-05-10 03:25:00, running 9286.70 2018-05-10 03:26:00, running 9291.65 2018-05-10 03:27:00, running 9291.65 2018-05-10 03:28:00, running 9291.70 2018-05-10 03:29:00, running 9291.70 2018-05-10 03:30:00, running 9291.70 2018-05-10 03:31:00, running 9291.70 2018-05-10 03:32:00, running 9291.70 2018-05-10 03:33:00, running 9291.70 2018-05-10 03:34:00, running 9291.70 2018-05-10 03:35:00, running 9291.70 2018-05-10 03:36:00, running 9291.70 2018-05-10 03:37:00, running 9291.70 2018-05-10 03:38:00, running 9291.70 2018-05-10 03:39:00, running 9291.70 2018-05-10 03:40:00, running 9291.70 2018-05-10 03:41:00, running 9291.70 2018-05-10 03:42:00, running 9291.70 2018-05-10 03:43:00, running 9291.70 2018-05-10 03:44:00, running 9291.70 2018-05-10 03:45:00, running 9291.70 2018-05-10 03:46:00, running 9291.70Edit:
One of the indicators that is causing a problem is the BollingerBandsPct. Looking into the code for the indicator it seems that if the close price doesn't change for more than the period of the indicator (default period=20) then theline.topandline.bottomwill both be equal to the close price as there is no stddev for a series of values that are all the same (stddev=0).If both
line.topandline.bottomare equal to each other then in the Percent calculation the denominator will betop - bottom = 0.class BollingerBands(Indicator): ''' Defined by John Bollinger in the 80s. It measures volatility by defining upper and lower bands at distance x standard deviations Formula: - midband = SimpleMovingAverage(close, period) - topband = midband + devfactor * StandardDeviation(data, period) - botband = midband - devfactor * StandardDeviation(data, period) See: - http://en.wikipedia.org/wiki/Bollinger_Bands ''' alias = ('BBands',) lines = ('mid', 'top', 'bot',) params = (('period', 20), ('devfactor', 2.0), ('movav', MovAv.Simple),) plotinfo = dict(subplot=False) plotlines = dict( mid=dict(ls='--'), top=dict(_samecolor=True), bot=dict(_samecolor=True), ) def _plotlabel(self): plabels = [self.p.period, self.p.devfactor] plabels += [self.p.movav] * self.p.notdefault('movav') return plabels def __init__(self): self.lines.mid = ma = self.p.movav(self.data, period=self.p.period) ######################################### # if the last 20 closing prices are all the same then StdDev will evaluate # to zero ######################################### stddev = self.p.devfactor * StdDev(self.data, ma, period=self.p.period, movav=self.p.movav) ######################################### # both self.lines.top and self.lines.bottom will be equal to the ma # which is now equal to the close price ######################################### self.lines.top = ma + stddev self.lines.bot = ma - stddev super(BollingerBands, self).__init__() class BollingerBandsPct(BollingerBands): ''' Extends the Bollinger Bands with a Percentage line ''' lines = ('pctb',) plotlines = dict(pctb=dict(_name='%B')) # display the line as %B on chart def __init__(self): super(BollingerBandsPct, self).__init__() ######################################### # The denominator here will evaluate to zero as self.l.top # and self.l.bot are the same. ######################################### self.l.pctb = (self.data - self.l.bot) / (self.l.top - self.l.bot)Edit2:
I duplicated the BollingerBandsPct in my own code and replaced the percent calculation withbt.DivByZeroand it resolved the problem for this indicator. Unfortunately there are many others similar to this.Is there a particular reason why only the Stochastic and the Rsi indicators use the safeDiv function and not the other indicators?
class MyBollingerBandsPct(btind.BollingerBands): ''' Extends the Bollinger Bands with a Percentage line ''' lines = ('pctb',) plotlines = dict(pctb=dict(_name='%B')) # display the line as %B on chart def __init__(self): super(MyBollingerBandsPct, self).__init__() self.l.pctb = bt.DivByZero((self.data - self.l.bot), (self.l.top - self.l.bot), zero=0)
-
@sfkiwi said in DivideZeroError on some indicators:
it seems that if the close price doesn't change for more than the period of the indicator (default period=20) then the line.top and line.bottom will both be equal to the close price as there is no stddev for a series of values that are all the same (stddev=0).
Indeed. Hence the reason that you show what you do and with what you do it.
-
Ok but, unlikely as it is, the price stayed stationary for more than 20mins so shouldn't the indicators still be protected from a divide by zero error?
I am adding this protection to my fork of the latest code in the repo although having some difficulty getting the latest code in the repo to run without error. Once I get it to work I can submit a PR unless there is a reason you don't want this protection on the indicators?
-
@sfkiwi said in DivideZeroError on some indicators:
Ok but, unlikely as it is, the price stayed stationary for more than 20mins so shouldn't the indicators still be protected from a divide by zero error?
Nobody has claimed nothing about the price or the protection. Only that you said something was broken without posting any code or data. And when the code and data were posted, the error was somewhere else. Hence the need to post something sensible.
class BollingerBandsPct(BollingerBands): ''' Extends the Bollinger Bands with a Percentage line ''' lines = ('pctb',) params = (('safediv', False), ('safezero', 0.0)) plotlines = dict(pctb=dict(_name='%B')) # display the line as %B on chart def __init__(self): super(BollingerBandsPct, self).__init__() if not self.p.safediv: self.l.pctb = (self.data - self.l.bot) / (self.l.top - self.l.bot) else: self.l.pctb = DivByZero(self.data - self.l.bot, self.l.top - self.l.bot, self.p.safezero) class BollingerBandsPct_SafeDiv(BollingerBandsPct): params = (('safediv', True),)That's something you can directly use in your code. There is no need for a fork to do that.