bt.observers.Benchmark

timzhang

Just trying to add a benchmark observer into my strategy and plot. My method is to add data to cerebro first, then addobserver using the same benchmark data. It all seems to be fine, but the plot still only shows the strategy TimeReturn. I had a look at the benchmark documentation, but couldn't figure it out. Just wondering if any additional example that I can reference besides the below link?

https://www.backtrader.com/docu/observer-benchmark/benchmarking.html#observers-benchmarking

backtrader

The introduction of the Benchmark was documented here: Blog - Benchmarking

And there is a sample in the sources: Sources - observer-benchmark

With the sample you can use --timeframe to compare either on a daily, weekly, monthly or yearly basis.

ab_trader

@backtrader
Looks like benchmark observer was affected by lastest changes. I am trying to use it with the sample from front page of the website and got the following error:

Traceback (most recent call last):
  File "test_benchmarking.py", line 21, in <module>
    cerebro.plot()
  File "C:\Python27\lib\site-packages\backtrader\cerebro.py", line 738, in plot
    start=start, end=end)
  File "C:\Python27\lib\site-packages\backtrader\plot\plot.py", line 189, in plot
    self.plotind(None, ptop, subinds=self.dplotsover[ptop])
  File "C:\Python27\lib\site-packages\backtrader\plot\plot.py", line 388, in plotind
    indlabel = ind.plotlabel()
  File "C:\Python27\lib\site-packages\backtrader\lineseries.py", line 450, in plotlabel
    sublabels = self._plotlabel()
  File "C:\Python27\lib\site-packages\backtrader\observers\benchmark.py", line 86, in _plotlabel
    labels.append(self.p.data._name)
AttributeError: 'NoneType' object has no attribute '_name'

Code:

from datetime import datetime
import backtrader as bt

class SmaCross(bt.SignalStrategy):
    def __init__(self):
        sma1, sma2 = bt.ind.SMA(period=10), bt.ind.SMA(period=30)
        crossover = bt.ind.CrossOver(sma1, sma2)
        self.signal_add(bt.SIGNAL_LONG, crossover)

cerebro = bt.Cerebro()
cerebro.addstrategy(SmaCross)

data0 = bt.feeds.YahooFinanceData(dataname='YHOO', fromdate=datetime(2015, 1, 1),
                                  todate=datetime(2016, 12, 31))
cerebro.adddata(data0)

cerebro.addobserver(bt.observers.Benchmark)
cerebro.addobserver(bt.observers.TimeReturn, timeframe=bt.TimeFrame.NoTimeFrame)

cerebro.run()
cerebro.plot()

TimeReturn observer works well.

ab_trader

With my code it throws the following error:

Traceback (most recent call last):
  File "template.py", line 590, in <module>
    cerebro, stats = single_backtest(single_run_params)
  File "template.py", line 437, in single_backtest
    strats = cerebro.run(stdstats=False)
  File "C:\Python27\lib\site-packages\backtrader\cerebro.py", line 873, in run
    runstrat = self.runstrategies(iterstrat)
  File "C:\Python27\lib\site-packages\backtrader\cerebro.py", line 1010, in runstrategies
    self._runnext(runstrats)
  File "C:\Python27\lib\site-packages\backtrader\cerebro.py", line 1326, in _runnext
    strat._next()
  File "C:\Python27\lib\site-packages\backtrader\strategy.py", line 300, in _next
    self._next_observers(minperstatus)
  File "C:\Python27\lib\site-packages\backtrader\strategy.py", line 328, in _next_observers
    observer._next()
  File "C:\Python27\lib\site-packages\backtrader\lineiterator.py", line 265, in _next
    self.nextstart()  # only called for the 1st value
  File "C:\Python27\lib\site-packages\backtrader\lineiterator.py", line 330, in nextstart
    self.next()
  File "C:\Python27\lib\site-packages\backtrader\observers\benchmark.py", line 100, in next
    super(Benchmark, self).next()
  File "C:\Python27\lib\site-packages\backtrader\observers\timereturn.py", line 78, in next
    self.lines.timereturn[0] = self.treturn.rets[self.treturn.dtkey]
KeyError: datetime.datetime(9999, 12, 31, 23, 59, 59, 999999)

TimeReturn also works well.

backtrader

See: Docs - Benchmarking

In the sample you can observe how a data is added. Something is needed as a benchmark.

ab_trader

My bad. I've expected that if no data specified, then it will just take first data imported. Added data, now it works for the sample, but still shows the same error for my script. Could you propose any idea based on error where I need to dig?

ab_trader

@backtrader

I was looking thru error I received with Benchmark observer and found out that it doesn't work correctly due to BarReplayer_Open filter. Looks like this filter prevent correct indexing. I've added this filter to the sample case and got the same error.

Code:

from datetime import datetime
import backtrader as bt

class SmaCross(bt.SignalStrategy):
    def __init__(self):
        sma1, sma2 = bt.ind.SMA(period=10), bt.ind.SMA(period=30)
        crossover = bt.ind.CrossOver(sma1, sma2)
        self.signal_add(bt.SIGNAL_LONG, crossover)

cerebro = bt.Cerebro()
cerebro.addstrategy(SmaCross)

data0 = bt.feeds.YahooFinanceData(dataname='YHOO', fromdate=datetime(2015, 1, 1),
                                  todate=datetime(2016, 12, 31))
data0.addfilter(bt.filters.BarReplayer_Open)
cerebro.adddata(data0)

benchdata = bt.feeds.YahooFinanceData(dataname='YHOO', fromdate=datetime(2015, 1, 1),
                                      todate=datetime(2016, 12, 31))
cerebro.adddata(benchdata)

cerebro.addobserver(bt.observers.Benchmark, data=benchdata, timeframe=bt.TimeFrame.NoTimeFrame)
cerebro.addobserver(bt.observers.TimeReturn, timeframe=bt.TimeFrame.NoTimeFrame)

cerebro.run()
cerebro.plot()

Error:

Traceback (most recent call last):
  File "test_benchmarking.py", line 25, in <module>
    cerebro.run()
  File "C:\Python27\lib\site-packages\backtrader\cerebro.py", line 873, in run
    runstrat = self.runstrategies(iterstrat)
  File "C:\Python27\lib\site-packages\backtrader\cerebro.py", line 1010, in runstrategies
    self._runnext(runstrats)
  File "C:\Python27\lib\site-packages\backtrader\cerebro.py", line 1326, in _runnext
    strat._next()
  File "C:\Python27\lib\site-packages\backtrader\strategy.py", line 300, in _next
    self._next_observers(minperstatus)
  File "C:\Python27\lib\site-packages\backtrader\strategy.py", line 328, in _next_observers
    observer._next()
  File "C:\Python27\lib\site-packages\backtrader\lineiterator.py", line 265, in _next
    self.nextstart()  # only called for the 1st value
  File "C:\Python27\lib\site-packages\backtrader\lineiterator.py", line 330, in nextstart
    self.next()
  File "C:\Python27\lib\site-packages\backtrader\observers\benchmark.py", line 100, in next
    super(Benchmark, self).next()
  File "C:\Python27\lib\site-packages\backtrader\observers\timereturn.py", line 78, in next
    self.lines.timereturn[0] = self.treturn.rets[self.treturn.dtkey]
KeyError: datetime.datetime(9999, 12, 31, 23, 59, 59, 999999)

Could you pleas take a look?

ab_trader

@backtrader
Sorry for bugging you. Did you have a chance to check out interference between Benchmark observer and BarReplay_Open filter?

backtrader

Use next_open and cheat_on_open

See: Community - Release 1.9.44.116

For Cheat-On-Open see Blog - Cheat On Open