bt.observers.Benchmark



  • Just trying to add a benchmark observer into my strategy and plot. My method is to add data to cerebro first, then addobserver using the same benchmark data. It all seems to be fine, but the plot still only shows the strategy TimeReturn. I had a look at the benchmark documentation, but couldn't figure it out. Just wondering if any additional example that I can reference besides the below link?

    https://www.backtrader.com/docu/observer-benchmark/benchmarking.html#observers-benchmarking


  • administrators

    The introduction of the Benchmark was documented here: Blog - Benchmarking

    And there is a sample in the sources: Sources - observer-benchmark

    With the sample you can use --timeframe to compare either on a daily, weekly, monthly or yearly basis.



  • @backtrader
    Looks like benchmark observer was affected by lastest changes. I am trying to use it with the sample from front page of the website and got the following error:

    Traceback (most recent call last):
      File "test_benchmarking.py", line 21, in <module>
        cerebro.plot()
      File "C:\Python27\lib\site-packages\backtrader\cerebro.py", line 738, in plot
        start=start, end=end)
      File "C:\Python27\lib\site-packages\backtrader\plot\plot.py", line 189, in plot
        self.plotind(None, ptop, subinds=self.dplotsover[ptop])
      File "C:\Python27\lib\site-packages\backtrader\plot\plot.py", line 388, in plotind
        indlabel = ind.plotlabel()
      File "C:\Python27\lib\site-packages\backtrader\lineseries.py", line 450, in plotlabel
        sublabels = self._plotlabel()
      File "C:\Python27\lib\site-packages\backtrader\observers\benchmark.py", line 86, in _plotlabel
        labels.append(self.p.data._name)
    AttributeError: 'NoneType' object has no attribute '_name'
    

    Code:

    from datetime import datetime
    import backtrader as bt
    
    class SmaCross(bt.SignalStrategy):
        def __init__(self):
            sma1, sma2 = bt.ind.SMA(period=10), bt.ind.SMA(period=30)
            crossover = bt.ind.CrossOver(sma1, sma2)
            self.signal_add(bt.SIGNAL_LONG, crossover)
    
    cerebro = bt.Cerebro()
    cerebro.addstrategy(SmaCross)
    
    data0 = bt.feeds.YahooFinanceData(dataname='YHOO', fromdate=datetime(2015, 1, 1),
                                      todate=datetime(2016, 12, 31))
    cerebro.adddata(data0)
    
    cerebro.addobserver(bt.observers.Benchmark)
    cerebro.addobserver(bt.observers.TimeReturn, timeframe=bt.TimeFrame.NoTimeFrame)
    
    cerebro.run()
    cerebro.plot()
    

    TimeReturn observer works well.



  • With my code it throws the following error:

    Traceback (most recent call last):
      File "template.py", line 590, in <module>
        cerebro, stats = single_backtest(single_run_params)
      File "template.py", line 437, in single_backtest
        strats = cerebro.run(stdstats=False)
      File "C:\Python27\lib\site-packages\backtrader\cerebro.py", line 873, in run
        runstrat = self.runstrategies(iterstrat)
      File "C:\Python27\lib\site-packages\backtrader\cerebro.py", line 1010, in runstrategies
        self._runnext(runstrats)
      File "C:\Python27\lib\site-packages\backtrader\cerebro.py", line 1326, in _runnext
        strat._next()
      File "C:\Python27\lib\site-packages\backtrader\strategy.py", line 300, in _next
        self._next_observers(minperstatus)
      File "C:\Python27\lib\site-packages\backtrader\strategy.py", line 328, in _next_observers
        observer._next()
      File "C:\Python27\lib\site-packages\backtrader\lineiterator.py", line 265, in _next
        self.nextstart()  # only called for the 1st value
      File "C:\Python27\lib\site-packages\backtrader\lineiterator.py", line 330, in nextstart
        self.next()
      File "C:\Python27\lib\site-packages\backtrader\observers\benchmark.py", line 100, in next
        super(Benchmark, self).next()
      File "C:\Python27\lib\site-packages\backtrader\observers\timereturn.py", line 78, in next
        self.lines.timereturn[0] = self.treturn.rets[self.treturn.dtkey]
    KeyError: datetime.datetime(9999, 12, 31, 23, 59, 59, 999999)
    

    TimeReturn also works well.


  • administrators

    See: Docs - Benchmarking

    In the sample you can observe how a data is added. Something is needed as a benchmark.



  • My bad. I've expected that if no data specified, then it will just take first data imported. Added data, now it works for the sample, but still shows the same error for my script. Could you propose any idea based on error where I need to dig?



  • @backtrader

    I was looking thru error I received with Benchmark observer and found out that it doesn't work correctly due to BarReplayer_Open filter. Looks like this filter prevent correct indexing. I've added this filter to the sample case and got the same error.

    Code:

    from datetime import datetime
    import backtrader as bt
    
    class SmaCross(bt.SignalStrategy):
        def __init__(self):
            sma1, sma2 = bt.ind.SMA(period=10), bt.ind.SMA(period=30)
            crossover = bt.ind.CrossOver(sma1, sma2)
            self.signal_add(bt.SIGNAL_LONG, crossover)
    
    cerebro = bt.Cerebro()
    cerebro.addstrategy(SmaCross)
    
    data0 = bt.feeds.YahooFinanceData(dataname='YHOO', fromdate=datetime(2015, 1, 1),
                                      todate=datetime(2016, 12, 31))
    data0.addfilter(bt.filters.BarReplayer_Open)
    cerebro.adddata(data0)
    
    benchdata = bt.feeds.YahooFinanceData(dataname='YHOO', fromdate=datetime(2015, 1, 1),
                                          todate=datetime(2016, 12, 31))
    cerebro.adddata(benchdata)
    
    cerebro.addobserver(bt.observers.Benchmark, data=benchdata, timeframe=bt.TimeFrame.NoTimeFrame)
    cerebro.addobserver(bt.observers.TimeReturn, timeframe=bt.TimeFrame.NoTimeFrame)
    
    cerebro.run()
    cerebro.plot()
    

    Error:

    Traceback (most recent call last):
      File "test_benchmarking.py", line 25, in <module>
        cerebro.run()
      File "C:\Python27\lib\site-packages\backtrader\cerebro.py", line 873, in run
        runstrat = self.runstrategies(iterstrat)
      File "C:\Python27\lib\site-packages\backtrader\cerebro.py", line 1010, in runstrategies
        self._runnext(runstrats)
      File "C:\Python27\lib\site-packages\backtrader\cerebro.py", line 1326, in _runnext
        strat._next()
      File "C:\Python27\lib\site-packages\backtrader\strategy.py", line 300, in _next
        self._next_observers(minperstatus)
      File "C:\Python27\lib\site-packages\backtrader\strategy.py", line 328, in _next_observers
        observer._next()
      File "C:\Python27\lib\site-packages\backtrader\lineiterator.py", line 265, in _next
        self.nextstart()  # only called for the 1st value
      File "C:\Python27\lib\site-packages\backtrader\lineiterator.py", line 330, in nextstart
        self.next()
      File "C:\Python27\lib\site-packages\backtrader\observers\benchmark.py", line 100, in next
        super(Benchmark, self).next()
      File "C:\Python27\lib\site-packages\backtrader\observers\timereturn.py", line 78, in next
        self.lines.timereturn[0] = self.treturn.rets[self.treturn.dtkey]
    KeyError: datetime.datetime(9999, 12, 31, 23, 59, 59, 999999)
    

    Could you pleas take a look?



  • @backtrader
    Sorry for bugging you. Did you have a chance to check out interference between Benchmark observer and BarReplay_Open filter?


  • administrators

    Use next_open and cheat_on_open

    See: Community - Release 1.9.44.116

    For Cheat-On-Open see Blog - Cheat On Open


Log in to reply
 

Looks like your connection to Backtrader Community was lost, please wait while we try to reconnect.