Backtrader Community

    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

    [Benchmark Observer] Issue when benchmarking with resampled data

    General Code/Help
    2
    7
    1781
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • dimitar-petrov
      dimitar-petrov last edited by

      Hello mementum/gents,

      I am facing an issue when adding benchmark observer with resampled data.
      Here is a short example.

      import backtrader as bt
      
      class Strat(bt.Strategy):
      
        def __init__(self):
          self.ema = bt.ind.ExponentialMovingAverage(period=2)
      
        def next(self):
            pass
      
      cerebro = bt.Cerebro()
      dfile = '/tmp/test_data.csv'
      
      data = bt.feeds.GenericCSVData(dataname=dfile)
      cerebro.resampledata(data, timeframe=bt.TimeFrame.Days, compression=1)
      cerebro.addobserver(bt.observers.Benchmark,
                          data=cerebro.datas[0],
                          timeframe=bt.TimeFrame.NoTimeFrame)
      
      cerebro.addstrategy(Strat)
      
      import pdb; pdb.set_trace()
      cerebro.run()
      

      and that is the data feeded to the platform:

      2014-08-04 10:00:00,91.3808,91.4188,90.4414,90.356,551022849.0,288493
      2014-08-04 11:00:00,90.4224,90.6407,90.3466,90.5268,972065815.0,177483
      2014-08-04 12:00:00,90.5363,90.821,90.375,90.84,1245762598.0,80312
      2014-08-04 13:00:00,90.8305,90.8314,90.6075,90.7251,1453963196.0,44641
      2014-08-04 14:00:00,90.7545,90.9348,90.6312,90.9633,1628333296.0,46629
      2014-08-04 15:00:00,90.9729,91.3171,90.783,90.7924,1894081674.0,297779
      2014-08-04 16:00:00,90.802,90.6976,90.764,90.7071,36304379.0,927375
      2014-08-05 09:00:00,90.5743,90.4319,89.5589,89.7013,119958953.0,150692
      2014-08-05 10:00:00,89.7107,90.2516,89.6158,90.0048,757768836.0,144909
      2014-08-05 11:00:00,90.0618,90.1662,89.8673,89.8866,1153074048.0,85007
      2014-08-05 12:00:00,89.9859,90.0998,89.8151,90.0114,1437677659.0,26554
      2014-08-05 13:00:00,90.0808,90.6881,89.91,90.318,1890843871.0,138986
      2014-08-05 14:00:00,90.3465,90.3275,89.7202,90.2638,2557173264.0,56034
      2014-08-05 15:00:00,90.2611,90.3845,90.0248,90.2706,2934212054.0,272037
      2014-08-05 16:00:00,90.3086,90.2232,90.2706,90.2801,53558659.0,740163
      2014-08-06 09:00:00,90.1472,90.5268,89.91,90.1188,110307201.0,176703
      2014-08-06 10:00:00,90.1235,90.3086,89.9385,90.337,623044588.0,95524
      2014-08-06 11:00:00,90.4129,90.4034,90.0903,90.3086,997681083.0,34679
      2014-08-06 12:00:00,90.356,90.356,90.0902,90.1852,1240101549.0,21805
      2014-08-06 13:00:00,90.2326,90.2801,90.0799,90.1273,1438332050.0,39124
      2014-08-06 14:00:00,90.1378,90.2829,89.9764,90.2687,1662506155.0,42895
      2014-08-06 15:00:00,90.2706,90.337,90.1188,90.1661,1933269928.0,182204
      2014-08-06 16:00:00,90.1662,90.0903,90.1567,90.1093,35204618.0,470100
      2014-08-07 09:00:00,90.5957,90.7101,90.3477,90.7673,105480730.0,95062
      2014-08-07 10:00:00,90.8055,91.4349,90.5957,91.0677,661284861.0,120101
      2014-08-07 11:00:00,91.0725,91.2346,90.6624,90.6481,1191828033.0,150209
      2014-08-07 12:00:00,90.7387,90.8818,90.3954,90.5194,1602115067.0,80080
      2014-08-07 13:00:00,90.5194,90.8551,90.5099,90.8344,1855378746.0,27218
      2014-08-07 14:00:00,90.8532,90.8341,90.3668,90.3954,2029750367.0,38943
      2014-08-07 15:00:00,90.405,90.4717,89.7469,89.9854,2379287631.0,169671
      2014-08-07 16:00:00,90.0998,89.9758,89.9854,90.0998,43999921.0,430020
      2014-08-08 09:00:00,90.0712,89.786,89.3273,89.7284,132831979.0,81711
      2014-08-08 10:00:00,89.7279,89.7183,89.0603,89.5085,657399391.0,42598
      2014-08-08 11:00:00,89.5467,89.5943,89.3021,89.4322,993243343.0,78475
      2014-08-08 12:00:00,89.4609,89.5181,89.3273,89.5371,1198958712.0,27913
      2014-08-08 13:00:00,89.5467,90.1952,89.4895,90.0903,1463132880.0,63001
      2014-08-08 14:00:00,90.1067,90.3764,90.0855,90.3572,1806865549.0,66153
      2014-08-08 15:00:00,90.362,90.3887,90.1093,90.3764,2095341481.0,268884
      2014-08-08 16:00:00,90.424,90.3096,90.3668,90.3382,39785070.0,1019502.0
      

      Exception:

      
      /home/dpetrov/develop/python/py-envs/py3env/lib/python3.6/site-packages/backtrader/cerebro.py in _runnext(self, runstrats)
         1555                 self._check_timers(runstrats, dt0, cheat=False)
         1556                 for strat in runstrats:
      -> 1557                     strat._next()
         1558                     if self._event_stop:  # stop if requested
         1559                         return
      
      /home/dpetrov/develop/python/py-envs/py3env/lib/python3.6/site-packages/backtrader/strategy.py in _next(self)
          326         minperstatus = self._getminperstatus()
          327         self._next_analyzers(minperstatus)
      --> 328         self._next_observers(minperstatus)
          329 
          330         self.clear()
      
      /home/dpetrov/develop/python/py-envs/py3env/lib/python3.6/site-packages/backtrader/strategy.py in _next_observers(self, minperstatus, once)
          354                     observer.prenext()
          355             else:
      --> 356                 observer._next()
          357 
          358     def _next_analyzers(self, minperstatus, once=False):
      
      /home/dpetrov/develop/python/py-envs/py3env/lib/python3.6/site-packages/backtrader/lineiterator.py in _next(self)
          262                 self.next()
          263             elif clock_len == self._minperiod:
      --> 264                 self.nextstart()  # only called for the 1st value
          265             elif clock_len:
          266                 self.prenext()
      
      /home/dpetrov/develop/python/py-envs/py3env/lib/python3.6/site-packages/backtrader/lineiterator.py in nextstart(self)
          327 
          328         # Called once for 1st full calculation - defaults to regular next
      --> 329         self.next()
          330 
          331     def next(self):
      
      /home/dpetrov/develop/python/py-envs/py3env/lib/python3.6/site-packages/backtrader/observers/benchmark.py in next(self)
          101 
          102     def next(self):
      --> 103         super(Benchmark, self).next()
          104         self.lines.benchmark[0] = self.tbench.rets[self.treturn.dtkey]
          105 
      
      /home/dpetrov/develop/python/py-envs/py3env/lib/python3.6/site-packages/backtrader/observers/timereturn.py in next(self)
           76 
           77     def next(self):
      ---> 78         self.lines.timereturn[0] = self.treturn.rets[self.treturn.dtkey]
      
      KeyError: datetime.datetime(9999, 12, 31, 23, 59, 59, 999999)
      

      Something is getting mixed with the slave timereturn analyzers of benchmark.
      I have been looking at it for couple of hours but unable to pin point the exact issue.
      The difference I am seeing when I remove the resampler and just feed the data is here:

              self._doreplay = self._doreplay or any(x.replaying for x in self.datas)
              if self._doreplay:
                  # preloading is not supported with replay. full timeframe bars
                  # are constructed in realtime
                  self._dopreload = False
      
      

      If one case dopreload is True and benchmark is working in the othercase due to the resampled data it is False and benchmark observer is getting confused.

      BR,
      Dimitar

      B 1 Reply Last reply Reply Quote 0
      • dimitar-petrov
        dimitar-petrov last edited by

        Anyone has an idea?

        1 Reply Last reply Reply Quote 0
        • B
          backtrader administrators @dimitar-petrov last edited by

          @dimitar-petrov said in [Benchmark Observer] Issue when benchmarking with resampled data:

          KeyError: datetime.datetime(9999, 12, 31, 23, 59, 59, 999999)
          

          This datetime is used as an initial key before the analyzer has seen enough data to calculate the returns. This shows that the analyzer has actually not calculated anything.

          It seems that the timeframe being considered is NoTimeFrame, which will be because the resampled data has no actual parameter timeframe.

          It might be because the _timeframe internal attribute in the resampled data is being set late by the filter. Too late for the analyzer to catch up.

          Will be looked into.

          1 Reply Last reply Reply Quote 0
          • dimitar-petrov
            dimitar-petrov last edited by

            Great, thanks!

            1 Reply Last reply Reply Quote 0
            • B
              backtrader administrators last edited by

              Nothing to do with the assumption but rather with the distinction between runonce=True and runonce=False. It is actually a chain of coincidences in that changes to the Benchmark analyzer to skip the prenext phase and only make a comparison from the actual trading starting point, impact when the analyzer actually delivers values. Supporting the two aforementioned runonce modes for observers and analyzers is done at different points in the code (unluckily it cannot be done with the same internal tools) and the observer by skipping prenext was coming out of sync with the strategy.

              Resampling data triggers this chain of events because it automatically disables preloading in the background, which in turns set runonce=False

              Soon to be corrected.

              1 Reply Last reply Reply Quote 0
              • B
                backtrader administrators last edited by

                There is a fix in the latest commit of the development branch. It seems in any case to have a side effect for writers. So the release that was planned with it will be delayed.

                1 Reply Last reply Reply Quote 0
                • dimitar-petrov
                  dimitar-petrov last edited by

                  Works for me,
                  Thanks

                  1 Reply Last reply Reply Quote 0
                  • 1 / 1
                  • First post
                    Last post
                  Copyright © 2016, 2017, 2018, 2019, 2020, 2021 NodeBB Forums | Contributors