Navigation

    Backtrader Community

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

    Custom Indicator - Mixing Timeframes - different scenarios

    General Discussion
    1
    2
    235
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • monstrar
      monstrar last edited by

      Hey all,

      First of all, thank you for the amazing backtesting library! I've been trying to use a custom indicator that uses multiple timeframes and I have found some things I dont understand. I've read a lot on the forum and still don't understand some behaviour that occurs with multiple timeframes. Below I'll try to as accurately possible explain the different code I've tried and results I've gotten. I on purpose do not use the cerebro.resample function as I read in another post that if you want custom lines to your datafeed, you should not use the resample function. In my simplified example I have not added any extra lines, am just printing out data, and have NOT used the multiple timeframes in calculations, just loaded the timeframes in the indicator. My goal is to understand why I am getting different results when using different code in this simplified example.

      Data used (first 4 rows):

      1 minute dataframe
      date_of_close open high low close
      2018-01-01 00:01:00 13715.65 13715.65 13681.00 13707.92
      2018-01-01 00:02:00 13707.91 13707.91 13666.11 13694.92
      2018-01-01 00:03:00 13682.00 13694.94 13680.00 13680.00
      2018-01-01 00:04:00 13679.98 13679.98 13601.00 13645.99

      60 minute dataframe
      date_of_close open high low close
      2018-01-01 01:00:00 13715.65 13715.65 13400.01 13529.01
      2018-01-01 02:00:00 13528.99 13595.89 13155.38 13203.06
      2018-01-01 03:00:00 13203.00 13418.43 13200.00 13330.18
      2018-01-01 04:00:00 13330.26 13611.27 13290.00 13410.03

      Scenario 1 code:

      1. runonce=True
      2. 1 datafeed passed to indicator
      class TestCrossIndicator(bt.Indicator):
          lines = ('crossline',)
          params = dict(period=25)
          
          def __init__(self):
              self.ema = btind.ExponentialMovingAverage(self.datas[0].close, period=self.params.period)
              self.lines.crossline = self.datas[0].close > self.ema
      
      class TestCrossStrategy(bt.Strategy):
          def __init__(self):
              self.indicatorline = TestCrossIndicator(self.datas[1])
              
              self.ema = btind.ExponentialMovingAverage(self.datas[1].close, period=25)
              self.strategyline = self.datas[1].close > self.ema        
          def next(self):
              print('LTF Strategy: ', len(self.datas[0]))
              print('HTF Strategy: ', len(self.datas[1]))
              print('Indicator line', self.indicatorline[0])
              print('Strategy line', self.strategyline[0])
              print(self.indicatorline[0] == self.strategyline[0])
      
      cerebro = bt.Cerebro()
      ltf_data = PandasData(dataname=ltf_train, timeframe=bt.TimeFrame.Minutes, compression=1, plot=True)
      
      htf_data = PandasData(dataname=htf_train, timeframe=bt.TimeFrame.Minutes, compression=60, plot=True)
      data = cerebro.adddata(ltf_data)
      data2 = cerebro.adddata(htf_data)
      
      cerebro.addstrategy(TestCrossStrategy)
      cerebro.broker.set_cash(10000)
      
      print('Starting Portfolio Value: %.2f' % cerebro.broker.getvalue())
       
      cerebro.run(runonce=True)
      
      print('Final Portfolio Value: %.2f' % cerebro.broker.getvalue())
      

      Output Scenario 1:

      LTF Strategy:  1500
      HTF Strategy:  25
      Indicator line 1.0
      Strategy line 1.0
      True
      LTF Strategy:  1501
      HTF Strategy:  25
      Indicator line 1.0
      Strategy line 1.0
      True
      LTF Strategy:  1502
      HTF Strategy:  25
      Indicator line 1.0
      Strategy line 1.0
      True
      

      Code runs as I would expect. Minimum period of 25 based on datas[1], indicator line and strategy line are equivalent.

      Scenario 2 code:

      1. runonce=True
      2. 2 datafeeds passed to indicator

      Code adjusted for scenario 2:
      A. Add a datafeed to the indicator.

      self.indicatorline = TestCrossIndicator(self.datas[0], self.datas[1])
      

      B. Adjust indicator to take into account second datafeed

      self.ema = btind.ExponentialMovingAverage(self.datas[1].close, period=self.params.period)
      self.lines.crossline = self.datas[1].close > self.ema
      

      Output scenario 2:

      LTF Strategy:  1500
      HTF Strategy:  25
      Indicator line nan
      Strategy line 1.0
      False
      LTF Strategy:  1501
      HTF Strategy:  25
      Indicator line nan
      Strategy line 1.0
      False
      LTF Strategy:  1502
      HTF Strategy:  25
      Indicator line nan
      Strategy line 1.0
      False
      

      The indicator line is NaN as opposed to the expected 1. I do not understand why it is is NaN. I read https://www.backtrader.com/docu/mixing-timeframes/indicators-mixing-timeframes/ and based upon that reading I thought I only needed the runonce=False argument in celebro.run() if I am doing certain calculations based on multiple datafeeds (such as making comparisons between data feeds). However, I have just loaded the 2 datafeeds in the indicator and am using only 1 datafeed for calculations.

      I had a closer look at the above and added the following code to TestCrossIndicator.

      def next(self):
              print('LTF Indicator:', len(self.datas[0]))
              print('HTF Indicator:', len(self.datas[1]))
      

      Output:

      LTF Indicator: 25
      HTF Indicator: 25
      LTF Indicator: 26
      HTF Indicator: 26
      LTF Indicator: 27
      HTF Indicator: 27
      LTF Indicator: 28
      HTF Indicator: 28
      LTF Indicator: 29
      HTF Indicator: 29
      ..........
      ..........
      LTF Indicator: 7074
      HTF Indicator: 7074
      LTF Indicator: 7075
      HTF Indicator: 7075
      
      LTF Strategy:  1500
      HTF Strategy:  25
      Indicator line nan
      Strategy line 1.0
      False
      LTF Strategy:  1501
      HTF Strategy:  25
      Indicator line nan
      Strategy line 1.0
      False
      LTF Strategy:  1502
      HTF Strategy:  25
      Indicator line nan
      Strategy line 1.0
      False
      

      The above output was somewhat unexpected to me that the indicator output was printed first and that the LEN of the HTF datafeed is equal to the LEN of the LTF datafeed. I assume this has something to do with the indicator being pre-processed due to runonce=True, but I expected the datafeeds LEN in Indicator to still be as in Strategy. Also not sure why the next() method was ran before the strategy next() method, as I thought only the init_ method would be pre-processed of the custom indicator.

      Scenario 3:

      1. Runonce=False
      2. 2 datafeeds passed to indicator
      LTF Indicator: 25
      HTF Indicator: 0
      LTF Indicator: 26
      HTF Indicator: 0
      LTF Indicator: 27
      HTF Indicator: 0
      LTF Indicator: 28
      HTF Indicator: 0
      LTF Indicator: 29
      HTF Indicator: 0
      .......
      .......
      .......
      
      LTF Strategy:  1500
      HTF Strategy:  25
      Indicator line 1.0
      Strategy line 1.0
      True
      LTF Indicator: 1501
      HTF Indicator: 25
      LTF Strategy:  1501
      HTF Strategy:  25
      Indicator line 1.0
      Strategy line 1.0
      True
      LTF Indicator: 1502
      HTF Indicator: 25
      LTF Strategy:  1502
      HTF Strategy:  25
      Indicator line 1.0
      Strategy line 1.0
      True
      

      Output is as I expected, with next() method of Indicator running along with next() method of Strategy. LEN of HTF datafeed is correct in custom indicator. Indicator line equals Strategy line.

      Summary
      Summarized based on the different scenarios above it seems that I should always use runonce=False when using a custom indicator with multiple data feeds. Is this true? I would love to better understand why the behaviour in scenario 2 occurs.

      In the Strategy class, the strategyline seems to work fine even with runonce=True. As runonce=True is faster based on the docs, should one implement indicators in the Strategy class and not use the seperate Indicator class if highest speed is preferred?

      For completeness sake: the above behaviour in scenario 2 does not occur when I am using the celebro.resample method to load the second datafeed. In such case, the indicator works fine even if not using runonce=False. I do have to manually set the minimum period though in the indicator when using the resample method. As mentioned in the introduction, I am on purpose not using the resample method as my production code will have custom lines (e.g. see https://community.backtrader.com/topic/266/line-info-lost-during-resampling)

      Im still a beginner in programming, so I hope I posted all the relevant information. If there is any additional information you need, please let me know :) Thanks a lot in advance and once again thanks for the platform!

      Kind regards,

      Monstrar

      1 Reply Last reply Reply Quote 0
      • monstrar
        monstrar last edited by

        It seems that the behaviour is for me unexpected in scenario 3 (runonce=False, multiple timeframes) if I do not have the strategyline defined (see in code below the part I commented out). Used a 15 minute data feed as opposed to 1hour data feed here, as I found this error while working on 15 minute data feed.

        Code used (Scenario 4):

        class TestCrossIndicator(bt.Indicator):
            lines = ('crossline',)
            params = dict(period=25)
            
            def __init__(self):
                self.ema = btind.ExponentialMovingAverage(self.datas[1].close, period=self.params.period)
                self.lines.crossline = self.datas[1].close > self.ema
            def next(self):
                print('LTF Indicator:', len(self.datas[0]))
                print('HTF Indicator:', len(self.datas[1]))
            
        
        class TestCrossStrategy(bt.Strategy):
            def __init__(self):
                self.indicatorline = TestCrossIndicator(self.datas[0], self.datas[1])
                
        #         self.ema = btind.ExponentialMovingAverage(self.datas[1].close, period=25)
        #         self.strategyline = self.datas[1].close > self.ema
                
            def next(self):
                print('LTF Strategy: ', len(self.datas[0]))
                print('HTF Strategy: ', len(self.datas[1]))
                print('Indicator line', self.indicatorline[0])
        #        print('Strategy line', self.strategyline[0])
        #        print(self.indicatorline[0] == self.strategyline[0])
        
        cerebro = bt.Cerebro()
        ltf_data = PandasData(dataname=ltf_train, timeframe=bt.TimeFrame.Minutes, compression=1, plot=True)
        
        htf_data = PandasData(dataname=htf_train, timeframe=bt.TimeFrame.Minutes, compression=15, plot=True)
        data = cerebro.adddata(ltf_data)
        data2 = cerebro.adddata(htf_data)
        
        cerebro.addstrategy(TestCrossStrategy)
        cerebro.broker.set_cash(10000)
        
        print('Starting Portfolio Value: %.2f' % cerebro.broker.getvalue())
         
        cerebro.run(runonce=False)
        
        print('Final Portfolio Value: %.2f' % cerebro.broker.getvalue())
        

        Output Scenario 4:

        LTF Indicator: 25
        HTF Indicator: 1
        LTF Strategy:  25
        HTF Strategy:  1
        Indicator line nan
        LTF Indicator: 26
        HTF Indicator: 1
        LTF Strategy:  26
        HTF Strategy:  1
        Indicator line nan
        LTF Indicator: 27
        HTF Indicator: 1
        LTF Strategy:  27
        HTF Strategy:  1
        Indicator line nan
        LTF Indicator: 28
        HTF Indicator: 1
        LTF Strategy:  28
        HTF Strategy:  1
        Indicator line nan
        LTF Indicator: 29
        HTF Indicator: 1
        LTF Strategy:  29
        HTF Strategy:  1
        Indicator line nan
        .....
        .....
        LTF Indicator: 374
        HTF Indicator: 24
        LTF Strategy:  374
        HTF Strategy:  24
        Indicator line nan
        LTF Indicator: 375
        HTF Indicator: 25
        LTF Strategy:  375
        HTF Strategy:  25
        Indicator line 1.0
        LTF Indicator: 376
        HTF Indicator: 25
        LTF Strategy:  376
        HTF Strategy:  25
        Indicator line 1.0
        

        In this case, the indicator seems to work as expected, providing NaN for the first 24 bars of the HTF data feed. However, the indicator and strategy next method seem to use the minimum period based on the LTF data feed (LTF starts at 25) and not on the HTF data feed. I don't understand why this happens.

        Once again thanks!

        1 Reply Last reply Reply Quote 0
        • 1 / 1
        • First post
          Last post
        Copyright © 2016, 2017, 2018 NodeBB Forums | Contributors
        $(document).ready(function () { app.coldLoad(); }); }