Navigation

    Backtrader Community

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

    Different results with resample data and add data methods!

    General Code/Help
    multi datafeed timeframe multiple data resampled data dataframes
    4
    17
    788
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • A
      aqa20 last edited by

      I get very different results between the two approaches shouldn't both of them deliver the same result!?

      Approach one:

      import backtrader as bt
      from datetime import datetime
      
      
      class CustomDataset(bt.feeds.GenericCSVData):
          ''' Instinitate CustomDataset '''
          params = (
              ('time', -1),
              ('datetime', 0),
              ('open', 1),
              ('high', 2),
              ('low', 3),
              ('close', 4),
              ('volume', 5),
              ('openinterest', 6),
          )
      
      
      cerebro = bt.Cerebro()
      
      data = CustomDataset(
          name='data',
          dataname='./5m-timeframe.csv',
          timeframe=bt.TimeFrame.Minutes,
          fromdate=datetime(2021, 3, 1),
          todate=datetime(2021, 3, 10)
      )
      
      
      data1 = CustomDataset(
          name='data1',
          dataname='./1h-timeframe.csv',
          timeframe=bt.TimeFrame.Minutes,
          fromdate=datetime(2021, 3, 1),
          todate=datetime(2021, 3, 10)
      )
      
      cerebro.adddata(data)
      cerebro.adddata(data1)
      

      Approach 2

      import backtrader as bt
      from datetime import datetime
      
      
      class CustomDataset(bt.feeds.GenericCSVData):
          ''' Instinitate CustomDataset '''
          params = (
              ('time', -1),
              ('datetime', 0),
              ('open', 1),
              ('high', 2),
              ('low', 3),
              ('close', 4),
              ('volume', 5),
              ('openinterest', 6),
          )
      
      
      cerebro = bt.Cerebro()
      
      data = CustomDataset(
          name='data',
          dataname='./1m-timeframe.csv',
          timeframe=bt.TimeFrame.Minutes,
          fromdate=datetime(2021, 3, 1),
          todate=datetime(2021, 3, 10)
      )
      
      cerebro.resampledata(data, timeframe=bt.TimeFrame.Minutes, compression=5, name='5m')
      cerebro.resampledata(data, timeframe=bt.TimeFrame.Minutes, compression=60, name='60m')
      
      D E 2 Replies Last reply Reply Quote 0
      • D
        dasch @aqa20 last edited by

        @aqa20 in your approach one you did not define the data compression, so all your csv are defined as 1 minute data, which is wrong. Try approach one with adding the param compression.

        A 1 Reply Last reply Reply Quote 1
        • A
          aqa20 @dasch last edited by

          @dasch said in Different results with resample data and add data methods!:

          our approach one you did not define the data compression, so all your csv are defined as 1 minute data, which is wrong. Try approach one with adding the param compression.

          I'm loading data from 2 different files, the first one is 5m data, and the second is 1h data, and I haven't added the compression parameter in the first approach because the default value for compression is one

          D 1 Reply Last reply Reply Quote 0
          • D
            dasch @aqa20 last edited by

            @aqa20 so for 5 minutes you would use these params:

            compression=5, timeframe=bt.TimeFrame.Minutes

            for 1 hour data you would use these params:

            compression=60, timeframe=bt.TimeFrame.Minutes

            A D 2 Replies Last reply Reply Quote 1
            • A
              aqa20 @dasch last edited by

              @dasch Well that's what I've done already, check the code

              D 2 Replies Last reply Reply Quote 0
              • D
                dasch @dasch last edited by

                but you need to define, what kind of data you are using. so in approach one you just don't define the data granularity which is required. the result should be the same after defining the compression in approach one.

                1 Reply Last reply Reply Quote 1
                • D
                  dasch @aqa20 last edited by

                  @aqa20 said in Different results with resample data and add data methods!:

                  @dasch Well that's what I've done already, check the code

                  no, you did not define the data correctly in approach one. you left out the compression, which is needed there.

                  1 Reply Last reply Reply Quote 1
                  • D
                    dasch @aqa20 last edited by

                    @aqa20 approach two works because there you defined to what compreession you want to resample your data to. backtrader will check if the timestamp of current candle is over the period and forward the data when its over (which is basically every row, since you already use the matching timeframe/compression in your source data).
                    If you just define a data source and add it to bbacktrader, then backtrader does not know, what granularity your data has.
                    this is why you need to define in approach one also the compression of your data.

                    A 1 Reply Last reply Reply Quote 1
                    • E
                      EMR @aqa20 last edited by

                      @aqa20

                      • What are the differences that you find between these 2 approaches ?
                      • In approach 1 you load datas which are already in 5 minutes & 60 minutes compression. Do you know if this compression is the same as when done by backtrader (ie bar2edge, boundoff, rightedge etc.)
                      • Do you have a different result in approach 1 if you add, along with the timeframe parameter, the parameters compression = 5 & compression = 60 to data & data1 ?
                      • Have you checked for missing minutes in the 1 minute data file ?
                      A 1 Reply Last reply Reply Quote 1
                      • A
                        aqa20 @dasch last edited by

                        @dasch I've tried to add compression to the first approach, but I still get different result from 2 approaches

                        1 Reply Last reply Reply Quote 0
                        • A
                          aqa20 @EMR last edited by

                          @emr
                          The differences is in the chart and the time execution orders,
                          In approach one if the buy order was executed on [05-03-21 12:00], the buy order will be executed on [05-03-21 13:00] on the the second approach, and same for sell orders.

                          Do you know if this compression is the same as when done by backtrader (ie bar2edge, boundoff, rightedge etc?

                          I believe so here is a sample of the 1m data

                          2021-03-24 00:00:00,54342.8,54359.0,54230.0,54260.35,1616544059999,2153
                          2021-03-24 00:01:00,54260.36,54422.43,54248.53,54408.13,1616544119999,1364
                          2021-03-24 00:02:00,54402.67,54515.73,54394.46,54515.73,1616544179999,1194
                          2021-03-24 00:03:00,54511.02,54606.15,54511.02,54592.13,1616544239999,1324
                          2021-03-24 00:04:00,54592.12,54669.35,54563.21,54597.62,1616544299999,1381
                          2021-03-24 00:05:00,54595.26,54626.39,54518.44,54555.23,1616544359999,909
                          

                          Sample from the 1h data:

                          2021-03-24 00:00:00,255.5191,256.9375,250.8104,251.6466,1616547599999,51155
                          2021-03-24 01:00:00,251.5354,256.0,251.1514,255.6516,1616551199999,20791
                          2021-03-24 02:00:00,255.6426,256.9996,254.8947,255.6458,1616554799999,19972
                          2021-03-24 03:00:00,255.6458,256.6457,254.2957,256.1307,1616558399999,21009
                          2021-03-24 04:00:00,256.1242,256.98,253.9556,255.0727,1616561999999,22185
                          2021-03-24 05:00:00,255.0662,255.7192,252.8269,255.5432,1616565599999,18944
                          

                          Sample from 5m data:

                          2021-03-24 00:00:00,255.5191,256.9375,254.6,256.813,1616544299999,3381
                          2021-03-24 00:05:00,256.8,256.9372,254.11,254.45,1616544599999,4276
                          2021-03-24 00:10:00,254.3532,255.0489,253.05,253.6577,1616544899999,4797
                          2021-03-24 00:15:00,253.8025,253.809,252.102,252.1554,1616545199999,4867
                          2021-03-24 00:20:00,252.305,253.4392,251.7721,252.3381,1616545499999,6368
                          2021-03-24 00:25:00,252.3289,252.63,250.8104,252.0226,1616545799999,6691
                          2021-03-24 00:30:00,252.0087,253.0,251.5001,252.4069,1616546099999,4125
                          

                          Do you have a different result in approach 1 if you add, along with the timeframe parameter, the parameters compression = 5 & compression = 60 to data & data1 ?

                          Yes!

                          Have you checked for missing minutes in the 1 minute data file ?
                          Yes i checked for that using pandas

                          A run-out E 3 Replies Last reply Reply Quote 0
                          • A
                            aqa20 @aqa20 last edited by

                            @aqa20 I'm sorry actually It don't get different result when adding, 5 and 60 compression permeates.

                            so the answer for this question:
                            " [Do you have a different result in approach 1 if you add, along with the timeframe parameter, the parameters compression = 5 & compression = 60 to data & data1 ?](link url)"
                            Is no

                            1 Reply Last reply Reply Quote 0
                            • run-out
                              run-out @aqa20 last edited by

                              @aqa20
                              The only way you will resolve this with certainty is to turn off/comment out/don't print the trading algo part, and just print the data and carefully compare the two versions.

                              You will immediately see where the discrepencies are and can start to debug from there.

                              I believe the previous posters have all given you good clues. Do some investigation on the data itself and try to find where the problem lies.

                              It might be possible your data/datasource methodology is not lined up with backtrader, since there are variables that can change when resampling.

                              RunBacktest.com

                              A 1 Reply Last reply Reply Quote 2
                              • A
                                aqa20 @run-out last edited by

                                @run-out I have done that and in the first approach the data was correct,
                                in next method I print the datetime along with other values OHLVC, and the values were matching perfectly with the data
                                source in the first approach, however in the second approach it was completely different values! but I still don't know where is the problem in approach 2!

                                D 1 Reply Last reply Reply Quote 0
                                • D
                                  dasch @aqa20 last edited by

                                  @aqa20 take a look here: https://community.backtrader.com/topic/1629/problem-with-resampling-1m-data-to-5m-15m/

                                  A 1 Reply Last reply Reply Quote 1
                                  • A
                                    aqa20 @dasch last edited by

                                    @dasch Thanks man, but for seek of simplicity I will stick with the first approach for now since it's working and I already have the data of time frames that I need, but now I'm trying to use the first approach in the live trading, so let me know if you can help with that but without using the resample data, I want to achieve that with first approach

                                    1 Reply Last reply Reply Quote 0
                                    • E
                                      EMR @aqa20 last edited by

                                      @aqa20 said in Different results with resample data and add data methods!:

                                      @emr

                                      Do you know if this compression is the same as when done by backtrader (ie bar2edge, boundoff, rightedge etc?

                                      I believe so

                                      I noticed that you can have very different presentation of 5mn, 1h etc data depending of the data provider. For me, it was differences between data from MT5 & data resampled by BT. For example, 1h data from MT5 had a different close value. You may want to check the M5 & H1 open & close values to find if it is the one of neighboring minutes and also check the doc to find if some additional parameters (boundoff, rightedge etc.) can bring you the results you want

                                      Also, I noticed during my tests that if some minutes are missing, the resampling to for ex. H8 may lead to errors (closure of the bar occur later than expected - I still have to find the time to log an issue in BT2 for that)

                                      1 Reply Last reply Reply Quote 1
                                      • 1 / 1
                                      • First post
                                        Last post
                                      Copyright © 2016, 2017, 2018, 2019, 2020, 2021 NodeBB Forums | Contributors