Navigation

    Backtrader Community

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

    cerebro.resampledata() vs pandas .resample()

    General Code/Help
    2
    4
    1002
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • qwert666
      qwert666 last edited by qwert666

      I'm trying to figure out what way is the proper way of aggregating the data.
      I have some minute data from the market and I want to do some daily TA, i've tried two approaches that gives me a bit different final output.

      First attempt using the resampledata() method:

      exampleDataWithoutPreResample = coinBaseUSD.loc['2015-02-01':'2015-03-14'][['Open','High','Low','Close','Volume']]
      data = bt.feeds.PandasData(dataname=exampleDataWithoutPreResample, timeframe=bt.TimeFrame.Minutes, openinterest=None)
      foo = cerebro.resampledata(data, timeframe=bt.TimeFrame.Days)
      

      Logs that i print

      2015-03-13, Sell created, 287.57
      2015-03-14, SELL EXECUTED, Price: 295.74, Cost: 4350.45, Comm 44.36 << ??
      2015-03-14, OPERATION PROFIT, GROSS 85.65, NET -2.22
      2015-03-14, (MA Period  6) Ending Value 99773.58
      

      The second approach with pre aggregation in Pandas:

      ohlc_dict = {                                                                                                             
          'Open':'first',
          'High':'max',
          'Low':'min',
          'Close': 'last',
          'Volume': 'sum'
      }
      exampleData = coinBaseUSD.loc['2015-02-01':'2015-03-14'][['Open','High','Low','Close','Volume']].resample('D', how=ohlc_dict)
      data = bt.feeds.PandasData(dataname=exampleData)
      foo2 = cerebro.adddata(data)
      

      Everything match to the last Sell execution

      2015-03-13, Sell created, 287.57
      2015-03-14, SELL EXECUTED, Price: 287.65, Cost: 4350.45, Comm 43.15 << ?
      2015-03-14, OPERATION PROFIT, GROSS -35.70, NET -122.35
      2015-03-14, (MA Period  6) Ending Value 99653.44
      
      >> print(foo.getwritervalues(), '\n',foo2.getwritervalues())
      ['', 42, datetime.datetime(2015, 3, 14, 23, 59, 59, 999989), 287.65, 289.0, 280.86, 284.83, 7898.626718840009, nan] 
       ['', 42, datetime.datetime(2015, 3, 14, 0, 0), 287.65, 289.0, 280.86, 284.83, 7898.626718840009, nan]
      
      exampleData.tail(5)
      Timestamp       Open	High	Low	Close	Volume
      2015-03-12	296.83	298.90	291.87	295.91	9698.128688
      2015-03-13	295.74	296.52	284.86	287.57	11889.603667
      2015-03-14	287.65	289.00	280.86	284.83	7898.626719
      

      The difference is with in the SELL EXECUTED, Price: xxx
      First example takes the open value from 2015-03-13 and the second example takes the open value from 2015-03-14
      Ah and why is that the resampledata is way slower when running

      logic behind sell execution:

      def next(self):
          # ....
          if self.dataclose[0] < self.sma[0]:
              self.log("Sell created, %.2f" % self.dataclose[0])
      
      def notify_order():
          # ....
          if order.issell():
              self.log('SELL EXECUTED, Price: %.2f, Cost: %.2f, Comm %.2f' %
                          (order.executed.price,
                          order.executed.value,
                          order.executed.comm))
      
      1 Reply Last reply Reply Quote 0
      • B
        backtrader administrators last edited by

        The following would actually help:

        • print the actual datetime and open, close prices in your next.

        Additionally

        @qwert666 said in cerebro.resampledata() vs pandas .resample():

        def notify_order():
            # ....
            if order.issell():
                self.log('SELL EXECUTED, Price: %.2f, Cost: %.2f, Comm %.2f' %
                            (order.executed.price,
                            order.executed.value,
                            order.executed.comm))
        

        Should one assume that you are checking the actual notification status before printing? (Accepted, Completed)

        Of course it is also assumed that both things are running with the same parameters and not one with cheat-on-close (or similar approach) and not the other.

        qwert666 1 Reply Last reply Reply Quote 0
        • qwert666
          qwert666 @backtrader last edited by

          @backtrader said in cerebro.resampledata() vs pandas .resample():

          datetime and open, close

          Yes both are running with the same parameters i've paste only the part that seemed relevant to solve the problem

          if self.dataclose[0] < self.sma[0]:
              self.log("DEBUG %s, %.2f" % (bt.num2date(self.datetime[0]).isoformat(), self.dataopen[0]))
              self.log("Sell created, %.2f" % self.dataclose[0])
          
          2015-03-13, DEBUG 2015-03-13T23:59:59.999989, 295.74
          2015-03-13, Sell created, 287.57
          2015-03-14, SELL EXECUTED, Price: 295.74, Cost: 4350.45, Comm 44.36
          2015-03-14, OPERATION PROFIT, GROSS 85.65, NET -2.22
          2015-03-14, (MA Period  6) Ending Value 99773.58
          

          and the second approach with Pandas

          2015-03-13, DEBUG 2015-03-13T00:00:00, 295.74
          2015-03-13, Sell created, 287.57
          2015-03-14, SELL EXECUTED, Price: 287.65, Cost: 4350.45, Comm 43.15
          2015-03-14, OPERATION PROFIT, GROSS -35.70, NET -122.35
          2015-03-14, (MA Period  6) Ending Value 99653.44
          

          .resample('D', how=ohlc_dict) cut the hours and the resampledata() leave it with 23:59 it's also visible in the values returned by getwritervalues could it be the reason why it behave in this way?

          1 Reply Last reply Reply Quote 0
          • B
            backtrader administrators last edited by

            Seeing how the data is actually resampled (what pandas gives you and what backtrader gives you) would seem relevant. Not only when buying and selling.

            In addition to that and since the differences you see seem to be in the transition from 2015-03-13 to 2015-03-14, it would be ideal to see how the 1-minute bars around the transition look like.

            backtrader gives you a resampled 1-day bar which is at the end of the day. The rationale behind:

            • If you mix that with a smaller timeframe, the 1-day bar is after the smaller timeframes.

            pandas is giving you a resampled data at the beginning of the day. Now you mix 2 timeframes and:

            • 2015-03-13 10:01:00 (1-minute timeframe) happens actually later than 2015-03-13 00:00:00 (1-day resampled by pandas)

            Which wouldn't really make sense.

            1 Reply Last reply Reply Quote 0
            • 1 / 1
            • First post
              Last post
            Copyright © 2016, 2017, 2018, 2019, 2020, 2021 NodeBB Forums | Contributors