Navigation

    Backtrader Community

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

    Resampling tick data - missing candles

    General Code/Help
    2
    4
    527
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • H
      hodl last edited by

      Hi,
      First things first: Thank you for the great code! I'm new on boat and already very impressed.

      Here's a problem:
      I'm trying to import / resample tick data in following format:

      1546939820,3997.560000000000,1.000000000000
      1546939827,3997.550000000000,0.030602000000
      1546939828,3997.550000000000,0.008605000000
      1546939828,3999.740000000000,0.608984570000
      1546939828,3999.740000000000,0.005633000000
      1546939840,4002.930000000000,0.512500000000
      1546939840,4003.620000000000,3.748600000000
      1546939840,4003.720000000000,2.000000000000
      1546939840,4004.470000000000,0.010530000000
      1546939840,4005.640000000000,4.000000000000
      1546939846,4000.540000000000,0.351588260000
      1546939849,3998.220000000000,2.000000000000
      1546939850,3997.740000000000,1.306291820000
      1546939851,4004.220000000000,1.500000000000
      1546939851,4004.870000000000,2.000000000000
      

      my code is:

      from __future__ import (absolute_import, division, print_function,
                              unicode_literals)
      
      import datetime  # For datetime objects
      import os.path  # To manage paths
      import sys  # To find out the script name (in argv[0])
      
      # Import the backtrader platform
      import backtrader as bt
      import backtrader.feeds as btfeeds
      
      
      # Create a Stratey
      class TestStrategy(bt.Strategy):
      
          def log(self, txt, dt=None):
              ''' Logging function for this strategy'''
              dt = dt or self.datas[0].datetime.date(0)
              ti = self.datas[0].datetime.time(0)
              print('%s %s, %s' % (dt.isoformat(), ti.isoformat(), txt))
      
          def __init__(self):
              # Keep a reference to the "close" line in the data[0] dataseries
              self.open = self.datas[0].open
              self.high = self.datas[0].high
              self.low = self.datas[0].low
              self.close = self.datas[0].close
              self.volume = self.datas[0].volume
      
          def next(self):
              # Simply log the closing price of the series from the reference
              self.log('Open, %.2f' % self.open[0])
              self.log('Close, %.2f' % self.close[0])
              self.log('____________')
      
      
      
      
      if __name__ == '__main__':
          # Create a cerebro entity
          cerebro = bt.Cerebro()
      
          # Add a strategy
          cerebro.addstrategy(TestStrategy)
      
          # Datas are in a subfolder of the samples. Need to find where the script is
          # because it could have been called from anywhere
          modpath = '../'
      
          #    datapath = os.path.join(modpath, './datas/orcl-1995-2014.txt')
          datapath = os.path.join(modpath, './datas/bitstampUSDsmall.csv')
      
          data = btfeeds.GenericCSVData(
              dataname=datapath,
              dtformat=1,
              timeframe=bt.TimeFrame.Ticks,
              #        fromdate=datetime.datetime(2019, 1, 10),
              openinterest=-1,
              open=1,
              high=1,
              low=1,
              close=1,
              volume=2,
              header=False,
              separator=',',
              compression=1,
            )
      
      
      
          # Resample the data
          cerebro.resampledata(
              data,
              timeframe=bt.TimeFrame.Minutes,
              compression=240,
              bar2edge=True,
              adjbartime=True,
              rightedge=True)
      
          # Add the Data Feed to Cerebro
          cerebro.adddata(data)
      
          # Set our desired cash start
          cerebro.broker.setcash(100000.0)
      
          # Print out the starting conditions
          print('Starting Portfolio Value: %.2f' % cerebro.broker.getvalue())
      
          # Run over everything
          cerebro.run()
      
          # Print out the final result
          print('Final Portfolio Value: %.2f' % cerebro.broker.getvalue())
      

      and the result is:

      Starting Portfolio Value: 100000.00
      2019-01-08 16:00:00, Open, 4031.78
      2019-01-08 16:00:00, Close, 4031.82
      2019-01-08 16:00:00, ____________
      2019-01-08 23:59:59.999989, Open, 4015.90
      2019-01-08 23:59:59.999989, Close, 3994.90
      2019-01-08 23:59:59.999989, ____________
      2019-01-09 08:00:00, Open, 4018.84
      2019-01-09 08:00:00, Close, 4019.33
      2019-01-09 08:00:00, ____________
      2019-01-09 16:00:00, Open, 4025.00
      2019-01-09 16:00:00, Close, 3993.42
      2019-01-09 16:00:00, ____________
      2019-01-09 23:59:59.999989, Open, 4014.37
      2019-01-09 23:59:59.999989, Close, 4002.13
      2019-01-09 23:59:59.999989, ____________
      2019-01-10 08:00:00, Open, 4013.44
      2019-01-10 08:00:00, Close, 3803.42
      2019-01-10 08:00:00, ____________
      2019-01-10 16:00:00, Open, 3791.06
      2019-01-10 16:00:00, Close, 3782.40
      2019-01-10 16:00:00, ____________
      2019-01-11 00:00:00, Open, 3637.34
      2019-01-11 00:00:00, Close, 3627.65
      2019-01-11 00:00:00, ____________
      2019-01-11 08:00:00, Open, 3620.10
      2019-01-11 08:00:00, Close, 3643.65
      2019-01-11 08:00:00, ____________
      2019-01-11 16:00:00, Open, 3639.99
      2019-01-11 16:00:00, Close, 3635.20
      2019-01-11 16:00:00, ____________
      2019-01-12 04:00:00, Open, 3650.40
      2019-01-12 04:00:00, Close, 3638.49
      2019-01-12 04:00:00, ____________
      2019-01-12 12:00:00, Open, 3622.78
      2019-01-12 12:00:00, Close, 3617.21
      2019-01-12 12:00:00, ____________
      2019-01-12 20:00:00, Open, 3629.61
      2019-01-12 20:00:00, Close, 3629.11
      2019-01-12 20:00:00, ____________
      2019-01-13 08:00:00, Open, 3622.47
      2019-01-13 08:00:00, Close, 3613.77
      2019-01-13 08:00:00, ____________
      2019-01-13 16:00:00, Open, 3615.01
      2019-01-13 16:00:00, Close, 3611.06
      2019-01-13 16:00:00, ____________
      2019-01-14 04:00:00, Open, 3502.81
      2019-01-14 04:00:00, Close, 3520.01
      2019-01-14 04:00:00, ____________
      2019-01-14 12:00:00, Open, 3538.24
      2019-01-14 12:00:00, Close, 3528.26
      2019-01-14 12:00:00, ____________
      2019-01-14 20:00:00, Open, 3668.54
      2019-01-14 20:00:00, Close, 3683.76
      2019-01-14 20:00:00, ____________
      2019-01-15 08:00:00, Open, 3668.97
      2019-01-15 08:00:00, Close, 3667.96
      2019-01-15 08:00:00, ____________
      2019-01-15 16:00:00, Open, 3646.76
      2019-01-15 16:00:00, Close, 3632.47
      2019-01-15 16:00:00, ____________
      2019-01-16 00:00:00, Open, 3631.24
      2019-01-16 00:00:00, Close, 3576.92
      2019-01-16 00:00:00, ____________
      2019-01-16 08:00:00, Open, 3597.37
      2019-01-16 08:00:00, Close, 3578.60
      2019-01-16 08:00:00, ____________
      2019-01-16 12:00:00, Open, 3578.60
      2019-01-16 12:00:00, Close, 3615.24
      2019-01-16 12:00:00, ____________
      Final Portfolio Value: 100000.00
      
      Process finished with exit code 0
      
      

      One can see that resampler skips every other candle except for the last two which are ok.
      There is also strange time ending with '59:59.999989' . I tried with different timeframes (minutes, days) and the output is always the same, every second candle is missing.
      Anyone has idea what went wrong?
      I can provide full sample data (4mb) if needed.

      thnx,
      T

      H 1 Reply Last reply Reply Quote 0
      • H
        hodl @hodl last edited by

        Solved.
        Resampledata adds data to cerebro automagically. I added it manually with cerebro.adddata().

        thnx anyway,
        T.

        B 1 Reply Last reply Reply Quote 0
        • B
          backtrader administrators @hodl last edited by backtrader

          @hodl said in Resampling tick data - missing candles:

          Resampledata adds data to cerebro automagically

          There is nothing automagic in resampledata adding the data feed. The documentation says (Docs - Cerebro)

          resampledata(dataname, name=None, **kwargs)
          Adds a Data Feed to be resampled by the system
          
          If name is not None it will be put into data._name which is meant for decoration/plotting purposes.
          
          Any other kwargs like timeframe, compression, todate which are supported by the resample filter will be passed transparently
          
          H 1 Reply Last reply Reply Quote 0
          • H
            hodl @backtrader last edited by

            @backtrader thanks for pointing to relevant docs. I know it was you and not some magic :)

            1 Reply Last reply Reply Quote 1
            • 1 / 1
            • First post
              Last post
            Copyright © 2016, 2017, 2018, 2019, 2020, 2021 NodeBB Forums | Contributors