For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

Resampling tick data - missing candles



  • Hi,
    First things first: Thank you for the great code! I'm new on boat and already very impressed.

    Here's a problem:
    I'm trying to import / resample tick data in following format:

    1546939820,3997.560000000000,1.000000000000
    1546939827,3997.550000000000,0.030602000000
    1546939828,3997.550000000000,0.008605000000
    1546939828,3999.740000000000,0.608984570000
    1546939828,3999.740000000000,0.005633000000
    1546939840,4002.930000000000,0.512500000000
    1546939840,4003.620000000000,3.748600000000
    1546939840,4003.720000000000,2.000000000000
    1546939840,4004.470000000000,0.010530000000
    1546939840,4005.640000000000,4.000000000000
    1546939846,4000.540000000000,0.351588260000
    1546939849,3998.220000000000,2.000000000000
    1546939850,3997.740000000000,1.306291820000
    1546939851,4004.220000000000,1.500000000000
    1546939851,4004.870000000000,2.000000000000
    

    my code is:

    from __future__ import (absolute_import, division, print_function,
                            unicode_literals)
    
    import datetime  # For datetime objects
    import os.path  # To manage paths
    import sys  # To find out the script name (in argv[0])
    
    # Import the backtrader platform
    import backtrader as bt
    import backtrader.feeds as btfeeds
    
    
    # Create a Stratey
    class TestStrategy(bt.Strategy):
    
        def log(self, txt, dt=None):
            ''' Logging function for this strategy'''
            dt = dt or self.datas[0].datetime.date(0)
            ti = self.datas[0].datetime.time(0)
            print('%s %s, %s' % (dt.isoformat(), ti.isoformat(), txt))
    
        def __init__(self):
            # Keep a reference to the "close" line in the data[0] dataseries
            self.open = self.datas[0].open
            self.high = self.datas[0].high
            self.low = self.datas[0].low
            self.close = self.datas[0].close
            self.volume = self.datas[0].volume
    
        def next(self):
            # Simply log the closing price of the series from the reference
            self.log('Open, %.2f' % self.open[0])
            self.log('Close, %.2f' % self.close[0])
            self.log('____________')
    
    
    
    
    if __name__ == '__main__':
        # Create a cerebro entity
        cerebro = bt.Cerebro()
    
        # Add a strategy
        cerebro.addstrategy(TestStrategy)
    
        # Datas are in a subfolder of the samples. Need to find where the script is
        # because it could have been called from anywhere
        modpath = '../'
    
        #    datapath = os.path.join(modpath, './datas/orcl-1995-2014.txt')
        datapath = os.path.join(modpath, './datas/bitstampUSDsmall.csv')
    
        data = btfeeds.GenericCSVData(
            dataname=datapath,
            dtformat=1,
            timeframe=bt.TimeFrame.Ticks,
            #        fromdate=datetime.datetime(2019, 1, 10),
            openinterest=-1,
            open=1,
            high=1,
            low=1,
            close=1,
            volume=2,
            header=False,
            separator=',',
            compression=1,
          )
    
    
    
        # Resample the data
        cerebro.resampledata(
            data,
            timeframe=bt.TimeFrame.Minutes,
            compression=240,
            bar2edge=True,
            adjbartime=True,
            rightedge=True)
    
        # Add the Data Feed to Cerebro
        cerebro.adddata(data)
    
        # Set our desired cash start
        cerebro.broker.setcash(100000.0)
    
        # Print out the starting conditions
        print('Starting Portfolio Value: %.2f' % cerebro.broker.getvalue())
    
        # Run over everything
        cerebro.run()
    
        # Print out the final result
        print('Final Portfolio Value: %.2f' % cerebro.broker.getvalue())
    

    and the result is:

    Starting Portfolio Value: 100000.00
    2019-01-08 16:00:00, Open, 4031.78
    2019-01-08 16:00:00, Close, 4031.82
    2019-01-08 16:00:00, ____________
    2019-01-08 23:59:59.999989, Open, 4015.90
    2019-01-08 23:59:59.999989, Close, 3994.90
    2019-01-08 23:59:59.999989, ____________
    2019-01-09 08:00:00, Open, 4018.84
    2019-01-09 08:00:00, Close, 4019.33
    2019-01-09 08:00:00, ____________
    2019-01-09 16:00:00, Open, 4025.00
    2019-01-09 16:00:00, Close, 3993.42
    2019-01-09 16:00:00, ____________
    2019-01-09 23:59:59.999989, Open, 4014.37
    2019-01-09 23:59:59.999989, Close, 4002.13
    2019-01-09 23:59:59.999989, ____________
    2019-01-10 08:00:00, Open, 4013.44
    2019-01-10 08:00:00, Close, 3803.42
    2019-01-10 08:00:00, ____________
    2019-01-10 16:00:00, Open, 3791.06
    2019-01-10 16:00:00, Close, 3782.40
    2019-01-10 16:00:00, ____________
    2019-01-11 00:00:00, Open, 3637.34
    2019-01-11 00:00:00, Close, 3627.65
    2019-01-11 00:00:00, ____________
    2019-01-11 08:00:00, Open, 3620.10
    2019-01-11 08:00:00, Close, 3643.65
    2019-01-11 08:00:00, ____________
    2019-01-11 16:00:00, Open, 3639.99
    2019-01-11 16:00:00, Close, 3635.20
    2019-01-11 16:00:00, ____________
    2019-01-12 04:00:00, Open, 3650.40
    2019-01-12 04:00:00, Close, 3638.49
    2019-01-12 04:00:00, ____________
    2019-01-12 12:00:00, Open, 3622.78
    2019-01-12 12:00:00, Close, 3617.21
    2019-01-12 12:00:00, ____________
    2019-01-12 20:00:00, Open, 3629.61
    2019-01-12 20:00:00, Close, 3629.11
    2019-01-12 20:00:00, ____________
    2019-01-13 08:00:00, Open, 3622.47
    2019-01-13 08:00:00, Close, 3613.77
    2019-01-13 08:00:00, ____________
    2019-01-13 16:00:00, Open, 3615.01
    2019-01-13 16:00:00, Close, 3611.06
    2019-01-13 16:00:00, ____________
    2019-01-14 04:00:00, Open, 3502.81
    2019-01-14 04:00:00, Close, 3520.01
    2019-01-14 04:00:00, ____________
    2019-01-14 12:00:00, Open, 3538.24
    2019-01-14 12:00:00, Close, 3528.26
    2019-01-14 12:00:00, ____________
    2019-01-14 20:00:00, Open, 3668.54
    2019-01-14 20:00:00, Close, 3683.76
    2019-01-14 20:00:00, ____________
    2019-01-15 08:00:00, Open, 3668.97
    2019-01-15 08:00:00, Close, 3667.96
    2019-01-15 08:00:00, ____________
    2019-01-15 16:00:00, Open, 3646.76
    2019-01-15 16:00:00, Close, 3632.47
    2019-01-15 16:00:00, ____________
    2019-01-16 00:00:00, Open, 3631.24
    2019-01-16 00:00:00, Close, 3576.92
    2019-01-16 00:00:00, ____________
    2019-01-16 08:00:00, Open, 3597.37
    2019-01-16 08:00:00, Close, 3578.60
    2019-01-16 08:00:00, ____________
    2019-01-16 12:00:00, Open, 3578.60
    2019-01-16 12:00:00, Close, 3615.24
    2019-01-16 12:00:00, ____________
    Final Portfolio Value: 100000.00
    
    Process finished with exit code 0
    
    

    One can see that resampler skips every other candle except for the last two which are ok.
    There is also strange time ending with '59:59.999989' . I tried with different timeframes (minutes, days) and the output is always the same, every second candle is missing.
    Anyone has idea what went wrong?
    I can provide full sample data (4mb) if needed.

    thnx,
    T



  • Solved.
    Resampledata adds data to cerebro automagically. I added it manually with cerebro.adddata().

    thnx anyway,
    T.


  • administrators

    @hodl said in Resampling tick data - missing candles:

    Resampledata adds data to cerebro automagically

    There is nothing automagic in resampledata adding the data feed. The documentation says (Docs - Cerebro)

    resampledata(dataname, name=None, **kwargs)
    Adds a Data Feed to be resampled by the system
    
    If name is not None it will be put into data._name which is meant for decoration/plotting purposes.
    
    Any other kwargs like timeframe, compression, todate which are supported by the resample filter will be passed transparently
    


  • @backtrader thanks for pointing to relevant docs. I know it was you and not some magic :)


Log in to reply
 

});