Resampling tick data - missing candles
-
Hi,
First things first: Thank you for the great code! I'm new on boat and already very impressed.Here's a problem:
I'm trying to import / resample tick data in following format:1546939820,3997.560000000000,1.000000000000 1546939827,3997.550000000000,0.030602000000 1546939828,3997.550000000000,0.008605000000 1546939828,3999.740000000000,0.608984570000 1546939828,3999.740000000000,0.005633000000 1546939840,4002.930000000000,0.512500000000 1546939840,4003.620000000000,3.748600000000 1546939840,4003.720000000000,2.000000000000 1546939840,4004.470000000000,0.010530000000 1546939840,4005.640000000000,4.000000000000 1546939846,4000.540000000000,0.351588260000 1546939849,3998.220000000000,2.000000000000 1546939850,3997.740000000000,1.306291820000 1546939851,4004.220000000000,1.500000000000 1546939851,4004.870000000000,2.000000000000
my code is:
from __future__ import (absolute_import, division, print_function, unicode_literals) import datetime # For datetime objects import os.path # To manage paths import sys # To find out the script name (in argv[0]) # Import the backtrader platform import backtrader as bt import backtrader.feeds as btfeeds # Create a Stratey class TestStrategy(bt.Strategy): def log(self, txt, dt=None): ''' Logging function for this strategy''' dt = dt or self.datas[0].datetime.date(0) ti = self.datas[0].datetime.time(0) print('%s %s, %s' % (dt.isoformat(), ti.isoformat(), txt)) def __init__(self): # Keep a reference to the "close" line in the data[0] dataseries self.open = self.datas[0].open self.high = self.datas[0].high self.low = self.datas[0].low self.close = self.datas[0].close self.volume = self.datas[0].volume def next(self): # Simply log the closing price of the series from the reference self.log('Open, %.2f' % self.open[0]) self.log('Close, %.2f' % self.close[0]) self.log('____________') if __name__ == '__main__': # Create a cerebro entity cerebro = bt.Cerebro() # Add a strategy cerebro.addstrategy(TestStrategy) # Datas are in a subfolder of the samples. Need to find where the script is # because it could have been called from anywhere modpath = '../' # datapath = os.path.join(modpath, './datas/orcl-1995-2014.txt') datapath = os.path.join(modpath, './datas/bitstampUSDsmall.csv') data = btfeeds.GenericCSVData( dataname=datapath, dtformat=1, timeframe=bt.TimeFrame.Ticks, # fromdate=datetime.datetime(2019, 1, 10), openinterest=-1, open=1, high=1, low=1, close=1, volume=2, header=False, separator=',', compression=1, ) # Resample the data cerebro.resampledata( data, timeframe=bt.TimeFrame.Minutes, compression=240, bar2edge=True, adjbartime=True, rightedge=True) # Add the Data Feed to Cerebro cerebro.adddata(data) # Set our desired cash start cerebro.broker.setcash(100000.0) # Print out the starting conditions print('Starting Portfolio Value: %.2f' % cerebro.broker.getvalue()) # Run over everything cerebro.run() # Print out the final result print('Final Portfolio Value: %.2f' % cerebro.broker.getvalue())
and the result is:
Starting Portfolio Value: 100000.00 2019-01-08 16:00:00, Open, 4031.78 2019-01-08 16:00:00, Close, 4031.82 2019-01-08 16:00:00, ____________ 2019-01-08 23:59:59.999989, Open, 4015.90 2019-01-08 23:59:59.999989, Close, 3994.90 2019-01-08 23:59:59.999989, ____________ 2019-01-09 08:00:00, Open, 4018.84 2019-01-09 08:00:00, Close, 4019.33 2019-01-09 08:00:00, ____________ 2019-01-09 16:00:00, Open, 4025.00 2019-01-09 16:00:00, Close, 3993.42 2019-01-09 16:00:00, ____________ 2019-01-09 23:59:59.999989, Open, 4014.37 2019-01-09 23:59:59.999989, Close, 4002.13 2019-01-09 23:59:59.999989, ____________ 2019-01-10 08:00:00, Open, 4013.44 2019-01-10 08:00:00, Close, 3803.42 2019-01-10 08:00:00, ____________ 2019-01-10 16:00:00, Open, 3791.06 2019-01-10 16:00:00, Close, 3782.40 2019-01-10 16:00:00, ____________ 2019-01-11 00:00:00, Open, 3637.34 2019-01-11 00:00:00, Close, 3627.65 2019-01-11 00:00:00, ____________ 2019-01-11 08:00:00, Open, 3620.10 2019-01-11 08:00:00, Close, 3643.65 2019-01-11 08:00:00, ____________ 2019-01-11 16:00:00, Open, 3639.99 2019-01-11 16:00:00, Close, 3635.20 2019-01-11 16:00:00, ____________ 2019-01-12 04:00:00, Open, 3650.40 2019-01-12 04:00:00, Close, 3638.49 2019-01-12 04:00:00, ____________ 2019-01-12 12:00:00, Open, 3622.78 2019-01-12 12:00:00, Close, 3617.21 2019-01-12 12:00:00, ____________ 2019-01-12 20:00:00, Open, 3629.61 2019-01-12 20:00:00, Close, 3629.11 2019-01-12 20:00:00, ____________ 2019-01-13 08:00:00, Open, 3622.47 2019-01-13 08:00:00, Close, 3613.77 2019-01-13 08:00:00, ____________ 2019-01-13 16:00:00, Open, 3615.01 2019-01-13 16:00:00, Close, 3611.06 2019-01-13 16:00:00, ____________ 2019-01-14 04:00:00, Open, 3502.81 2019-01-14 04:00:00, Close, 3520.01 2019-01-14 04:00:00, ____________ 2019-01-14 12:00:00, Open, 3538.24 2019-01-14 12:00:00, Close, 3528.26 2019-01-14 12:00:00, ____________ 2019-01-14 20:00:00, Open, 3668.54 2019-01-14 20:00:00, Close, 3683.76 2019-01-14 20:00:00, ____________ 2019-01-15 08:00:00, Open, 3668.97 2019-01-15 08:00:00, Close, 3667.96 2019-01-15 08:00:00, ____________ 2019-01-15 16:00:00, Open, 3646.76 2019-01-15 16:00:00, Close, 3632.47 2019-01-15 16:00:00, ____________ 2019-01-16 00:00:00, Open, 3631.24 2019-01-16 00:00:00, Close, 3576.92 2019-01-16 00:00:00, ____________ 2019-01-16 08:00:00, Open, 3597.37 2019-01-16 08:00:00, Close, 3578.60 2019-01-16 08:00:00, ____________ 2019-01-16 12:00:00, Open, 3578.60 2019-01-16 12:00:00, Close, 3615.24 2019-01-16 12:00:00, ____________ Final Portfolio Value: 100000.00 Process finished with exit code 0
One can see that resampler skips every other candle except for the last two which are ok.
There is also strange time ending with '59:59.999989' . I tried with different timeframes (minutes, days) and the output is always the same, every second candle is missing.
Anyone has idea what went wrong?
I can provide full sample data (4mb) if needed.thnx,
T -
Solved.
Resampledata adds data to cerebro automagically. I added it manually with cerebro.adddata().thnx anyway,
T. -
@hodl said in Resampling tick data - missing candles:
Resampledata adds data to cerebro automagically
There is nothing automagic in
resampledata
adding the data feed. The documentation says (Docs - Cerebro)resampledata(dataname, name=None, **kwargs) Adds a Data Feed to be resampled by the system If name is not None it will be put into data._name which is meant for decoration/plotting purposes. Any other kwargs like timeframe, compression, todate which are supported by the resample filter will be passed transparently
-
@backtrader thanks for pointing to relevant docs. I know it was you and not some magic :)