Problem with resampling of intraday data



  • I am using 1min data for EURUSD, from the csv file. Data are read correctly and plotted fine.
    Ia m also resampling these data to daily timeframe but something doesn't work correctly. I am getting peculiar chart for daily data.

    0_1485528401703_1min_resample.png

    In next() function I print this:

    txt = ','.join(
                [self.data.datetime.date(0).isoformat() +" "
                 + self.data.datetime.time(0).isoformat()+", 1min data (OHLC): "
                '%.5f' % self.data.open[0]+", "
                '%.5f' % self.data.high[0]+", "
                '%.5f' % self.data.low[0]+", "
                '%.5f' % self.data.close[0]+", Daily data (OHLC):"
                '%.5f' % self.data1.open[0]+", "
                '%.5f' % self.data1.high[0]+", "
                '%.5f' % self.data1.low[0]+", "
                '%.5f' % self.data1.close[0]
                ])
    
    print(txt)
    

    The excerpt of the output:

    2016-01-04 15:53:00, 1min data (OHLC): 1.08115, 1.08151, 1.08112, 1.08147, Daily data (OHLC):1.08718, 1.08727, 1.08451, 1.08516
    2016-01-04 15:54:00, 1min data (OHLC): 1.08146, 1.08151, 1.08122, 1.08128, Daily data (OHLC):1.08718, 1.08727, 1.08451, 1.08516
    2016-01-04 15:55:00, 1min data (OHLC): 1.08128, 1.08133, 1.08113, 1.08131, Daily data (OHLC):1.08718, 1.08727, 1.08451, 1.08516
    2016-01-04 15:56:00, 1min data (OHLC): 1.08135, 1.08146, 1.08104, 1.08118, Daily data (OHLC):1.08718, 1.08727, 1.08451, 1.08516
    2016-01-04 15:57:00, 1min data (OHLC): 1.08117, 1.08148, 1.08079, 1.08083, Daily data (OHLC):1.08718, 1.08727, 1.08451, 1.08516
    2016-01-04 15:58:00, 1min data (OHLC): 1.08082, 1.08083, 1.08037, 1.08048, Daily data (OHLC):1.08718, 1.08727, 1.08451, 1.08516
    2016-01-04 15:59:00, 1min data (OHLC): 1.08048, 1.08054, 1.08012, 1.08015, Daily data (OHLC):1.08718, 1.08727, 1.08451, 1.08516
    2016-01-04 16:00:00, 1min data (OHLC): 1.08016, 1.08046, 1.08016, 1.08046, Daily data (OHLC):1.08718, 1.08727, 1.08451, 1.08516
    2016-01-04 16:01:00, 1min data (OHLC): 1.08045, 1.08048, 1.08021, 1.08037, Daily data (OHLC):1.08718, 1.08727, 1.08451, 1.08516
    2016-01-04 16:02:00, 1min data (OHLC): 1.08039, 1.08099, 1.08037, 1.08097, Daily data (OHLC):1.08718, 1.08727, 1.08451, 1.08516
    2016-01-04 16:03:00, 1min data (OHLC): 1.08098, 1.08123, 1.08064, 1.08076, Daily data (OHLC):1.08718, 1.08727, 1.08451, 1.08516
    2016-01-04 16:04:00, 1min data (OHLC): 1.08076, 1.08093, 1.08064, 1.08085, Daily data (OHLC):1.08718, 1.08727, 1.08451, 1.08516
    2016-01-04 16:05:00, 1min data (OHLC): 1.08085, 1.08102, 1.08021, 1.08022, Daily data (OHLC):1.08718, 1.08727, 1.08451, 1.08516
    2016-01-04 16:06:00, 1min data (OHLC): 1.08016, 1.08062, 1.08016, 1.08062, Daily data (OHLC):1.08718, 1.08727, 1.08451, 1.08516
    2016-01-04 16:07:00, 1min data (OHLC): 1.08060, 1.08060, 1.08011, 1.08032, Daily data (OHLC):1.08718, 1.08727, 1.08451, 1.08516
    2016-01-04 16:08:00, 1min data (OHLC): 1.08031, 1.08047, 1.07985, 1.08000, Daily data (OHLC):1.08718, 1.08727, 1.08451, 1.08516
    2016-01-04 16:09:00, 1min data (OHLC): 1.08000, 1.08059, 1.07998, 1.08050, Daily data (OHLC):1.08718, 1.08727, 1.08451, 1.08516
    2016-01-04 16:10:00, 1min data (OHLC): 1.08050, 1.08051, 1.07998, 1.08007, Daily data (OHLC):1.08718, 1.08727, 1.08451, 1.08516
    2016-01-04 16:11:00, 1min data (OHLC): 1.08008, 1.08013, 1.08000, 1.08002, Daily data (OHLC):1.08718, 1.08727, 1.08451, 1.08516
    2016-01-04 16:12:00, 1min data (OHLC): 1.08006, 1.08009, 1.07992, 1.08003, Daily data (OHLC):1.08718, 1.08727, 1.08451, 1.08516
    2016-01-04 16:13:00, 1min data (OHLC): 1.08003, 1.08005, 1.07873, 1.07884, Daily data (OHLC):1.08718, 1.08727, 1.08451, 1.08516
    2016-01-04 16:14:00, 1min data (OHLC): 1.07885, 1.07912, 1.07846, 1.07855, Daily data (OHLC):1.08718, 1.08727, 1.08451, 1.08516
    2016-01-04 16:15:00, 1min data (OHLC): 1.07855, 1.07871, 1.07841, 1.07854, Daily data (OHLC):1.08718, 1.08727, 1.08451, 1.08516
    2016-01-04 16:16:00, 1min data (OHLC): 1.07853, 1.07886, 1.07847, 1.07852, Daily data (OHLC):1.08718, 1.08727, 1.08451, 1.08516
    2016-01-04 16:17:00, 1min data (OHLC): 1.07852, 1.07879, 1.07850, 1.07861, Daily data (OHLC):1.08718, 1.08727, 1.08451, 1.08516
    2016-01-04 16:18:00, 1min data (OHLC): 1.07861, 1.07865, 1.07835, 1.07846, Daily data (OHLC):1.08718, 1.08727, 1.08451, 1.08516
    2016-01-04 16:19:00, 1min data (OHLC): 1.07848, 1.07860, 1.07824, 1.07836, Daily data (OHLC):1.08718, 1.08727, 1.08451, 1.08516
    2016-01-04 16:20:00, 1min data (OHLC): 1.07836, 1.07857, 1.07826, 1.07848, Daily data (OHLC):1.08718, 1.08727, 1.08451, 1.08516
    2016-01-04 16:21:00, 1min data (OHLC): 1.07848, 1.07868, 1.07840, 1.07860, Daily data (OHLC):1.08718, 1.08727, 1.08451, 1.08516
    2016-01-04 16:22:00, 1min data (OHLC): 1.07862, 1.07866, 1.07820, 1.07820, Daily data (OHLC):1.08718, 1.08727, 1.08451, 1.08516
    2016-01-04 16:23:00, 1min data (OHLC): 1.07821, 1.07841, 1.07816, 1.07818, Daily data (OHLC):1.08718, 1.08727, 1.08451, 1.08516
    2016-01-04 16:24:00, 1min data (OHLC): 1.07818, 1.07829, 1.07812, 1.07818, Daily data (OHLC):1.08718, 1.08727, 1.08451, 1.08516
    2016-01-04 16:25:00, 1min data (OHLC): 1.07818, 1.07853, 1.07815, 1.07848, Daily data (OHLC):1.08718, 1.08727, 1.08451, 1.08516
    

    Resampling of 1min data is done with this command:

    cerebro.resampledata(data, timeframe=bt.TimeFrame.Days)
    

    What could be the problem here?



  • your daily resampled data will have the previous days price because you don't know todays values
    you might want to try and replay the data if you are wanting the daily values to update through out the day



  • I've seen what I believe are some similar issues with .replaydata()

    https://community.backtrader.com/topic/147/getting-daily-replay-data


  • administrators

    The problem is not so much with the output data as with the plot. What may seem as dirt on one's screen, are actually the daily bars on the chart, which have a very small presence on the screen.

    The last minute of 2016-01-04 has these corresponding values for the daily resampling:

    Daily data (OHLC):1.08718, 1.08727, 1.08451, 1.08516
    

    And there is a spot on the chart which shows the bar. The problem here is the scale. Zooming in:

    0_1485607151651_upload-92049b4d-ad21-4844-befe-d28e812aa92c

    The 1st spot on the left seems to actually go from 1.08451 (low) to 1.08727 (*high). The gray dashed lines are the markers for 1.080 and 1.095.

    It seems to boil down to a scale representation issue. It could have to do with matplotlib (which has code in place to check for scaling) or the resampling code missing some of your packets or something else.

    Try plotting with volume=False (although it seems already to be there, either manually or automatically) to avoid having an overlay axis on the data.


  • administrators

    On the other hand and looking again at the chart it seems the daily variation may be rather small. Because a full year is on the chart, the 1-minute bars are compressed give a sense of continuity.

    Some more data would be needed.


Log in to reply
 

Looks like your connection to Backtrader Community was lost, please wait while we try to reconnect.