For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/
2019-10-02: The community is currently in read-only mode

Question on resampledata with 1min data to 60 mins



  • Hi,

    I'm resampling the 1 min data into 60 mins, with session start at 9:15 AM,

    data = bt.feeds.GenericCSVData(
            dataname=datapath,
            fromdate=datetime.datetime(2019, 2, 26),
            todate=datetime.datetime(2019, 2, 27),
            timeframe=bt.TimeFrame.Minutes,
            sessionstart=datetime.time(9, 15),
            sessionend=datetime.time(15, 30),
            dtformat=('%Y%m%d %H%M%S'),
            headers=False,
            separator=';',
            datetime=0,
            time=-1,
            open=1,
            high=2,
            low=3,
            close=4,
            volume=5,
            openinterest=-1)
    
    cerebro.resampledata(data, timeframe=bt.TimeFrame.Minutes, compression=60)
    
    

    1min data as following

    2019-02-26 09:15:00, Open: 10794.00, High: 10818.00, Low: 10793.00, Close: 10817.60, Volume: 384900
    2019-02-26 09:16:00, Open: 10817.00, High: 10833.00, Low: 10816.55, Close: 10833.00, Volume: 145650
    2019-02-26 09:17:00, Open: 10831.70, High: 10843.45, Low: 10830.60, Close: 10830.60, Volume: 137250
    2019-02-26 09:18:00, Open: 10831.40, High: 10836.00, Low: 10820.10, Close: 10820.10, Volume: 128400
    2019-02-26 09:19:00, Open: 10820.10, High: 10821.05, Low: 10805.75, Close: 10815.05, Volume: 107550
    2019-02-26 09:20:00, Open: 10816.10, High: 10816.10, Low: 10798.00, Close: 10798.00, Volume: 125550
    .
    .
    .
    2019-02-26 15:24:00, Open: 10826.85, High: 10827.35, Low: 10824.00, Close: 10826.25, Volume: 32400
    2019-02-26 15:25:00, Open: 10826.40, High: 10826.90, Low: 10825.00, Close: 10825.00, Volume: 49050
    2019-02-26 15:26:00, Open: 10825.00, High: 10825.50, Low: 10823.60, Close: 10823.65, Volume: 82800
    2019-02-26 15:27:00, Open: 10823.75, High: 10825.00, Low: 10823.65, Close: 10824.50, Volume: 53100
    2019-02-26 15:28:00, Open: 10824.00, High: 10825.90, Low: 10824.00, Close: 10824.95, Volume: 84750
    2019-02-26 15:29:00, Open: 10825.00, High: 10825.00, Low: 10822.00, Close: 10822.00, Volume: 114750
    

    60mins resample as following

    2019-02-26 10:00:00, Open: 10794.00, High: 10843.45, Low: 10732.00, Close: 10732.75, Volume: 3788550
    2019-02-26 11:00:00, Open: 10734.00, High: 10806.35, Low: 10725.20, Close: 10793.80, Volume: 3184800
    2019-02-26 12:00:00, Open: 10794.95, High: 10847.05, Low: 10792.75, Close: 10832.75, Volume: 2198400
    2019-02-26 13:00:00, Open: 10832.75, High: 10874.00, Low: 10815.75, Close: 10856.05, Volume: 1879650
    2019-02-26 14:00:00, Open: 10856.05, High: 10862.00, Low: 10827.00, Close: 10845.95, Volume: 1197675
    2019-02-26 15:00:00, Open: 10845.95, High: 10892.80, Low: 10817.00, Close: 10836.25, Volume: 2613675
    2019-02-26 16:00:00, Open: 10835.55, High: 10844.00, Low: 10821.85, Close: 10822.00, Volume: 1374750
    

    60mins resample starts frin 9 AM instead of 9:15 AM thats specified in the session start.

    Can the resample to start from 9:15, instead of 9:00 Hours?

    thanks in advance
    Sathish


  • administrators

    @tradersatz said in Question on resampledata with 1min data to 60 mins:

    60mins resample starts frin 9 AM instead of 9:15 AM thats specified in the session start.
    Can the resample to start from 9:15, instead of 9:00 Hours?

    Sorry, but you see something which the rest of us probably doesn't ...

    @tradersatz said in Question on resampledata with 1min data to 60 mins:

    2019-02-26 10:00:00, Open: 10794.00, High: 10843.45, Low: 10732.00, Close: 10732.75, Volume: 3788550
    2019-02-26 11:00:00, Open: 10734.00, High: 10806.35, Low: 10725.20, Close: 10793.80, Volume: 3184800
    ...
    

    The first resampled bar has a timestamp of 10:00:00 ...



  • @backtrader said in Question on resampledata with 1min data to 60 mins:

    @tradersatz said in Question on resampledata with 1min data to 60 mins:

    60mins resample starts frin 9 AM instead of 9:15 AM thats specified in the session start.
    Can the resample to start from 9:15, instead of 9:00 Hours?

    Sorry, but you see something which the rest of us probably doesn't ...

    I did search in the community to see if anyone ran into similar issue, I didn't!

    @tradersatz said in Question on resampledata with 1min data to 60 mins:

    2019-02-26 10:00:00, Open: 10794.00, High: 10843.45, Low: 10732.00, Close: 10732.75, Volume: 3788550
    2019-02-26 11:00:00, Open: 10734.00, High: 10806.35, Low: 10725.20, Close: 10793.80, Volume: 3184800
    ...
    

    The first resampled bar has a timestamp of 10:00:00 ...

    Yes right, it starts resample at 10:00 hours where I'm expecting 10:15, I can share the 1min data, if thats going to help to know on this issue.


  • administrators

    @tradersatz said in Question on resampledata with 1min data to 60 mins:

    @backtrader said in Question on resampledata with 1min data to 60 mins:

    @tradersatz said in Question on resampledata with 1min data to 60 mins:

    60mins resample starts frin 9 AM instead of 9:15 AM thats specified in the session start.
    Can the resample to start from 9:15, instead of 9:00 Hours?

    Sorry, but you see something which the rest of us probably doesn't ...

    I did search in the community to see if anyone ran into similar issue, I didn't!

    This was meant as that you said 09:00 and there was no 09:00 in your post.

    @tradersatz said in Question on resampledata with 1min data to 60 mins:

    2019-02-26 10:00:00, Open: 10794.00, High: 10843.45, Low: 10732.00, Close: 10732.75, Volume: 3788550
    2019-02-26 11:00:00, Open: 10734.00, High: 10806.35, Low: 10725.20, Close: 10793.80, Volume: 3184800
    ...
    

    The first resampled bar has a timestamp of 10:00:00 ...

    Yes right, it starts resample at 10:00 hours where I'm expecting 10:15, I can share the 1min data, if thats going to help to know on this issue.

    The bar ends at 10:00 it doesn't start there. The timestamp is the last timestamp that was considered for the bar.

    See if the boundoff parameter can help. Docs - Data Resampling



  • @backtrader said in Question on resampledata with 1min data to 60 mins:

    @tradersatz said in Question on resampledata with 1min data to 60 mins:

    @backtrader said in Question on resampledata with 1min data to 60 mins:

    @tradersatz said in Question on resampledata with 1min data to 60 mins:

    60mins resample starts frin 9 AM instead of 9:15 AM thats specified in the session start.
    Can the resample to start from 9:15, instead of 9:00 Hours?

    Sorry, but you see something which the rest of us probably doesn't ...

    I did search in the community to see if anyone ran into similar issue, I didn't!

    This was meant as that you said 09:00 and there was no 09:00 in your post.

    Sorry, I didn't say it right, let me rephrase it,

    Trading session starts at 9:15 AM and ends at 3:30 PM.

    And my 1 min data starts at 9:15 AM and goes till 3:30 PM.

    when i resample this data to 60 mins, I expect data to start 9:15 , 10:15, 11:15, 12:15 ... 2:15, 3:15, but the actual result is 10:00 AM to 4:00 PM.

    expected resample data like following (Exported from Amibroker)

    2019-02-26 09:15:00,10794.00,10843.45,10725.20,10758.00,4740525
    2019-02-26 10:15:00,10756.90,10808.75,10743.20,10804.75,2665425
    2019-02-26 11:15:00,10803.05,10847.05,10796.10,10831.00,2006025
    2019-02-26 12:15:00,10830.05,10874.00,10815.75,10853.00,1806600
    2019-02-26 13:15:00,10853.00,10860.40,10827.00,10858.25,1344075
    2019-02-26 14:15:00,10858.25,10892.80,10817.00,10829.00,2825550
    2019-02-26 15:15:00,10828.05,10835.90,10821.85,10822.00,849300
    

    I do set the session start and end dates on the initial data load, is this expected?

    @tradersatz said in Question on resampledata with 1min data to 60 mins:

    2019-02-26 10:00:00, Open: 10794.00, High: 10843.45, Low: 10732.00, Close: 10732.75, Volume: 3788550
    2019-02-26 11:00:00, Open: 10734.00, High: 10806.35, Low: 10725.20, Close: 10793.80, Volume: 3184800
    ...
    

    The first resampled bar has a timestamp of 10:00:00 ...

    Yes right, it starts resample at 10:00 hours where I'm expecting 10:15, I can share the 1min data, if thats going to help to know on this issue.

    The bar ends at 10:00 it doesn't start there. The timestamp is the last timestamp that was considered for the bar.

    See if the boundoff parameter can help. Docs - Data Resampling

    with boundoff=-15,

    2019-02-26 09:15:00, 10794.00,10818.00,10793.00,10817.60,384900
    2019-02-26 10:15:00, 10817.00,10843.45,10725.20,10757.00,4399200
    2019-02-26 11:15:00, 10757.75,10808.75,10743.20,10805.00,2651175
    2019-02-26 12:15:00, 10805.00,10847.05,10796.10,10829.75,2014575
    2019-02-26 13:15:00, 10829.75,10874.00,10815.75,10850.25,1785600
    2019-02-26 14:15:00, 10852.25,10860.40,10827.00,10857.20,1417125
    2019-02-26 15:15:00, 10857.25,10892.80,10817.00,10835.90,2774700
    2019-02-26 16:00:00, 10835.10,10835.10,10821.85,10822.00,810225
    

    this is better, but not matching whats expected.

    thanks


  • administrators

    The easiest to would be to substract 15 minutes to all your timestamps with a filter and then resample.



  • @backtrader said in Question on resampledata with 1min data to 60 mins:

    The easiest to would be to substract 15 minutes to all your timestamps with a filter and then resample.

    If you can point me documentation or code sample on the filter approach will be of great help.

    I'm still curious why Session start time provided on the data source not considered while resampling, is that per design?



  • @backtrader said in Question on resampledata with 1min data to 60 mins:

    The easiest to would be to substract 15 minutes to all your timestamps with a filter and then resample.

    Playing around more with different resample mintues,

    15 mins setup

    cerebro.resampledata(data, timeframe=bt.TimeFrame.Minutes, compression=15)
    

    result:

    2019-02-26 09:30:00, 10794.00,10843.45,10789.55,10805.00,1605225
    2019-02-26 09:45:00, 10805.30,10807.30,10775.05,10782.00,912975
    2019-02-26 10:00:00, 10782.00,10786.25,10761.00,10764.80,825375
    .
    .
    .
    2019-02-26 15:00:00, 10878.05,10879.90,10817.00,10834.90,822975
    2019-02-26 15:15:00, 10833.20,10844.00,10823.00,10829.00,563325
    2019-02-26 15:30:00, 10828.05,10835.90,10821.85,10822.00,849300
    

    30 mins setup

    cerebro.resampledata(data, timeframe=bt.TimeFrame.Minutes, compression=30)
    

    result:

    2019-02-26 09:30:00, 10794.00,10843.45,10789.55,10805.00,1605225
    2019-02-26 10:00:00, 10805.30,10807.30,10761.00,10764.80,1738350
    2019-02-26 10:30:00, 10764.45,10777.35,10725.20,10775.20,2370375
    .
    .
    .
    2019-02-26 14:30:00, 10849.90,10866.00,10837.05,10864.75,931950
    2019-02-26 15:00:00, 10865.80,10892.80,10817.00,10834.90,1656675
    2019-02-26 15:30:00, 10833.20,10844.00,10821.85,10822.00,1412625
    

    60 mins setup

    cerebro.resampledata(data, timeframe=bt.TimeFrame.Minutes, compression=60)
    

    result:

    2019-02-26 10:00:00, 10794.00,10843.45,10761.00,10764.80,3343575
    2019-02-26 11:00:00, 10764.45,10806.35,10725.20,10794.85,3613125
    2019-02-26 12:00:00, 10793.00,10847.05,10792.75,10827.85,2194800
    2019-02-26 13:00:00, 10828.00,10874.00,10815.75,10853.10,1884750
    2019-02-26 14:00:00, 10851.15,10862.00,10827.00,10849.90,1200000
    2019-02-26 15:00:00, 10849.90,10892.80,10817.00,10834.90,2588625
    2019-02-26 15:30:00, 10833.20,10844.00,10821.85,10822.00,1412625
    

    As per my findings, It's working fine for 5, 10, 15 mins!
    It's not working for 30 mins and 60 mins.

    When I says not working, first bar in 60 mins should be 10:15, instead of 10, and for 30 mins, first bar to be 9:45.

    I'll dig into the code as well.



});