Navigation

    Backtrader Community

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

    LIVE data resampling issue

    General Code/Help
    1
    3
    546
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • S
      sylvain last edited by sylvain

      Hello all, I got ran into an issue with live data resampling from 1 min to 1 day. As you can see from some of my results below, the resampling works very well during backfilling, as well as resampling from 1min to 10min or 30min. However, it ran into this issue, the 1day timeframe data feeds did not get updated to the previous day's value.

      My code for getdata() and resampling:

      data = ibstore.getdata(dataname='TSLA-STK-SMART-USD', timeframe = bt.TimeFrame.Minutes, compression =1, rtbar=False,
                             qcheck=1.0, backfill_start=True, backfill=True, backfill_from = dataHist, useRTH=True, latethrough =False,
                             sessionstart=datetime.time(9,30,00), sessionend=datetime.time(16,00,00))
      
      
      data.addfilter(bt.filters.SessionFilter(data))
      
      cerebro.resampledata(data, timeframe=bt.TimeFrame.Minutes, compression=1) ## Adding 1min bar (data0)
      cerebro.resampledata(data, timeframe=bt.TimeFrame.Minutes, compression=10) ## Adding 10min bar (data1)
      cerebro.resampledata(data, timeframe=bt.TimeFrame.Minutes, compression=30) ## Adding 30min bar (data2)
      cerebro.resampledata(data, timeframe=bt.TimeFrame.Days, compression=1) ## Adding 1day bar (data3)
      

      datafeeds during backfilling:

      .......
      next run started @ 2019-02-15T09:01:17.282031
      Data0, 24209, 2019-02-13T15:58:00.000000, 308.53, 308.55, 308.14, 308.34, 366.0,
      Data1, 2421, 2019-02-13T15:50:00.000000, 307.97, 307.97, 307.37, 307.92, 1027.0,
      Data2, 0807, 2019-02-13T15:30:00.000000, 309.31, 309.47, 307.79, 307.91, 1514.0
      Data3, 0062, 2019-02-12T16:00:00.000000, 316.2, 318.19, 309.62, 311.81, 41598.0,

      next run started @ 2019-02-15T09:01:17.332751
      Data0, 24210, 2019-02-13T15:59:00.000000, 308.36, 308.43, 308.0, 308.11, 273.0, nan
      Data1, 2421, 2019-02-13T15:50:00.000000, 307.97, 307.97, 307.37, 307.92, 1027.0, nan
      Data2, 0807, 2019-02-13T15:30:00.000000, 309.31, 309.47, 307.79, 307.91, 1514.0
      Data3, 0062, 2019-02-12T16:00:00.000000, 316.2, 318.19, 309.62, 311.81, 41598.0,

      next run started @ 2019-02-15T09:01:17.386498
      Data0, 24210, 2019-02-13T15:59:00.000000, 308.36, 308.43, 308.0, 308.11, 273.0, nan
      Data1, 2421, 2019-02-13T15:50:00.000000, 307.97, 307.97, 307.37, 307.92, 1027.0, nan
      Data2, 0807, 2019-02-13T15:30:00.000000, 309.31, 309.47, 307.79, 307.91, 1514.0
      Data3, 0063, 2019-02-13T16:00:00.000000, 312.35, 312.75, 305.57, 308.11, 38587.

      next run started @ 2019-02-15T09:01:17.435305
      Data0, 24211, 2019-02-14T09:30:00.000000, 303.38, 304.62, 302.95, 304.1, 1294.0, nan
      Data1, 2422, 2019-02-14T09:30:00.000000, 307.99, 308.75, 302.95, 304.1, 3439.0, -1.0
      Data2, 0808, 2019-02-14T09:30:00.000000, 307.81, 308.75, 302.95, 304.1, 4910.0
      Data3, 0063, 2019-02-13T16:00:00.000000, 312.35, 312.75, 305.57, 308.11, 38587.0,

      next run started @ 2019-02-15T09:01:17.481209
      Data0, 24212, 2019-02-14T09:31:00.000000, 304.11, 304.63, 303.21, 303.38, 387.0, nan
      Data1, 2422, 2019-02-14T09:30:00.000000, 307.99, 308.75, 302.95, 304.1, 3439.0, -1.0
      Data2, 0808, 2019-02-14T09:30:00.000000, 307.81, 308.75, 302.95, 304.1, 4910.0
      Data3, 0063, 2019-02-13T16:00:00.000000, 312.35, 312.75, 305.57, 308.11, 38587.0,
      .......

      Resampling while turn into live:

      .......
      next run started @ 2019-02-15T09:01:39.194076
      Data0, 24600, 2019-02-14T15:59:00.000000, 304.0, 304.05, 303.35, 303.83, 635.0, nan
      Data1, 2460, 2019-02-14T15:50:00.000000, 304.6, 304.86, 304.34, 304.42, 907.0, nan
      Data2, 0820, 2019-02-14T15:30:00.000000, 304.37, 305.87, 304.2, 304.88, 1797.0
      Data3, 0063, 2019-02-13T16:00:00.000000, 312.35, 312.75, 305.57, 308.11, 38587.0,

      ***** DATA NOTIF: DELAYED
      ***** DATA NOTIF: LIVE
      next run started @ 2019-02-15T09:30:05.705076
      Data0, 24601, 2019-02-15T09:30:00.000000, 304.4, 304.56, 304.15, 304.39, 681.0, nan
      Data1, 2461, 2019-02-15T09:30:00.000000, 304.44, 304.56, 303.35, 304.39, 2784.0, nan
      Data2, 0821, 2019-02-15T09:30:00.000000, 304.95, 305.09, 303.35, 304.39, 4159.0
      Data3, 0063, 2019-02-13T16:00:00.000000, 312.35, 312.75, 305.57, 308.11, 38587.0,

      next run started @ 2019-02-15T09:31:01.533144
      Data0, 24602, 2019-02-15T09:31:00.000000, 304.31, 304.53, 304.09, 304.51, 204.0, nan
      Data1, 2461, 2019-02-15T09:30:00.000000, 304.44, 304.56, 303.35, 304.39, 2784.0, nan
      Data2, 0821, 2019-02-15T09:30:00.000000, 304.95, 305.09, 303.35, 304.39, 4159.0
      Data3, 0063, 2019-02-13T16:00:00.000000, 312.35, 312.75, 305.57, 308.11, 38587.0,
      .......

      One can see the daily bar(Data3) is not getting update to the correct timestamp. For the information, the dataHist I am backfilling from is the RTH 1min bar stored locally, it will get update once a day after market close, so it alwasy up to yesterday's 15:59:00.

      I really appreciate if you could provide any guidance.

      S 1 Reply Last reply Reply Quote 0
      • S
        sylvain @sylvain last edited by

        @sylvain I think one possible solution can be: change the timestamp of the historical data to 1 minute later. this would make sense since, IB 1 min hist bar starts at 9:30 ends at 15:59, and after the shift, it will starts at 9:31 ends at 16:00, this will line up with the resampled 1 min live bar. However still unclear about the issue mentioned above.
        Also, it looks like the first 1min LIVE bar of the day is:

        next run started @ 2019-02-15T09:30:05.705076
        Data0, 24601, 2019-02-15T09:30:00.000000, 304.4, 304.56, 304.15, 304.39, 681.0, nan

        should it be at 9:31:XX ?
        Since the 1min bar would have enough info only after 1min of the mark opened, really not sure how to interpret this. Any help would be appreciated @backtrader

        S 1 Reply Last reply Reply Quote 0
        • S
          sylvain @sylvain last edited by

          @sylvain Problem solved after making the change of shift the timestamp of hist 1 min bar to 1 min later. Thank you !

          1 Reply Last reply Reply Quote 1
          • 1 / 1
          • First post
            Last post
          Copyright © 2016, 2017, 2018, 2019, 2020, 2021 NodeBB Forums | Contributors