Backtrader Community

    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

    Gaps in historical minute data

    General Code/Help
    2
    3
    1165
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • J
      jstin last edited by jstin

      I'm seeing what appears to be gaps when requesting historical data. Notice how it jumps from 2017-04-11 T23:59:00 to 2017-04-12 T18:00:00. I'm getting this across multiple weekdays. Any help with what I am missing would be appreciated.

      python ibtest.py --port 4002 --clientId 555 --data0 CL-201705-NYMEX --timeframe Minutes --compression 1 --fromdate 2017-04-10 --historical

      Data0, 0717, 736431.163889, 2017-04-11T23:56:00.000000, 53.56, 53.56, 53.55, 53.55, 12, 0, 53.552
      Data0, 0718, 736431.164583, 2017-04-11T23:57:00.000000, 53.55, 53.55, 53.54, 53.54, 30, 0, 53.55
      Data0, 0719, 736431.165278, 2017-04-11T23:58:00.000000, 53.54, 53.55, 53.54, 53.55, 28, 0, 53.548
      Data0, 0720, 736431.165972, 2017-04-11T23:59:00.000000, 53.55, 53.56, 53.55, 53.56, 4, 0, 53.55
      Data0, 0721, 736431.916667, 2017-04-12T18:00:00.000000, 52.85, 52.9, 52.82, 52.87, 255, 0, 53.414
      Data0, 0722, 736431.917361, 2017-04-12T18:01:00.000000, 52.87, 52.89, 52.87, 52.89, 35, 0, 53.282
      Data0, 0723, 736431.918056, 2017-04-12T18:02:00.000000, 52.89, 52.91, 52.88, 52.91, 70, 0, 53.156
      Data0, 0724, 736431.91875, 2017-04-12T18:03:00.000000, 52.9, 52.91, 52.9, 52.9, 79, 0, 53.026
      Data0, 0725, 736431.919444, 2017-04-12T18:04:00.000000, 52.89, 52.9, 52.88, 52.89, 34, 0, 52.892
      
      1 Reply Last reply Reply Quote 0
      • B
        backtrader administrators last edited by

        CL seems to be the ticker for the Crude Oil futures which after some perusal seem to operate over the midnight barrier.

        From here: http://www.cmegroup.com/trading/energy/crude-oil/light-sweet-crude_contractSpecs_futures.html

        During development a behavior in the Interactive Brokers API reqHistoricalData was observed, where times seem to play an invisible role in filtering out some bars (they are not received by the client, you can check the lack of those messages with --debug)

        Given the date of the message (2017-04-19) and the fromdate request (2017-04-10), the calculated duration to pass to the API is in Days (the IB API has not a regular mechanism to say "from this date" to "this other date", but works with a backwards calculation from a given time).

        That means that because you have not specified a time in --fromdate 2017-04-10, the request to the API goes with a time of 00:00:00 which is midnight from today and trying to download "x Days backwards. This 00:00:00 may be creating the invisible filter mentioned above.

        You may try the following line where the start time of the session is given.

        python ibtest.py --port 4002 --clientId 555 --data0 CL-201705-NYMEX --timeframe Minutes --compression 1 --fromdate 2017-04-10T18:00:00 --historical
        
        1 Reply Last reply Reply Quote 0
        • J
          jstin last edited by

          Thanks so much!

          1 Reply Last reply Reply Quote 0
          • 1 / 1
          • First post
            Last post
          Copyright © 2016, 2017, 2018, 2019, 2020, 2021 NodeBB Forums | Contributors