Navigation

    Backtrader Community

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

    Backfill periods

    General Code/Help
    2
    3
    1107
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • M
      monoid last edited by

      Good day, and firstly thank you very much to the developers of Backtrader for the hours you have put in.

      I have a question concerning the implementation of backfilling of data, specifically the initial backfill before LIVE data are processed for IB. The relevant documentation is:
      https://www.backtrader.com/docu/live/ib/ib.html#ibdata-feeds

      Where "maximum possible duration" implicitly references:
      https://interactivebrokers.github.io/tws-api/historical_limitations.html

      For my use case, many times I do not need to backfill 1800s of data, 60s is sufficient. This can be determined in fact from my strategies _minperiod, so my question is: why the greedy backfill? Can we not be more frugal and take only what we need? Was this done because the correct _minperiod is not accessible to the feed?

      Code ref example:
      After _minperiod was set on the strategies, cerebro calls:
      https://github.com/backtrader/backtrader/blob/master/backtrader/cerebro.py#L1538
      which goes to
      https://github.com/backtrader/backtrader/blob/master/backtrader/feeds/ibdata.py#L457
      which goes to
      https://github.com/backtrader/backtrader/blob/master/backtrader/stores/ibstore.py#L673

      It seems from that 2nd link like maybe I could hack this in by setting my IBData objects datetime to:
      now() - datetime(minperiod*timeframe*compression)
      or something like that? Sorry if that code is offensive :')

      And thanks again.

      B 1 Reply Last reply Reply Quote 0
      • B
        backtrader administrators @monoid last edited by

        @monoid said in Backfill periods:

        why the greedy backfill?

        It's not greedy. It's what's offered in a single request and it takes basically the same amount of time to get the greedy data as to get less.

        @monoid said in Backfill periods:

        Was this done because the correct _minperiod is not accessible to the feed?

        It's done because it's what's fits in a single request. It has no other implications. Yes, big data has for sure made Trump win the election and the brits to vote for Brexit, but it didn't have any influence here.

        @monoid said in Backfill periods:

        Can we not be more frugal and take only what we need?

        Specify fromdate to the data feed and less will be taken.

        1 Reply Last reply Reply Quote 0
        • M
          monoid last edited by

          @backtrader said in Backfill periods:

          it takes basically the same amount of time to get the greedy data as to get less

          Today yes, but I worry what will happen with net neutrality being repealed. Jokes aside, in my case the data were taking maybe 50ms to process each datapoint but:
          a) I realise that was probably an issue with the processing of the received data not the downloading of that data and
          b) that issue seems to have magically disappeared today and the 1800s is no longer a bottleneck.

          Thank you for your reply.

          1 Reply Last reply Reply Quote 0
          • 1 / 1
          • First post
            Last post
          Copyright © 2016, 2017, 2018, 2019, 2020, 2021 NodeBB Forums | Contributors