Navigation

    Backtrader Community

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

    Error: deque index out of range when running optstrategy with memory saving

    General Code/Help
    2
    4
    836
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • B
      Boning Gong last edited by

      Dear community,

      We are currently using Backtrader and so far it has been a great tool!!

      Today we encountered a bug and after spending many hours on it, were not able to fix it ourselves. The error is as follows:

      multiprocessing.pool.RemoteTraceback:
      """
      Traceback (most recent call last):
        File "c:\users\boning\appdata\local\programs\python\python37\Lib\multiprocessing\pool.py", line 121, in worker
          result = (True, func(*args, **kwds))
        File "C:\Users\Boning\Documents\Agga\venv\lib\site-packages\backtrader\cerebro.py", line 1007, in __call__
          return self.runstrategies(iterstrat, predata=predata)
        File "C:\Users\Boning\Documents\Agga\venv\lib\site-packages\backtrader\cerebro.py", line 1298, in runstrategies
          self._runnext(runstrats)
        File "C:\Users\Boning\Documents\Agga\venv\lib\site-packages\backtrader\cerebro.py", line 1630, in _runnext
          strat._next()
        File "C:\Users\Boning\Documents\Agga\venv\lib\site-packages\backtrader\strategy.py", line 325, in _next
          super(Strategy, self)._next()
        File "C:\Users\Boning\Documents\Agga\venv\lib\site-packages\backtrader\lineiterator.py", line 255, in _next
          clock_len = self._clk_update()
        File "C:\Users\Boning\Documents\Agga\venv\lib\site-packages\backtrader\strategy.py", line 310, in _clk_update
          for d in self.datas if len(d))
        File "C:\Users\Boning\Documents\Agga\venv\lib\site-packages\backtrader\strategy.py", line 310, in <genexpr>
          for d in self.datas if len(d))
        File "C:\Users\Boning\Documents\Agga\venv\lib\site-packages\backtrader\linebuffer.py", line 165, in __getitem__
          return self.array[self.idx + ago]
      IndexError: deque index out of range
      """
      
      The above exception was the direct cause of the following exception:
      
      Traceback (most recent call last):
        File "Main.py", line 813, in <module>
          binance_run(res_dict)
        File "Main.py", line 772, in binance_run
          run_save_results(source, pair, res_dict)
        File "Main.py", line 722, in run_save_results
          run_strat(pair, source)
        File "Main.py", line 620, in run_strat
          call_all_EMA_combinations(list_train_test, dn1, dn2, dn3, source, pair, test_index)
        File "Main.py", line 527, in call_all_EMA_combinations
          res = trainer.run()
        File "C:\Users\Boning\Documents\Agga\venv\lib\site-packages\backtrader\cerebro.py", line 1143, in run
          for r in pool.imap(self, iterstrats):
        File "c:\users\boning\appdata\local\programs\python\python37\Lib\multiprocessing\pool.py", line 748, in next
          raise value
      IndexError: deque index out of range
      

      After some debugging within linebuffer.py we found out the following:

      Trying to get index, idx is 0 and ago is 0 while size is 1
      Trying to get index, idx is 11 and ago is 0 while size is 12
      Trying to get index, idx is -1 and ago is 0 while size is 0
      

      So somehow the collections.deque has a size of 0 and the function getitem within linebuffer.py tries to access it with an index of -1.

      Some other information that might be useful:

       cerebro = bt.Cerebro(stdstats=False,
                                  runonce=False,
                                  preload=False,
                                  cheat_on_open=True,
                                  maxcpus=args.maxcpus,
                                  exactbars=args.save)
      

      Where we use 1 as exactbars input and None for maxcpus. Also we are running optstrategy of Cerebro with three data streams (smallest time frames loaded earlier than larger time frames) and optimize a set of 5 different values for one parameter (even though this error appears as well when running optstrategy with only 1 specific value for the parameter).

      Thank you in advance!

      1 Reply Last reply Reply Quote 0
      • B
        backtrader administrators last edited by

        Yes, this is bound to happend because the data streams misalign. Disable exactbars.

        1 Reply Last reply Reply Quote 0
        • B
          Boning Gong last edited by

          Thank you for the quick reply!

          Is there another way to gain the same memory saving results as when exactbars = 1 when using multiple data streams within optstrategy? We are having out of memory exceptions when running the code for multiple hours.

          1 Reply Last reply Reply Quote 0
          • B
            backtrader administrators last edited by

            No, there isn't.

            1 Reply Last reply Reply Quote 0
            • 1 / 1
            • First post
              Last post
            Copyright © 2016, 2017, 2018, 2019, 2020, 2021 NodeBB Forums | Contributors