Navigation

    Backtrader Community

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

    IndexError: array assignment index out of range

    General Code/Help
    3
    8
    93
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Matt Wilson
      Matt Wilson last edited by

      I'm getting a similar error as in this post and this post, however the solutions provided there do not work on my script. Unless I'm misunderstanding of course.

      The dataset I'm using can be downloaded from my google drive here for reproducibility:

      https://drive.google.com/file/d/1fSyEPE9FszABg7ej_7mbB9ts4oHjFmMg/view?usp=sharing

      It's a self-made dataset that re-creates "ticks" from a 1 minute dataset of the SPY. So the dataset is in 15 second intervals (re: 4 x 15 seconds in 1 minute). I am using this dataset with replaydata to upsample it to 5 minutes.

      I've created a minimal working example below to reproduce the error, just ensure the dataset provided is in the same directory as the script.

      In my real implementation, I'm using a genetic algorithm to help with optimization of parameters, NOT BT's optimize function, so you will see terms pertaining to GA's in there, like gene_space and chromosome etc. It's well commented and easy to follow.

      Basically, the code runs for the first combination of parameters, but then gives this error before starting the second one:

      Traceback (most recent call last):
        File "test5.py", line 169, in <module>
          results = cerebro.run(tradehistory=True)
        File "C:\Users\chalu\AppData\Roaming\Python\Python37\lib\site-packages\backtrader\cerebro.py", line 1127, in run
          runstrat = self.runstrategies(iterstrat)
        File "C:\Users\chalu\AppData\Roaming\Python\Python37\lib\site-packages\backtrader\cerebro.py", line 1298, in runstrategies
          self._runnext(runstrats)
        File "C:\Users\chalu\AppData\Roaming\Python\Python37\lib\site-packages\backtrader\feed.py", line 523, in load
          retff = ff(self, *fargs, **fkwargs)
        File "C:\Users\chalu\AppData\Roaming\Python\Python37\lib\site-packages\backtrader\resamplerfilter.py", line 697, in __call__
          data._updatebar(self.bar.lvalues(), forward=False, ago=0)
        File "C:\Users\chalu\AppData\Roaming\Python\Python37\lib\site-packages\backtrader\feed.py", line 570, in _updatebar
          line[0 + ago] = val
        File "C:\Users\chalu\AppData\Roaming\Python\Python37\lib\site-packages\backtrader\linebuffer.py", line 222, in __setitem__
          self.array[self.idx + ago] = value
      IndexError: array assignment index out of range
      

      ...which doesn't really help ME much, other than I think it has something to do with replaydata.

      I've read through the two posts and ensured that I'm not using [] brackets when I should be using () brackets, particularly in the __init__ function, but I do use [0] in a few spots in the next, but this isn't the problem as I've tested this script with another dataset WITHOUT using replay and it works fine.

      Hopefully someone who has come across this can spot the issue? Here is the code:

      from __future__ import (absolute_import, division, print_function, unicode_literals)
      from numpy import where
      import backtrader as bt
      import json
      
      # Create a Stratey Class
      class StrategyClass(bt.Strategy):
      
          # Define a self.gene_space. I'll be manually assigning values
          # in the main program start below, but this emulates my real 
          # code in my project. In other words, don't mind this for now
          gene_space={'lenSMA':{'low':10, 'high':500, 'step': 10},
                      'lenEMA':{'low':10, 'high':500, 'step': 10},
                      'keltnerLenK':{'low':20, 'high':1000, 'step': 20},
                      'keltnerMult':{'low':0.1, 'high':20, 'step': 0.1},
                      'keltnerATRLen':{'low':5, 'high':200, 'step': 5},
                      'stochPerK':{'low':5, 'high':200, 'step': 5},
                      'stochSmoothk':{'low':5, 'high':200, 'step': 5},
                      'stochPerD':{'low':5, 'high':200, 'step': 5},
                      'macdFastLen':{'low':10, 'high':500, 'step': 10},
                      'macdSlowLen':{'low':20, 'high':1000, 'step': 20},
                      'macdSignalLen':{'low':10, 'high':500, 'step': 10},
              }
      
          # Define the __init__ function, whereby we pass in a list of
          # current "gene" values
          def __init__(self, current_chromosome):
      
              self.current_chromosome = current_chromosome
      
              # Assign the current_chromosome values to the strategy class before backtesting them
              for i, key in enumerate(self.gene_space):
                  self.gene_space[key] = self.current_chromosome[i]
      
              self.out = bt.indicators.MovingAverageSimple(self.data1.close, period=self.gene_space['lenSMA'])
              self.out2 = bt.indicators.ExponentialMovingAverage(self.data1.close, period=self.gene_space['lenEMA'])
              self.ma = bt.indicators.MovingAverageSimple(self.data1.close, period=self.gene_space['keltnerLenK'])
              self.rangema = bt.indicators.AverageTrueRange(self.data1, period=self.gene_space['keltnerATRLen'])
              # self.stoch = bt.talib.STOCH(self.data1.high, self.data1.low, self.data1.close, self.gene_space['stochPerK'])
              self.highest_high = bt.indicators.Highest(self.data1.high, period=self.gene_space['stochPerK'])
              self.lowest_low = bt.indicators.Lowest(self.data1.low, period=self.gene_space['stochPerK'])
              self.fast_ma = bt.indicators.MovingAverageSimple(self.data1.close, period=self.gene_space['macdFastLen'])
              self.slow_ma = bt.indicators.MovingAverageSimple(self.data1.close, period=self.gene_space['macdSlowLen'])
      
              self.upper = self.ma + self.rangema * self.gene_space['keltnerMult']
              self.lower = self.ma - self.rangema * self.gene_space['keltnerMult']
              self.kden = self.highest_high - self.lowest_low
              self.knum = self.data1.close - self.lowest_low
              self.stoch = 100.0 * bt.DivByZero(self.knum, self.kden, zero=0.0)
              self.k = bt.indicators.MovingAverageSimple(self.stoch, period=self.gene_space['stochSmoothk'])
              self.subtracted = self.fast_ma - self.slow_ma
              self.signal = bt.indicators.MovingAverageSimple(self.subtracted, period=self.gene_space['macdSignalLen'])
              self.hist = (self.fast_ma - self.slow_ma) - self.signal
      
          # Next Function
          def next(self):
      
              # Order conditions
              self.enter_long = where((self.data1.high[0] > self.out[0]) & 
                                      (self.data1.high[0] < self.upper[0]) & 
                                      (self.data1.high[0] > self.lower[0]) & 
                                      (self.hist[0] < 0) & 
                                      (self.k[0] < 50) & 
                                      (self.data1.high[0] > self.out2[0]), 
                                      True, False)
              self.enter_short = where((self.data1.low[0] < self.out[0]) & 
                                       (self.data1.low[0] < self.upper[0]) & 
                                       (self.data1.low[0] > self.lower[0]) & 
                                       (self.hist[0] > 0) & 
                                       (self.k[0] > 50) & 
                                       (self.data1.low[0] < self.out2[0]), 
                                       True, False)
              self.exit_long = self.enter_short
              self.exit_short = self.enter_long
      
              # If we are NOT currently holding a position
              if not self.position or self.position.size == 0:
                  if self.enter_long:
                      self.buy(data=self.data1, 
                               price=self.data1.close[0] + self.limit_order_threshold, 
                               exectype=bt.Order.Limit)
                  elif self.enter_short:
                      self.sell(data=self.data1, 
                                price=self.data1.close[0] - self.limit_order_threshold, 
                                exectype=bt.Order.Limit)
      
              # Check if we are LONG
              if self.position.size > 0:
                  if self.enter_short or self.exit_long:
                      self.close()
                      self.sell(data=self.data1, 
                                price=self.data1.close[0] - self.limit_order_threshold, 
                                exectype=bt.Order.Limit)
              # Check if we are SHORT
              elif self.position.size < 0:
                  if self.enter_long or self.exit_short:
                      self.close()
                      self.buy(data=self.data1, 
                               price=self.data1.close[0] + self.limit_order_threshold, 
                               exectype=bt.Order.Limit)
      
      
      # MAIN PROGRAM START
      # Just assign 2 sets of random chromosome values, the code
      # will error out after the first one is done.
      chromosomes_to_test = [[90, 390, 860, 5.9, 175, 55, 95, 150, 320, 720, 120],
                             [390, 90, 740, 18.7, 190, 195, 70, 150, 450, 380, 80]]
      
      # Generate the cerebro dataset from the df's
      data = bt.feeds.GenericCSVData(
          dataname="SPY_5_1_252_days_backtest_dataset.csv",
          reverse=False,
          dtformat=('%Y-%m-%d %H:%M:%S'),
          timeframe=bt.TimeFrame.Seconds,
          compression=15, # It's a 15 second "tick" dataset
          datetime=0,
          open=1,
          high=2,
          low=3,
          close=4,
          volume=5,
          openinterest=-1
      )
      
      # Now backtest both
      for current_chromosome in chromosomes_to_test:
          
          print("Current Chromosome:", current_chromosome)
      
          # Create a cerebro entity
          cerebro = bt.Cerebro(stdstats=False)
      
          # Add some price slippage to the broker
          cerebro.broker = bt.brokers.BackBroker(slip_fixed=0.03)
          
          # Create orders on Close of current bar or not
          cerebro.broker.set_coc(False)
          
          # Add a strategy class, along with the current_chromosome to test
          cerebro.addstrategy(StrategyClass, current_chromosome=current_chromosome)
      
          # Add the Replay Data Feed to Cerebro
          cerebro.replaydata(data)
          data.plotinfo.plotmaster = data # ignore: this just ensures that it plots both intervals on the same plot
          cerebro.replaydata(data, timeframe=bt.TimeFrame.Minutes, compression=5) # Upsampled to 5 minutes
          
          # Set our desired cash start
          cerebro.broker.setcash(1000)
          
          # Add a FixedSize/AllInSizer/PercentSizer sizer according to the stake
          # 100% will get rejects due to comission, so set to less than 100
          cerebro.addsizer(bt.sizers.PercentSizer, percents=90) #, stake=10
          
          # Set a commission %
          cerebro.broker.setcommission(commission=0.001)
      
          # Add an analyzer or two to get some stats after the run
          cerebro.addanalyzer(bt.analyzers.TradeAnalyzer, _name='trade_analyzer')
          
          # Run
          results = cerebro.run(tradehistory=True)
          
          # Kowalski, analysis!
          analysis = results[0].analyzers.getbyname("trade_analyzer").get_analysis()
      
          # If you want to inspect the analysis dict, use this, but will clutter
          # up genetipy's output
          print(json.dumps(analysis, indent=4))
      
          # Print out the final result
          print('Final Portfolio Value: %.2f' % cerebro.broker.getvalue())
      
      Matt Wilson 1 Reply Last reply Reply Quote 0
      • Matt Wilson
        Matt Wilson @Matt Wilson last edited by

        I should re-iterate that the code runs for the first combo of parameters, and starts the 2nd one, but then gives the error, so when debugging, make sure it'll run through both without error. Trades being placed or not is irrelevant as this is just an example code, but running all the way through while getting around this error is the goal. Thanks!

        1 Reply Last reply Reply Quote 0
        • Matt Wilson
          Matt Wilson last edited by

          No takers?

          Pierre Cilliers 0 1 Reply Last reply Reply Quote 0
          • Pierre Cilliers 0
            Pierre Cilliers 0 @Matt Wilson last edited by

            @matt-wilson Trying to replicate something on my side, but struggling.

            Have you tried it with only 2 chromosomes before attempting the whole lot?

            Matt Wilson 1 Reply Last reply Reply Quote 0
            • Matt Wilson
              Matt Wilson @Pierre Cilliers 0 last edited by

              Hey @pierre-cilliers-0 !

              Haha another issue on my side only? Lovely.

              I don't suppose BackTrader stores a cache or anything does it? I've restarted my computer and and running the latest version of BackTrader on Python 3.7.9 on Windows 10 in case that's relevant.

              I've provided a bigger list of chromosomes to try below, so just replace my 2 list with this bigger one and see what you see. If you run into the same error with this new list, that'll tell me it's something to do with the chromosomes, but it's odd because I still get that error with just these two.

              Another thing I've tried is to replace this line:

              results = cerebro.run(tradehistory=True)

              with

              try:
                  results = cerebro.run(tradehistory=True)
              except:
                  print("DOH!")
                  continue
              

              ...and see what happens, but the same thing, after the first run, it just prints the "DOH!" for every chromosome after the first one. Strange...

              Anyway, let me know how you make out when you get a chance!

              chromosomes_to_test = [[260, 200, 400, 11.6, 150, 135, 100, 55, 320, 120, 35],
                                      [30, 380, 530, 0.7, 40, 80, 195, 145, 40, 660, 16],
                                      [380, 260, 350, 3.2, 175, 75, 160, 50, 210, 960, 37],
                                      [330, 430, 40, 4.1, 115, 35, 55, 35, 350, 400, 27],
                                      [410, 130, 210, 17.9, 105, 40, 160, 105, 460, 200, 31],
                                      [460, 460, 240, 13.8, 40, 130, 90, 165, 470, 440, 32],
                                      [340, 430, 50, 18.0, 60, 75, 10, 30, 470, 400, 22],
                                      [70, 430, 500, 12.2, 110, 50, 150, 95, 270, 260, 49],
                                      [420, 250, 860, 16.9, 75, 110, 15, 20, 380, 180, 8],
                                      [420, 350, 490, 13.6, 60, 175, 160, 130, 410, 920, 39],
                                      [240, 240, 90, 6.5, 170, 65, 155, 135, 360, 60, 18],
                                      [170, 80, 280, 2.0, 80, 100, 35, 155, 420, 900, 5],
                                      [190, 280, 290, 9.2, 110, 195, 135, 170, 350, 800, 46],
                                      [370, 370, 740, 1.2, 85, 160, 50, 145, 360, 180, 11],
                                      [310, 60, 450, 2.2, 15, 45, 175, 30, 490, 200, 18],
                                      [410, 400, 860, 4.4, 85, 30, 80, 100, 460, 140, 36],
                                      [160, 130, 980, 2.0, 25, 125, 165, 40, 60, 780, 29],
                                      [120, 120, 520, 7.5, 75, 5, 95, 195, 440, 840, 48],
                                      [60, 100, 30, 5.5, 130, 155, 80, 150, 150, 260, 5],
                                      [110, 200, 260, 8.7, 75, 5, 175, 45, 10, 700, 32],
                                      [120, 160, 890, 3.2, 65, 120, 125, 185, 200, 660, 36],
                                      [330, 190, 80, 10.7, 170, 140, 105, 110, 400, 300, 16],
                                      [440, 50, 280, 16.5, 45, 170, 5, 30, 240, 740, 30],
                                      [140, 60, 730, 18.6, 110, 15, 195, 115, 110, 100, 18],
                                      [120, 30, 650, 9.0, 55, 30, 85, 65, 60, 940, 43],
                                      [80, 100, 300, 18.2, 45, 100, 30, 75, 80, 60, 4],
                                      [280, 320, 620, 3.6, 115, 90, 80, 195, 110, 800, 38],
                                      [40, 130, 110, 15.3, 90, 105, 110, 150, 160, 300, 29],
                                      [390, 40, 80, 11.9, 125, 70, 90, 25, 260, 420, 18],
                                      [100, 410, 410, 14.8, 190, 75, 180, 165, 360, 240, 5],
                                      [80, 360, 490, 15.5, 15, 60, 85, 95, 280, 840, 36],
                                      [20, 340, 150, 7.8, 20, 130, 15, 10, 200, 120, 37],
                                      [490, 40, 700, 1.9, 15, 105, 25, 45, 430, 680, 37],
                                      [310, 30, 430, 17.4, 95, 125, 15, 115, 50, 440, 18],
                                      [160, 430, 720, 9.5, 100, 140, 110, 60, 400, 740, 35],
                                      [370, 50, 360, 4.8, 90, 165, 150, 195, 490, 840, 11],
                                      [340, 450, 680, 19.1, 160, 130, 115, 5, 320, 540, 11],
                                      [20, 270, 660, 19.6, 110, 165, 120, 190, 250, 280, 5],
                                      [400, 460, 700, 18.5, 105, 165, 185, 135, 430, 880, 1],
                                      [240, 200, 30, 18.1, 10, 125, 15, 185, 290, 940, 31],
                                      [260, 390, 780, 2.7, 125, 10, 15, 170, 450, 340, 44],
                                      [70, 290, 150, 10.6, 130, 170, 110, 150, 220, 960, 22],
                                      [270, 260, 190, 8.9, 40, 135, 95, 155, 150, 820, 36],
                                      [300, 30, 30, 8.8, 115, 105, 35, 100, 30, 680, 31],
                                      [280, 60, 920, 4.6, 55, 140, 70, 15, 370, 580, 32],
                                      [240, 220, 440, 9.6, 60, 65, 165, 175, 130, 480, 39],
                                      [400, 20, 970, 8.2, 105, 55, 90, 95, 70, 720, 23],
                                      [40, 170, 980, 4.4, 170, 5, 80, 100, 40, 700, 44],
                                      [450, 100, 420, 13.7, 65, 130, 75, 145, 220, 740, 18],
                                      [330, 190, 130, 3.1, 10, 70, 5, 5, 230, 600, 19],
                                      [320, 30, 560, 7.4, 60, 145, 160, 20, 220, 140, 47],
                                      [290, 40, 830, 19.4, 75, 60, 120, 55, 170, 540, 21],
                                      [420, 160, 940, 16.7, 45, 110, 70, 160, 200, 680, 17],
                                      [20, 460, 880, 18.7, 90, 95, 5, 70, 380, 460, 6],
                                      [390, 100, 50, 8.5, 60, 100, 90, 120, 50, 400, 43],
                                      [440, 360, 840, 13.8, 100, 60, 185, 100, 40, 320, 23],
                                      [440, 330, 740, 5.5, 80, 170, 90, 130, 270, 540, 47],
                                      [380, 100, 290, 15.7, 90, 65, 175, 190, 200, 840, 16],
                                     # Original two here
                                     [90, 390, 860, 5.9, 175, 55, 95, 150, 320, 720, 120],
                                     [390, 90, 740, 18.7, 190, 195, 70, 150, 450, 380, 80]]
              
              1 Reply Last reply Reply Quote 0
              • sq N
                sq N last edited by

                I am facing the same error, which is strange.

                1 Reply Last reply Reply Quote 1
                • Matt Wilson
                  Matt Wilson last edited by

                  @backtrader Any ideas?

                  sq N 1 Reply Last reply Reply Quote 0
                  • sq N
                    sq N @Matt Wilson last edited by

                    @matt-wilson recently solved this problem, but I think it could be caused by many reasons. In my situation, I used open[1] which is a future data when buying and I changed to close[0] and solved. I think may be you can review your strategy. It might be an idea mistake rather than code.

                    1 Reply Last reply Reply Quote 0
                    • 1 / 1
                    • First post
                      Last post
                    Copyright © 2016, 2017, 2018, 2019, 2020, 2021 NodeBB Forums | Contributors