Navigation

    Backtrader Community

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

    Quickstart with generic CSV - IndexError: array assignment index out of range

    General Code/Help
    5
    7
    407
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • niqnil
      niqnil last edited by

      Hi, I am a newbie trying to work through the quickstart guide.

      I get IndexError: array assignment index out of range when I try to use a generic CSV with a date range. It works without date range, meaning if I don't pass

          fromdate=datetime.datetime(2019, 1, 2),
          todate=datetime.datetime(2020, 12, 31),
      

      I have also tried variations of different dates for the date range with this generic CSV and got the same IndexError. I have checked that the CSV data has the dates.

      There is no index error with using the original file and code from the guide.

          data = bt.feeds.YahooFinanceCSVData(
              dataname='orcl-1995-2014.txt',
              # Do not pass values before this date
              fromdate=datetime.datetime(2000, 1, 1),
              # Do not pass values before this date
              todate=datetime.datetime(2000, 12, 31),
              # Do not pass values after this date
              reverse=False)
      

      Can someone help point out what I am doing wrong? My csv(snipped), code and traceback:

      timestamp,open,high,low,close,adjusted_close,volume,dividend_amount,split_coefficient
      2021-03-03,687.99,700.7,651.705,653.2,653.2,29693988,0.0000,1.0
      2021-03-02,718.28,721.11,685.0,686.44,686.44,23732158,0.0000,1.0
      2021-03-01,690.11,719.0,685.05,718.43,718.43,27136239,0.0000,1.0
      ...
      2020-12-31,699.99,718.72,691.12,705.67,705.67,49649928,0.0000,1.0
      2020-12-30,672.0,696.6,668.3603,694.78,694.78,42846021,0.0000,1.0
      2020-12-29,661.0,669.9,655.0,665.99,665.99,22910811,0.0000,1.0
      ...
      2019-01-04,306.0,318.0,302.73,317.69,63.538,7394100,0.0000,1.0
      2019-01-03,307.0,309.4,297.38,300.36,60.072,6965200,0.0000,1.0
      2019-01-02,306.1,315.13,298.8,310.12,62.024,11658600,0.0000,1.0
      
      from __future__ import (absolute_import, division, print_function,
                              unicode_literals)
      
      %matplotlib widget
      import datetime  # For datetime objects
      import os.path  # To manage paths
      import sys  # To find out the script name (in argv[0])
      
      # Import the backtrader platform
      import backtrader as bt
      
      
      # Create a Stratey
      class TestStrategy(bt.Strategy):
          params = (
              ('maperiod', 15),
          )
      
          def log(self, txt, dt=None):
              ''' Logging function fot this strategy'''
              dt = dt or self.datas[0].datetime.date(0)
              print('%s, %s' % (dt.isoformat(), txt))
      
          def __init__(self):
              # Keep a reference to the "close" line in the data[0] dataseries
              self.dataclose = self.datas[0].close
      
              # To keep track of pending orders and buy price/commission
              self.order = None
              self.buyprice = None
              self.buycomm = None
      
              # Add a MovingAverageSimple indicator
              self.sma = bt.indicators.SimpleMovingAverage(
                  self.datas[0], period=self.params.maperiod)
      
              # Indicators for the plotting show
              bt.indicators.ExponentialMovingAverage(self.datas[0], period=25)
              bt.indicators.WeightedMovingAverage(self.datas[0], period=25,
                                                  subplot=True)
              bt.indicators.StochasticSlow(self.datas[0])
              bt.indicators.MACDHisto(self.datas[0])
              rsi = bt.indicators.RSI(self.datas[0])
              bt.indicators.SmoothedMovingAverage(rsi, period=10)
              bt.indicators.ATR(self.datas[0], plot=False)
      
          def notify_order(self, order):
              if order.status in [order.Submitted, order.Accepted]:
                  # Buy/Sell order submitted/accepted to/by broker - Nothing to do
                  return
      
              # Check if an order has been completed
              # Attention: broker could reject order if not enough cash
              if order.status in [order.Completed]:
                  if order.isbuy():
                      self.log(
                          'BUY EXECUTED, Price: %.2f, Cost: %.2f, Comm %.2f' %
                          (order.executed.price,
                           order.executed.value,
                           order.executed.comm))
      
                      self.buyprice = order.executed.price
                      self.buycomm = order.executed.comm
                  else:  # Sell
                      self.log('SELL EXECUTED, Price: %.2f, Cost: %.2f, Comm %.2f' %
                               (order.executed.price,
                                order.executed.value,
                                order.executed.comm))
      
                  self.bar_executed = len(self)
      
              elif order.status in [order.Canceled, order.Margin, order.Rejected]:
                  self.log('Order Canceled/Margin/Rejected')
      
              # Write down: no pending order
              self.order = None
      
          def notify_trade(self, trade):
              if not trade.isclosed:
                  return
      
              self.log('OPERATION PROFIT, GROSS %.2f, NET %.2f' %
                       (trade.pnl, trade.pnlcomm))
      
          def next(self):
              # Simply log the closing price of the series from the reference
              self.log('Close, %.2f' % self.dataclose[0])
      
              # Check if an order is pending ... if yes, we cannot send a 2nd one
              if self.order:
                  return
      
              # Check if we are in the market
              if not self.position:
      
                  # Not yet ... we MIGHT BUY if ...
                  if self.dataclose[0] > self.sma[0]:
      
                      # BUY, BUY, BUY!!! (with all possible default parameters)
                      self.log('BUY CREATE, %.2f' % self.dataclose[0])
      
                      # Keep track of the created order to avoid a 2nd order
                      self.order = self.buy()
      
              else:
      
                  if self.dataclose[0] < self.sma[0]:
                      # SELL, SELL, SELL!!! (with all possible default parameters)
                      self.log('SELL CREATE, %.2f' % self.dataclose[0])
      
                      # Keep track of the created order to avoid a 2nd order
                      self.order = self.sell()
      
      
      if __name__ == '__main__':
          # Create a cerebro entity
          cerebro = bt.Cerebro()
      
          # Add a strategy
          cerebro.addstrategy(TestStrategy)
      
          # Create a Data Feed
          data = bt.feeds.GenericCSVData(
          dataname='alpha_vantage_TSLA.csv',
      
          fromdate=datetime.datetime(2019, 1, 2),
          todate=datetime.datetime(2020, 12, 31),
              
          nullvalue=0.0,
      
          dtformat=('%Y-%m-%d'),
      
          datetime=0,
          time=-1,
          open=1,
          high=2,
          low=3,
          close=5,
          volume=6,
          openinterest=-1
          )
          
          # Add the Data Feed to Cerebro
          cerebro.adddata(data)
      
          # Set our desired cash start
          cerebro.broker.setcash(1000.0)
      
          # Add a FixedSize sizer according to the stake
          cerebro.addsizer(bt.sizers.FixedSize, stake=10)
      
          # Set the commission
          cerebro.broker.setcommission(commission=0.002)
      
          # Print out the starting conditions
          print('Starting Portfolio Value: %.2f' % cerebro.broker.getvalue())
      
          # Run over everything
          cerebro.run()
      
          # Print out the final result
          print('Final Portfolio Value: %.2f' % cerebro.broker.getvalue())
      
          # Plot the result
          cerebro.plot()
      
      Starting Portfolio Value: 1000.00
      ---------------------------------------------------------------------------
      IndexError                                Traceback (most recent call last)
      <ipython-input-11-63f0d7e87f7b> in <module>
          157 
          158     # Run over everything
      --> 159     cerebro.run()
          160 
          161     # Print out the final result
      
      ~\anaconda3\envs\Backtrader\lib\site-packages\backtrader\cerebro.py in run(self, **kwargs)
         1125             # let's skip process "spawning"
         1126             for iterstrat in iterstrats:
      -> 1127                 runstrat = self.runstrategies(iterstrat)
         1128                 self.runstrats.append(runstrat)
         1129                 if self._dooptimize:
      
      ~\anaconda3\envs\Backtrader\lib\site-packages\backtrader\cerebro.py in runstrategies(self, iterstrat, predata)
         1291                     self._runonce_old(runstrats)
         1292                 else:
      -> 1293                     self._runonce(runstrats)
         1294             else:
         1295                 if self.p.oldsync:
      
      ~\anaconda3\envs\Backtrader\lib\site-packages\backtrader\cerebro.py in _runonce(self, runstrats)
         1650         '''
         1651         for strat in runstrats:
      -> 1652             strat._once()
         1653             strat.reset()  # strat called next by next - reset lines
         1654 
      
      ~\anaconda3\envs\Backtrader\lib\site-packages\backtrader\lineiterator.py in _once(self)
          295 
          296         for indicator in self._lineiterators[LineIterator.IndType]:
      --> 297             indicator._once()
          298 
          299         for observer in self._lineiterators[LineIterator.ObsType]:
      
      ~\anaconda3\envs\Backtrader\lib\site-packages\backtrader\lineiterator.py in _once(self)
          295 
          296         for indicator in self._lineiterators[LineIterator.IndType]:
      --> 297             indicator._once()
          298 
          299         for observer in self._lineiterators[LineIterator.ObsType]:
      
      ~\anaconda3\envs\Backtrader\lib\site-packages\backtrader\lineiterator.py in _once(self)
          315         # indicators are each called with its min period
          316         self.preonce(0, self._minperiod - 1)
      --> 317         self.oncestart(self._minperiod - 1, self._minperiod)
          318         self.once(self._minperiod, self.buflen())
          319 
      
      ~\anaconda3\envs\Backtrader\lib\site-packages\backtrader\lineiterator.py in oncestart(self, start, end)
          325 
          326     def oncestart(self, start, end):
      --> 327         self.once(start, end)
          328 
          329     def once(self, start, end):
      
      ~\anaconda3\envs\Backtrader\lib\site-packages\backtrader\indicators\basicops.py in once(self, start, end)
          362 
          363         for i in range(start, end):
      --> 364             dst[i] = math.fsum(src[i - period + 1:i + 1]) / period
          365 
          366 
      
      IndexError: array assignment index out of range
      
      
      1 Reply Last reply Reply Quote 0
      • R
        rajanprabu last edited by

        @niqnil

        I think your csv is reversed chronologically... Try with reverse=True

        niqnil 1 Reply Last reply Reply Quote 3
        • niqnil
          niqnil @rajanprabu last edited by

          @rajanprabu Thanks for your response! I tried your suggestion and still got the error.
          Do you know what else might be wrong?

          fromdate=datetime.datetime(2019, 1, 2),
          todate=datetime.datetime(2020, 12, 31),
          reverse=True,
          
          run-out 1 Reply Last reply Reply Quote 0
          • D
            dasch last edited by

            @niqnil as @rajanprabu already said, get your data into the right order. You seem to use YahooFinanceCSVData which should support the reverse param, but in your example you just load a csv file. so there may be your issue. For a local csv file just use GenericCSV feed and ensure your data loaded is correct.

            1 Reply Last reply Reply Quote 2
            • run-out
              run-out @niqnil last edited by

              @niqnil Try swapping out the datafeed to see if your data is the problem. Try using this:

              data = bt.feeds.YahooFinanceData(
                              dataname="FB",
                              timeframe=bt.TimeFrame.Days,
                              fromdate=datetime.date(2020, 1, 1),
                              todate=datetime.date(2020, 12, 31),
                              reverse=False,
                          )
              

              RunBacktest.com

              1 Reply Last reply Reply Quote 2
              • A
                ab_trader last edited by

                The issue is your reversed data feed, I was able to get the same error message. But you use GenericCSVData in your script and it doesn't have reverse parameter. In order to continue to use your Alphavantage data feed, you need to reverse the data in the data files.

                I can suggest to use https://github.com/ab-trader/backtrader_addons/blob/master/backtrader_addons/datafeeds/alphavantage.py, may work for you.

                • If my answer helped, hit reputation up arrow at lower right corner of the post.
                • Python Debugging With Pdb
                • New to python and bt - check this out
                1 Reply Last reply Reply Quote 2
                • niqnil
                  niqnil last edited by

                  @dasch @run-out Yes the problem was the wrong order in my CSV.
                  @ab_trader thanks for the suggestion. I will check it out.

                  I got it to work by reversing the order in the CSV. Thanks everyone for your help:)
                  In case anyone wants to use my quick and dirty way:

                  import pandas as pd
                  
                  # load csv and use row 0 as headers
                  df = pd.read_csv("file_name.csv", header = 0)
                  
                  # reverse data and save
                  df=df.iloc[::-1]
                  df.set_index('timestamp', inplace=True)
                  df.to_csv('reversed.csv')
                  
                  1 Reply Last reply Reply Quote 0
                  • 1 / 1
                  • First post
                    Last post
                  Copyright © 2016, 2017, 2018, 2019, 2020, 2021 NodeBB Forums | Contributors