For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/
Quickstart with generic CSV - IndexError: array assignment index out of range
-
Hi, I am a newbie trying to work through the quickstart guide.
I get IndexError: array assignment index out of range when I try to use a generic CSV with a date range. It works without date range, meaning if I don't pass
fromdate=datetime.datetime(2019, 1, 2), todate=datetime.datetime(2020, 12, 31),I have also tried variations of different dates for the date range with this generic CSV and got the same IndexError. I have checked that the CSV data has the dates.
There is no index error with using the original file and code from the guide.
data = bt.feeds.YahooFinanceCSVData( dataname='orcl-1995-2014.txt', # Do not pass values before this date fromdate=datetime.datetime(2000, 1, 1), # Do not pass values before this date todate=datetime.datetime(2000, 12, 31), # Do not pass values after this date reverse=False)Can someone help point out what I am doing wrong? My csv(snipped), code and traceback:
timestamp,open,high,low,close,adjusted_close,volume,dividend_amount,split_coefficient 2021-03-03,687.99,700.7,651.705,653.2,653.2,29693988,0.0000,1.0 2021-03-02,718.28,721.11,685.0,686.44,686.44,23732158,0.0000,1.0 2021-03-01,690.11,719.0,685.05,718.43,718.43,27136239,0.0000,1.0 ... 2020-12-31,699.99,718.72,691.12,705.67,705.67,49649928,0.0000,1.0 2020-12-30,672.0,696.6,668.3603,694.78,694.78,42846021,0.0000,1.0 2020-12-29,661.0,669.9,655.0,665.99,665.99,22910811,0.0000,1.0 ... 2019-01-04,306.0,318.0,302.73,317.69,63.538,7394100,0.0000,1.0 2019-01-03,307.0,309.4,297.38,300.36,60.072,6965200,0.0000,1.0 2019-01-02,306.1,315.13,298.8,310.12,62.024,11658600,0.0000,1.0from __future__ import (absolute_import, division, print_function, unicode_literals) %matplotlib widget import datetime # For datetime objects import os.path # To manage paths import sys # To find out the script name (in argv[0]) # Import the backtrader platform import backtrader as bt # Create a Stratey class TestStrategy(bt.Strategy): params = ( ('maperiod', 15), ) def log(self, txt, dt=None): ''' Logging function fot this strategy''' dt = dt or self.datas[0].datetime.date(0) print('%s, %s' % (dt.isoformat(), txt)) def __init__(self): # Keep a reference to the "close" line in the data[0] dataseries self.dataclose = self.datas[0].close # To keep track of pending orders and buy price/commission self.order = None self.buyprice = None self.buycomm = None # Add a MovingAverageSimple indicator self.sma = bt.indicators.SimpleMovingAverage( self.datas[0], period=self.params.maperiod) # Indicators for the plotting show bt.indicators.ExponentialMovingAverage(self.datas[0], period=25) bt.indicators.WeightedMovingAverage(self.datas[0], period=25, subplot=True) bt.indicators.StochasticSlow(self.datas[0]) bt.indicators.MACDHisto(self.datas[0]) rsi = bt.indicators.RSI(self.datas[0]) bt.indicators.SmoothedMovingAverage(rsi, period=10) bt.indicators.ATR(self.datas[0], plot=False) def notify_order(self, order): if order.status in [order.Submitted, order.Accepted]: # Buy/Sell order submitted/accepted to/by broker - Nothing to do return # Check if an order has been completed # Attention: broker could reject order if not enough cash if order.status in [order.Completed]: if order.isbuy(): self.log( 'BUY EXECUTED, Price: %.2f, Cost: %.2f, Comm %.2f' % (order.executed.price, order.executed.value, order.executed.comm)) self.buyprice = order.executed.price self.buycomm = order.executed.comm else: # Sell self.log('SELL EXECUTED, Price: %.2f, Cost: %.2f, Comm %.2f' % (order.executed.price, order.executed.value, order.executed.comm)) self.bar_executed = len(self) elif order.status in [order.Canceled, order.Margin, order.Rejected]: self.log('Order Canceled/Margin/Rejected') # Write down: no pending order self.order = None def notify_trade(self, trade): if not trade.isclosed: return self.log('OPERATION PROFIT, GROSS %.2f, NET %.2f' % (trade.pnl, trade.pnlcomm)) def next(self): # Simply log the closing price of the series from the reference self.log('Close, %.2f' % self.dataclose[0]) # Check if an order is pending ... if yes, we cannot send a 2nd one if self.order: return # Check if we are in the market if not self.position: # Not yet ... we MIGHT BUY if ... if self.dataclose[0] > self.sma[0]: # BUY, BUY, BUY!!! (with all possible default parameters) self.log('BUY CREATE, %.2f' % self.dataclose[0]) # Keep track of the created order to avoid a 2nd order self.order = self.buy() else: if self.dataclose[0] < self.sma[0]: # SELL, SELL, SELL!!! (with all possible default parameters) self.log('SELL CREATE, %.2f' % self.dataclose[0]) # Keep track of the created order to avoid a 2nd order self.order = self.sell() if __name__ == '__main__': # Create a cerebro entity cerebro = bt.Cerebro() # Add a strategy cerebro.addstrategy(TestStrategy) # Create a Data Feed data = bt.feeds.GenericCSVData( dataname='alpha_vantage_TSLA.csv', fromdate=datetime.datetime(2019, 1, 2), todate=datetime.datetime(2020, 12, 31), nullvalue=0.0, dtformat=('%Y-%m-%d'), datetime=0, time=-1, open=1, high=2, low=3, close=5, volume=6, openinterest=-1 ) # Add the Data Feed to Cerebro cerebro.adddata(data) # Set our desired cash start cerebro.broker.setcash(1000.0) # Add a FixedSize sizer according to the stake cerebro.addsizer(bt.sizers.FixedSize, stake=10) # Set the commission cerebro.broker.setcommission(commission=0.002) # Print out the starting conditions print('Starting Portfolio Value: %.2f' % cerebro.broker.getvalue()) # Run over everything cerebro.run() # Print out the final result print('Final Portfolio Value: %.2f' % cerebro.broker.getvalue()) # Plot the result cerebro.plot()Starting Portfolio Value: 1000.00 --------------------------------------------------------------------------- IndexError Traceback (most recent call last) <ipython-input-11-63f0d7e87f7b> in <module> 157 158 # Run over everything --> 159 cerebro.run() 160 161 # Print out the final result ~\anaconda3\envs\Backtrader\lib\site-packages\backtrader\cerebro.py in run(self, **kwargs) 1125 # let's skip process "spawning" 1126 for iterstrat in iterstrats: -> 1127 runstrat = self.runstrategies(iterstrat) 1128 self.runstrats.append(runstrat) 1129 if self._dooptimize: ~\anaconda3\envs\Backtrader\lib\site-packages\backtrader\cerebro.py in runstrategies(self, iterstrat, predata) 1291 self._runonce_old(runstrats) 1292 else: -> 1293 self._runonce(runstrats) 1294 else: 1295 if self.p.oldsync: ~\anaconda3\envs\Backtrader\lib\site-packages\backtrader\cerebro.py in _runonce(self, runstrats) 1650 ''' 1651 for strat in runstrats: -> 1652 strat._once() 1653 strat.reset() # strat called next by next - reset lines 1654 ~\anaconda3\envs\Backtrader\lib\site-packages\backtrader\lineiterator.py in _once(self) 295 296 for indicator in self._lineiterators[LineIterator.IndType]: --> 297 indicator._once() 298 299 for observer in self._lineiterators[LineIterator.ObsType]: ~\anaconda3\envs\Backtrader\lib\site-packages\backtrader\lineiterator.py in _once(self) 295 296 for indicator in self._lineiterators[LineIterator.IndType]: --> 297 indicator._once() 298 299 for observer in self._lineiterators[LineIterator.ObsType]: ~\anaconda3\envs\Backtrader\lib\site-packages\backtrader\lineiterator.py in _once(self) 315 # indicators are each called with its min period 316 self.preonce(0, self._minperiod - 1) --> 317 self.oncestart(self._minperiod - 1, self._minperiod) 318 self.once(self._minperiod, self.buflen()) 319 ~\anaconda3\envs\Backtrader\lib\site-packages\backtrader\lineiterator.py in oncestart(self, start, end) 325 326 def oncestart(self, start, end): --> 327 self.once(start, end) 328 329 def once(self, start, end): ~\anaconda3\envs\Backtrader\lib\site-packages\backtrader\indicators\basicops.py in once(self, start, end) 362 363 for i in range(start, end): --> 364 dst[i] = math.fsum(src[i - period + 1:i + 1]) / period 365 366 IndexError: array assignment index out of range -
I think your csv is reversed chronologically... Try with reverse=True
-
@rajanprabu Thanks for your response! I tried your suggestion and still got the error.
Do you know what else might be wrong?fromdate=datetime.datetime(2019, 1, 2), todate=datetime.datetime(2020, 12, 31), reverse=True, -
@niqnil as @rajanprabu already said, get your data into the right order. You seem to use YahooFinanceCSVData which should support the reverse param, but in your example you just load a csv file. so there may be your issue. For a local csv file just use GenericCSV feed and ensure your data loaded is correct.
-
@niqnil Try swapping out the datafeed to see if your data is the problem. Try using this:
data = bt.feeds.YahooFinanceData( dataname="FB", timeframe=bt.TimeFrame.Days, fromdate=datetime.date(2020, 1, 1), todate=datetime.date(2020, 12, 31), reverse=False, ) -
The issue is your reversed data feed, I was able to get the same error message. But you use
GenericCSVDatain your script and it doesn't havereverseparameter. In order to continue to use yourAlphavantagedata feed, you need to reverse the data in the data files.I can suggest to use https://github.com/ab-trader/backtrader_addons/blob/master/backtrader_addons/datafeeds/alphavantage.py, may work for you.
-
@dasch @run-out Yes the problem was the wrong order in my CSV.
@ab_trader thanks for the suggestion. I will check it out.I got it to work by reversing the order in the CSV. Thanks everyone for your help:)
In case anyone wants to use my quick and dirty way:import pandas as pd # load csv and use row 0 as headers df = pd.read_csv("file_name.csv", header = 0) # reverse data and save df=df.iloc[::-1] df.set_index('timestamp', inplace=True) df.to_csv('reversed.csv')
