Index out of range error using CSV file

Zugba

If anybody knows how to fix this I would very much appreciate it.

I'm trying to implement a CSV file but I keep getting this error:

Traceback (most recent call last):
  File "c:\Users\dnzug\Zugba_Projects\Projects\backtest.py", line 46, in <module>
    cerebro.run()
  File "C:\Users\dnzug\Zugba_Projects\work_env\lib\site-packages\backtrader\cerebro.py", line 1127, in run
    runstrat = self.runstrategies(iterstrat)
  File "C:\Users\dnzug\Zugba_Projects\work_env\lib\site-packages\backtrader\cerebro.py", line 1212, in runstrategies
    data.preload()
  File "C:\Users\dnzug\Zugba_Projects\work_env\lib\site-packages\backtrader\feed.py", line 688, in preload
    while self.load():
  File "C:\Users\dnzug\Zugba_Projects\work_env\lib\site-packages\backtrader\feed.py", line 479, in load
    _loadret = self._load()
  File "C:\Users\dnzug\Zugba_Projects\work_env\lib\site-packages\backtrader\feed.py", line 710, in _load
    return self._loadline(linetokens)
  File "C:\Users\dnzug\Zugba_Projects\work_env\lib\site-packages\backtrader\feeds\csvgeneric.py", line 148, in _loadline
    csvfield = linetokens[csvidx]
IndexError: list index out of range

I'm using this class to implement the CSV file:

class CSV(bt.feeds.GenericCSVData):

    params = (
        ('dtformat', '%Y-%m-%d'),

        ('datetime', 0),
        ('open', 1),
        ('high', 2),
        ('low', 3),
        ('close', 4),
        ('volume', -1),
        ('openintrest', -1)
    )

feed =  CSV(dataname ="ETH_data.csv")

The CSV file I am using:

datetime,open,high,low,close
2020-05-04,209.73,210.29,195.2,206.65
2020-05-05,206.64,210.88,201.33,204.8
2020-05-06,205.58,210.76,198.61,198.88
2020-05-07,197.84,215.0,197.12,212.68
2020-05-08,213.01,216.25,207.12,211.6
2020-05-09,210.96,214.5,208.68,211.55
2020-05-10,210.0,210.0,174.43,188.27
2020-05-11,188.38,193.5,175.76,186.42
2020-05-12,185.94,192.06,185.94,189.9
2020-05-13,190.0,201.19,188.72,199.12
2020-05-14,200.1,205.97,196.06,203.1
2020-05-15,203.48,204.15,191.2,194.23
2020-05-16,193.41,203.27,193.41,200.51
2020-05-17,200.38,209.99,199.94,207.74
2020-05-18,208.35,217.01,208.03,214.49
2020-05-19,214.17,215.82,209.49,214.51
2020-05-20,214.46,215.6,205.96,210.0
2020-05-21,210.0,211.5,191.84,198.52

It's also weird that I can implement the CSV file with no problems using this method:

data = bt.feeds.GenericCSVData(
        dataname='ETH_data.csv',

        dtformat=('%Y-%m-%d'),
        datetime=0,
        high=2,
        low=3,
        open=1,
        close=4,
        volume=-1,
        openinterest=-1,

    )

but I would like to expand the file so this is not a practical solution.

Thanks in advance :)

vladisld

@zugba seems strange indeed. Are you sure exactly the same csv data file was used in both implementations ? The error suggests that there is a problem with the data itself.

Zugba

@vladisld Yes I am certain that I am using the same CSV file

Zugba

@zugba
Here is the whole code

cerebro = bt.Cerebro()

class CSV(bt.feeds.GenericCSVData):

    params = (
        ('dtformat', '%Y-%m-%d'),

        ('datetime', 0),
        ('open', 1),
        ('high', 2),
        ('low', 3),
        ('close', 4),
        ('volume', -1),
        ('openintrest', -1)
    )


data = bt.feeds.GenericCSVData(
        dataname='ETH_data.csv',

        dtformat=('%Y-%m-%d'),
        datetime=0,
        high=2,
        low=3,
        open=1,
        close=4,
        volume=-1,
        openinterest=-1,

    )


feed =  CSV(dataname ="ETH_data.csv")

cerebro.adddata(feed)
    
#cerebro.addstrategy(strategy)

cerebro.run()

cerebro.plot(volume = False)

vladisld

@zugba hmm, currently the only way I could see is to debug the _loadline method in csvgeneric.py file ( setting the breakpoint upon IndexError exception raise may help) and see what values you get in linetokens and csvidx - this way you may get an idea where the wrong data line is coming from. Unless others may have a better idea of cause.

davidavr

@zugba Could you show the last lines of the file (e.g. the output of tail ETH_data.csv from the terminal)? Maybe there is a bad trailing line that is messing it up.

run-out

@zugba Thank you for your code. I copied and ran your code using both your data and a sample of backtrader data. I used both the standard csv loader and your custom class.

All ran smoothly with backtrader data. The only error was when I use your data with your custom class. This suggest an error in your data. Try using a different data source.

Here's the data I used. 2006-volume-day-001.txt

Zugba

@run-out Thanks! I used a different data source and it worked

vcfriend

是数据格式的问题，修正一下数据格式可以了，在尾部添加1列分隔符。
@zugba said in Index out of range error using CSV file:

datetime,open,high,low,close,
2020-05-04,209.73,210.29,195.2,206.65,
2020-05-05,206.64,210.88,201.33,204.8,
2020-05-06,205.58,210.76,198.61,198.88,
2020-05-07,197.84,215.0,197.12,212.68,
2020-05-08,213.01,216.25,207.12,211.6,
2020-05-09,210.96,214.5,208.68,211.55,
2020-05-10,210.0,210.0,174.43,188.27,
2020-05-11,188.38,193.5,175.76,186.42,
2020-05-12,185.94,192.06,185.94,189.9,
2020-05-13,190.0,201.19,188.72,199.12,
2020-05-14,200.1,205.97,196.06,203.1,
2020-05-15,203.48,204.15,191.2,194.23,
2020-05-16,193.41,203.27,193.41,200.51,
2020-05-17,200.38,209.99,199.94,207.74,
2020-05-18,208.35,217.01,208.03,214.49,
2020-05-19,214.17,215.82,209.49,214.51,
2020-05-20,214.46,215.6,205.96,210.0,
2020-05-21,210.0,211.5,191.84,198.52,