Backtrader Community

    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

    Index out of range error using CSV file

    General Code/Help
    6
    9
    1732
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Z
      Zugba last edited by

      If anybody knows how to fix this I would very much appreciate it.

      I'm trying to implement a CSV file but I keep getting this error:

      Traceback (most recent call last):
        File "c:\Users\dnzug\Zugba_Projects\Projects\backtest.py", line 46, in <module>
          cerebro.run()
        File "C:\Users\dnzug\Zugba_Projects\work_env\lib\site-packages\backtrader\cerebro.py", line 1127, in run
          runstrat = self.runstrategies(iterstrat)
        File "C:\Users\dnzug\Zugba_Projects\work_env\lib\site-packages\backtrader\cerebro.py", line 1212, in runstrategies
          data.preload()
        File "C:\Users\dnzug\Zugba_Projects\work_env\lib\site-packages\backtrader\feed.py", line 688, in preload
          while self.load():
        File "C:\Users\dnzug\Zugba_Projects\work_env\lib\site-packages\backtrader\feed.py", line 479, in load
          _loadret = self._load()
        File "C:\Users\dnzug\Zugba_Projects\work_env\lib\site-packages\backtrader\feed.py", line 710, in _load
          return self._loadline(linetokens)
        File "C:\Users\dnzug\Zugba_Projects\work_env\lib\site-packages\backtrader\feeds\csvgeneric.py", line 148, in _loadline
          csvfield = linetokens[csvidx]
      IndexError: list index out of range
      

      I'm using this class to implement the CSV file:

      class CSV(bt.feeds.GenericCSVData):
      
          params = (
              ('dtformat', '%Y-%m-%d'),
      
              ('datetime', 0),
              ('open', 1),
              ('high', 2),
              ('low', 3),
              ('close', 4),
              ('volume', -1),
              ('openintrest', -1)
          )
      
      feed =  CSV(dataname ="ETH_data.csv")
      

      The CSV file I am using:

      datetime,open,high,low,close
      2020-05-04,209.73,210.29,195.2,206.65
      2020-05-05,206.64,210.88,201.33,204.8
      2020-05-06,205.58,210.76,198.61,198.88
      2020-05-07,197.84,215.0,197.12,212.68
      2020-05-08,213.01,216.25,207.12,211.6
      2020-05-09,210.96,214.5,208.68,211.55
      2020-05-10,210.0,210.0,174.43,188.27
      2020-05-11,188.38,193.5,175.76,186.42
      2020-05-12,185.94,192.06,185.94,189.9
      2020-05-13,190.0,201.19,188.72,199.12
      2020-05-14,200.1,205.97,196.06,203.1
      2020-05-15,203.48,204.15,191.2,194.23
      2020-05-16,193.41,203.27,193.41,200.51
      2020-05-17,200.38,209.99,199.94,207.74
      2020-05-18,208.35,217.01,208.03,214.49
      2020-05-19,214.17,215.82,209.49,214.51
      2020-05-20,214.46,215.6,205.96,210.0
      2020-05-21,210.0,211.5,191.84,198.52
      

      It's also weird that I can implement the CSV file with no problems using this method:

      data = bt.feeds.GenericCSVData(
              dataname='ETH_data.csv',
      
              dtformat=('%Y-%m-%d'),
              datetime=0,
              high=2,
              low=3,
              open=1,
              close=4,
              volume=-1,
              openinterest=-1,
      
          )
      

      but I would like to expand the file so this is not a practical solution.

      Thanks in advance :)

      vladisld 1 Reply Last reply Reply Quote 0
      • vladisld
        vladisld @Zugba last edited by

        @zugba seems strange indeed. Are you sure exactly the same csv data file was used in both implementations ? The error suggests that there is a problem with the data itself.

        Z 1 Reply Last reply Reply Quote 0
        • Z
          Zugba @vladisld last edited by

          @vladisld Yes I am certain that I am using the same CSV file

          Z 1 Reply Last reply Reply Quote 0
          • Z
            Zugba @Zugba last edited by

            @zugba
            Here is the whole code

            cerebro = bt.Cerebro()
            
            class CSV(bt.feeds.GenericCSVData):
            
                params = (
                    ('dtformat', '%Y-%m-%d'),
            
                    ('datetime', 0),
                    ('open', 1),
                    ('high', 2),
                    ('low', 3),
                    ('close', 4),
                    ('volume', -1),
                    ('openintrest', -1)
                )
            
            
            data = bt.feeds.GenericCSVData(
                    dataname='ETH_data.csv',
            
                    dtformat=('%Y-%m-%d'),
                    datetime=0,
                    high=2,
                    low=3,
                    open=1,
                    close=4,
                    volume=-1,
                    openinterest=-1,
            
                )
            
            
            feed =  CSV(dataname ="ETH_data.csv")
            
            cerebro.adddata(feed)
                
            #cerebro.addstrategy(strategy)
            
            cerebro.run()
            
            cerebro.plot(volume = False)
            
            vladisld D run-out 3 Replies Last reply Reply Quote 0
            • vladisld
              vladisld @Zugba last edited by

              @zugba hmm, currently the only way I could see is to debug the _loadline method in csvgeneric.py file ( setting the breakpoint upon IndexError exception raise may help) and see what values you get in linetokens and csvidx - this way you may get an idea where the wrong data line is coming from. Unless others may have a better idea of cause.

              1 Reply Last reply Reply Quote 1
              • D
                davidavr @Zugba last edited by

                @zugba Could you show the last lines of the file (e.g. the output of tail ETH_data.csv from the terminal)? Maybe there is a bad trailing line that is messing it up.

                1 Reply Last reply Reply Quote 0
                • run-out
                  run-out @Zugba last edited by

                  @zugba Thank you for your code. I copied and ran your code using both your data and a sample of backtrader data. I used both the standard csv loader and your custom class.

                  All ran smoothly with backtrader data. The only error was when I use your data with your custom class. This suggest an error in your data. Try using a different data source.

                  Here's the data I used. 2006-volume-day-001.txt

                  RunBacktest.com

                  Z 1 Reply Last reply Reply Quote 0
                  • Z
                    Zugba @run-out last edited by

                    @run-out Thanks! I used a different data source and it worked

                    1 Reply Last reply Reply Quote 1
                    • vcfriend
                      vcfriend last edited by

                      是数据格式的问题,修正一下数据格式可以了,在尾部添加1列分隔符。
                      @zugba said in Index out of range error using CSV file:

                      datetime,open,high,low,close,
                      2020-05-04,209.73,210.29,195.2,206.65,
                      2020-05-05,206.64,210.88,201.33,204.8,
                      2020-05-06,205.58,210.76,198.61,198.88,
                      2020-05-07,197.84,215.0,197.12,212.68,
                      2020-05-08,213.01,216.25,207.12,211.6,
                      2020-05-09,210.96,214.5,208.68,211.55,
                      2020-05-10,210.0,210.0,174.43,188.27,
                      2020-05-11,188.38,193.5,175.76,186.42,
                      2020-05-12,185.94,192.06,185.94,189.9,
                      2020-05-13,190.0,201.19,188.72,199.12,
                      2020-05-14,200.1,205.97,196.06,203.1,
                      2020-05-15,203.48,204.15,191.2,194.23,
                      2020-05-16,193.41,203.27,193.41,200.51,
                      2020-05-17,200.38,209.99,199.94,207.74,
                      2020-05-18,208.35,217.01,208.03,214.49,
                      2020-05-19,214.17,215.82,209.49,214.51,
                      2020-05-20,214.46,215.6,205.96,210.0,
                      2020-05-21,210.0,211.5,191.84,198.52,

                      1 Reply Last reply Reply Quote 0
                      • 1 / 1
                      • First post
                        Last post
                      Copyright © 2016, 2017, 2018, 2019, 2020, 2021 NodeBB Forums | Contributors