For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

AttributeError: 'numpy.int64' object has no attribute 'lower' when using pandas with the --noheaders argument



  • When I try the post "Pandas DataFeed Support" Link

    When type in the command:

    >(backtrader37) D:\Python\Jupyter\BackTrader\Test>python ./data-pandas.py --noheaders
    

    It reports:

    > (backtrader37) D:\Python\Jupyter\BackTrader\Test>python ./data-pandas.py --noheaders
    --------------------------------------------------
                     1        2        3        4  5  6
    0
    2006-01-02  3578.73  3605.95  3578.73  3604.33  0  0
    2006-01-03  3604.08  3638.42  3601.84  3614.34  0  0
    2006-01-04  3615.23  3652.46  3615.23  3652.46  0  0
    2006-01-05  3652.19  3661.65  3643.17  3650.24  0  0
    2006-01-06  3650.54  3666.99  3647.66  3666.99  0  0
    ...             ...      ...      ...      ... .. ..
    2006-12-21  4111.85  4125.27  4104.46  4112.10  0  0
    2006-12-22  4109.86  4109.86  4072.62  4073.50  0  0
    2006-12-27  4079.70  4134.86  4079.70  4134.86  0  0
    2006-12-28  4137.44  4142.06  4125.14  4130.66  0  0
    2006-12-29  4130.12  4142.01  4119.94  4119.94  0  0
    [255 rows x 6 columns]
    --------------------------------------------------
    Traceback (most recent call last):
      File "./data-pandas.py", line 92, in <module>
        runstrat()
      File "./data-pandas.py", line 71, in runstrat
        cerebro.run()
      File "C:\Users\WEI\.conda\envs\backtrader37\lib\site-packages\backtrader\cerebro.py", line 1127, in run
        runstrat = self.runstrategies(iterstrat)
      File "C:\Users\WEI\.conda\envs\backtrader37\lib\site-packages\backtrader\cerebro.py", line 1210, in runstrategies
        data._start()
      File "C:\Users\WEI\.conda\envs\backtrader37\lib\site-packages\backtrader\feed.py", line 203, in _start
        self.start()
      File "C:\Users\WEI\.conda\envs\backtrader37\lib\site-packages\backtrader\feeds\pandafeed.py", line 212, in start
        colnames = [x.lower() for x in self.p.dataname.columns.values]
      File "C:\Users\WEI\.conda\envs\backtrader37\lib\site-packages\backtrader\feeds\pandafeed.py", line 212, in <listcomp>
        colnames = [x.lower() for x in self.p.dataname.columns.values]
    AttributeError: 'numpy.int64' object has no attribute 'lower'
    

    Without the argument of --noheaders:

    >(backtrader37) D:\Python\Jupyter\BackTrader\Test>python ./data-pandas.py
    --------------------------------------------------
                   Open     High      Low    Close  Volume  OpenInterest
    Date
    2006-01-02  3578.73  3605.95  3578.73  3604.33       0             0
    2006-01-03  3604.08  3638.42  3601.84  3614.34       0             0
    2006-01-04  3615.23  3652.46  3615.23  3652.46       0             0
    2006-01-05  3652.19  3661.65  3643.17  3650.24       0             0
    2006-01-06  3650.54  3666.99  3647.66  3666.99       0             0
    ...             ...      ...      ...      ...     ...           ...
    2006-12-21  4111.85  4125.27  4104.46  4112.10       0             0
    2006-12-22  4109.86  4109.86  4072.62  4073.50       0             0
    2006-12-27  4079.70  4134.86  4079.70  4134.86       0             0
    2006-12-28  4137.44  4142.06  4125.14  4130.66       0             0
    2006-12-29  4130.12  4142.01  4119.94  4119.94       0             0
    
    [255 rows x 6 columns]
    --------------------------------------------------
    

    Everything is OK! But from the output, we can see that, with the --noheaders argument, the pandas data is successfully loaded, but Cerebro cannot deal with it.

    What's the problem?

    Appendix: data-pandas.py

    from __future__ import (absolute_import, division, print_function,
                            unicode_literals)
    import argparse
    import backtrader as bt
    import backtrader.feeds as btfeeds
    import pandas
    def runstrat():
        args = parse_args()
    
        # Create a cerebro entity
        cerebro = bt.Cerebro(stdstats=False)
    
        # Add a strategy
        cerebro.addstrategy(bt.Strategy)
    
        # Get a pandas dataframe
        datapath = ('../../datas/2006-day-001.txt')
    
        # Simulate the header row isn't there if noheaders requested
        skiprows = 1 if args.noheaders else 0
        header = None if args.noheaders else 0
    
        dataframe = pandas.read_csv(
            datapath,
            skiprows=skiprows,
            header=header,
            # parse_dates=[0],
            parse_dates=True,
            index_col=0,
        )
    
        if not args.noprint:
            print('--------------------------------------------------')
            print(dataframe)
            print('--------------------------------------------------')
    
        # Pass it to the backtrader datafeed and add it to the cerebro
        data = bt.feeds.PandasData(dataname=dataframe,
                                   # datetime='Date',
                                   nocase=True,
                                   )
    
        cerebro.adddata(data)
    
        # Run over everything
        cerebro.run()
    
        # Plot the result
        cerebro.plot(style='bar')
    
    
    def parse_args():
        parser = argparse.ArgumentParser(
            description='Pandas test script')
    
        parser.add_argument('--noheaders', action='store_true', default=False,
                            required=False,
                            help='Do not use header rows')
    
        parser.add_argument('--noprint', action='store_true', default=False,
                            help='Print the dataframe')
    
        return parser.parse_args()
    
    
    if __name__ == '__main__':
        runstrat()
    


  • It seems that PandasData column index auto-detection mechanism is broken in case the input dataframe contains no column names ( using indexes instead).

    I suspect the following commit has introduced the regression:

    05051890efe527ee22919d354038b4dfc1ffe7ca: Rework ix -> iloc pull request and autodetection algorithm in PandasData

    Opening the issue to track this:

    https://github.com/backtrader2/backtrader/issues/43


Log in to reply
 

});