Navigation

    Backtrader Community

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

    Reading Pandas DataFrame

    General Code/Help
    3
    7
    121
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • T
      tralaritralara last edited by

      Greetings,

      I am just starting with Backtrader, I think it is a really complete tool, but I am finding some difficulties to get started with it. I am trying to follow the Quickstart Guide in documentation, but I am not even able to load price history. This is my code:

      from __future__ import (absolute_import, division, print_function,
                              unicode_literals)
      
      import datetime  # For datetime objects
      import os.path  # To manage paths
      import sys  # To find out the script name (in argv[0])
      
      # Import the backtrader platform
      import backtrader as bt
      
      class dataFeed(bt.feeds.PandasData):
          params = (
              ('datetime', None),
              ('open', None),
              ('high', None),
              ('low', None),
              ('close', 1),
              ('volume', None),
              ('openinterest', None)
          )
      
      def data_mining():
          start_date = pd.to_datetime('1970-01-01')
          end_date = pd.to_datetime('2020-12-31')
          auth_tok = '##########'
          data = quandl.get('LBMA/GOLD', trim_start=start_date, trim_end=end_date, authtoken=auth_tok)
          close = data['USD (AM)']
          all_weekdays = pd.date_range(start=start_date, end=end_date, freq='B')  ## B = Business days (L-V)
          close = close.reindex(all_weekdays)
          close = close.fillna(method='ffill')
          close = close.to_frame().rename(columns={'index': 'Date', 'USD (AM)': 'Close'})
          return close  ## With this I return a DataFrame of 1 column, I only want close prices
      
      if __name__ == '__main__':
          # Create a cerebro entity
          cerebro = bt.Cerebro()
      
          close = data_mining() ## My data in Pandas DataFrame format
          # Create a Data Feed
          data = dataFeed(dataname=close)  ### Class defined above
      
          # Add the Data Feed to Cerebro
          cerebro.adddata(data)
      
          # Set our desired cash start
          cerebro.broker.setcash(100000.0)
      
          # Print out the starting conditions
          print('Starting Portfolio Value: %.2f' % cerebro.broker.getvalue())
      
          # Run over everything
          cerebro.run()
      
          # Print out the final result
          print('Final Portfolio Value: %.2f' % cerebro.broker.getvalue())
      

      Whatever I do (I tried many more similar ideas), I get this error:

      Traceback (most recent call last):
        File "<input>", line 13, in <module>
        File "C:\Proyectos\cockroach_portfolio\venv\lib\site-packages\backtrader\cerebro.py", line 1127, in run
          runstrat = self.runstrategies(iterstrat)
        File "C:\Proyectos\cockroach_portfolio\venv\lib\site-packages\backtrader\cerebro.py", line 1212, in runstrategies
          data.preload()
        File "C:\Proyectos\cockroach_portfolio\venv\lib\site-packages\backtrader\feed.py", line 688, in preload
          while self.load():
        File "C:\Proyectos\cockroach_portfolio\venv\lib\site-packages\backtrader\feed.py", line 479, in load
          _loadret = self._load()
        File "C:\Proyectos\cockroach_portfolio\venv\lib\site-packages\backtrader\feed.py", line 710, in _load
          return self._loadline(linetokens)
        File "C:\Proyectos\cockroach_portfolio\venv\lib\site-packages\backtrader\feeds\csvgeneric.py", line 148, in _loadline
          csvfield = linetokens[csvidx]
      IndexError: list index out of range
      

      I would like to know how to import Pandas data, I am mainly interested only in close prices and large history backtestings.

      Thanks in advance

      D 1 Reply Last reply Reply Quote 0
      • D
        dasch @tralaritralara last edited by

        @tralaritralara try thhis:

        class dataFeed(bt.feeds.PandasData):
            params = (
                ('datetime', 0),
                ('open', None),
                ('high', None),
                ('low', None),
                ('close', 1),
                ('volume', None),
                ('openinterest', None)
            )
        

        you may need to define the dtparam for your data. but you need to provide some date time information to backtrader.

        T 1 Reply Last reply Reply Quote 0
        • T
          tralaritralara @dasch last edited by

          @dasch My datetime is given in index column (according to documentation, that should work), it has %Y-%m-%d format, I will update my OP with a sample

          T 1 Reply Last reply Reply Quote 0
          • T
            tralaritralara @tralaritralara last edited by

            It seems I cannot edit my posts, so I will post it here. This is a sample of my data, historical prices for Gold obtained from Quandl. As I said before, I am only interested in close prices, so there is no OHLC structure.

                          Close
            1970-01-01      NaN
            1970-01-02    35.12
            1970-01-05    35.10
            1970-01-06    35.12
            1970-01-07    35.00
                         ...
            2020-12-25  1872.55
            2020-12-28  1872.55
            2020-12-29  1873.90
            2020-12-30  1877.55
            2020-12-31  1891.10
            [13306 rows x 1 columns]
            
            D 1 Reply Last reply Reply Quote 0
            • D
              dasch @tralaritralara last edited by

              @tralaritralara you can reset the index by:

              df.reset_index(inplace=True)
              

              before creating the pandas feed, set datetime to 0.

              class dataFeed(bt.feeds.PandasData):
                  params = (
                      ('datetime', 0),
                      ('open', None),
                      ('high', None),
                      ('low', None),
                      ('close', 1),
                      ('volume', None),
                      ('openinterest', None)
                  )
              
              run-out T 2 Replies Last reply Reply Quote 0
              • run-out
                run-out @dasch last edited by

                @dasch I haven't tried this, but I think from previous reading you have to set the open/high/low to the close value not None

                RunBacktest.com

                1 Reply Last reply Reply Quote 0
                • T
                  tralaritralara @dasch last edited by

                  @dasch @run-out That made the fix. run-out is right, although dasch's proposal also works out, it will later create other problems as soon as cerebro tools get into action. So it is better to leave all the columns with close prices.

                  Thanks a lot to both of you :)

                  1 Reply Last reply Reply Quote 1
                  • 1 / 1
                  • First post
                    Last post
                  Copyright © 2016, 2017, 2018, 2019, 2020, 2021 NodeBB Forums | Contributors