Navigation

    Backtrader Community

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

    How to filter large stock universe then run trading system on small group

    General Code/Help
    2
    9
    2470
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • S
      Sagittarius19 last edited by

      Hello,

      I am new to backtrader and have been looking through the forums and docs to try to find a solution to my problem but i couldn't find anything that solves it completely.

      I have two different datafeeds avaiable to me. The first being daily OHLC data for 100 stocks and then I have tick data for those same 100 stocks.

      My idea is to run each stock through a filter daily, and then use that smaller universe of stocks to run my algo.

      So far I have been able to filter the large universe of stocks each day into a dictionary that will contain only a handful of names. The trouble is that I have no idea how to use this new dictionary to run a second strategy as backtrader doesn't seem to be able to support that. I also am not sure how to get the name of ticker that is being triggered by my filters. Without the ticker name, im not sure how I can identify the datas.

      Does anyone have a solution?

      B 1 Reply Last reply Reply Quote 0
      • B
        backtrader administrators @Sagittarius19 last edited by

        You want to run two different algos and you need to run backtrader twice. As you would need to run other platforms twice.

        1. You run your first algo on a cerebro and end up with a dictionary
        2. You use the dictionary to make a 2nd cerebro run with that restricted set of data feeds

        @sagittarius19 said in How to filter large stock universe then run trading system on small group:

        I also am not sure how to get the name of ticker that is being triggered by my filters

        You have created the 1st set of data feeds, so it seems you already have access to the ticker names. We probably won't understand your worries without seeing some code.

        1 Reply Last reply Reply Quote 0
        • S
          Sagittarius19 last edited by Sagittarius19

          list item@backtrader thank you for the quick reply.

          @sagittarius19 said in How to filter large stock universe then run trading system on small group:

          I also am not sure how to get the name of ticker that is being triggered by my filters

          You have created the 1st set of data feeds, so it seems you already have access to the ticker names. We probably won't understand your worries without seeing some code.

          What i mean to say is, that there is no column in datas that says which ticker I am using. Here is a sample of code.

          def __init__(self): 
                 self.Universe = []
             def next(self):    
                 for i, d in enumerate(self.datas):  
                     self.day_open = self.datas[i].open
                     self.day_close = self.datas[i].close
                     self.date = self.datas[i].datetime.date(0)
                     if self.day_open[0]> ((self.day_close[-1])*1.05):
                         self.Universe.append(self.date)
             def stop(self):
                 print ("Stock list: %s" % self.Universe)
          

          This is only able to append the dates of times where the stocks are over 5% up on the day at the open. I send those date values to the list 'self.Universe' but I dont know how to record in the same list or a separate list the ticker that the filter was running since there is no "symbol" or "ticker" column in the dataframe.

          B 1 Reply Last reply Reply Quote 0
          • B
            backtrader administrators @Sagittarius19 last edited by

            @sagittarius19 said in How to filter large stock universe then run trading system on small group:

                   self.date = self.datas[i].datetime.date(0)
                   if self.day_open[0]> ((self.day_close[-1])*1.05):
                       self.Universe.append(self.date)
            

            You are appending a datetime.date instance to your universe. It would really be a miracle if you could actually find out to which data feed it belongs.

            You can add each data feed with a name and use it with self.dnames or self.data._name.

            See:

            • Docs - Cerebro - adddata
            • Docs - Stratey - dnames
            1 Reply Last reply Reply Quote 0
            • S
              Sagittarius19 last edited by Sagittarius19

              @backtrader I have updated my code below:

              import datetime 
              from datetime import timedelta
              import os.path  
              import sys 
              import pandas as pd
              import backtrader.feeds as btfeeds
              import pandas_datareader.data as web
              import backtrader as bt 
              import backtrader.indicators as btind
              import backtrader.analyzers as btanalyzers
              
              class Pandasclass(btfeeds.PandasData):
                  lines = ('symbol')
                  params = (('symbol',-1)) 
                  datafields = btfeeds.PandasData.datafields + (['symbol'])
              class Universe_filter(bt.Strategy):
                  Universe_date = []
                  Universe_data = []
                  def __init__(self):
                      pass
                  def next(self):    
                      for i, d in enumerate(self.datas): 
                          self.day_open = self.datas[i].open
                          self.day_close = self.datas[i].close
                          self.date = self.datas[i].datetime.date(0)
                          self.symbol = self.datas[i].symbol
                          if self.day_open[0]> ((self.day_close[-1])*1.05):
                              self.Universe_date.append(self.date[0])
                              self.Universe_data.append(self.symbol[0])
                  def stop(self):
                      print ("Stock dates: %s" % self.Universe_date)
                      print ("Stock datas: %s" % self.Universe_data)
                      print (help(btfeeds.PandasData)) 
              if __name__ == '__main__':  
                  cerebro = bt.Cerebro()
                  cerebro.addstrategy(Universe_filter)
                  start = datetime.date(2017,4,1)
                  end = datetime.date(2018,3,19) 
                  #stocks = ["AAPL","MSFT",'TSLA','FB','AMZN','GOOGL','BABA','PYPL','OSTK']
                  stocks = ['FB','AAPL'] 
                  for i in stocks: 
                      DF_1 = web.DataReader([i], "morningstar", start, end)
                      DF_1= pd.DataFrame(DF_1)
                      DF_1.reset_index(level = 'Symbol', inplace=True,drop=False)
                      data1 = Pandasclass(dataname = DF_1)
                      cerebro.adddata(data1)       
                  cerebro.run()        
              

              I am still getting this error though. Any idea what to do?

              Traceback (most recent call last):
                File "c:\Users\Robert\Desktop\Python docs\Extended pandas import attempt.py", line 12, in <module>
                  class Pandasclass(btfeeds.PandasData):
                File "C:\pyth27 fold\pyth27path\lib\site-packages\backtrader\lineseries.py", line 350, in __new__
                  cls = super(MetaLineSeries, meta).__new__(meta, name, bases, dct)
                File "C:\pyth27 fold\pyth27path\lib\site-packages\backtrader\metabase.py", line 239, in __new__
                  cls.params = params._derive(name, newparams, morebasesparams)
                File "C:\pyth27 fold\pyth27path\lib\site-packages\backtrader\metabase.py", line 116, in _derive
                  clsinfo.update(info)
                File "C:\pyth27 fold\pyth27path\lib\_abcoll.py", line 571, in update
                  for key, value in other:
              ValueError: too many values to unpack
              
              1 Reply Last reply Reply Quote 0
              • B
                backtrader administrators last edited by

                Your params definition is wrong. You are defining a tuple of 2 elements and not a tuple in a tuple.

                1 Reply Last reply Reply Quote 0
                • S
                  Sagittarius19 last edited by Sagittarius19

                  @backtrader thanks for the help regarding this. I know its prolly a very simple fix. Here is what I updated to, and im still getting an error.

                  class Pandasclass(btfeeds.PandasData):
                         lines = ('symbol')
                         params = (('symbol', -1),)            
                         btfeeds.PandasData.datafields = btfeeds.PandasData.datafields + [u'symbol']
                  
                  File "c:\Python27_path\Python27 Scritps\Extended pandas import attempt4.py", line 13, in <module>
                     class Pandasclass(btfeeds.PandasData):
                   File "c:\Python36\lib\site-packages\backtrader\lineseries.py", line 364, in __new__
                     linesoverride, lalias=la)
                   File "c:\Python36\lib\site-packages\backtrader\lineseries.py", line 130, in _derive
                     clslines = baselines + lines
                  TypeError: can only concatenate tuple (not "str") to tuple
                  

                  When i put another dummy value next to it, I got this error.

                  class Pandasclass(btfeeds.PandasData):
                     
                         lines = ('symbol','blank')
                         params = (('symbol', -1),('blank', -1),)            
                         btfeeds.PandasData.datafields = btfeeds.PandasData.datafields + [u'symbol', u'blank']
                  
                  File "c:\Python27_path\Python27 Scritps\Extended pandas import attempt4.py", line 49, in <module>
                     cerebro.run()        
                   File "c:\Python36\lib\site-packages\backtrader\cerebro.py", line 1127, in run
                     runstrat = self.runstrategies(iterstrat)
                   File "c:\Python36\lib\site-packages\backtrader\cerebro.py", line 1209, in runstrategies
                     data.preload()
                   File "c:\Python36\lib\site-packages\backtrader\feed.py", line 435, in preload
                     while self.load():
                   File "c:\Python36\lib\site-packages\backtrader\feed.py", line 476, in load
                     _loadret = self._load()
                   File "c:\Python36\lib\site-packages\backtrader\feeds\pandafeed.py", line 255, in _load
                     line[0] = self.p.dataname.iloc[self._idx, colindex]
                   File "c:\Python36\lib\site-packages\backtrader\linebuffer.py", line 222, in __setitem__
                     self.array[self.idx + ago] = value
                  TypeError: must be real number, not str
                  

                  Does datas not allow for non numerical values to be in the columns?

                  B 1 Reply Last reply Reply Quote 0
                  • S
                    Sagittarius19 last edited by

                    Update: I ended up just using the _names function as you suggested earlier. It works just as well. I would like to know thought, if i can call on none numerical values in the columns of the datas.

                    class Universe_filter(bt.Strategy):
                       Universe_date = []
                       Universe_data = []
                       def __init__(self):
                           pass
                       def next(self):    
                           for i, d in enumerate(self.datas): 
                               self.day_open = self.datas[i].open
                               self.day_close = self.datas[i].close
                               self.date = self.datas[i].datetime.date
                               self.symbol = self.datas[i]._name
                               if self.day_open[0]> ((self.day_close[-1])*1.05):
                                   self.Universe_date.append(self.date(0))
                                   self.Universe_data.append(self.symbol)
                       def stop(self):
                           print ("Stock dates: %s" % self.Universe_date)
                           print ("Stock datas: %s" % self.Universe_data)
                            
                    if __name__ == '__main__':  
                       cerebro = bt.Cerebro()
                       cerebro.addstrategy(Universe_filter)
                       start = datetime.date(2017,4,1)
                       end = datetime.date(2018,3,19) 
                       #stocks = ["AAPL","MSFT",'TSLA','FB','AMZN','GOOGL','BABA','PYPL','OSTK']
                       stocks = ['FB','AAPL'] 
                       for i in stocks: 
                           DF_1 = web.DataReader([i], "morningstar", start, end)
                           DF_1= pd.DataFrame(DF_1)
                           DF_1.reset_index(level = 'Symbol', inplace=True,drop=False)
                           data1 = btfeeds.PandasData(dataname = DF_1)
                           cerebro.adddata(data1, name = i)       
                       cerebro.run()        
                    
                    
                    
                    [Running] python "c:\Python27_path\Python27 Scritps\Extended pandas import attempt4.py"
                    Stock dates: [datetime.date(2017, 7, 27), datetime.date(2017, 8, 2)]
                    Stock datas: ['FB', 'AAPL']
                    
                    1 Reply Last reply Reply Quote 0
                    • B
                      backtrader administrators @Sagittarius19 last edited by

                      @sagittarius19 said in How to filter large stock universe then run trading system on small group:

                      @backtrader thanks for the help regarding this. I know its prolly a very simple fix. Here is what I updated to, and im still getting an error.

                      class Pandasclass(btfeeds.PandasData):
                             lines = ('symbol')
                             params = (('symbol', -1),)            
                             btfeeds.PandasData.datafields = btfeeds.PandasData.datafields + [u'symbol']
                      
                      File "c:\Python27_path\Python27 Scritps\Extended pandas import attempt4.py", line 13, in <module>
                         class Pandasclass(btfeeds.PandasData):
                       File "c:\Python36\lib\site-packages\backtrader\lineseries.py", line 364, in __new__
                         linesoverride, lalias=la)
                       File "c:\Python36\lib\site-packages\backtrader\lineseries.py", line 130, in _derive
                         clslines = baselines + lines
                      TypeError: can only concatenate tuple (not "str") to tuple
                      

                      When i put another dummy value next to it, I got this error.

                      class Pandasclass(btfeeds.PandasData):
                         
                             lines = ('symbol','blank')
                             params = (('symbol', -1),('blank', -1),)            
                             btfeeds.PandasData.datafields = btfeeds.PandasData.datafields + [u'symbol', u'blank']
                      
                      File "c:\Python27_path\Python27 Scritps\Extended pandas import attempt4.py", line 49, in <module>
                         cerebro.run()        
                       File "c:\Python36\lib\site-packages\backtrader\cerebro.py", line 1127, in run
                         runstrat = self.runstrategies(iterstrat)
                       File "c:\Python36\lib\site-packages\backtrader\cerebro.py", line 1209, in runstrategies
                         data.preload()
                       File "c:\Python36\lib\site-packages\backtrader\feed.py", line 435, in preload
                         while self.load():
                       File "c:\Python36\lib\site-packages\backtrader\feed.py", line 476, in load
                         _loadret = self._load()
                       File "c:\Python36\lib\site-packages\backtrader\feeds\pandafeed.py", line 255, in _load
                         line[0] = self.p.dataname.iloc[self._idx, colindex]
                       File "c:\Python36\lib\site-packages\backtrader\linebuffer.py", line 222, in __setitem__
                         self.array[self.idx + ago] = value
                      TypeError: must be real number, not str
                      

                      Does datas not allow for non numerical values to be in the columns?

                      All values are floats. This is not a database.

                      1 Reply Last reply Reply Quote 0
                      • 1 / 1
                      • First post
                        Last post
                      Copyright © 2016, 2017, 2018, 2019, 2020, 2021 NodeBB Forums | Contributors