Navigation

    Backtrader Community

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

    Analyzer increases process as more data loaded, using only stop

    General Discussion
    2
    6
    69
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Xavier Escudero
      Xavier Escudero last edited by

      Hi. I've developed an analyzer that uses a data list with 1700 stocks. Each stock has a pandas dataframe with information of his latest ticks OHLC and some indicators. All this information is created outside of cerebro.

      At stop method I look for different conditions, as for example if today's close is above the close of three days ago.

      As the length of each stock dataframe increases, the process time of the cerebro.run is worse (by seconds). I don't understand why does it happen if at stop I am iterating only the stocks, not each row of their pandas.

      How can improve the process time? (I've tried to define 'next' with pass, ...).

      The process is:

      1. For all symbols I've in a database, I load their ticks and compute his indicators
      ...
      df['rsi'] = btalib.rsi(df, period=14).df
      df['atr'] = btalib.atr(df).df
      
      df = df.tail(5)                
      data_list.append((df, symbol))
      
      1. This datafeed is added at cerebro:
      def execute_analysis(data_list, queries):        
              start_time = timeit.default_timer()
              cerebro = bt.Cerebro()
              
              for i in range(len(self.data_list)):            
                  data = PandasData(
                      dataname=data_list[i][0], # Pandas DataFrame
                      name=data_list[i][1] # The symbol
                      )   
                                   
                  cerebro.adddata(data)     
              cerebro.addanalyzer(ScreenerAnalyzer, _name="screener", queries=queries)
      
      1. The run analyzer has only defined the method 'stop':
      class ScreenerAnalyzer(bt.Analyzer):    
       
          def stop(self):
              print('{}: Results'.format(self.datas[0].datetime.date()))
              print('-'*80)
              self.rets = list()
                  
              for i, d in enumerate(self.datas):
                 ...
      1 Reply Last reply Reply Quote 0
      • Xavier Escudero
        Xavier Escudero last edited by

        Hi, again. Sorry, but I am stuck. Anyone has any suggestion?

        1 Reply Last reply Reply Quote 0
        • vladisld
          vladisld last edited by

          No sure I fully understand your confusion - but it is probably my confusion after all.

          The basic thing is that Cerebro.run will iterate through all of your datas' dataframes anyway - regardless of whether or not you are implementing next or stop in your strategy/analyzer or just leave the default next or stop implementation (which does nothing btw) in the corresponding inherited classes. So if the size of the dataframe increases so will the time it takes to iterate it.

          Is it what you've been missing? or is it me that doesn't understand your question?

          Xavier Escudero 1 Reply Last reply Reply Quote 1
          • Xavier Escudero
            Xavier Escudero @vladisld last edited by

            @vladisld My goal is to find stocks by technical indicators and other conditions (integrated in a React web app). My first implementation was to use a single cached pandas dataframe with one row for each stock. Each row had all the information needed for each stock (ATR, ATR(-1), ...). Each search lasted between 1 and 2 seconds.

            As I discovered Backtrader I liked a lot his architecture so I migrated everything to it. Now the query lasts 7-8 seconds, but if I increase one day the information for each stock dataframe, 1 second is increased (more or less).

            Are you telling then that all dataframes are iterated? I would like to create all information before run, so I can get information directly at stop.

            Thanks a lot for your support.

            1 Reply Last reply Reply Quote 0
            • vladisld
              vladisld last edited by

              Backtrader is working best for, well... back testing. So, neutralizing its main functionality in order to just iterate over all symbols just once seems to be sub optimal. I doubt it was designed for this scenario.

              IMHO it is better to just use native pandas/numpy apsi for whatever algo you are using for symbol selection.

              Xavier Escudero 1 Reply Last reply Reply Quote 1
              • Xavier Escudero
                Xavier Escudero @vladisld last edited by

                @vladisld OK, thanks for the explanation. I'll revert to my first solution with pandas.

                1 Reply Last reply Reply Quote 0
                • 1 / 1
                • First post
                  Last post
                Copyright © 2016, 2017, 2018 NodeBB Forums | Contributors
                $(document).ready(function () { app.coldLoad(); }); }