For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

Error with Custom Indicator - Rolling OLS



  • Thanks for the toolset, really quite indispensable.

    I wrote up a Strategy and would like to have a customer Indicator class instead of integrating it directly into the Strategy (so I can use it elsewhere possibly). It seems to me like I made the correct classes and also instantiated them correctly, but there seems to be an error, upon running Cerebro, with the indicator's internals.

    The reason I'm writing about it here and not in pyfinance is because I use the ols.PandasRollingOLS from pyfinance by itself without issue. It's only the way I've written it into my Indicator class that throws an error, and the error ends up in a backtrader module (lineseries.py). Sorry for length, shortened it as much as feasible:

    from __future__ import (absolute_import, division, print_function,
                            unicode_literals)
    
    import backtrader as bt
    import backtrader.indicators as btind
    import backtrader.feeds as btfeeds
    
    import pandas
    
    from pyfinance import ols
    
    # Create an Indicator
    class FloatingWindowResiduals(bt.Indicator):
        
        _mindatas = 2
        lines = ('zscore',)
        params = (
            ('wndw', 65),
        )
        
        def __init__(self):
            self.addminperiod(self.params.wndw)
        
        def next(self):
            
            rolling_beta = ols.PandasRollingOLS(y=self.data0, x=self.data1, 
                                          window=self.params.wndw)
            
            spread = self.data1 - (rolling_beta.beta['feature1'] * 
                                   self.data0 + rolling_beta.alpha)
    
            # Get the 1 day moving average of the price spread
            spread_mavg1 = spread.rolling(window=1, center=False).mean()
    
            # Get the X day moving average
            spread_mavgx = spread.rolling(window=self.params.wndw, center=False).mean()
            
            # Take a rolling X day standard deviation
            std_x = spread.rolling(window=self.params.wndw, center=False).std()
    
            # Compute the z score for each day
            self.lines.zscore[0] = (spread_mavg1 - spread_mavgx)/std_x
    
    # Create a Stratey
    class ResidualModel(bt.Strategy):
        params = (
            ('wndw', 65),
    ...
        )
        
        # Initialise upon class instantiation
        def __init__(self):
            
    ...
            
            # Add an indicator
            self.fwResid = FloatingWindowResiduals(self.data0, self.data1)
    
    ...
    
    if __name__ == '__main__':
        
        # Create a cerebro entity
        cerebro = bt.Cerebro()
        
        # Add a strategy
        cerebro.addstrategy(ResidualModel)
        
        # Get a pandas dataframe
        datapath0 = ('ABC.csv')
    
        dataframe0 = pandas.read_csv(datapath0,
                                     names=['date', 'open', 'high', 'low', 'close'],
                                     skiprows=1,
                                     header=None,
                                     index_col='date',
                                     usecols=['date','close'],
                                     parse_dates=True)
    
        data0 = bt.feeds.PandasData(dataname=dataframe0)
    
        cerebro.adddata(data0)
        
        datapath1 = ('XYZ.csv')
    
        dataframe1 = pandas.read_csv(datapath1,
                                     names=['date', 'open', 'high', 'low', 'close'],
                                     skiprows=1,
                                     header=None,
                                     index_col='date',
                                     usecols=['date','close'],
                                     parse_dates=True)
    
        data1 = bt.feeds.PandasData(dataname=dataframe1)
    
        cerebro.adddata(data1)
    
        cerebro.run()
    
    ---------------------------------------------------------------------------
    AttributeError                            Traceback (most recent call last)
    <ipython-input-3-3938288c90b6> in <module>()
        261     cerebro.adddata(data1)
        262 
    --> 263     cerebro.run()
        264
    
    ~\Anaconda3\lib\site-packages\backtrader\cerebro.py in run(self, **kwargs)
       1125             # let's skip process "spawning"
       1126             for iterstrat in iterstrats:
    -> 1127                 runstrat = self.runstrategies(iterstrat)
       1128                 self.runstrats.append(runstrat)
       1129                 if self._dooptimize:
    
    ~\Anaconda3\lib\site-packages\backtrader\cerebro.py in runstrategies(self, iterstrat, predata)
       1291                     self._runonce_old(runstrats)
       1292                 else:
    -> 1293                     self._runonce(runstrats)
       1294             else:
       1295                 if self.p.oldsync:
    
    ~\Anaconda3\lib\site-packages\backtrader\cerebro.py in _runonce(self, runstrats)
       1650         
       1651         for strat in runstrats:
    -> 1652             strat._once()
       1653             strat.reset()  # strat called next by next - reset lines
       1654 
    
    ~\Anaconda3\lib\site-packages\backtrader\lineiterator.py in _once(self)
        290 
        291         for indicator in self._lineiterators[LineIterator.IndType]:
    --> 292             indicator._once()
        293 
        294         for observer in self._lineiterators[LineIterator.ObsType]:
    
    ~\Anaconda3\lib\site-packages\backtrader\lineiterator.py in _once(self)
        310         # indicators are each called with its min period
        311         self.preonce(0, self._minperiod - 1)
    --> 312         self.oncestart(self._minperiod - 1, self._minperiod)
        313         self.once(self._minperiod, self.buflen())
        314 
    
    ~\Anaconda3\lib\site-packages\backtrader\indicator.py in oncestart_via_nextstart(self, start, end)
        122 
        123             self.advance()
    --> 124             self.nextstart()
        125 
        126     def once_via_next(self, start, end):
    
    ~\Anaconda3\lib\site-packages\backtrader\lineiterator.py in nextstart(self)
        340 
        341         # Called once for 1st full calculation - defaults to regular next
    --> 342         self.next()
        343 
        344     def next(self):
    
    <ipython-input-3-3938288c90b6> in next(self)
         34 
         35         rolling_beta = ols.PandasRollingOLS(y=self.data0, x=self.data1, 
    ---> 36                                       window=self.params.wndw)
         37 
         38         spread = self.data1 - (rolling_beta.beta['feature1'] * 
    
    ~\Anaconda3\lib\site-packages\pyfinance\ols.py in __init__(self, y, x, window, has_const, use_const, names)
        703                         k = x.shape[-1] - 1
        704                     else:
    --> 705                         if x.ndim == 1:
        706                             k = 1
        707                         else:
    
    ~\Anaconda3\lib\site-packages\backtrader\lineseries.py in __getattr__(self, name)
        459         # in this object if we set an attribute in this object it will be
        460         # found before we end up here
    --> 461         return getattr(self.lines, name)
        462 
        463     def __len__(self):
    
    AttributeError: 'Lines_LineSeries_DataSeries_OHLC_OHLCDateTime_Abst' object has no attribute 'ndim'
    

    So what I'm not sure is why is it looking for a numpy ndmin attribute in what looks like the window parameter in my rolling_beta = ols.PandasRollingOLS(y=self.data0, x=self.data1, window=self.params.wndw) calculation? This PandasRollingOLS method takes 3 parameters, and I figure I'm assigning them more or less as intended in the Strategy parameters, right?



  • Sorry, looks like I found a discussion regarding floating window OLS here, but I'm not clear as to the status? As I understand, pandas deprecated it/is deprecating it and it's not clear how it works in statsmodels to me yet.

    Perhaps I should just go with your existing indicator and work on it? At the moment I don't see a rolling window option but rather 'full_sample'. Thanks.

    Edit: seems like OLS_TransformationN is exactly what I need, since this is pretty much the example from Quantopian which I also came across.

    I guess they removed the convenient “floating” window option in the statsmodels version but if you’re clever enough it can be done manually with slope, intercept and MAs.