MemoryError



  • I am getting this weird error when running a backtest of minute data. There are 129k rows and it works fine for the most part until i start adding code for trailing stops and take profits and it runs out of memory.

    I have a pretty powerful

    Traceback (most recent call last):
      File "C:/Users/oso/Dropbox/All Devices/Backtest v1.1.py", line 328, in <module>
        cerebro.plot(style='bar')
      File "C:\Users\oso\AppData\Roaming\Python\Python27\site-packages\backtrader\cerebro.py", line 999, in plot
        start=start, end=end, use=use)
      File "C:\Users\oso\AppData\Roaming\Python\Python27\site-packages\backtrader\plot\plot.py", line 222, in plot
        self.plotdata(data, self.dplotsover[data])
      File "C:\Users\oso\AppData\Roaming\Python\Python27\site-packages\backtrader\plot\plot.py", line 725, in plotdata
        self.plotind(data, ind, subinds=self.dplotsover[ind], masterax=ax)
      File "C:\Users\oso\AppData\Roaming\Python\Python27\site-packages\backtrader\plot\plot.py", line 420, in plotind
        lplot = line.plotrange(self.pinf.xstart, self.pinf.xend)
      File "C:\Users\oso\AppData\Roaming\Python\Python27\site-packages\backtrader\linebuffer.py", line 332, in plotrange
        return self.array[start:end]
    MemoryError
    

    Here is the TP/SL code:

    if (str(dt) in lst_ext) and tme == datetime.time(12,00,0,0):
                        self.close()
                        self.broker.cancel(self.sl_order)
                        self.broker.cancel(self.tp_order)
    
    
                else:
    
                    # set stop loss and take profit prices
                    # in case of trailing stops stop loss prices can be assigned based on current indicator value
                    #Price Stop
                    price_tp_long = self.position.price * 1.02
                    price_tp_short = self.position.price * 0.98
                    price_sl_long = .99
                    price_sl_short = 1.01
    
                    # cancel existing stop loss and take profit orders
                    if self.tp_order:
                        self.broker.cancel(self.tp_order)
    
                    #check & update stop loss order
                    sl_price = 0
                    # if price goes up and you are long move stop
                    if self.position.size > 0 and (self.dataclose[0] - self.dataclose[-1]) > 0: sl_price = price_sl_long * self.position.price
                    if self.position.size < 0 and (self.dataclose[0] - self.dataclose[-1]) < 0: sl_price = price_sl_short * self.position.price
    
                    if sl_price != 0:
                        self.broker.cancel(self.sl_order)
                        self.sl_order = self.order_target_value(target=0.0, exectype=bt.Order.Stop, price=sl_price)
    
    
                    # check & update take profit order
                    tp_price = 0.0
                    if self.position.size > 0 and price_tp_long != 0: tp_price = price_tp_long
                    if self.position.size < 0 and price_tp_short != 0: tp_price = price_tp_short
    
                    if tp_price != 0.0:
                        self.tp_order = self.order_target_value(target=0.0, exectype=bt.Order.Limit, price=tp_price)
    
    

    and my main

    if __name__ == '__main__':
        #Start the brain
        cerebro = bt.Cerebro()
        # Load Strat
        cerebro.addstrategy(Strat)
    
        #Start cash level
        cerebro.broker.setcash(10000.0)
    
        #Set multiplier and commish level
        cerebro.broker.setcommission(commission=0.79, margin=3000.0, mult=1000.0)
    
    
    
    
        #Get dataframe
        df = pd.DataFrame()
    
        #df = pd.DataFrame.from_csv('C:\\Users\\oso\\Dropbox\\Strats\\df.csv')
    
        data = bt.feeds.PandasData(dataname=df,timeframe=bt.TimeFrame.Minutes, compression=1)
    
        #Add Analyzers
    
        cerebro.addanalyzer(bt.analyzers.TradeAnalyzer, _name='tradeanalyizer')  # Analyzes individual trades
        cerebro.addanalyzer(bt.analyzers.SharpeRatio_A, _name='mysharpeA')  # Gets the annualized Sharpe ratio
        cerebro.addanalyzer(bt.analyzers.AnnualReturn, _name='myannualreturn')        # Annualized returns (does not work?)
        cerebro.addanalyzer(bt.analyzers.Returns, _name='myreturn')  # Returns
        cerebro.addanalyzer(bt.analyzers.DrawDown, _name='mydrawdown')  # Drawdown statistics
        cerebro.addanalyzer(bt.analyzers.VWR, _name='myvwr')  # Value at risk
        cerebro.addanalyzer(bt.analyzers.SharpeRatio, _name='mysharpe')  # Sharpe
        cerebro.addanalyzer(bt.analyzers.TimeDrawDown, _name='mydrawdowntime')
        cerebro.addanalyzer(bt.analyzers.SQN, _name='mysqn')
        cerebro.addanalyzer(bt.analyzers.Transactions, _name='trans')
        cerebro.addanalyzer(bt.analyzers.TimeReturn, _name='myreturntme',timeframe=bt.TimeFrame.Months)
    
        #Load data to brain
        cerebro.adddata(data)
    
        print('Starting Portfolio Value: %.2f' % cerebro.broker.getvalue())
    
        cerebro.addanalyzer(bt.analyzers.SharpeRatio, _name='mysharpe')
    
        results = cerebro.run()
    
    
    
        print('Final Portfolio Value: %.2f' % cerebro.broker.getvalue())
    
    
    
    
        cerebro.plot(style='bar')
    

    Any ideas how to solve?


  • administrators

    @Osofuego said in MemoryError:

    cerebro.plot(style='bar')

    Plotting runs out of memory. Backtesting has completed by then.

    It seems in any case odd to plot 129k minute bars, taking into account that matplotlib has no built-in scrolling.

    You can try to use numfigs=x to split the plot. This will also request smallest chunks from the underlying arrays (this doesn't guarantee you won't run out of memory)


Log in to reply
 

Looks like your connection to Backtrader Community was lost, please wait while we try to reconnect.