For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

MemoryError



  • I am getting this weird error when running a backtest of minute data. There are 129k rows and it works fine for the most part until i start adding code for trailing stops and take profits and it runs out of memory.

    I have a pretty powerful

    Traceback (most recent call last):
      File "C:/Users/oso/Dropbox/All Devices/Backtest v1.1.py", line 328, in <module>
        cerebro.plot(style='bar')
      File "C:\Users\oso\AppData\Roaming\Python\Python27\site-packages\backtrader\cerebro.py", line 999, in plot
        start=start, end=end, use=use)
      File "C:\Users\oso\AppData\Roaming\Python\Python27\site-packages\backtrader\plot\plot.py", line 222, in plot
        self.plotdata(data, self.dplotsover[data])
      File "C:\Users\oso\AppData\Roaming\Python\Python27\site-packages\backtrader\plot\plot.py", line 725, in plotdata
        self.plotind(data, ind, subinds=self.dplotsover[ind], masterax=ax)
      File "C:\Users\oso\AppData\Roaming\Python\Python27\site-packages\backtrader\plot\plot.py", line 420, in plotind
        lplot = line.plotrange(self.pinf.xstart, self.pinf.xend)
      File "C:\Users\oso\AppData\Roaming\Python\Python27\site-packages\backtrader\linebuffer.py", line 332, in plotrange
        return self.array[start:end]
    MemoryError
    

    Here is the TP/SL code:

    if (str(dt) in lst_ext) and tme == datetime.time(12,00,0,0):
                        self.close()
                        self.broker.cancel(self.sl_order)
                        self.broker.cancel(self.tp_order)
    
    
                else:
    
                    # set stop loss and take profit prices
                    # in case of trailing stops stop loss prices can be assigned based on current indicator value
                    #Price Stop
                    price_tp_long = self.position.price * 1.02
                    price_tp_short = self.position.price * 0.98
                    price_sl_long = .99
                    price_sl_short = 1.01
    
                    # cancel existing stop loss and take profit orders
                    if self.tp_order:
                        self.broker.cancel(self.tp_order)
    
                    #check & update stop loss order
                    sl_price = 0
                    # if price goes up and you are long move stop
                    if self.position.size > 0 and (self.dataclose[0] - self.dataclose[-1]) > 0: sl_price = price_sl_long * self.position.price
                    if self.position.size < 0 and (self.dataclose[0] - self.dataclose[-1]) < 0: sl_price = price_sl_short * self.position.price
    
                    if sl_price != 0:
                        self.broker.cancel(self.sl_order)
                        self.sl_order = self.order_target_value(target=0.0, exectype=bt.Order.Stop, price=sl_price)
    
    
                    # check & update take profit order
                    tp_price = 0.0
                    if self.position.size > 0 and price_tp_long != 0: tp_price = price_tp_long
                    if self.position.size < 0 and price_tp_short != 0: tp_price = price_tp_short
    
                    if tp_price != 0.0:
                        self.tp_order = self.order_target_value(target=0.0, exectype=bt.Order.Limit, price=tp_price)
    
    

    and my main

    if __name__ == '__main__':
        #Start the brain
        cerebro = bt.Cerebro()
        # Load Strat
        cerebro.addstrategy(Strat)
    
        #Start cash level
        cerebro.broker.setcash(10000.0)
    
        #Set multiplier and commish level
        cerebro.broker.setcommission(commission=0.79, margin=3000.0, mult=1000.0)
    
    
    
    
        #Get dataframe
        df = pd.DataFrame()
    
        #df = pd.DataFrame.from_csv('C:\\Users\\oso\\Dropbox\\Strats\\df.csv')
    
        data = bt.feeds.PandasData(dataname=df,timeframe=bt.TimeFrame.Minutes, compression=1)
    
        #Add Analyzers
    
        cerebro.addanalyzer(bt.analyzers.TradeAnalyzer, _name='tradeanalyizer')  # Analyzes individual trades
        cerebro.addanalyzer(bt.analyzers.SharpeRatio_A, _name='mysharpeA')  # Gets the annualized Sharpe ratio
        cerebro.addanalyzer(bt.analyzers.AnnualReturn, _name='myannualreturn')        # Annualized returns (does not work?)
        cerebro.addanalyzer(bt.analyzers.Returns, _name='myreturn')  # Returns
        cerebro.addanalyzer(bt.analyzers.DrawDown, _name='mydrawdown')  # Drawdown statistics
        cerebro.addanalyzer(bt.analyzers.VWR, _name='myvwr')  # Value at risk
        cerebro.addanalyzer(bt.analyzers.SharpeRatio, _name='mysharpe')  # Sharpe
        cerebro.addanalyzer(bt.analyzers.TimeDrawDown, _name='mydrawdowntime')
        cerebro.addanalyzer(bt.analyzers.SQN, _name='mysqn')
        cerebro.addanalyzer(bt.analyzers.Transactions, _name='trans')
        cerebro.addanalyzer(bt.analyzers.TimeReturn, _name='myreturntme',timeframe=bt.TimeFrame.Months)
    
        #Load data to brain
        cerebro.adddata(data)
    
        print('Starting Portfolio Value: %.2f' % cerebro.broker.getvalue())
    
        cerebro.addanalyzer(bt.analyzers.SharpeRatio, _name='mysharpe')
    
        results = cerebro.run()
    
    
    
        print('Final Portfolio Value: %.2f' % cerebro.broker.getvalue())
    
    
    
    
        cerebro.plot(style='bar')
    

    Any ideas how to solve?


  • administrators

    @Osofuego said in MemoryError:

    cerebro.plot(style='bar')

    Plotting runs out of memory. Backtesting has completed by then.

    It seems in any case odd to plot 129k minute bars, taking into account that matplotlib has no built-in scrolling.

    You can try to use numfigs=x to split the plot. This will also request smallest chunks from the underlying arrays (this doesn't guarantee you won't run out of memory)