MemoryError

Osofuego

I am getting this weird error when running a backtest of minute data. There are 129k rows and it works fine for the most part until i start adding code for trailing stops and take profits and it runs out of memory.

I have a pretty powerful

Traceback (most recent call last):
  File "C:/Users/oso/Dropbox/All Devices/Backtest v1.1.py", line 328, in <module>
    cerebro.plot(style='bar')
  File "C:\Users\oso\AppData\Roaming\Python\Python27\site-packages\backtrader\cerebro.py", line 999, in plot
    start=start, end=end, use=use)
  File "C:\Users\oso\AppData\Roaming\Python\Python27\site-packages\backtrader\plot\plot.py", line 222, in plot
    self.plotdata(data, self.dplotsover[data])
  File "C:\Users\oso\AppData\Roaming\Python\Python27\site-packages\backtrader\plot\plot.py", line 725, in plotdata
    self.plotind(data, ind, subinds=self.dplotsover[ind], masterax=ax)
  File "C:\Users\oso\AppData\Roaming\Python\Python27\site-packages\backtrader\plot\plot.py", line 420, in plotind
    lplot = line.plotrange(self.pinf.xstart, self.pinf.xend)
  File "C:\Users\oso\AppData\Roaming\Python\Python27\site-packages\backtrader\linebuffer.py", line 332, in plotrange
    return self.array[start:end]
MemoryError

Here is the TP/SL code:

if (str(dt) in lst_ext) and tme == datetime.time(12,00,0,0):
                    self.close()
                    self.broker.cancel(self.sl_order)
                    self.broker.cancel(self.tp_order)


            else:

                # set stop loss and take profit prices
                # in case of trailing stops stop loss prices can be assigned based on current indicator value
                #Price Stop
                price_tp_long = self.position.price * 1.02
                price_tp_short = self.position.price * 0.98
                price_sl_long = .99
                price_sl_short = 1.01

                # cancel existing stop loss and take profit orders
                if self.tp_order:
                    self.broker.cancel(self.tp_order)

                #check & update stop loss order
                sl_price = 0
                # if price goes up and you are long move stop
                if self.position.size > 0 and (self.dataclose[0] - self.dataclose[-1]) > 0: sl_price = price_sl_long * self.position.price
                if self.position.size < 0 and (self.dataclose[0] - self.dataclose[-1]) < 0: sl_price = price_sl_short * self.position.price

                if sl_price != 0:
                    self.broker.cancel(self.sl_order)
                    self.sl_order = self.order_target_value(target=0.0, exectype=bt.Order.Stop, price=sl_price)


                # check & update take profit order
                tp_price = 0.0
                if self.position.size > 0 and price_tp_long != 0: tp_price = price_tp_long
                if self.position.size < 0 and price_tp_short != 0: tp_price = price_tp_short

                if tp_price != 0.0:
                    self.tp_order = self.order_target_value(target=0.0, exectype=bt.Order.Limit, price=tp_price)

and my main

if __name__ == '__main__':
    #Start the brain
    cerebro = bt.Cerebro()
    # Load Strat
    cerebro.addstrategy(Strat)

    #Start cash level
    cerebro.broker.setcash(10000.0)

    #Set multiplier and commish level
    cerebro.broker.setcommission(commission=0.79, margin=3000.0, mult=1000.0)




    #Get dataframe
    df = pd.DataFrame()

    #df = pd.DataFrame.from_csv('C:\\Users\\oso\\Dropbox\\Strats\\df.csv')

    data = bt.feeds.PandasData(dataname=df,timeframe=bt.TimeFrame.Minutes, compression=1)

    #Add Analyzers

    cerebro.addanalyzer(bt.analyzers.TradeAnalyzer, _name='tradeanalyizer')  # Analyzes individual trades
    cerebro.addanalyzer(bt.analyzers.SharpeRatio_A, _name='mysharpeA')  # Gets the annualized Sharpe ratio
    cerebro.addanalyzer(bt.analyzers.AnnualReturn, _name='myannualreturn')        # Annualized returns (does not work?)
    cerebro.addanalyzer(bt.analyzers.Returns, _name='myreturn')  # Returns
    cerebro.addanalyzer(bt.analyzers.DrawDown, _name='mydrawdown')  # Drawdown statistics
    cerebro.addanalyzer(bt.analyzers.VWR, _name='myvwr')  # Value at risk
    cerebro.addanalyzer(bt.analyzers.SharpeRatio, _name='mysharpe')  # Sharpe
    cerebro.addanalyzer(bt.analyzers.TimeDrawDown, _name='mydrawdowntime')
    cerebro.addanalyzer(bt.analyzers.SQN, _name='mysqn')
    cerebro.addanalyzer(bt.analyzers.Transactions, _name='trans')
    cerebro.addanalyzer(bt.analyzers.TimeReturn, _name='myreturntme',timeframe=bt.TimeFrame.Months)

    #Load data to brain
    cerebro.adddata(data)

    print('Starting Portfolio Value: %.2f' % cerebro.broker.getvalue())

    cerebro.addanalyzer(bt.analyzers.SharpeRatio, _name='mysharpe')

    results = cerebro.run()



    print('Final Portfolio Value: %.2f' % cerebro.broker.getvalue())




    cerebro.plot(style='bar')

Any ideas how to solve?

backtrader

@Osofuego said in MemoryError:

cerebro.plot(style='bar')

Plotting runs out of memory. Backtesting has completed by then.

It seems in any case odd to plot 129k minute bars, taking into account that matplotlib has no built-in scrolling.

You can try to use numfigs=x to split the plot. This will also request smallest chunks from the underlying arrays (this doesn't guarantee you won't run out of memory)