For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

Beginner optimizer and analyzer help



  • Hi,

    Just got started with backtrader and I am blown away with the things you can do. I am trying to do a very simple strategy to make sure I can get the optimizer and analyzer working as expected but I cannot seem to have any luck with this.

    The strategy works 100% without the optimizer and analyzer, but I dont seem to understand how these work together.

    here is my code:

    from __future__ import (absolute_import, division, print_function,
                            unicode_literals)
    
    import datetime  # For datetime objects
    import os.path  # To manage paths
    import sys  # To find out the script name (in argv[0])
    from strategy import *
    
    # Import the backtrader platform
    import backtrader as bt
    import backtrader.indicators as btind
    
    def optimizer_callbacks(cb):
        print('optcallback executed')
    
    if __name__ == '__main__':
        cerebro = bt.Cerebro(optreturn=False)
        cerebro.addanalyzer(bt.analyzers.SharpeRatio, _name='sharpe_ratio')    
        cerebro.optcallback(optimizer_callbacks)    
        cerebro.broker.setcommission(commission=0.025)  # 2.5% commission
        cerebro.optstrategy(SMA_CrossOver,
                           fast=range(3,5),
                           slow=range(19,21)
                           )
        cerebro.addsizer(bt.sizers.SizerFix, stake=1)
        start_portfolio_value = 10000.0
        cerebro.broker.setcash(start_portfolio_value)
        #print('Starting Portfolio Value: %.2f' % cerebro.broker.getvalue())
        data = bt.feeds.CCXT(exchange='binance',
                                   symbol='BTC/USDT',
                                   name="btc_usd_tick",
                                   # Do not pass values before this date
                                   fromdate=datetime.datetime(2019, 1, 1),
                                   # Do not pass values before this date
                                   todate=datetime.datetime(2019, 6, 1),
                                   timeframe=bt.TimeFrame.Days,
                                   compression=1,
                                   )
    
        cerebro.adddata(data)
        optimized_runs = cerebro.run(maxcpus=1)
    
        # results
        final_results_list = []
        for run in optimized_runs:
            for strategy in run:
                PnL = round(strategy.broker.get_value() - 10000, 2)
                sharpe = strategy.analyzers.sharpe_ratio.get_analysis()
                final_results_list.append([strategy.params.fast,
                                           strategy.params.slow, PnL, sharpe['sharperatio']])
    
        for item in final_results_list:
            print(item)
        
        sort_by_sharpe = sorted(final_results_list, key=lambda x: x[3],
                                reverse=True)
        
        for line in sort_by_sharpe[:5]:
            print(line)
    
    

    and my very simple strategy:

    class SMA_CrossOver(bt.Strategy):
        params = (('fast', 3), ('slow', 20))
    
        def __init__(self):
            self.buysig = {}
            for d in self.getdatanames():
                sma_fast = btind.SMA(self.getdatabyname(d), period=self.p.fast)
                sma_slow = btind.SMA(self.getdatabyname(d), period=self.p.slow)
                self.buysig[d] = btind.CrossOver(sma_fast, sma_slow)
    
        def next(self):
            for d in self.getdatanames():
                if self.getpositionbyname(d).size:
                    if self.buysig[d] < 0:
                        self.sell(data=self.getdatabyname(d))
                        print('sell!')
    
                elif self.buysig[d] > 0:
                    self.buy(data=self.getdatabyname(d))
                    print('buy!')
    

    my output unfortunately gives me the following:

    buy!
    sell!
    buy!
    optcallback executed
    optcallback executed
    optcallback executed
    [3, 19, 0.0, None]
    [3, 20, 0.0, None]
    [4, 19, 0.0, None]
    [4, 20, 0.0, None]
    Traceback (most recent call last):
      File "...\trading_optimizer.py", line 56, in <module>
        reverse=True)
    TypeError: '<' not supported between instances of 'NoneType' and 'NoneType'
    

    To me it seems it is only executing the strategy once (as there is no buy! or sell! in my output), and in the analyzer results it doesnt even give me the output to that first run?

    Next steps once I have this working :

    1. compare different assets (e.g. BTC/USDT vs BTC/ETH)
    2. compare different strategies

    to see which gives a better results in my analyzers - is it possible to do this in 1 run?

    1. I need to work out how to print my optimizer_callbacks with the strategy name and parameters.

    Any assistance would be most appreciated.

    Thanks again for an amazing library.



  • So I have been working on this and have made some progress, making some changes along the way:

    I added TradeAnalyzer as well to the analyzers to see if I could try something different and I have made some progress - I am now able to successfully get the results of the first strategy run:

        cerebro.optstrategy(SMA_CrossOver,
                           fast=range(17,20)
                           )
    
    optimized_runs = cerebro.run(maxcpus=1)
        print('runs completed: ' + str(len(optimized_runs)))
        
        for optimized_run in optimized_runs:
            for strategy in optimized_run:
                print(strategy.params.fast)
                try:
                    print(strategy.analyzers.trades.get_analysis())
                    print('---------------------------------------')
                except:
                    print('not enough data...')
    

    but I am still unable to access the other analyzers results - it says there is not enough data

    run begin
    optcallback executed
    optcallback executed
    optcallback executed
    runs completed: 3
    17
    AutoOrderedDict([('total', AutoOrderedDict([('total', 10), ('open', 1), ('closed', 9)])), ('streak', AutoOrderedDict([('won', AutoOrderedDict([('current', 0), ('longest', 4)])), ('lost', AutoOrderedDict([('current', 2), ('longest', 2)]))])), ('pnl', AutoOrderedDict([('gross', AutoOrderedDict([('total', 639.347), ('average', 71.03855555555555)])), ('net', AutoOrderedDict([('total', 273.2601249999998), ('average', 30.362236111111088)]))])), ('won', AutoOrderedDict([('total', 5), ('pnl', AutoOrderedDict([('total', 511.07400000000007), ('average', 102.21480000000001), ('max', 228.05882499999996)]))])), ('lost', 
    AutoOrderedDict([('total', 4), ('pnl', AutoOrderedDict([('total', -237.81387500000028), ('average', -59.45346875000007), ('max', -204.41035000000005)]))])), ('long', AutoOrderedDict([('total', 9), ('pnl', AutoOrderedDict([('total', 273.2601249999998), ('average', 30.362236111111088), ('won', AutoOrderedDict([('total', 511.07400000000007), ('average', 102.21480000000001), ('max', 228.05882499999996)])), ('lost', AutoOrderedDict([('total', -237.81387500000028), ('average', -59.45346875000007), ('max', -204.41035000000005)]))])), ('won', 5), ('lost', 4)])), ('short', AutoOrderedDict([('total', 0), ('pnl', AutoOrderedDict([('total', 0.0), ('average', 0.0), ('won', AutoOrderedDict([('total', 0.0), ('average', 0.0), ('max', 0.0)])), ('lost', AutoOrderedDict([('total', 0.0), ('average', 0.0), ('max', 0.0)]))])), ('won', 0), ('lost', 0)])), ('len', AutoOrderedDict([('total', 297), ('average', 33.0), ('max', 67), ('min', 13), ('won', AutoOrderedDict([('total', 142), ('average', 
    28.4), ('max', 48), ('min', 13)])), ('lost', AutoOrderedDict([('total', 155), ('average', 38.75), ('max', 67), ('min', 20)])), ('long', AutoOrderedDict([('total', 297), ('average', 33.0), ('max', 67), ('min', 13), ('won', AutoOrderedDict([('total', 142), ('average', 28.4), ('max', 48), ('min', 13)])), ('lost', AutoOrderedDict([('total', 155), ('average', 38.75), ('max', 67), ('min', 20)]))])), ('short', AutoOrderedDict([('total', 0), ('average', 0.0), ('max', 0), ('min', 2147483647), ('won', AutoOrderedDict([('total', 0), ('average', 0.0), ('max', 0), ('min', 2147483647)])), ('lost', AutoOrderedDict([('total', 0), ('average', 0.0), ('max', 0), ('min', 2147483647)]))]))]))])
    ---------------------------------------
    18
    AutoOrderedDict([('total', AutoOrderedDict([('total', 0)]))])
    ---------------------------------------
    19
    AutoOrderedDict([('total', AutoOrderedDict([('total', 0)]))])
    --------------------------------------
    

    I know the other values do have results and trades if I set those first, I just dont know what I need to do different to access them?



  • It appears that your second and third trade analyzer have no trades. Is there a reason why there would be no trades with those settings?



  • @run-out said in Beginner optimizer and analyzer help:

    It appears that your second and third trade analyzer have no trades. Is there a reason why there would be no trades with those settings?

    No they definately should have trades - I have adjusted the parameters (the run above was 17,20) to 18,20 and now the results are:

    18
    AutoOrderedDict([('total', AutoOrderedDict([('total', 10), ('open', 1), ('closed', 9)])), ('streak', AutoOrderedDict([('won', AutoOrderedDict([('current', 0), ('longest', 4)])), ('lost', AutoOrderedDict([('current', 2), ('longest', 2)]))])), ('pnl', AutoOrderedDict([('gross', AutoOrderedDict([('total', 684.163), ('average', 76.01811111111111)])), ('net', AutoOrderedDict([('total', 323.06872499999986), ('average', 35.89652499999998)]))])), ('won', AutoOrderedDict([('total', 5), ('pnl', AutoOrderedDict([('total', 458.521875), ('average', 91.704375), ('max', 199.78830000000002)]))])), ('lost', AutoOrderedDict([('total', 4), ('pnl', AutoOrderedDict([('total', -135.45315000000014), ('average', -33.863287500000034), ('max', -52.87622500000009)]))])), ('long', AutoOrderedDict([('total', 9), ('pnl', AutoOrderedDict([('total', 323.06872499999986), ('average', 35.89652499999998), ('won', AutoOrderedDict([('total', 458.521875), ('average', 91.704375), ('max', 199.78830000000002)])), ('lost', AutoOrderedDict([('total', -135.45315000000014), ('average', -33.863287500000034), ('max', -52.87622500000009)]))])), ('won', 5), ('lost', 4)])), ('short', AutoOrderedDict([('total', 0), ('pnl', AutoOrderedDict([('total', 0.0), ('average', 0.0), ('won', AutoOrderedDict([('total', 0.0), ('average', 0.0), ('max', 0.0)])), ('lost', AutoOrderedDict([('total', 0.0), ('average', 0.0), ('max', 0.0)]))])), ('won', 0), ('lost', 0)])), ('len', AutoOrderedDict([('total', 259), ('average', 28.77777777777778), ('max', 48), ('min', 14), ('won', AutoOrderedDict([('total', 173), ('average', 34.6), ('max', 48), ('min', 18)])), ('lost', AutoOrderedDict([('total', 86), ('average', 21.5), ('max', 30), ('min', 14)])), ('long', AutoOrderedDict([('total', 259), ('average', 28.77777777777778), ('max', 48), ('min', 14), ('won', AutoOrderedDict([('total', 173), ('average', 34.6), ('max', 48), ('min', 18)])), ('lost', AutoOrderedDict([('total', 86), ('average', 21.5), ('max', 30), ('min', 14)]))])), ('short', AutoOrderedDict([('total', 0), ('average', 0.0), ('max', 0), ('min', 2147483647), ('won', AutoOrderedDict([('total', 0), ('average', 0.0), ('max', 0), ('min', 2147483647)])), ('lost', AutoOrderedDict([('total', 0), ('average', 0.0), ('max', 0), ('min', 2147483647)]))]))]))])
    ---------------------------------------
    19
    AutoOrderedDict([('total', AutoOrderedDict([('total', 0)]))])
    

    So it is only getting the results for the first run - eventhough 18 was second in the first range and gave no values but when it was first in the range it did give values.



  • Could you post your full code please?



  • Here is the full code with everything

    from __future__ import (absolute_import, division, print_function,
                            unicode_literals)
    
    import datetime  # For datetime objects
    import os.path  # To manage paths
    import backtrader as bt
    import backtrader.indicators as btind
    
    class BBands(bt.Strategy):
        params = (('BBandsperiod', 20),('logging',True))
    
        def log(self, txt, dt=None):
            ''' Logging function fot this strategy'''
            if self.params.logging:
                dt = dt or self.datas[0].datetime.date(0)
                print('%s, %s' % (dt.isoformat(), txt))
    
        def __init__(self):
            # Keep a reference to the "close" line in the data[0] dataseries
            self.dataclose = self.datas[0].close
    
            # To keep track of pending orders and buy price/commission
            self.order = None
            self.buyprice = None
            self.buycomm = None
            self.redline = None
            self.blueline = None
    
            # Add a BBand indicator
            self.bband = bt.indicators.BBands(self.datas[0], period=self.params.BBandsperiod)
    
        def notify_order(self, order):
            if order.status in [order.Submitted, order.Accepted]:
                # Buy/Sell order submitted/accepted to/by broker - Nothing to do
                return
    
            # Check if an order has been completed
            # Attention: broker could reject order if not enougth cash
            if order.status in [order.Canceled, order.Margin]:
                if order.isbuy():
                    self.log(
                        'BUY FAILED, Cancelled or Margin'
                        )
                self.log
            if order.status in [order.Completed, order.Canceled, order.Margin]:
                if order.isbuy():
                    self.log(
                        'BUY EXECUTED, Price: %.2f, Cost: %.2f, Comm %.2f' %
                        (order.executed.price,
                         order.executed.value,
                         order.executed.comm))
    
                    self.buyprice = order.executed.price
                    self.buycomm = order.executed.comm
                else:  # Sell
                    self.log('SELL EXECUTED, Price: %.2f, Cost: %.2f, Comm %.2f' %
                             (order.executed.price,
                              order.executed.value,
                              order.executed.comm))
    
                self.bar_executed = len(self)
    
            # Write down: no pending order
            self.order = None
    
        def notify_trade(self, trade):
            if not trade.isclosed:
                return
    
            self.log('OPERATION PROFIT, GROSS %.2f, NET %.2f' %
                     (trade.pnl, trade.pnlcomm))
    
        
    
        def next(self):
            # Simply log the closing price of the series from the reference
            #self.log('Close, %.2f' % self.dataclose[0])
    
            # Check if an order is pending ... if yes, we cannot send a 2nd one
            if self.order:
                return
    
            if self.dataclose < self.bband.lines.bot and not self.position:
            	self.redline = True
    
            if self.dataclose > self.bband.lines.top and self.position:
            	self.blueline = True
    
            if self.dataclose > self.bband.lines.mid and not self.position and self.redline:        	
            	# BUY, BUY, BUY!!! (with all possible default parameters)
                self.log('BUY CREATE, %.2f' % self.dataclose[0])
                # Keep track of the created order to avoid a 2nd order
                self.order = self.buy()
    
            if self.dataclose > self.bband.lines.top and not self.position:
                # BUY, BUY, BUY!!! (with all possible default parameters)
                self.log('BUY CREATE, %.2f' % self.dataclose[0])
                # Keep track of the created order to avoid a 2nd order
                self.order = self.buy()
    
            if self.dataclose < self.bband.lines.mid and self.position and self.blueline:
                # SELL, SELL, SELL!!! (with all possible default parameters)
                self.log('SELL CREATE, %.2f' % self.dataclose[0])
                self.blueline = False
                self.redline = False
                # Keep track of the created order to avoid a 2nd order
                self.order = self.sell()                
    
    def optimizer_callbacks(cb):
        print('optcallback executed')
    
    if __name__ == '__main__':
        print('Starting up Cerebro...')
        cerebro = bt.Cerebro(optreturn=False)
        cerebro.addanalyzer(bt.analyzers.SharpeRatio, _name='sharpe_ratio') 
        cerebro.addanalyzer(bt.analyzers.TradeAnalyzer, _name="trades")   
        cerebro.optcallback(optimizer_callbacks)    
        cerebro.broker.setcommission(commission=0.025)  # 2.5% commission
        
        cerebro.optstrategy(BBands,
                            BBandsperiod=range(13,15),
                            logging=False
                            )
    
        cerebro.addsizer(bt.sizers.SizerFix, stake=0.1)
        start_portfolio_value = 10000.0
        cerebro.broker.setcash(start_portfolio_value)
        #print('Starting Portfolio Value: %.2f' % cerebro.broker.getvalue())
        data = bt.feeds.CCXT(exchange='binance',
                                   symbol='BTC/USDT',
                                   name="btc_usd_tick",
                                   # Do not pass values before this date
                                   fromdate=datetime.datetime(2019, 1, 1),
                                   # Do not pass values before this date
                                   todate=datetime.datetime(2020, 5, 1),
                                   timeframe=bt.TimeFrame.Days,
                                   compression=1,
                                   )
    
        cerebro.adddata(data)
        print('run begin')
        optimized_runs = cerebro.run(maxcpus=1)
        print('runs completed: ' + str(len(optimized_runs)))
    
        
        for optimized_run in optimized_runs:
            for strategy in optimized_run:
                print(strategy.params.BBandsperiod)
                try:
                    print(strategy.analyzers.trades.get_analysis())
                    print(strategy.analyzers.sharpe_ratio.get_analysis())
                    print('---------------------------------------')
                except:
                    print('not enough data...')
    

    with my output:

    Starting up Cerebro...
    run begin
    optcallback executed
    optcallback executed
    runs completed: 2
    13
    AutoOrderedDict([('total', AutoOrderedDict([('total', 11), ('open', 1), ('closed', 10)])), ('streak', AutoOrderedDict([('won', AutoOrderedDict([('current', 0), ('longest', 4)])), ('lost', AutoOrderedDict([('current', 1), ('longest', 1)]))])), ('pnl', AutoOrderedDict([('gross', AutoOrderedDict([('total', 412.5020000000002), ('average', 41.25020000000002)])), ('net', AutoOrderedDict([('total', -2.847749999999877), ('average', -0.2847749999999877)]))])), ('won', AutoOrderedDict([('total', 7), ('pnl', AutoOrderedDict([('total', 524.1731250000003), ('average', 74.88187500000004), ('max', 228.45857500000002)]))])), ('lost', AutoOrderedDict([('total', 3), ('pnl', AutoOrderedDict([('total', -527.0208750000002), ('average', -175.67362500000004), ('max', -267.76225000000005)]))])), ('long', AutoOrderedDict([('total', 10), ('pnl', AutoOrderedDict([('total', -2.847749999999877), ('average', -0.2847749999999877), ('won', AutoOrderedDict([('total', 524.1731250000003), ('average', 74.88187500000004), ('max', 228.45857500000002)])), ('lost', AutoOrderedDict([('total', -527.0208750000002), ('average', -175.67362500000004), ('max', -267.76225000000005)]))])), ('won', 7), ('lost', 3)])), ('short', AutoOrderedDict([('total', 0), ('pnl', AutoOrderedDict([('total', 0.0), ('average', 0.0), ('won', AutoOrderedDict([('total', 0.0), ('average', 0.0), ('max', 0.0)])), ('lost', AutoOrderedDict([('total', 0.0), ('average', 0.0), ('max', 0.0)]))])), ('won', 0), ('lost', 0)])), ('len', AutoOrderedDict([('total', 305), ('average', 30.5), ('max', 55), ('min', 13), ('won', AutoOrderedDict([('total', 173), 
    ('average', 24.714285714285715), ('max', 55), ('min', 13)])), ('lost', AutoOrderedDict([('total', 132), ('average', 44.0), 
    ('max', 50), ('min', 37)])), ('long', AutoOrderedDict([('total', 305), ('average', 30.5), ('max', 55), ('min', 13), ('won', AutoOrderedDict([('total', 173), ('average', 24.714285714285715), ('max', 55), ('min', 13)])), ('lost', AutoOrderedDict([('total', 132), ('average', 44.0), ('max', 50), ('min', 37)]))])), ('short', AutoOrderedDict([('total', 0), ('average', 0.0), ('max', 0), ('min', 2147483647), ('won', AutoOrderedDict([('total', 0), ('average', 0.0), ('max', 0), ('min', 2147483647)])), ('lost', AutoOrderedDict([('total', 0), ('average', 0.0), ('max', 0), ('min', 2147483647)]))]))]))])
    OrderedDict([('sharperatio', -0.46681509014083633)])
    ---------------------------------------
    14
    AutoOrderedDict([('total', AutoOrderedDict([('total', 0)]))])
    OrderedDict([('sharperatio', None)])
    ---------------------------------------
    


  • I ran your code although using local ES mini data since I don't have the CCXT version installed. I did not have a problem and I received two results. This suggest then:
    a) CCXT doesn't like you pulling data so fast... try installing the data locally or...
    b) your installed version of Backtrader is doing something funny.

    Perhaps try running your code with some local data and see what happens.



  • @dracount Also... Not sure if this will help. but consider adding in optdatas=True when instatiating cerebro...

        cerebro = bt.Cerebro(optreturn=False, optdatas=True)
    

    Read here in the docs. This might help if you are having a data loading problem.



  • @run-out

    Thank you - I have managed to get some historical data and test it out and it works as expected.

    Is there a way to save the data that is used in bt.feeds.CCXT ?



  • Yes there is a way. You can create an analyzer that will track the ohlcv of the backtest, and then save the output after the test.

    Here is the one I sometimes use.

    class OHLCV(bt.analyzers.Analyzer):
        """This analyzer reports the OHLCV of each of datas.
        Params:
          - timeframe (default: ``None``)
            If ``None`` then the timeframe of the 1st data of the system will be
            used
          - compression (default: ``None``)
            Only used for sub-day timeframes to for example work on an hourly
            timeframe by specifying "TimeFrame.Minutes" and 60 as compression
            If ``None`` then the compression of the 1st data of the system will be
            used
        Methods:
          - get_analysis
            Returns a dictionary with returns as values and the datetime points for
            each return as keys
        """
    
        def start(self):
            tf = min(d._timeframe for d in self.datas)
            self._usedate = tf >= bt.TimeFrame.Days
            self.rets = {}
    
        def next(self):
    
            self.rets[self.data.datetime.datetime()] = [
                self.datas[0].open[0],
                self.datas[0].high[0],
                self.datas[0].low[0],
                self.datas[0].close[0],
                self.datas[0].volume[0],
            ]
    
        def get_analysis(self):
            return self.rets
    

    Add it to cerebro when you are starting cerebro up.

    cerebro.addanalyzer(OHLCV, _name="ohlcv")
    

    Then when you run cerebro, if:

    results = cerebro.run()
    

    Then you can get your data from results.

    results[0].analyzers.getbyname("ohlcv").get_analysis()
    

    This will yield a dictionary with the data. The datetime will be the key of the dictionary, and the ohlcv the values in a list.


Log in to reply
 

});