For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

How to set commission for order_target_value(target = size)



  • When using order_target_value(target = size)
    I can not cerebro.broker.setcommission(commission=0.001), why would that happen?


  • administrators

    Yes you can. First, because setting the commission is always done before issuing orders.

    Second, because the values you pass as commission are not type-checked, so you could pass a string and setcomission would still work.

    You may want to elaborate on what the actual problem, is:

    • Code
    • Data
    • Logs
    • What the expected outcome is
    • What the actual outcome is


  • Here is my code:

    import backtrader as bt
    import sys
    import matplotlib.pyplot as plt
    import os
    import pandas as pd
    from jinja2 import Environment, FileSystemLoader
    from weasyprint import HTML
    
    from datetime import datetime
    
    
    def timestamp2str(ts):
        """ Converts Timestamp object to str containing date and time
        """
        date = ts.date().strftime("%Y-%m-%d")
        time = ts.time().strftime("%H:%M:%S")
        return ' '.join([date, time])
    
    
    def get_now():
        """ Return current datetime as str
        """
        return timestamp2str(datetime.now())
    
    
    def dir_exists(foldername):
        """ Return True if folder exists, else False
        """
        return os.path.isdir(foldername)
    
    
    class PerformanceReport:
        """ Report with performce stats for given backtest run
        """
    
        def __init__(self, stratbt, infilename,
                     outputdir, user, memo):
            self.stratbt = stratbt  # works for only 1 stategy
            self.infilename = infilename
            self.outputdir = outputdir
            self.user = user
            self.memo = memo
            self.check_and_assign_defaults()
    
        def check_and_assign_defaults(self):
            """ Check initialization parameters or assign defaults
            """
            if not self.infilename:
                self.infilename = 'Not given'
            if not dir_exists(self.outputdir):
                msg = "*** ERROR: outputdir {} does not exist."
                print(msg.format(self.outputdir))
                sys.exit(0)
            if not self.user:
                self.user = 'Happy Canary'
            if not self.memo:
                self.memo = 'No comments'
    
        def get_performance_stats(self):
            """ Return dict with performace stats for given strategy withing backtest
            """
            st = self.stratbt
            dt = st.data._dataname['open'].index
            trade_analysis = st.analyzers.myTradeAnalysis.get_analysis()
            rpl = trade_analysis.pnl.net.total
            total_return = rpl / self.get_startcash()
            total_number_trades = trade_analysis.total.total
            trades_closed = trade_analysis.total.closed
            bt_period = dt[-1] - dt[0]
            bt_period_days = bt_period.days
            drawdown = st.analyzers.myDrawDown.get_analysis()
            sharpe_ratio = st.analyzers.mySharpe.get_analysis()['sharperatio']
            sqn_score = st.analyzers.mySqn.get_analysis()['sqn']
            kpi = {# PnL
                   'start_cash': self.get_startcash(),
                   'rpl': rpl,
                   'result_won_trades': trade_analysis.won.pnl.total,
                   'result_lost_trades': trade_analysis.lost.pnl.total,
                   'profit_factor': (-1 * trade_analysis.won.pnl.total / trade_analysis.lost.pnl.total),
                   'rpl_per_trade': rpl / trades_closed,
                   'total_return': 100 * total_return,
                   'annual_return': (100 * (1 + total_return)**(365.25 / bt_period_days) - 100),
                   'max_money_drawdown': drawdown['max']['moneydown'],
                   'max_pct_drawdown': drawdown['max']['drawdown'],
                   # trades
                   'total_number_trades': total_number_trades,
                   'trades_closed': trades_closed,
                   'pct_winning': 100 * trade_analysis.won.total / trades_closed,
                   'pct_losing': 100 * trade_analysis.lost.total / trades_closed,
                   'avg_money_winning': trade_analysis.won.pnl.average,
                   'avg_money_losing':  trade_analysis.lost.pnl.average,
                   'best_winning_trade': trade_analysis.won.pnl.max,
                   'worst_losing_trade': trade_analysis.lost.pnl.max,
                   #  performance
                   'sharpe_ratio': sharpe_ratio,
                   'sqn_score': sqn_score,
                   'sqn_human': self._sqn2rating(sqn_score)
                   }
            return kpi
    
        def get_equity_curve(self):
            """ Return series containing equity curve
            """
            st = self.stratbt
            dt = st.data._dataname['open'].index
            value = st.observers.broker.lines[1].array[:len(dt)]
            curve = pd.Series(data=value, index=dt)
            return 100 * curve / curve.iloc[0]
    
        def _sqn2rating(self, sqn_score):
            """ Converts sqn_score score to human readable rating
            See: http://www.vantharp.com/tharp-concepts/sqn.asp
            """
            if sqn_score < 1.6:
                return "Poor"
            elif sqn_score < 1.9:
                return "Below average"
            elif sqn_score < 2.4:
                return "Average"
            elif sqn_score < 2.9:
                return "Good"
            elif sqn_score < 5.0:
                return "Excellent"
            elif sqn_score < 6.9:
                return "Superb"
            else:
                return "Holy Grail"
    
        def __str__(self):
            msg = ("*** PnL: ***\n"
                   "Start capital         : {start_cash:4.2f}\n"
                   "Total net profit      : {rpl:4.2f}\n"
                   "Result winning trades : {result_won_trades:4.2f}\n"
                   "Result lost trades    : {result_lost_trades:4.2f}\n"
                   "Profit factor         : {profit_factor:4.2f}\n"
                   "Total return          : {total_return:4.2f}%\n"
                   "Annual return         : {annual_return:4.2f}%\n"
                   "Max. money drawdown   : {max_money_drawdown:4.2f}\n"
                   "Max. percent drawdown : {max_pct_drawdown:4.2f}%\n\n"
                   "*** Trades ***\n"
                   "Number of trades      : {total_number_trades:d}\n"
                   "    %winning          : {pct_winning:4.2f}%\n"
                   "    %losing           : {pct_losing:4.2f}%\n"
                   "    avg money winning : {avg_money_winning:4.2f}\n"
                   "    avg money losing  : {avg_money_losing:4.2f}\n"
                   "    best winning trade: {best_winning_trade:4.2f}\n"
                   "    worst losing trade: {worst_losing_trade:4.2f}\n\n"
                   "*** Performance ***\n"
                   "Sharpe ratio          : {sharpe_ratio:4.2f}\n"
                   "SQN score             : {sqn_score:4.2f}\n"
                   "SQN human             : {sqn_human:s}"
                   )
            kpis = self.get_performance_stats()
            # see: https://stackoverflow.com/questions/24170519/
            # python-# typeerror-non-empty-format-string-passed-to-object-format
            kpis = {k: -999 if v is None else v for k, v in kpis.items()}
            return msg.format(**kpis)
    
        def plot_equity_curve(self, fname='equity_curve.png'):
            """ Plots equity curve to png file
            """
            curve = self.get_equity_curve()
            buynhold = self.get_buynhold_curve()
            xrnge = [curve.index[0], curve.index[-1]]
            dotted = pd.Series(data=[100, 100], index=xrnge)
            fig, ax = plt.subplots(1, 1)
            ax.set_ylabel('Net Asset Value (start=100)')
            ax.set_title('Equity curve')
            _ = curve.plot(kind='line', ax=ax)
            _ = buynhold.plot(kind='line', ax=ax, color='grey')
            _ = dotted.plot(kind='line', ax=ax, color='grey', linestyle=':')
            return fig
    
        def _get_periodicity(self):
            """ Maps length backtesting interval to appropriate periodiciy for return plot
            """
            curve = self.get_equity_curve()
            startdate = curve.index[0]
            enddate = curve.index[-1]
            time_interval = enddate - startdate
            time_interval_days = time_interval.days
            if time_interval_days > 5 * 365.25:
                periodicity = ('Yearly', 'Y')
            elif time_interval_days > 365.25:
                periodicity = ('Monthly', 'M')
            elif time_interval_days > 50:
                periodicity = ('Weekly', '168H')
            elif time_interval_days > 5:
                periodicity = ('Daily', '24H')
            elif time_interval_days > 0.5:
                periodicity = ('Hourly', 'H')
            elif time_interval_days > 0.05:
                periodicity = ('Per 15 Min', '15M')
            else: periodicity = ('Per minute', '1M')
            return periodicity
    
        def plot_return_curve(self, fname='return_curve.png'):
            """ Plots return curve to png file
            """
            curve = self.get_equity_curve()
            period = self._get_periodicity()
            values = curve.resample(period[1]).ohlc()['close']
            # returns = 100 * values.diff().shift(-1) / values
            returns = 100 * values.diff() / values
            returns.index = returns.index.date
            is_positive = returns > 0
            fig, ax = plt.subplots(1, 1)
            ax.set_title("{} returns".format(period[0]))
            ax.set_xlabel("date")
            ax.set_ylabel("return (%)")
            _ = returns.plot.bar(color=is_positive.map({True: 'green', False: 'red'}), ax=ax)
            return fig
    
        def generate_html(self):
            """ Returns parsed HTML text string for report
            """
            # basedir = os.path.abspath(os.path.dirname(__file__))
            basedir = os.path.abspath('')
            images = os.path.join(basedir, 'templates')
            eq_curve = os.path.join(images, 'equity_curve.png')
            rt_curve = os.path.join(images, 'return_curve.png')
            fig_equity = self.plot_equity_curve()
            fig_equity.savefig(eq_curve)
            fig_return = self.plot_return_curve()
            fig_return.savefig(rt_curve)
            env = Environment(loader=FileSystemLoader('.'))
            template = env.get_template("templates/template.html")
            header = self.get_header_data()
            kpis = self.get_performance_stats()
            graphics = {'url_equity_curve': 'file://' + eq_curve,
                        'url_return_curve': 'file://' + rt_curve
                        }
            all_numbers = {**header, **kpis, **graphics}
            html_out = template.render(all_numbers)
            return html_out
    
        def generate_pdf_report(self):
            """ Returns PDF report with backtest results
            """
            html = self.generate_html()
            outfile = os.path.join(self.outputdir, 'report.pdf')
            HTML(string=html).write_pdf(outfile)
            msg = "See {} for report with backtest results."
            print(msg.format(outfile))
    
        def get_strategy_name(self):
            return self.stratbt.__class__.__name__
    
        def get_strategy_params(self):
            return self.stratbt.cerebro.strats[0][0][-1]
    
        def get_start_date(self):
            """ Return first datafeed datetime
            """
            dt = self.stratbt.data._dataname['open'].index
            return timestamp2str(dt[0])
    
        def get_end_date(self):
            """ Return first datafeed datetime
            """
            dt = self.stratbt.data._dataname['open'].index
            return timestamp2str(dt[-1])
    
        def get_header_data(self):
            """ Return dict with data for report header
            """
            header = {'strategy_name': self.get_strategy_name(),
                      'params': self.get_strategy_params(),
                      'file_name': self.infilename,
                      'start_date': self.get_start_date(),
                      'end_date': self.get_end_date(),
                      'name_user': self.user,
                      'processing_date': get_now(),
                      'memo_field': self.memo
                      }
            return header
    
        def get_series(self, column='close'):
            """ Return data series
            """
            return self.stratbt.data._dataname[column]
    
        def get_buynhold_curve(self):
            """ Returns Buy & Hold equity curve starting at 100
            """
            s = self.get_series(column='open')
            return 100 * s / s[0]
    
        def get_startcash(self):
            return self.stratbt.broker.startingcash
    
    
    class Cerebro(bt.Cerebro):
        def __init__(self, **kwds):
            super().__init__(**kwds)
            self.add_report_analyzers()
    
        def add_report_analyzers(self, riskfree=0.01):
                """ Adds performance stats, required for report
                """
                self.addanalyzer(bt.analyzers.SharpeRatio,
                                 _name="mySharpe",
                                 riskfreerate=riskfree,
                                 timeframe=bt.TimeFrame.Weeks)
                self.addanalyzer(bt.analyzers.DrawDown,
                                 _name="myDrawDown")
                self.addanalyzer(bt.analyzers.AnnualReturn,
                                 _name="myReturn")
                self.addanalyzer(bt.analyzers.TradeAnalyzer,
                                 _name="myTradeAnalysis")
                self.addanalyzer(bt.analyzers.SQN,
                                 _name="mySqn")
    
        def get_strategy_backtest(self):
            return self.runstrats[0][0]
    
        def report(self, outputdir,
                   infilename=None, user=None, memo=None):
            bt = self.get_strategy_backtest()
            rpt =PerformanceReport(bt, infilename=infilename,
                                   outputdir=outputdir, user=user,
                                   memo=memo)
            rpt.generate_pdf_report()
    
    
    
    # This defines not only the commission info, but some other aspects
    # of a given data asset like the "getsize" information from below
    # params = dict(stocklike=True)  # No margin, no multiplier
    
    class CommInfoFractional(bt.CommissionInfo):
        def getsize(self, price, cash):
            '''Returns fractional size for cash operation @price'''
            return self.p.leverage * (cash / price)
    
    class testStrat(bt.Strategy):
        def __init__(self):
          self.openinterest = self.datas[0].openinterest
          self.set_tradehistory(True)
    
        def next(self):
          size = 1000
          if not self.position:
              if self.openinterest[0] == 2:
                  self.order_target_value(target = -size)
              if self.openinterest[0] == 1:
                  self.order_target_value(target = size)
          else:
              if self.position.size > 0:
                if self.openinterest[0] == 2:
                  self.close()
                  self.order_target_value(target = -size)
                elif self.openinterest[0] == 1:
                  self.order_target_value(target = size)
              else:
                if self.openinterest[0] == 1:
                  self.close()
                  self.order_target_value(target = size)
                elif self.openinterest[0] == 2:
                  self.order_target_value(target = -size)
          
          
     
        def notify_trade(self,trade):
          if trade.isclosed:
            print('OPERATION PROFIT, GROSS %.2f, NET %.2f' %(trade.pnl, trade.pnlcomm))
    
    if __name__ == "__main__":   
        cerebro = Cerebro()
        # Add our strategy
        cerebro.addstrategy(testStrat)
        # Add the Data Feed to Cerebro
        data = bt.feeds.PandasData(dataname=newdf)
        cerebro.adddata(data)
        #Variable for our starting cash
        startcash = 10000
        # Set our desired cash start
        cerebro.broker.setcash(startcash)
        # cerebro.broker.set_coc(True)
        # 0.1% ... divide by 100 to remove the %
        cerebro.broker.setcommission(commission=0.001)
    
        # use the fractional scheme if requested
        cerebro.broker.addcommissioninfo(CommInfoFractional())
        # Run over everything
        cerebro.run()
        #Get final portfolio Value
        portvalue = cerebro.broker.getvalue()
        pnl = portvalue - startcash
        #Print out the final result
        print('Final Portfolio Value: ${}'.format(round(portvalue,2)))
        print('P/L: ${}'.format(round(pnl,2)))
    
        # Finally plot the end results
        cerebro.report('/content/drive/My Drive/',
                       user='Steve',
                        memo='Test')
    

    The problem is when I not addcommissioninfo(CommInfoFractional()), which I get from Target order example, cerebro excecutes no trade at all

    And when I add addcommissioninfo(CommInfoFractional()), the result appears as below, there is no commission added

    OPERATION PROFIT, GROSS 0.59, NET 0.59
    OPERATION PROFIT, GROSS 8.16, NET 8.16
    OPERATION PROFIT, GROSS 13.92, NET 13.92
    OPERATION PROFIT, GROSS -3.18, NET -3.18
    OPERATION PROFIT, GROSS 0.57, NET 0.57
    OPERATION PROFIT, GROSS -5.86, NET -5.86
    OPERATION PROFIT, GROSS -32.51, NET -32.51
    

  • administrators

    @Sang said in How to set commission for order_target_value(target = size):

        cerebro.broker.setcommission(commission=0.001)
    
        # use the fractional scheme if requested
        cerebro.broker.addcommissioninfo(CommInfoFractional())
    

    And CommInfoFractional

    @Sang said in How to set commission for order_target_value(target = size):

    class CommInfoFractional(bt.CommissionInfo):
        def getsize(self, price, cash):
            '''Returns fractional size for cash operation @price'''
            return self.p.leverage * (cash / price)
    

    defines no commission at all. So you basically overwrite the commission you have set with setcommissioninfo which is meant for a simple usage without having to create any object with an object which carries no commission. Define the commission value for your commission info scheme.

    Nothing to do with order_target_xxx



  • But when I delete cerebro.broker.addcommissioninfo(CommInfoFractional())
    The program got error at rpl = trade_analysis.pnl.net.total in
    a3d221b3-7f2b-46a8-be54-5c214dcb99c1-image.png
    How to fix this?
    Can I make an Object that simulate cerebro.broker.setcommission(commission=0.001)?


  • administrators

    @Sang said in How to set commission for order_target_value(target = size):

    The program got error at rpl = trade_analysis.pnl.net.total in

    And don't you think that the error is even more relevant than the line at which the error happened?

    Don't you think that simply deleting the addition of a commission info scheme is not the source of some other errors? Just like order_target_xxx has nothing to do directly with the commissions.

    @Sang said in How to set commission for order_target_value(target = size):

    Can I make an Object that simulate cerebro.broker.setcommission(commission=0.001)?

    @backtrader said in How to set commission for order_target_value(target = size):

    Define the commission value for your commission info scheme.

    As stated above. Add the commission to the commission info defining the fractional scheme. Use the same parameter

    Please read this: Docs - Commission Schemes - Custom Schemes - https://www.backtrader.com/docu/user-defined-commissions/commission-schemes-subclassing/



  • I've tried CommInfo_Stocks_PercAbs, CommInfo_Futures_Fixed and CommInfo_Stocks_Perc.
    Nothing works and I don't know why. The program excecuted no trade at all
    Please help, I just want to add 0.1% commission to all trades, why is it so hard

    class CommInfo_Stocks_PercAbs(bt.CommInfoBase):
        params = (
            ('stocklike', True),
            ('commtype', bt.CommInfoBase.COMM_PERC),
            ('percabs', True),
        )
        # def _getcommission(self, size, price, pseudoexec):
        #     return size * price * self.p.commission 
    
    if __name__ == "__main__":   
        cerebro = Cerebro()
        # Add our strategy
        cerebro.addstrategy(testStrat)
        # Add the Data Feed to Cerebro
        data = bt.feeds.PandasData(dataname=newdf)
        cerebro.adddata(data)
        #Variable for our starting cash
        startcash = 10000
        # Set our desired cash start
        cerebro.broker.setcash(startcash)
    
        comminfo = CommInfo_Stocks_PercAbs(commission=0.001)  
        cerebro.broker.addcommissioninfo(comminfo)
        # Run over everything
        cerebro.run()
        #Get final portfolio Value
        portvalue = cerebro.broker.getvalue()
        pnl = portvalue - startcash
        #Print out the final result
        print('Final Portfolio Value: ${}'.format(round(portvalue,2)))
        print('P/L: ${}'.format(round(pnl,2)))
    

  • administrators

    @Sang said in How to set commission for order_target_value(target = size):

    I've tried CommInfo_Stocks_PercAbs, CommInfo_Futures_Fixed and CommInfo_Stocks_Perc.

    Apparently you wanted to use the fractional sizes CommInfo object.

    @Sang said in How to set commission for order_target_value(target = size):

    Nothing works and I don't know why. The program excecuted no trade at all

    You were asked for data and provided none, which obscures your reasons to use the fractional sizes commission info.

    Now, you go out to the wild and publish data without being requested here: https://www.reddit.com/r/algotrading/comments/e852wr/is_there_an_easier_way_to_backtest_ml_based_algo/

    Which of course clarifies why you need that for a value=1000.

    But of course ...

    backtrader is very difficult.

    If the fractional sizes commission info object is a commission info object (just like CommInfo_Stocks_PercAbs, CommInfo_Futures_Fixed and CommInfo_Stocks_Perc.) why don't you follow the suggestion from above and apply the commission to that object

    @Sang said in How to set commission for order_target_value(target = size):

    class CommInfoFractional(bt.CommissionInfo):
        def getsize(self, price, cash):
            '''Returns fractional size for cash operation @price'''
            return self.p.leverage * (cash / price)
    

    as in ...

    class CommInfoFractional(bt.CommissionInfo):
        params = dict(commission=0.001)
        def getsize(self, price, cash):
            '''Returns fractional size for cash operation @price'''
            return self.p.leverage * (cash / price)
    

    Incredibly difficult.

    Things are very difficult when no information is shared at all, suggestions are ignores, and when the documentation is not read at all.



  • @backtrader hi Sir,
    At first I want to sorry because I forgot to provide you the data as I thought it was not important

    Which of course clarifies why you need that for a value=1000.

    I don't understand this?

    Also I've tried to do as your suggestions and read the documentation, but it is really hard for me.
    But it is okay now, thank you so much. I will be more careful next time


Log in to reply
 

});