How to set commission for order_target_value(target = size)

[Sang]

When using order_target_value(target = size)
I can not cerebro.broker.setcommission(commission=0.001), why would that happen?

[backtrader]

Yes you can. First, because setting the commission is always done before issuing orders.

Second, because the values you pass as commission are not type-checked, so you could pass a string and setcomission would still work.

You may want to elaborate on what the actual problem, is:

• Code
• Data
• Logs
• What the expected outcome is
• What the actual outcome is

[Sang]

Here is my code:

import backtrader as bt
import sys
import matplotlib.pyplot as plt
import os
import pandas as pd
from jinja2 import Environment, FileSystemLoader
from weasyprint import HTML

from datetime import datetime


def timestamp2str(ts):
    """ Converts Timestamp object to str containing date and time
    """
    date = ts.date().strftime("%Y-%m-%d")
    time = ts.time().strftime("%H:%M:%S")
    return ' '.join([date, time])


def get_now():
    """ Return current datetime as str
    """
    return timestamp2str(datetime.now())


def dir_exists(foldername):
    """ Return True if folder exists, else False
    """
    return os.path.isdir(foldername)


class PerformanceReport:
    """ Report with performce stats for given backtest run
    """

    def __init__(self, stratbt, infilename,
                 outputdir, user, memo):
        self.stratbt = stratbt  # works for only 1 stategy
        self.infilename = infilename
        self.outputdir = outputdir
        self.user = user
        self.memo = memo
        self.check_and_assign_defaults()

    def check_and_assign_defaults(self):
        """ Check initialization parameters or assign defaults
        """
        if not self.infilename:
            self.infilename = 'Not given'
        if not dir_exists(self.outputdir):
            msg = "*** ERROR: outputdir {} does not exist."
            print(msg.format(self.outputdir))
            sys.exit(0)
        if not self.user:
            self.user = 'Happy Canary'
        if not self.memo:
            self.memo = 'No comments'

    def get_performance_stats(self):
        """ Return dict with performace stats for given strategy withing backtest
        """
        st = self.stratbt
        dt = st.data._dataname['open'].index
        trade_analysis = st.analyzers.myTradeAnalysis.get_analysis()
        rpl = trade_analysis.pnl.net.total
        total_return = rpl / self.get_startcash()
        total_number_trades = trade_analysis.total.total
        trades_closed = trade_analysis.total.closed
        bt_period = dt[-1] - dt[0]
        bt_period_days = bt_period.days
        drawdown = st.analyzers.myDrawDown.get_analysis()
        sharpe_ratio = st.analyzers.mySharpe.get_analysis()['sharperatio']
        sqn_score = st.analyzers.mySqn.get_analysis()['sqn']
        kpi = {# PnL
               'start_cash': self.get_startcash(),
               'rpl': rpl,
               'result_won_trades': trade_analysis.won.pnl.total,
               'result_lost_trades': trade_analysis.lost.pnl.total,
               'profit_factor': (-1 * trade_analysis.won.pnl.total / trade_analysis.lost.pnl.total),
               'rpl_per_trade': rpl / trades_closed,
               'total_return': 100 * total_return,
               'annual_return': (100 * (1 + total_return)**(365.25 / bt_period_days) - 100),
               'max_money_drawdown': drawdown['max']['moneydown'],
               'max_pct_drawdown': drawdown['max']['drawdown'],
               # trades
               'total_number_trades': total_number_trades,
               'trades_closed': trades_closed,
               'pct_winning': 100 * trade_analysis.won.total / trades_closed,
               'pct_losing': 100 * trade_analysis.lost.total / trades_closed,
               'avg_money_winning': trade_analysis.won.pnl.average,
               'avg_money_losing':  trade_analysis.lost.pnl.average,
               'best_winning_trade': trade_analysis.won.pnl.max,
               'worst_losing_trade': trade_analysis.lost.pnl.max,
               #  performance
               'sharpe_ratio': sharpe_ratio,
               'sqn_score': sqn_score,
               'sqn_human': self._sqn2rating(sqn_score)
               }
        return kpi

    def get_equity_curve(self):
        """ Return series containing equity curve
        """
        st = self.stratbt
        dt = st.data._dataname['open'].index
        value = st.observers.broker.lines[1].array[:len(dt)]
        curve = pd.Series(data=value, index=dt)
        return 100 * curve / curve.iloc[0]

    def _sqn2rating(self, sqn_score):
        """ Converts sqn_score score to human readable rating
        See: http://www.vantharp.com/tharp-concepts/sqn.asp
        """
        if sqn_score < 1.6:
            return "Poor"
        elif sqn_score < 1.9:
            return "Below average"
        elif sqn_score < 2.4:
            return "Average"
        elif sqn_score < 2.9:
            return "Good"
        elif sqn_score < 5.0:
            return "Excellent"
        elif sqn_score < 6.9:
            return "Superb"
        else:
            return "Holy Grail"

    def __str__(self):
        msg = ("*** PnL: ***\n"
               "Start capital         : {start_cash:4.2f}\n"
               "Total net profit      : {rpl:4.2f}\n"
               "Result winning trades : {result_won_trades:4.2f}\n"
               "Result lost trades    : {result_lost_trades:4.2f}\n"
               "Profit factor         : {profit_factor:4.2f}\n"
               "Total return          : {total_return:4.2f}%\n"
               "Annual return         : {annual_return:4.2f}%\n"
               "Max. money drawdown   : {max_money_drawdown:4.2f}\n"
               "Max. percent drawdown : {max_pct_drawdown:4.2f}%\n\n"
               "*** Trades ***\n"
               "Number of trades      : {total_number_trades:d}\n"
               "    %winning          : {pct_winning:4.2f}%\n"
               "    %losing           : {pct_losing:4.2f}%\n"
               "    avg money winning : {avg_money_winning:4.2f}\n"
               "    avg money losing  : {avg_money_losing:4.2f}\n"
               "    best winning trade: {best_winning_trade:4.2f}\n"
               "    worst losing trade: {worst_losing_trade:4.2f}\n\n"
               "*** Performance ***\n"
               "Sharpe ratio          : {sharpe_ratio:4.2f}\n"
               "SQN score             : {sqn_score:4.2f}\n"
               "SQN human             : {sqn_human:s}"
               )
        kpis = self.get_performance_stats()
        # see: https://stackoverflow.com/questions/24170519/
        # python-# typeerror-non-empty-format-string-passed-to-object-format
        kpis = {k: -999 if v is None else v for k, v in kpis.items()}
        return msg.format(**kpis)

    def plot_equity_curve(self, fname='equity_curve.png'):
        """ Plots equity curve to png file
        """
        curve = self.get_equity_curve()
        buynhold = self.get_buynhold_curve()
        xrnge = [curve.index[0], curve.index[-1]]
        dotted = pd.Series(data=[100, 100], index=xrnge)
        fig, ax = plt.subplots(1, 1)
        ax.set_ylabel('Net Asset Value (start=100)')
        ax.set_title('Equity curve')
        _ = curve.plot(kind='line', ax=ax)
        _ = buynhold.plot(kind='line', ax=ax, color='grey')
        _ = dotted.plot(kind='line', ax=ax, color='grey', linestyle=':')
        return fig

    def _get_periodicity(self):
        """ Maps length backtesting interval to appropriate periodiciy for return plot
        """
        curve = self.get_equity_curve()
        startdate = curve.index[0]
        enddate = curve.index[-1]
        time_interval = enddate - startdate
        time_interval_days = time_interval.days
        if time_interval_days > 5 * 365.25:
            periodicity = ('Yearly', 'Y')
        elif time_interval_days > 365.25:
            periodicity = ('Monthly', 'M')
        elif time_interval_days > 50:
            periodicity = ('Weekly', '168H')
        elif time_interval_days > 5:
            periodicity = ('Daily', '24H')
        elif time_interval_days > 0.5:
            periodicity = ('Hourly', 'H')
        elif time_interval_days > 0.05:
            periodicity = ('Per 15 Min', '15M')
        else: periodicity = ('Per minute', '1M')
        return periodicity

    def plot_return_curve(self, fname='return_curve.png'):
        """ Plots return curve to png file
        """
        curve = self.get_equity_curve()
        period = self._get_periodicity()
        values = curve.resample(period[1]).ohlc()['close']
        # returns = 100 * values.diff().shift(-1) / values
        returns = 100 * values.diff() / values
        returns.index = returns.index.date
        is_positive = returns > 0
        fig, ax = plt.subplots(1, 1)
        ax.set_title("{} returns".format(period[0]))
        ax.set_xlabel("date")
        ax.set_ylabel("return (%)")
        _ = returns.plot.bar(color=is_positive.map({True: 'green', False: 'red'}), ax=ax)
        return fig

    def generate_html(self):
        """ Returns parsed HTML text string for report
        """
        # basedir = os.path.abspath(os.path.dirname(__file__))
        basedir = os.path.abspath('')
        images = os.path.join(basedir, 'templates')
        eq_curve = os.path.join(images, 'equity_curve.png')
        rt_curve = os.path.join(images, 'return_curve.png')
        fig_equity = self.plot_equity_curve()
        fig_equity.savefig(eq_curve)
        fig_return = self.plot_return_curve()
        fig_return.savefig(rt_curve)
        env = Environment(loader=FileSystemLoader('.'))
        template = env.get_template("templates/template.html")
        header = self.get_header_data()
        kpis = self.get_performance_stats()
        graphics = {'url_equity_curve': 'file://' + eq_curve,
                    'url_return_curve': 'file://' + rt_curve
                    }
        all_numbers = {**header, **kpis, **graphics}
        html_out = template.render(all_numbers)
        return html_out

    def generate_pdf_report(self):
        """ Returns PDF report with backtest results
        """
        html = self.generate_html()
        outfile = os.path.join(self.outputdir, 'report.pdf')
        HTML(string=html).write_pdf(outfile)
        msg = "See {} for report with backtest results."
        print(msg.format(outfile))

    def get_strategy_name(self):
        return self.stratbt.__class__.__name__

    def get_strategy_params(self):
        return self.stratbt.cerebro.strats[0][0][-1]

    def get_start_date(self):
        """ Return first datafeed datetime
        """
        dt = self.stratbt.data._dataname['open'].index
        return timestamp2str(dt[0])

    def get_end_date(self):
        """ Return first datafeed datetime
        """
        dt = self.stratbt.data._dataname['open'].index
        return timestamp2str(dt[-1])

    def get_header_data(self):
        """ Return dict with data for report header
        """
        header = {'strategy_name': self.get_strategy_name(),
                  'params': self.get_strategy_params(),
                  'file_name': self.infilename,
                  'start_date': self.get_start_date(),
                  'end_date': self.get_end_date(),
                  'name_user': self.user,
                  'processing_date': get_now(),
                  'memo_field': self.memo
                  }
        return header

    def get_series(self, column='close'):
        """ Return data series
        """
        return self.stratbt.data._dataname[column]

    def get_buynhold_curve(self):
        """ Returns Buy & Hold equity curve starting at 100
        """
        s = self.get_series(column='open')
        return 100 * s / s[0]

    def get_startcash(self):
        return self.stratbt.broker.startingcash


class Cerebro(bt.Cerebro):
    def __init__(self, **kwds):
        super().__init__(**kwds)
        self.add_report_analyzers()

    def add_report_analyzers(self, riskfree=0.01):
            """ Adds performance stats, required for report
            """
            self.addanalyzer(bt.analyzers.SharpeRatio,
                             _name="mySharpe",
                             riskfreerate=riskfree,
                             timeframe=bt.TimeFrame.Weeks)
            self.addanalyzer(bt.analyzers.DrawDown,
                             _name="myDrawDown")
            self.addanalyzer(bt.analyzers.AnnualReturn,
                             _name="myReturn")
            self.addanalyzer(bt.analyzers.TradeAnalyzer,
                             _name="myTradeAnalysis")
            self.addanalyzer(bt.analyzers.SQN,
                             _name="mySqn")

    def get_strategy_backtest(self):
        return self.runstrats[0][0]

    def report(self, outputdir,
               infilename=None, user=None, memo=None):
        bt = self.get_strategy_backtest()
        rpt =PerformanceReport(bt, infilename=infilename,
                               outputdir=outputdir, user=user,
                               memo=memo)
        rpt.generate_pdf_report()



# This defines not only the commission info, but some other aspects
# of a given data asset like the "getsize" information from below
# params = dict(stocklike=True)  # No margin, no multiplier

class CommInfoFractional(bt.CommissionInfo):
    def getsize(self, price, cash):
        '''Returns fractional size for cash operation @price'''
        return self.p.leverage * (cash / price)

class testStrat(bt.Strategy):
    def __init__(self):
      self.openinterest = self.datas[0].openinterest
      self.set_tradehistory(True)

    def next(self):
      size = 1000
      if not self.position:
          if self.openinterest[0] == 2:
              self.order_target_value(target = -size)
          if self.openinterest[0] == 1:
              self.order_target_value(target = size)
      else:
          if self.position.size > 0:
            if self.openinterest[0] == 2:
              self.close()
              self.order_target_value(target = -size)
            elif self.openinterest[0] == 1:
              self.order_target_value(target = size)
          else:
            if self.openinterest[0] == 1:
              self.close()
              self.order_target_value(target = size)
            elif self.openinterest[0] == 2:
              self.order_target_value(target = -size)
      
      
 
    def notify_trade(self,trade):
      if trade.isclosed:
        print('OPERATION PROFIT, GROSS %.2f, NET %.2f' %(trade.pnl, trade.pnlcomm))

if __name__ == "__main__":   
    cerebro = Cerebro()
    # Add our strategy
    cerebro.addstrategy(testStrat)
    # Add the Data Feed to Cerebro
    data = bt.feeds.PandasData(dataname=newdf)
    cerebro.adddata(data)
    #Variable for our starting cash
    startcash = 10000
    # Set our desired cash start
    cerebro.broker.setcash(startcash)
    # cerebro.broker.set_coc(True)
    # 0.1% ... divide by 100 to remove the %
    cerebro.broker.setcommission(commission=0.001)

    # use the fractional scheme if requested
    cerebro.broker.addcommissioninfo(CommInfoFractional())
    # Run over everything
    cerebro.run()
    #Get final portfolio Value
    portvalue = cerebro.broker.getvalue()
    pnl = portvalue - startcash
    #Print out the final result
    print('Final Portfolio Value: ${}'.format(round(portvalue,2)))
    print('P/L: ${}'.format(round(pnl,2)))

    # Finally plot the end results
    cerebro.report('/content/drive/My Drive/',
                   user='Steve',
                    memo='Test')

The problem is when I not addcommissioninfo(CommInfoFractional()), which I get from Target order example, cerebro excecutes no trade at all

And when I add addcommissioninfo(CommInfoFractional()), the result appears as below, there is no commission added

OPERATION PROFIT, GROSS 0.59, NET 0.59
OPERATION PROFIT, GROSS 8.16, NET 8.16
OPERATION PROFIT, GROSS 13.92, NET 13.92
OPERATION PROFIT, GROSS -3.18, NET -3.18
OPERATION PROFIT, GROSS 0.57, NET 0.57
OPERATION PROFIT, GROSS -5.86, NET -5.86
OPERATION PROFIT, GROSS -32.51, NET -32.51

[backtrader]

@Sang said in How to set commission for order_target_value(target = size):

    cerebro.broker.setcommission(commission=0.001)

    # use the fractional scheme if requested
    cerebro.broker.addcommissioninfo(CommInfoFractional())

And CommInfoFractional

@Sang said in How to set commission for order_target_value(target = size):

class CommInfoFractional(bt.CommissionInfo):
    def getsize(self, price, cash):
        '''Returns fractional size for cash operation @price'''
        return self.p.leverage * (cash / price)

defines no commission at all. So you basically overwrite the commission you have set with setcommissioninfo which is meant for a simple usage without having to create any object with an object which carries no commission. Define the commission value for your commission info scheme.

Nothing to do with order_target_xxx

[Sang]

But when I delete cerebro.broker.addcommissioninfo(CommInfoFractional())
The program got error at rpl = trade_analysis.pnl.net.total in

How to fix this?
Can I make an Object that simulate cerebro.broker.setcommission(commission=0.001)?

[backtrader]

@Sang said in How to set commission for order_target_value(target = size):

The program got error at rpl = trade_analysis.pnl.net.total in

And don't you think that the error is even more relevant than the line at which the error happened?

Don't you think that simply deleting the addition of a commission info scheme is not the source of some other errors? Just like order_target_xxx has nothing to do directly with the commissions.

@Sang said in How to set commission for order_target_value(target = size):

Can I make an Object that simulate cerebro.broker.setcommission(commission=0.001)?

@backtrader said in How to set commission for order_target_value(target = size):

Define the commission value for your commission info scheme.

As stated above. Add the commission to the commission info defining the fractional scheme. Use the same parameter

Please read this: Docs - Commission Schemes - Custom Schemes - https://www.backtrader.com/docu/user-defined-commissions/commission-schemes-subclassing/

[Sang]

I've tried CommInfo_Stocks_PercAbs, CommInfo_Futures_Fixed and CommInfo_Stocks_Perc.
Nothing works and I don't know why. The program excecuted no trade at all
Please help, I just want to add 0.1% commission to all trades, why is it so hard

class CommInfo_Stocks_PercAbs(bt.CommInfoBase):
    params = (
        ('stocklike', True),
        ('commtype', bt.CommInfoBase.COMM_PERC),
        ('percabs', True),
    )
    # def _getcommission(self, size, price, pseudoexec):
    #     return size * price * self.p.commission 

if __name__ == "__main__":   
    cerebro = Cerebro()
    # Add our strategy
    cerebro.addstrategy(testStrat)
    # Add the Data Feed to Cerebro
    data = bt.feeds.PandasData(dataname=newdf)
    cerebro.adddata(data)
    #Variable for our starting cash
    startcash = 10000
    # Set our desired cash start
    cerebro.broker.setcash(startcash)

    comminfo = CommInfo_Stocks_PercAbs(commission=0.001)  
    cerebro.broker.addcommissioninfo(comminfo)
    # Run over everything
    cerebro.run()
    #Get final portfolio Value
    portvalue = cerebro.broker.getvalue()
    pnl = portvalue - startcash
    #Print out the final result
    print('Final Portfolio Value: ${}'.format(round(portvalue,2)))
    print('P/L: ${}'.format(round(pnl,2)))

[backtrader]

@Sang said in How to set commission for order_target_value(target = size):

I've tried CommInfo_Stocks_PercAbs, CommInfo_Futures_Fixed and CommInfo_Stocks_Perc.

Apparently you wanted to use the fractional sizes CommInfo object.

@Sang said in How to set commission for order_target_value(target = size):

Nothing works and I don't know why. The program excecuted no trade at all

You were asked for data and provided none, which obscures your reasons to use the fractional sizes commission info.

Now, you go out to the wild and publish data without being requested here: https://www.reddit.com/r/algotrading/comments/e852wr/is_there_an_easier_way_to_backtest_ml_based_algo/

Which of course clarifies why you need that for a value=1000.

But of course ...

backtrader is very difficult.

If the fractional sizes commission info object is a commission info object (just like CommInfo_Stocks_PercAbs, CommInfo_Futures_Fixed and CommInfo_Stocks_Perc.) why don't you follow the suggestion from above and apply the commission to that object

@Sang said in How to set commission for order_target_value(target = size):

class CommInfoFractional(bt.CommissionInfo):
    def getsize(self, price, cash):
        '''Returns fractional size for cash operation @price'''
        return self.p.leverage * (cash / price)

as in ...

class CommInfoFractional(bt.CommissionInfo):
    params = dict(commission=0.001)
    def getsize(self, price, cash):
        '''Returns fractional size for cash operation @price'''
        return self.p.leverage * (cash / price)

Incredibly difficult.

Things are very difficult when no information is shared at all, suggestions are ignores, and when the documentation is not read at all.

[Sang]

@backtrader hi Sir,
At first I want to sorry because I forgot to provide you the data as I thought it was not important

Which of course clarifies why you need that for a value=1000.

I don't understand this?

Also I've tried to do as your suggestions and read the documentation, but it is really hard for me.
But it is okay now, thank you so much. I will be more careful next time