Hi all

I've used Amibroker for some time and liked their trade list output. So written the analyzer for closed trades which provides similar information:

  chng%    pnl%    mae%    pricein  ticker      pnl/bar    size     value    mfe%    priceout       pnl    cumpnl  dateout       ref    nbars  dir    datein
-------  ------  ------  ---------  --------  ---------  ------  --------  ------  ----------  --------  --------  ----------  -----  -------  -----  ----------
  -6.07   -2.82  -10.12    281.397  SPY         -271.68     159   44742.1    0.68      264.31  -2716.78  -2716.78  2018-02-14      1       10  long   2018-01-31
  -1.59   -0.71   -2.2     119.128  TLT          -68.55     363   43243.6    1.01      117.24   -685.47  -3402.25  2018-02-22      2       10  long   2018-02-07
   1.21   -0.54   -4.01    268.163  SPY          -51.3     -158  -42369.7    1.44      271.41   -513.04  -3915.29  2018-03-01      3       10  short  2018-02-14
   1.89   -0.84   -2.16    117.946  TLT          -53.82    -363  -42814.4    0.8       120.17   -807.27  -4722.56  2018-03-15      4       15  short  2018-02-22
  -1.98   -1.26   -6       270.915  SPY          -49.35     158   42804.6    3.5       265.55  -1184.35  -5906.91  2018-04-05      5       24  long   2018-03-01
  -1.57    0.71   -0.41    265.55   SPY          336.49    -161  -42753.6    2.84      261.37    672.98  -5233.93  2018-04-09      7        2  short  2018-04-05
   0.93    0.42   -1.09    119.413  TLT           19.87     359   42869.2    2.47      120.52    397.45  -4836.48  2018-04-13      6       20  long   2018-03-15
  -0.89    0.4    -1.35    119.962  TLT           77.43    -361  -43306.4    1.28      118.89    387.13  -4449.35  2018-04-20      9        5  short  2018-04-13
   2.9     1.76   -2.06    264.486  SPY           71.27     165   43126.1    3.28      272.16   1710.47  -2738.88  2018-05-11      8       24  long   2018-04-09
  -0.16    0.07   -0.61    272.585  SPY            6.96    -160  -43613.6    0.94      272.15     69.6   -2669.28  2018-05-25     11       10  short  2018-05-11
   0.95    0.32   -2.43    118.985  TLT           10.37     365   43429.6    2.97      120.11    310.98  -2358.3   2018-06-04     10       30  long   2018-04-20
  -1.18    0.26   -0.32    120.11   TLT           86.15    -182  -21860      1.41      118.69    258.44  -2099.86  2018-06-07     13        3  short  2018-06-04
   0.2     0.09   -0.01    118.69   TLT           29.76     372   44152.7    1.52      118.93     89.28  -2010.58  2018-06-12     14        3  long   2018-06-07
   2.36   -1.08   -3       118.93   TLT          -74.67    -372  -44242      0.25      121.74  -1045.32  -3055.9   2018-07-02     15       14  short  2018-06-12
  -0.78   -0.33   -2.73    275.275  SPY          -11.28     159   43768.8    1.53      273.14   -315.8   -3371.7   2018-07-06     12       28  long   2018-05-25
   1.88   -0.85   -2.3     273.14   SPY         -204.31    -159  -43429.3    0.16      278.28   -817.26  -4188.96  2018-07-12     17        4  short  2018-07-06
  -2.24   -1.01   -2.35    122.013  TLT          -64.31     353   43070.5    0.74      119.28   -964.63  -5153.59  2018-07-24     16       15  long   2018-07-02
   2.55    0.86   -0.24    278.28   SPY           45.82     156   43411.7    2.78      285.39    824.76  -4328.83  2018-08-07     18       18  long   2018-07-12
  -0.02    0.01   -1.01    119.053  TLT            0.81    -362  -43097      0.83      119.03      8.15  -4320.68  2018-08-07     19       10  short  2018-07-24

Feel free to use/test it, let me know about any bugs found. The script:

# Trade list similar to Amibroker output
class trade_list(bt.Analyzer):

    def get_analysis(self):

        return self.trades


    def __init__(self):

        self.trades = []
        self.cumprofit = 0.0


    def notify_trade(self, trade):

        if trade.isclosed:

            brokervalue = self.strategy.broker.getvalue()

            dir = 'short'
            if trade.history[0].event.size > 0: dir = 'long'

            pricein = trade.history[len(trade.history)-1].status.price
            priceout = trade.history[len(trade.history)-1].event.price
            datein = bt.num2date(trade.history[0].status.dt)
            dateout = bt.num2date(trade.history[len(trade.history)-1].status.dt)
            if trade.data._timeframe >= bt.TimeFrame.Days:
                datein = datein.date()
                dateout = dateout.date()

            pcntchange = 100 * priceout / pricein - 100
            pnl = trade.history[len(trade.history)-1].status.pnlcomm
            pnlpcnt = 100 * pnl / brokervalue
            barlen = trade.history[len(trade.history)-1].status.barlen
            pbar = pnl / barlen
            self.cumprofit += pnl

            size = value = 0.0
            for record in trade.history:
                if abs(size) < abs(record.status.size):
                    size = record.status.size
                    value = record.status.value

            highest_in_trade = max(trade.data.high.get(ago=0, size=barlen+1))
            lowest_in_trade = min(trade.data.low.get(ago=0, size=barlen+1))
            hp = 100 * (highest_in_trade - pricein) / pricein
            lp = 100 * (lowest_in_trade - pricein) / pricein
            if dir == 'long':
                mfe = hp
                mae = lp
            if dir == 'short':
                mfe = -lp
                mae = -hp

            self.trades.append({'ref': trade.ref, 'ticker': trade.data._name, 'dir': dir,
                 'datein': datein, 'pricein': pricein, 'dateout': dateout, 'priceout': priceout,
                 'chng%': round(pcntchange, 2), 'pnl': pnl, 'pnl%': round(pnlpcnt, 2),
                 'size': size, 'value': value, 'cumpnl': self.cumprofit,
                 'nbars': barlen, 'pnl/bar': round(pbar, 2),
                 'mfe%': round(mfe, 2), 'mae%': round(mae, 2)})

It returns the list of dictionaries. In order to print it as a table above I've used tabulate module:

from tabulate import tabulate
...
    # add analyzers
    cerebro.addanalyzer(trade_list, _name='trade_list')
...
    # run backtest
    strats = cerebro.run(tradehistory=True)
...
    # get analyzers data
    trade_list = strats[0].analyzers.trade_list.get_analysis()
    print (tabulate(trade_list, headers="keys"))

Updated on 09/17/2018

I've tried pyevolve library for evolutionary optimization. This is first shot for the system shown above (2 params only, but lets be consistent), takes longer time than particle swarm optimization.

# example of optimizing SMA crossover strategy parameters using 
# evolutionary Optimization in the pyevolve python library
# http://pyevolve.sourceforge.net/

from datetime import datetime
import backtrader as bt

from pyevolve import G1DList
from pyevolve import GSimpleGA

class SmaCross(bt.SignalStrategy):
    params = (
        ('sma1', 10),
        ('sma2', 30),
    )

    def __init__(self):
        SMA1 = bt.ind.SMA(period=int(self.params.sma1))
        SMA2 = bt.ind.SMA(period=int(self.params.sma2))
        crossover = bt.ind.CrossOver(SMA1, SMA2)
        self.signal_add(bt.SIGNAL_LONG, crossover)

data0 = bt.feeds.YahooFinanceData(dataname='YHOO',
                                  fromdate=datetime(2011, 1, 1),
                                  todate=datetime(2012, 12, 31))

def runstrat(chromosome):
    
    cerebro = bt.Cerebro()
    cerebro.addstrategy(SmaCross, sma1=chromosome[0], sma2=chromosome[1])

    cerebro.adddata(data0)
    cerebro.run()
    return cerebro.broker.getvalue()

genome = G1DList.G1DList(2)
genome.setParams(rangemin=2, rangemax=55)
genome.evaluator.set(runstrat)
ga = GSimpleGA.GSimpleGA(genome)
ga.setGenerations(5)
ga.setMutationRate(0.2)
ga.setCrossoverRate(1.0)
ga.setPopulationSize(10)
ga.evolve(freq_stats=1)
print ga.bestIndividual()

I am not familiar with real trading approach using bt. @RandyT probably can tell more on this.

Below is full code for sample trading system, implementing fixed stop loss and take profit orders.

from __future__ import (absolute_import, division, print_function, unicode_literals)

import datetime as dt
import argparse

import backtrader as bt

def parse_args(pargs=None):

    parser = argparse.ArgumentParser(formatter_class=argparse.ArgumentDefaultsHelpFormatter,
                                     description=('Strategy auto-generation framework'))

    parser.add_argument('--data', required=False, default='SPY',
                        metavar='TICKER', help='Yahoo ticker to download')

    parser.add_argument('--data_from', required=False, default='2015-01-01',
                        metavar='YYYY-MM-DD[THH:MM:SS]',
                        help='Date[time] in YYYY-MM-DD[THH:MM:SS] format')

    parser.add_argument('--data_to', required=False, default='2016-12-31',
                        metavar='YYYY-MM-DD[THH:MM:SS]',
                        help='Ending date[time]')

    parser.add_argument('--plot', required=False, default='',
                        nargs='?', const='volume=True',
                        help='kwargs in key=value format')

    return parser.parse_args(pargs)

# script arguments parsing
args = parse_args(None)

# quotes feed for one ticker
data = bt.feeds.YahooFinanceCSVData(dataname=args.data+'.csv',
                                    fromdate=dt.datetime.strptime(args.data_from, '%Y-%m-%d'),
                                    todate=dt.datetime.strptime(args.data_to, '%Y-%m-%d'),
                                    reverse=True, adjclose=False, plot=True)

# observer showing stop loss and take profit levels
class SLTPTracking(bt.Observer):

    lines = ('stop', 'take')

    plotinfo = dict(plot=True, subplot=False)

    plotlines = dict(stop=dict(ls=':', linewidth=1.5),
                     take=dict(ls=':', linewidth=1.5))

    def next(self):

        if self._owner.sl_price != 0.0:
            self.lines.stop[0] = self._owner.sl_price

        if self._owner.tp_price != 0.0:
            self.lines.take[0] = self._owner.tp_price

# master strategy
class MasterStrategy(bt.Strategy):

    params = (
    ('ema_period', 200),
    ('sl', 1.0), ('rr_ratio', 3), ('n', 5),
             )

    def __init__(self):

        # init order variables
        self.order, self.sl_order, self.tp_order = None, None, None
        self.sl_price, self.tp_price = 0.0, 0.0

        self.ema = bt.indicators.ExponentialMovingAverage(period=int(self.p.ema_period),
                                                          subplot=False)
        
        # entry signals
        self.long = bt.indicators.CrossUp(self.data.close, self.ema)
        self.short = bt.indicators.CrossDown(self.data.close, self.ema)

    def notify_trade(self, trade):

        if trade.isclosed:

            # clear stop loss and take profit order variables for no position state
            if self.sl_order:
                self.broker.cancel(self.sl_order)
                self.sl_order = None
                self.sl_price = 0.0

            if self.tp_order:
                self.broker.cancel(self.tp_order)
                self.tp_order = None
                self.tp_price = 0.0

    def notify_order(self, order):

        if order.status in [order.Completed, order.Margin]:
            
            # clear order variable
            self.order = None

    def next(self):

        # process entries at bar open if not in position
        if not self.position:
            
           # check entry to long position
           if self.long[0] == True:
                self.order = self.order_target_percent(target=1.0)

           # check entry to short position
           elif self.short[0] == True:
                self.order = self.order_target_percent(target=-1.0)
                
        # process exits and position support orders if in position
        if self.position:

            price_sl_long = self.position.price * (1 - self.p.sl / 100)
            price_sl_short = self.position.price * (1 + self.p.sl / 100)

            price_tp_long = self.position.price * (1 + self.p.sl / 100 * self.p.rr_ratio)
            price_tp_short = self.position.price * (1 - self.p.sl / 100 * self.p.rr_ratio)
            
            # cancel existing stop loss and take profit orders
            if self.sl_order:
                self.broker.cancel(self.sl_order)

            if self.tp_order:
                self.broker.cancel(self.tp_order)

            # check & update stop loss order
            self.sl_price = 0.0
            if self.position.size > 0 and price_sl_long !=0: self.sl_price = price_sl_long
            if self.position.size < 0 and price_sl_short !=0: self.sl_price = price_sl_short

            if self.sl_price != 0.0:
                self.sl_order = self.order_target_value(target=0.0, exectype=bt.Order.Stop,
                                                        price=self.sl_price)
            # check & update take profit order
            self.tp_price = 0.0
            if self.position.size > 0 and price_tp_long !=0: self.tp_price = price_tp_long
            if self.position.size < 0 and price_tp_short !=0: self.tp_price = price_tp_short

            if self.tp_price != 0.0:
                self.tp_order = self.order_target_value(target=0.0, exectype=bt.Order.Limit,
                                                        price=self.tp_price)

# single backtest run
def single_backtest():

    cerebro = bt.Cerebro()
    cerebro.addstrategy(MasterStrategy)
    cerebro.addobserver(SLTPTracking)

    cerebro.adddata(data)
    cerebro.broker.set_cash(1000000.0)
    cerebro.broker.set_coc(True)
    cerebro.run()

    print ('Broker value %0.2f' % cerebro.broker.getvalue())

    return cerebro

if __name__ == '__main__':

    cerebro = single_backtest()

    if args.plot:
        cerebro.plot(style='bar', numfigs=1,
                        barup = 'black', bardown = 'black',
                        barupfill = True, bardownfill = False,
                        volup = 'green', voldown = 'red', voltrans = 50.0, voloverlay = False)

Outputs -
ref - bt's unique trade identifier
ticker - data feed name
datein - date and time of trade opening
pricein - price of trade entry
dir - long or short
dateout - date and time of trade closing
priceout - price of trade exit
chng% - exit price to entry price ratio
pnl - money profit/loss per trade
pnl% - proft/loss in %s to broker's value at the trade closing
cumpnl - cumulative profit/loss
size - max position size during trade
value - max trade value
nbars - trade duration in bars
pnl/bar - profit/loss per bar
mfe% - max favorable excursion
mae% - max adverse excursion

You can also run optimization having data feed as an optimization parameter. In this case it will be single cerebro run.

In order to have something automatically plotted in bt it should be data feed, indicator or observer. So you may want to develop an indicator or observer.

Docs - Indicator Development
DOcs - Observers and Statistics

On the other side, formula you are using will not give you a picture shown, it will be only a channel moving together with the close price. It will never cross the price. Hope you understand it and this formula is only first step.

Some time ago in order to develop similar picture I developed stop loss/take profit observer. Here is the link, hope it might help

Stop loss/Take profit observer

Data feed class need to be re-written as follows:

class dataFeed(btfeed.GenericCSVData):
    params = (
        ('dtformat', '%Y%m%d %H:%M:%S'),
        ('datetime', 0),
        ('open', 1),
        ('high', 2),
        ('low', 3),
        ('close', 4),
        ('volume', -1),
        ('openinterest', -1)
    )

and then data initiated as follows:

    data = dataFeed(dataname=path_to_file_with_quotes, timeframe=bt.TimeFrame.Minutes, compression=60)

Put self.tradesize = None or self.tradesize = 0 in the __init()__. It is undefined for first next() calls before you get your trade open.

@tracy it was discussed several times, try to search forums with the word multi. Also check the following post in the blog.

Straight forward way is to load all stocks into bt and do your calculations.

bt.If, bt.And, bt.Or

Docs - Platform Concepts

@rapha why the number of 24th is larger than the number of 28th? Should be the same, right? How many 23rds do you have.

@zuj said in Trades not getting executed using order_target_percent:

order_target_percent is faulty af.

That's possible, but needs to be proved.

On the other side, it looks like you are trying to open position on 5% from 500$, which is 25$. But 2ndcdsta stock price (EWC, right?) is more than 25$.

@zuj said in Trades not getting executed using order_target_percent:

I really don't get why the full code is required

In order to debug something, than this something need to be presented. Otherwise there is nothing to debug.

@zuj said in Trades not getting executed using order_target_percent:

Longcondition and short condition might as well be sma crossover for data0 and vice versa.

If you don't want to show the original code, than no problem, just provide the code with some general trading rules which has the same issue.

@spread_thin

change log function to

    def log(self, txt, dt=None):
        dt = dt or self.datas[0].datetime.datetime(0)
        print('%s, %s' % (dt.isoformat(), txt))

@amit said in Percent Price Oscillator conversion:

but get nan in the self.fractal.fractal_bearish[0]. what am I doing wrong? I am using fractals for StopLoss.

you are not trying to figure out what fractal indicator actually delivers and why it should deliver non-nanvalue. honestly i have no idea what does that indicator delivers, but simple fractal definition suggests that most of the bars should have nan and only rare fractal bars should deliver true or certain price values.

@rapha

I can't use your script since it requires your data feeds, therefore I made some tests based on the basic bt scripts. I was able to have two calls of the next per one date only when the data feed has that date twice. So i would check the data feeds around that dates first. Also i would check if 28-03-2021 date is present.

@amit i think it will be more convenient to have it as an indicator, but nobody prevents you from making calcs in the strategy. By the way, if you check the indicators in the docs, than you will find out that some of the indicators are already included in bt:

https://backtrader.com/docu/indautoref/#laguerrefilter
https://backtrader.com/docu/indautoref/#laguerrersi

I don't think you need a trading calendar to be added in case if you don't use resampling and you have no data on the holidays.

Also there is a lot of things going on in your script (necessary or not), so I would split it on a simpler pieces to debug.

And it us useful to have output as well.

@ppsev I didn't use mt4, so can say nothing on comparison.

@ppsev bt is event-driven backtester which simulates real trading. In real life you send order to broker and it opens for you position (or trade). Then you send another order to broker and broker can increase/decrease size of the position/trade or liquidate it. Also broker returns you prices which was used for order execution in notify_order.