Why are trades getting executed twice?



  • Hi,

    I have added to my own data to a simple MA cross strategy sample code in order to get used to backtrader. Everything seems to work fine, except the executions are duplicated. I searched through the blog about this and there was a mention about data time index not correct, but I am using a very simple CSV data here for illustration. I don't see this data can be the cause, if anyone can spot anything or offer some insight.. I will be very grateful.

    Here is the full 2.7 code (sorry if markdown is not showing this properly):

    import backtrader as bt
    import backtrader.indicators as btind
    import datetime as dt
    import pandas as pd
    import pandas_datareader as web
    from pandas import Series, DataFrame
    import random
    from copy import deepcopy
    import argparse
    import sys  # To find out the script name (in argv[0])
    import os.path  # To manage paths
    
    
    class SMAC(bt.Strategy):
        params = {"fast": 10, "slow": 20,  "optim": False, "optim_fs": (20, 50)} 
    
        def log(self, txt, dt=None):
            ''' Logging function for this strategy'''
            dt = dt or self.datas[0].datetime.date(0)
            print('%s, %s' % (dt.isoformat(), txt))
            
            
        def __init__(self):
            """Initialize the strategy"""
     
            self.fastma = dict()
            self.slowma = dict()
            self.regime = dict()
     
            if self.params.optim: 
                self.params.fast, self.params.slow = self.params.optim_fs    
     
            if self.params.fast > self.params.slow:
                raise ValueError(
                    "A SMAC strategy cannot have the fast moving average's window be " + \
                     "greater than the slow moving average window.")
     
            for d in self.getdatanames():
     
                # The moving averages
                self.fastma[d] = btind.SimpleMovingAverage(self.getdatabyname(d),      # The symbol for the moving average
                                                           period=self.params.fast,    # Fast moving average
                                                           plotname="FastMA: " + d)
                self.slowma[d] = btind.SimpleMovingAverage(self.getdatabyname(d),      # The symbol for the moving average
                                                           period=self.params.slow,    # Slow moving average
                                                           plotname="SlowMA: " + d)
     
                # Get the regime
                self.regime[d] = self.fastma[d] - self.slowma[d]    # Positive when bullish
     
        def next(self):
            """Define what will be done in a single step, including creating and closing trades"""
            for d in self.getdatanames():    # Looping through all symbols
                
                self.data = self.getdatabyname(d)
                # print the data name
                # print(d)
                # Simply log the closing price of the series from the reference
                self.log('Close, %.2f' % self.data.close[0])
                
                pos = self.getpositionbyname(d).size or 0
                self.log('Current position is: , %.2f' % pos)     
                           
                if pos == 0:    # Are we out of the market?
                    # Consider the possibility of entrance
                    # Notice the indexing; [0] always mens the present bar, and [-1] the bar immediately preceding
                    # Thus, the condition below translates to: "If today the regime is bullish (greater than
                    # 0) and yesterday the regime was not bullish"
                    if self.regime[d][0] > 0 and self.regime[d][-1] <= 0:    # A buy signal
                        self.buy(data=self.getdatabyname(d))
                        self.log('BUY CREATED, %.2f' % self.data.close[0])
                        # Keep track of the created order to avoid a 2nd order
                        self.order = self.buy()  
     
                else:    # We have an open position
                    if self.regime[d][0] <= 0 and self.regime[d][-1] > 0:    # A sell signal
                        self.sell(data=self.getdatabyname(d))
                        self.log('SELL CREATED, %.2f' % self.data.close[0])
                        # Keep track of the created order to avoid a 2nd order
                        self.order = self.sell()
                        
        def notify_order(self, order):
            if order.status in [order.Submitted, order.Accepted]:
                # Buy/Sell order submitted/accepted to/by broker - Nothing to do
                return
    
            # Check if an order has been completed
            # Attention: broker could reject order if not enougth cash
            if order.status in [order.Completed]:
                if order.isbuy():
                    self.log(
                        'BUY EXECUTED, Price: %.2f, Cost: %.2f, Comm %.2f' %
                        (order.executed.price,
                         order.executed.value,
                         order.executed.comm))
    
                    self.buyprice = order.executed.price
                    self.buycomm = order.executed.comm
                else:  # Sell
                    self.log('SELL EXECUTED, Price: %.2f, Cost: %.2f, Comm %.2f' %
                             (order.executed.price,
                              order.executed.value,
                              order.executed.comm))
    
                self.bar_executed = len(self)
    
            elif order.status in [order.Canceled, order.Margin, order.Rejected]:
                self.log('Order Canceled/Margin/Rejected')
    
            self.order = None
           
        def notify_trade(self, trade):
            if not trade.isclosed:
                return
            self.log('OPERATION PROFIT, GROSS %.2f, NET %.2f' %
                     (trade.pnl, trade.pnlcomm))
    
     
    class PropSizer(bt.Sizer):
        """A position sizer that will buy as many stocks as necessary for a certain proportion of the portfolio
           to be committed to the position, while allowing stocks to be bought in batches (say, 100)"""
        params = {"prop": 0.1, "batch": 1}
     
        def _getsizing(self, comminfo, cash, data, isbuy):
            """Returns the proper sizing"""
     
            if isbuy:    # Buying
                target = self.broker.getvalue() * self.params.prop    # Ideal total value of the position
                price = data.close[0]
                shares_ideal = target / price    # How many shares are needed to get target
                batches = int(shares_ideal / self.params.batch)    # How many batches is this trade?
                shares = batches * self.params.batch    # The actual number of shares bought
     
                if shares * price > cash:
                    return 0    # Not enough money for this trade
                else:
                    return shares
     
            else:    # Selling
                                             
                                                 
                return self.broker.getposition(data).size    # Clear the position
     
     
    class AcctValue(bt.Observer):
        alias = ('Value',)
        lines = ('value',)
     
        plotinfo = {"plot": True, "subplot": True}
     
        def next(self):
            self.lines.value[0] = self._owner.broker.getvalue()    # Get today's account value (cash + stocks)
     
     
    class AcctStats(bt.Analyzer):
        """A simple analyzer that gets the gain in the value of the account; should be self-explanatory"""
     
        def __init__(self):
            self.start_val = self.strategy.broker.get_value()
            self.end_val = None
     
        def stop(self):
            self.end_val = self.strategy.broker.get_value()
     
        def get_analysis(self):
            return {"start": self.start_val, "end": self.end_val,
                    "growth": self.end_val - self.start_val, "return": self.end_val / self.start_val}
            
    
    ### Data feed from CSV files
    
    #datapath
    thePath = "D:\\Michael\\Python\\codes\\"  #Path to your code
    dataPath = "D:\\Michael\\Python\\codes\\Data\\"  #Path to your data 
    
    
    def parse_args():
        parser = argparse.ArgumentParser(
            description='Pandas test script')
    
        parser.add_argument('--noheaders', action='store_true', default=False,
                            required=False,
                            help='Do not use header rows')
    
        parser.add_argument('--noprint', action='store_true', default=False,
                            help='Print the dataframe')
    
        return parser.parse_args()
    
    
    args = parse_args()
    
    skiprows = 1 if args.noheaders else 0
    header = None if args.noheaders else 0
    
    
    symbols = ["AU.SHF"]
    
    datafeeds = {s: pd.read_csv(dataPath +s+ '.csv',
                                    skiprows=skiprows,
                                    header=header,
                                    parse_dates=True,
                                    index_col=0) for s in symbols}
    
    
    for df in datafeeds.values():
        df["OpenInterest"] = 0    # PandasData reader expects an OpenInterest column;
                                  # not provided by Google and we don't use it so set to 0
    
    if __name__ == '__main__':
        
        cerebro = bt.Cerebro(stdstats=False)
         
        plot_symbols = ["AU.SHF", "AL.SHF", "IF.CFE"]
        is_first = True
        #plot_symbols = []
        for s, df in datafeeds.items():
            data = bt.feeds.PandasData(dataname=df, name=s)
            if s in plot_symbols:
                if is_first:
                    data_main_plot = data
                    is_first = False
                else:
                    data.plotinfo.plotmaster = data_main_plot
            else:
                data.plotinfo.plot = False
            cerebro.adddata(data)    # Give the data to cerebro
         
        cerebro.broker.setcash(1000000)
        cerebro.broker.setcommission(0.02)
        cerebro.addstrategy(SMAC)
        cerebro.addobserver(AcctValue)
        cerebro.addobservermulti(bt.observers.BuySell)    # Plots up/down arrows
        cerebro.addsizer(PropSizer)
        cerebro.addanalyzer(AcctStats)
         
        cerebro.run()        
        
        cerebro.plot(iplot=True, volume=False)
        endValue=cerebro.broker.getvalue()
        print(endValue)
    

    I'll include the csv Data I used here :

     DATETIME	WINDCODE	OPEN	HIGH	LOW	CLOSE	VOLUME
    2017/3/27	AU.SHF	281.8056818	284.8556818	281.3556818	284.6056818	162888
    2017/3/28	AU.SHF	284.6556818	284.9556818	283.5556818	283.8056818	148832
    2017/3/29	AU.SHF	284.6556818	284.7556818	282.6556818	283.2056818	159592
    2017/3/30	AU.SHF	283.5556818	283.8056818	282.7056818	283.2056818	126756
    2017/3/31	AU.SHF	282.9556818	282.9556818	280.8056818	281.2056818	178510
    2017/4/5	AU.SHF	283.8056818	284.3556818	283.3556818	283.8556818	74992
    2017/4/6	AU.SHF	282.8056818	284.3056818	281.8056818	283.8556818	198064
    2017/4/7	AU.SHF	283.6556818	288.0556818	283.0056818	286.1056818	294194
    2017/4/10	AU.SHF	287.0556818	287.6556818	283.7556818	284.8056818	254310
    2017/4/11	AU.SHF	283.8556818	285.5556818	283.5056818	285.3556818	169940
    2017/4/12	AU.SHF	286.0056818	289.7056818	285.8556818	288.4556818	316430
    2017/4/13	AU.SHF	288.7056818	290.6056818	287.7056818	289.7556818	242774
    2017/4/14	AU.SHF	290.1556818	290.6556818	289.3556818	290.3056818	152170
    2017/4/17	AU.SHF	290.6556818	292.6556818	289.5556818	289.8056818	196830
    2017/4/18	AU.SHF	289.5056818	290.5056818	288.4056818	289.0556818	194112
    2017/4/19	AU.SHF	289.2056818	290.3056818	287.8056818	288.9056818	250222
    2017/4/20	AU.SHF	289.1556818	289.3556818	287.3556818	288.7056818	216358
    2017/4/21	AU.SHF	288.5556818	288.9556818	287.8556818	288.5056818	170642
    2017/4/24	AU.SHF	288.9556818	290.5556818	286.1056818	286.5056818	310052
    2017/4/25	AU.SHF	285.9556818	287.0056818	285.5556818	286.5056818	166292
    2017/4/26	AU.SHF	285.3556818	286.1556818	284.7556818	285.0556818	183112
    2017/4/27	AU.SHF	285.1556818	286.1556818	284.6556818	285.4556818	154594
    2017/4/28	AU.SHF	285.7056818	285.8556818	284.9056818	285.8056818	129946
    2017/5/2	AU.SHF	283.8	284.4	283.55	283.9	39920
    2017/5/3	AU.SHF	283.05	283.7	282.85	283.3	30714
    2017/5/4	AU.SHF	283	283.25	278.7	279.65	92606
    2017/5/5	AU.SHF	278.05	280	277.4	278.8	102698
    2017/5/8	AU.SHF	278.3	279.35	278.05	278.5	86346
    2017/5/9	AU.SHF	278.65	279.4	277.5	278.45	128798
    2017/5/10	AU.SHF	277.35	277.4	275.65	277.05	118918
    2017/5/11	AU.SHF	277.25	277.55	276.3	277.35	92586
    2017/5/12	AU.SHF	277	278.5	276.95	278.1	119980
    2017/5/15	AU.SHF	278.75	279.55	277.75	279.4	124840
    2017/5/16	AU.SHF	280.15	280.65	279	280	118992
    2017/5/17	AU.SHF	280.05	281.3	279.5	280.95	133590
    2017/5/18	AU.SHF	282.5	284.75	282.35	283.25	230310
    2017/5/19	AU.SHF	283.55	284.2	281.6	282.65	186950
    2017/5/22	AU.SHF	283.3	283.6	282.15	283.3	114482
    2017/5/23	AU.SHF	284.05	285.2	283.45	285.2	123256
    2017/5/24	AU.SHF	285	285.1	281.5	281.85	170178
    2017/5/25	AU.SHF	282.5	283.8	281.75	283.3	136700
    2017/5/26	AU.SHF	282.75	283.5	282	283.35	104532
    2017/5/31	AU.SHF	283.05	283.3	281.65	281.65	74258
    2017/6/1	AU.SHF	282.35	283.15	281.4	281.85	157458
    2017/6/2	AU.SHF	281.25	282.2	280.6	280.85	129192
    2017/6/5	AU.SHF	283	285.2	282.75	284.55	215452
    2017/6/6	AU.SHF	284.8	286.5	284.05	286.3	135910
    2017/6/7	AU.SHF	286.6	287.2	286.25	286.6	138618
    2017/6/8	AU.SHF	286.55	286.9	285.05	285.75	141700
    2017/6/9	AU.SHF	285.15	285.4	282.15	282.5	223770
    2017/6/12	AU.SHF	281.8	281.9	281	281.05	151544
    2017/6/13	AU.SHF	280.85	281.6	280.35	281.15	105884
    2017/6/14	AU.SHF	280.35	282.4	280.05	281.7	131122
    2017/6/15	AU.SHF	283	284.65	279.7	279.85	238286
    2017/6/16	AU.SHF	278.5	279.8	278.3	279.35	147956
    2017/6/19	AU.SHF	279.65	279.8	278.5	278.85	99854
    2017/6/20	AU.SHF	278.55	278.9	277.6	278.25	111782
    2017/6/21	AU.SHF	278.05	278.5	277.3	278.15	104238
    2017/6/22	AU.SHF	277.85	280.3	277.3	279.9	135198
    2017/6/23	AU.SHF	280.15	280.55	279.1	280.45	119404
    2017/6/26	AU.SHF	281.55	281.55	280.35	280.6	78260
    2017/6/27	AU.SHF	278.35	279.45	277.9	279.4	170412
    2017/6/28	AU.SHF	278.6	279.3	278.05	278.75	168850
    2017/6/29	AU.SHF	278.95	279.15	277.7	278.15	158992
    2017/6/30	AU.SHF	276.4	277.35	275.4	275.5	221690
    2017/7/3	AU.SHF	275.3	276.3	274.35	274.6	142246
    2017/7/4	AU.SHF	273.25	273.35	271.6	272.6	190506
    2017/7/5	AU.SHF	272.7	273.3	272.05	272.4	102604
    2017/7/6	AU.SHF	271.1	273.2	271.05	272.85	129270
    2017/7/7	AU.SHF	272.9	273.25	271.6	272.2	115020
    2017/7/10	AU.SHF	272.25	272.35	268	268.8	241862
    2017/7/11	AU.SHF	269.35	270.7	268.95	270	139294
    2017/7/12	AU.SHF	270.1	271.65	269.25	271	156136
    2017/7/13	AU.SHF	272.35	272.45	271.05	271.75	154644
    2017/7/14	AU.SHF	271.5	271.5	269.85	270.65	113550
    2017/7/17	AU.SHF	273.15	273.45	272.3	273	163458
    2017/7/18	AU.SHF	273.7	274.4	273.5	273.7	125924
    2017/7/19	AU.SHF	274.3	274.75	273.85	274.05	113988
    2017/7/20	AU.SHF	274.55	274.75	273.9	274.2	100138
    2017/7/21	AU.SHF	274.3	275.85	273.5	275.8	173552
    2017/7/24	AU.SHF	276.7	277.45	276.05	276.45	124830
    2017/7/25	AU.SHF	277.35	277.4	276.15	276.8	99046
    2017/7/26	AU.SHF	276.1	276.5	274.15	274.4	142662
    2017/7/27	AU.SHF	274.8	278.05	274.6	277.6	164112
    2017/7/28	AU.SHF	277.9	277.95	276.6	277.25	149302
    2017/7/31	AU.SHF	277.95	278.95	277.65	278.05	147076
    2017/8/1	AU.SHF	278.3	278.5	277.75	278	92472
    2017/8/2	AU.SHF	278	279.05	277.55	277.95	160750
    2017/8/3	AU.SHF	277.6	278.8	276.6	276.9	147540
    2017/8/4	AU.SHF	277.55	278.55	277.4	278.45	143116
    2017/8/7	AU.SHF	277	277.25	275.65	275.95	175064
    2017/8/8	AU.SHF	275.85	276.5	275.35	275.7	112310
    2017/8/9	AU.SHF	276.2	277.6	274.15	276.45	186480
    2017/8/10	AU.SHF	278.1	278.8	277.4	278.45	187428
    2017/8/11	AU.SHF	279.35	280.3	278.85	279.85	222826
    2017/8/14	AU.SHF	279.75	280.3	278.55	279.75	175078
    2017/8/15	AU.SHF	278.9	279.75	277	277.85	182520
    2017/8/16	AU.SHF	276.55	277.9	276.35	276.95	119916
    2017/8/17	AU.SHF	277.1	280.45	276.65	279.95	192184
    2017/8/18	AU.SHF	279.45	280.5	278.75	280.5	144594
    2017/8/21	AU.SHF	281.8	282.6	278.75	279.7	238136
    2017/8/22	AU.SHF	279.8	280.95	279	279.1	149470
    2017/8/23	AU.SHF	279.65	280.4	278.35	279.15	148734
    2017/8/24	AU.SHF	279.75	279.9	279.05	279.25	119042
    2017/8/25	AU.SHF	279.3	279.75	278.6	279.05	111830
    2017/8/28	AU.SHF	278.95	280.9	276.5	280.45	263900
    2017/8/29	AU.SHF	280.3	283.8	279.75	283.8	215014
    2017/8/30	AU.SHF	283.8	284.3	280.25	280.35	220274
    2017/8/31	AU.SHF	280.35	281.3	279.2	279.9	141360
    2017/9/1	AU.SHF	280.9	282.65	280.7	281.8	173324
    

    The results are:

    runfile('D:/Michael/Python/codes/ntguardianExample.py', wdir='D:/Michael/Python/codes')
    2017-04-25, Close, 286.51
    2017-04-25, Current position is: , 0.00
    2017-04-26, Close, 285.06
    2017-04-26, Current position is: , 0.00
    2017-04-27, Close, 285.46
    2017-04-27, Current position is: , 0.00
    2017-04-28, Close, 285.81
    2017-04-28, Current position is: , 0.00
    2017-05-02, Close, 283.90
    2017-05-02, Current position is: , 0.00
    2017-05-03, Close, 283.30
    2017-05-03, Current position is: , 0.00
    2017-05-04, Close, 279.65
    2017-05-04, Current position is: , 0.00
    2017-05-05, Close, 278.80
    2017-05-05, Current position is: , 0.00
    2017-05-08, Close, 278.50
    2017-05-08, Current position is: , 0.00
    2017-05-09, Close, 278.45
    2017-05-09, Current position is: , 0.00
    2017-05-10, Close, 277.05
    2017-05-10, Current position is: , 0.00
    2017-05-11, Close, 277.35
    2017-05-11, Current position is: , 0.00
    2017-05-12, Close, 278.10
    2017-05-12, Current position is: , 0.00
    2017-05-15, Close, 279.40
    2017-05-15, Current position is: , 0.00
    2017-05-16, Close, 280.00
    2017-05-16, Current position is: , 0.00
    2017-05-17, Close, 280.95
    2017-05-17, Current position is: , 0.00
    2017-05-18, Close, 283.25
    2017-05-18, Current position is: , 0.00
    2017-05-19, Close, 282.65
    2017-05-19, Current position is: , 0.00
    2017-05-22, Close, 283.30
    2017-05-22, Current position is: , 0.00
    2017-05-23, Close, 285.20
    2017-05-23, Current position is: , 0.00
    2017-05-24, Close, 281.85
    2017-05-24, Current position is: , 0.00
    2017-05-25, Close, 283.30
    2017-05-25, Current position is: , 0.00
    2017-05-25, BUY CREATED, 283.30
    2017-05-26, BUY EXECUTED, Price: 282.75, Cost: 99528.00, Comm 1990.56
    2017-05-26, BUY EXECUTED, Price: 282.75, Cost: 99528.00, Comm 1990.56
    2017-05-26, Close, 283.35
    2017-05-26, Current position is: , 704.00
    2017-05-31, Close, 281.65
    2017-05-31, Current position is: , 704.00
    2017-06-01, Close, 281.85
    2017-06-01, Current position is: , 704.00
    2017-06-02, Close, 280.85
    2017-06-02, Current position is: , 704.00
    2017-06-05, Close, 284.55
    2017-06-05, Current position is: , 704.00
    2017-06-06, Close, 286.30
    2017-06-06, Current position is: , 704.00
    2017-06-07, Close, 286.60
    2017-06-07, Current position is: , 704.00
    2017-06-08, Close, 285.75
    2017-06-08, Current position is: , 704.00
    2017-06-09, Close, 282.50
    2017-06-09, Current position is: , 704.00
    2017-06-12, Close, 281.05
    2017-06-12, Current position is: , 704.00
    2017-06-13, Close, 281.15
    2017-06-13, Current position is: , 704.00
    2017-06-14, Close, 281.70
    2017-06-14, Current position is: , 704.00
    2017-06-15, Close, 279.85
    2017-06-15, Current position is: , 704.00
    2017-06-16, Close, 279.35
    2017-06-16, Current position is: , 704.00
    2017-06-19, Close, 278.85
    2017-06-19, Current position is: , 704.00
    2017-06-19, SELL CREATED, 278.85
    2017-06-20, SELL EXECUTED, Price: 278.55, Cost: 199056.00, Comm 3921.98
    2017-06-20, SELL EXECUTED, Price: 278.55, Cost: -196099.20, Comm 3921.98
    2017-06-20, OPERATION PROFIT, GROSS -2956.80, NET -10859.90
    2017-06-20, Close, 278.25
    2017-06-20, Current position is: , -704.00
    2017-06-21, Close, 278.15
    2017-06-21, Current position is: , -704.00
    2017-06-22, Close, 279.90
    2017-06-22, Current position is: , -704.00
    2017-06-23, Close, 280.45
    

    As one can see, only two signals were generated for this short data. One for buy and one for sell to close, yet each time the order was executed twice.

    Thanks for the help,

    Michael


  • administrators

    See the comment at the top of the page for formatting code blocks:

    For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

    @Michael172 said in Why are trades getting executed twice?:

    self.data = self.getdatabyname(d)
    

    An guess would be that with that line you are overriding (or at least trying to override) the default built-in in the platform which is self.data which always points to self.datas[0].

    In the loop you'd be better of with

    data = self.getdatabyname(d)
    

    And then referencing your opertions to data rather than always call getdatabyname

    because there is no need to keep that temporary loop inside the object.

    For working with names you may also consider dnames. See Docs - Strategy



  • Thanks for the help. I changed it but that didn't solve the problem. But, you are right, I should be more careful with variable names.

    I think the problem is with my looping logic with the data or backtrader has a problem with it. Since I have mulitiple symbols in my dataset, I have to loop through each. In that case, only one symbol gets duplicated executions, I have to dig into it and find out why. It's a puzzler. I'll report back if I find anything. At the mean time, any suggestions are greatly appreciated.


  • administrators

    On a second look: there are two almost consecutive buy statements when the signal is recognized.

    Consider also using CrossUp to detect tbe condition. It will greatly simplify the logic.



  • Yup, that's it. That did it. I thought the second "order" was to keep track of the order, not to create another one. Lesson learned! Thanks.

    By the way, thanks creating backtrader, I am sure everyone in the community is very thankful for you guys' great work.

    Michael


Log in to reply
 

Looks like your connection to Backtrader Community was lost, please wait while we try to reconnect.