For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

Trades are not executed as expected



  • Hi there

    I have a CSV with pricing data and signals.

    date,signal,close,high,low,open
    2018-01-02,1.0,16477.6,15444.6,13163.6,13625.0
    2018-01-03,1.0,15170.1,15572.8,14844.5,14978.2
    2018-01-04,1.0,14595.4,15739.7,14522.2,15270.7
    2018-01-05,1.0,14973.3,17705.2,15202.8,15477.2
    2018-01-06,1.0,13405.8,17712.4,16764.6,17462.1
    2018-01-07,1.0,13980.6,17579.6,16087.7,17527.3
    2018-01-08,1.0,14360.2,16537.9,14208.2,16476.2
    2018-01-09,1.0,13772.0,15497.5,14424.0,15123.7
    2018-01-10,1.0,13819.8,14973.3,13691.2,14588.5
    2018-01-11,0.0,11490.5,15018.8,13105.9,14968.2
    2018-01-12,0.0,11188.6,14229.9,13158.1,13453.9
    2018-01-13,0.0,11474.9,14659.5,13952.4,13952.4
    2018-01-14,1.0,11607.4,14511.8,13268.0,14370.8
    2018-01-15,0.0,12899.2,14445.5,13641.7,13767.3
    2018-01-16,0.0,11600.1,13843.1,10194.9,13836.1
    2018-01-17,0.0,10931.4,11678.0,9402.29,11431.1
    2018-01-18,0.0,10868.4,12107.3,10942.5,11198.8
    2018-01-19,1.0,11359.4,11992.8,11172.1,11429.8
    2018-01-20,-1.0,11259.4,13103.0,11656.2,11656.2
    2018-01-21,-1.0,11171.4,12895.9,11288.2,12889.2
    ...
    

    The log:

    Starting Portfolio Value: 1000000.00
    2018-01-02 23:59:59.999989,BUY
    2018-01-20 23:59:59.999989,SELL
    2018-01-21, OPERATION PROFIT, GROSS -2089.00, NET -2089.00
    2018-01-25 23:59:59.999989,BUY
    2018-01-26 23:59:59.999989,SELL
    2018-01-27, OPERATION PROFIT, GROSS -81.10, NET -81.10
    2018-01-27 23:59:59.999989,BUY
    2018-01-28 23:59:59.999989,SELL
    2018-01-29, OPERATION PROFIT, GROSS 280.20, NET 280.20
    2018-01-29 23:59:59.999989,BUY
    2018-01-30 23:59:59.999989,SELL
    ...
    

    From the log I learn two things which I do not understand.

    1. As you can see the buy signal is on the 2018-01-02, which means that the order should be executed on the next time frame which is the opening price on 2018-01-03.
      Why is the buy executed on the same time frame as the signal (exexution = 2018 01-02 23:59:59.999989) and not on the next time frame?

    2. Not all signals trigger the intended trades. In the CSV you can see the first few rows. There are quite some buying/selling signals (-1 = sell, 1 = buy, 0 = hold) — why are only certain ones executed and how do i change that?

    The used code:

    import datetime
    import backtrader as bt
    import backtrader.feeds as btfeed
    from backtrader.feeds import GenericCSVData
    
    class firstStrategy(bt.Strategy):
        
    #Logging function fot this strategy'''
    
        def log(self, txt, dt=None):
            dt = dt or self.datas[0].datetime.date(0)
            print('%s, %s' % (dt.isoformat(), txt))
        
    #buy sell logic of strategy    
        def next(self):
            if not self.position:
                if self.data.signal > 0:
                    self.buy(size = 1)
                    print('{},BUY'.format(self.datetime.datetime()))
                    
            else:
                if self.data.signal < 0:
                    self.sell(size = 1)
                    print('{},SELL'.format(self.datetime.datetime()))
                    
        def notify_trade(self, trade):
            if not trade.isclosed:
                return
    
            self.log('OPERATION PROFIT, GROSS %.2f, NET %.2f' %
                     (trade.pnl, trade.pnlcomm))
                    
    startcash = 1000000
    
    cerebro = bt.Cerebro()
    
    
    cerebro.addstrategy(firstStrategy)
        
    class GenericCSV_PE(GenericCSVData): 
    
        lines = ('signal',)
    
        params = (('signal', 1),)
    
        
    data = GenericCSV_PE(
        dataname='btctwitterpricepred.csv',
    
        fromdate=datetime.datetime(2018, 1, 2),
        todate=datetime.datetime(2018, 11, 11),
    
        nullvalue=0.0,
    
        dtformat=('%Y-%m-%d'),
    
    
        datetime=0,
        high=3,
        low=4,
        open=5,
        close=2,
        volume = -1,
        openinterest=-1,
    )
    
    cerebro.adddata(data)
    
    cerebro.broker.setcash(startcash)
    
    print('Starting Portfolio Value: %.2f' % cerebro.broker.getvalue())
    
    cerebro.run()
    
    # portfolio Value
    portvalue = cerebro.broker.getvalue()
    pnl = portvalue - startcash
    
    #Print results
    print('Final Portfolio Value: ${}'.format(portvalue))
    print('P/L: ${}'.format(pnl))
    
    cerebro.plot(style = 'line')
    

    Thank you


  • administrators

    @valentin said in Trades are not executed as expected:

    Trades are not executed as expected

    Yes, they are.

    @valentin said in Trades are not executed as expected:

    1. As you can see the buy signal is on the 2018-01-02, which means that the order should be executed on the next time frame which is the opening price on 2018-01-03.
      Why is the buy executed on the same time frame as the signal (exexution = 2018 01-02 23:59:59.999989) and not on the next time frame?

    @valentin said in Trades are not executed as expected:

                    self.buy(size = 1)
                    print('{},BUY'.format(self.datetime.datetime()))
    

    You are logging the order creation datetime. Order execution will be notified in notify_order. See:

    @valentin said in Trades are not executed as expected:

    1. Not all signals trigger the intended trades. In the CSV you can see the first few rows. There are quite some buying/selling signals (-1 = sell, 1 = buy, 0 = hold) — why are only certain ones executed and how do i change that?

    @valentin said in Trades are not executed as expected:

    if not self.position:
        if self.data.signal > 0:
            self.buy(size = 1)
    

    You are restricting yourself to a single entry.



  • Thank you for the feedback.

    I integrated the executed logging attribute and made sure not to restrict to a single entry:

    class firstStrategy(bt.Strategy):
        
        
        def log(self, txt, dt=None):
            ''' Logging function fot this strategy'''
            dt = dt or self.datas[0].datetime.date(0)
            print('%s, %s' % (dt.isoformat(), txt))
    
            
        def notify_order(self, order):
            if order.status in [order.Completed]:
                if order.isbuy():
                    self.log('BUY EXECUTED, Price: %.2f, Cost: %.2f' %
                        (order.executed.price,
                         order.executed.value
                         ))
    
                else:  # Sell
                    self.log('SELL EXECUTED, Price: %.2f, Cost: %.2f' %
                             (order.executed.price,
                              order.executed.value
                              ))
              
        def next(self):      
            if self.data.signal > 0:
                self.buy(size = 1)
                print('{},BUY'.format(self.datetime.datetime()))    
                      
            elif self.data.signal < 0:
                self.sell(size = 1)
                print('{},SELL'.format(self.datetime.datetime()))
        
        
                    
        def notify_trade(self, trade):
            if not trade.isclosed:
                return
    
            self.log('OPERATION PROFIT, GROSS %.2f, NET %.2f' %
                     (trade.pnl, trade.pnlcomm))
                    
    startcash = 100000
    
    cerebro = bt.Cerebro()
    
    
    cerebro.addstrategy(firstStrategy)
        
    class GenericCSV_PE(GenericCSVData): 
    
        lines = ('signal',)
    
        params = (('signal', 1),)
    
        
    data = GenericCSV_PE(
        dataname='btctwitterpricepred.csv',
    
        fromdate=datetime.datetime(2018, 1, 1),
        todate=datetime.datetime(2018, 11, 10),
    
        nullvalue=0.0,
    
        dtformat=('%Y-%m-%d'),
    
    
        datetime=0,
        high=3,
        low=4,
        open=5,
        close=2,
        volume = -1,
        openinterest=-1,
    )
    
    cerebro.adddata(data)
    
    cerebro.broker.setcash(startcash)
    
    print('Starting Portfolio Value: %.2f' % cerebro.broker.getvalue())
    
    cerebro.run()
    
    #Get final portfolio Value
    portvalue = cerebro.broker.getvalue()
    pnl = portvalue - startcash
    
    #Print out the final result
    print('Final Portfolio Value: ${}'.format(portvalue))
    print('P/L: ${}'.format(pnl))
    
    cerebro.plot(style = 'line')
    

    One issue remains though.
    The trades are triggered on a given day and then executed on the next day's closing price –but not on the next days open price, which is what I want to it to do.

    Starting Portfolio Value: 1000000.00
    2018-01-02 23:59:59.999989,BUY
    2018-01-03 23:59:59.999989,BUY EXECUTED
    2018-01-03, Price: 14978.20, Cost: 14978.20
    2018-01-03 23:59:59.999989,BUY
    2018-01-04 23:59:59.999989,BUY EXECUTED
    2018-01-04, Price: 15270.70, Cost: 15270.70
    2018-01-04 23:59:59.999989,BUY
    2018-01-05 23:59:59.999989,BUY EXECUTED
    2018-01-05, Price: 15477.20, Cost: 15477.20
    2018-01-05 23:59:59.999989,BUY
    2018-01-06 23:59:59.999989,BUY EXECUTED
    2018-01-06, Price: 17462.10, Cost: 17462.10
    

    I see that my data does not have a time for the open price. Is that the issue?
    If yes, how do I implement cheat_on_open? I tried this already but did not get it to work.

    Thank you


  • administrators

    @valentin said in Trades are not executed as expected:

    The trades are triggered on a given day and then executed on the next day's closing price –but not on the next days open price, which is what I want to it to do.

    You are not logging the prices and no data sample is provided. How can anyone know if the matched price is the closing or the opening price?

    @valentin said in Trades are not executed as expected:

    I see that my data does not have a time for the open price. Is that the issue?

    You have daily bars, how could you have a time for the opening price? A bar has a single timestamp, which for a daily bar is the day. You can have session opening times and session closing times, but they matter for things like resampling, to separate bars.

    @valentin said in Trades are not executed as expected:

    how do I implement cheat_on_open?

    cheat-on-open has a different use case. How you implemented it is unknown. The sample is here: Docs - Cheat On Open

    Do you care to explain how this code ...

    @valentin said in Trades are not executed as expected:

                    self.log('BUY EXECUTED, Price: %.2f, Cost: %.2f' %
                        (order.executed.price,
                         order.executed.value
                         ))
    

    produces this output

    @valentin said in Trades are not executed as expected:

    2018-01-03 23:59:59.999989,BUY EXECUTED
    2018-01-03, Price: 14978.20, Cost: 14978.20
    

    It seems really strange that things end up in two different lines. It would seem that the presented code doesn't match the executed code.



  • Thanks, you are right: the code I used was:

    class firstStrategy(bt.Strategy):
        
        
        def log(self, txt, dt=None):
            ''' Logging function fot this strategy'''
            dt = dt or self.datas[0].datetime.date(0)
            print('%s, %s' % (dt.isoformat(), txt))
    
            
        def notify_order(self, order):
            if order.status in [order.Completed]:
                if order.isbuy():
                    print('{},BUY EXECUTED'.format(self.datetime.datetime()))
                    self.log('Price: %.2f, Cost: %.2f' %
                        (order.executed.price,
                         order.executed.value
                         ))
    
                else:  # Sell
                    print('{},SELL EXECUTED'.format(self.datetime.datetime()))
                    self.log('SELL EXECUTED, Price: %.2f, Cost: %.2f' %
                             (order.executed.price,
                              order.executed.value
                              ))
             
        
        def next(self):      
            if self.data.signal > 0:
                self.buy(size = 1)
                print('{},BUY'.format(self.datetime.datetime()))    
                      
            elif self.data.signal < 0:
                self.sell(size = 1)
                print('{},SELL'.format(self.datetime.datetime()))
        
        
                    
        def notify_trade(self, trade):
            if not trade.isclosed:
                return
    
            self.log('OPERATION PROFIT, GROSS %.2f, NET %.2f' %
                     (trade.pnl, trade.pnlcomm))
                    
    startcash = 1000000
    
    cerebro = bt.Cerebro()
    
    
    cerebro.addstrategy(firstStrategy)
        
    class GenericCSV_PE(GenericCSVData): 
    
        lines = ('signal',)
    
        params = (('signal', 1),)
    
        
    data = GenericCSV_PE(
        dataname='shiftedttwice_witterpricepred.csv',
    
        fromdate=datetime.datetime(2018, 1, 1),
        todate=datetime.datetime(2018, 11, 10),
    
        nullvalue=0.0,
    
        dtformat=('%Y-%m-%d'),
    
    
        datetime=0,
        high=3,
        low=4,
        open=5,
        close=2,
        volume = -1,
        openinterest=-1,
    )
    
    cerebro.adddata(data)
    
    cerebro.broker.setcash(startcash)
    
    print('Starting Portfolio Value: %.2f' % cerebro.broker.getvalue())
    
    cerebro.run()
    
    #Get final portfolio Value
    portvalue = cerebro.broker.getvalue()
    pnl = portvalue - startcash
    
    #Print out the final result
    print('Final Portfolio Value: ${}'.format(portvalue))
    print('P/L: ${}'.format(pnl))
    
    cerebro.plot(style = 'line')
    

    Is there a mistake?

    2018-01-03 23:59:59.999989,BUY EXECUTED
    2018-01-03, Price: 14978.20, Cost: 14978.20
    

    This indicates that the price it was executed on was actually the open price (see data csv at the top), but why was it executed on 23:59:59.999989 isn't that the closing time?


  • administrators

    @valentin said in Trades are not executed as expected:

    but why was it executed on 23:59:59.999989 isn't that the closing time?

    1. You are not logging execution time, because you are logging the timestamp of the current bar. Execution is recorded in order.executed.dt. Ok, not much of a difference.
    2. You are printing the timestamp of the bar and bars have only 1 timestamp.
    3. You have daily bars and that's the end of the day when no sessionend has been specified and hence when the bar is delivered, which means it is the timestamp. of the bar.