Execute on bid/ask



  • First of all, many thanks to the developer and the contributors to this project!

    Hopefully somebody can help me with a problem I have and can't seem to be able to fix. I've got intra day bid/ask index data (sampled every 5 seconds for now), and I am trying to figure out how to properly execute on the bid or the offer at the next bid/ask quote coming in after order creation.

    Currently, I can sell the bid (market sell order) on the next quote, but the order to buy the offer is only executed at 23:59:55, at the end of the day (even when order creation was much earlier).

    Here is my data feed class:

    class BidAskCSV(btfeeds.GenericCSVData):
        linesoverride = True
        lines = ('bid', 'ask', 'datetime', 'close', 'open', 'high', 'low')
        params = (('datetime', 0), ('bid', 2), ('ask', 3), ('close', 3), ('open', 2), ('low', 2), ('high',3))
    
        def __init__(self):
            super(BidAskCSV, self).__init__()
    

    And some toy order creation/execution logic in a strategy class:

    
            # Check if we are in the market
            position = self.getposition(data=self.datas[0])
            rand = random.random()
            if rand < 0.2:
                if position:
                    # SELL (with all possible default parameters)
                    self.log('SELL CREATE {}'.format(self.datas[0].bid[0]))
    
                    # Keep track of the created order to avoid a 2nd order
                    self.order = self.sell(data=self.datas[0])
            elif rand > 0.8:
                if not position:
                    # BUY (with all possible default parameters)
                    self.log('BUY CREATE {}'.format(self.datas[0].ask[0]))
    
                    # Keep track of the created order to avoid a 2nd order
                    self.order = self.buy(data=self.datas[0], exectype=bt.Order.Close)
    

    The sell order gets executed on the bid of the next quote. It is a market order and the 'open' price is set to the bid-column of my data in the data feed class. Everything is fine so far....

    But, for buy orders I am trying to take the offer with a 'close' order (the 'close' price being set to the ask column). However, this order gets executed at 23:59:55 and not at the quote following the order generation. I would like it to be executed at the offer of the following quote that comes in.

    Should I be using a different order type, structure the CSV differently, is there an extra parameter to pass, etc? I much appreciate any help!

    Cheers,
    N


  • administrators

    Glad to see that linesoverride has found a use case.

    The problem here is the interpretation of the execution type Order.Close you are making. This is not going to give you the close price of a specific bar. It is meant to give you the close price which corresponds to the end of the session.

    Each broker/platform has a naming for it. Interactive Brokers, for example, calls it Market-On-Close or MOC.

    See the description: https://www.interactivebrokers.com/en/index.php?f=599

    Quoting directly from the BackBroker (the simulation broker in backtrader):

    • Close: meant for intraday in which the order is executed with the closing price of the last bar of the session

    Or directly at the documentation: Docs - Broker

    This order execution type is for intraday data and will be translated to the proper order type if you decide to connect to a live broker (in the aforementioned Interactive Brokers case it will be translated to MOC)

    If you wish to get the close price of the bar at which the order is generated (your ask price) you have to activate the cheat-on-close mode in the broker and then simply issue a Market order, which is the default.

    See here for example: https://community.backtrader.com/topic/15/convincing-strategy-to-buy-at-close-of-current-bar-in-backtest

    Or directly at the docs: https://www.backtrader.com/docu/broker.html?highlight=cheat



  • @backtrader
    Many thanks for your reply! I experimented with the coc parameter:

    cerebro.broker.set_coc(True)
    

    This does execute a market buy order on the offer, but at the same time will also execute a market sell order on the offer price. However, the behaviour I am after is to execute a buy order on the offer and a sell order on the bid.

    An idea I've got is to just rework my data to represent the midpoint of the bid/ask data, use the cheat-on-close param and allow for the spread in the form of an extra commission charge.

    Cheers,
    N


  • administrators

    This really grants a small addition to the broker, because by activating cheat-on-close you get twice the closing price, and the 2nd price you want is the incoming opening price.

    Let's add the possibility to selectively disable coc for orders. Your sell action would now be as follows:

     self.order = self.sell(data=self.datas[0], coc=False)
    

    In regular code with coc deactivated, this plays no role. But if you activate it to get the closing price of the bar for your buy, the sell deactivates it to make sure the execution waits until the next bar.

    In the development branch now with this commit id: 959f6cf46623d750c73f75bde9457269b0d4aed2



  • This is great, thank you very much!



  • @Nigel said in Execute on bid/ask:

    BidAskCSV

    Hello,

    I am also using Bid/ask data. Wanted to do a test where I buy and sell like @Nigel did.
    But it seems like the orders are not being executed because when I access the order.Exuecuted.price is always 0.

    from __future__ import (absolute_import, division, print_function,
                            unicode_literals)
    
    import argparse
    
    import backtrader as bt
    import backtrader.feeds as btfeeds
    #import backtrader.indicators as btind
    import random
    
    
    class BidAskCSV(btfeeds.GenericCSVData):
        linesoverride = True  # discard usual OHLC structure
        # datetime must be present and last
        lines = ('bid', 'ask', 'close', 'open','low', 'high', 'datetime')
        # datetime (always 1st) and then the desired order for
        params = (
            # (datetime, 0), # inherited from parent class
            ('bid', 1),  # default field pos 1
            ('ask', 2),  # default field pos 2
            ('close', 2), ('open', 1), ('low', 1), ('high',2)
        )
        
        def __init__(self):
            super(BidAskCSV, self).__init__()
    

    The strategy:

    class St(bt.Strategy):
    
        def log(self, txt, dt= None):
            """Logging function """
            dt= dt or self.datas[0].datetime[0]
            if isinstance(dt, float):
                dt = bt.num2date(dt)
            print('%s, %s' % (dt.isoformat(), txt))
    
    
        def __init__(self):
            self.order = None
            self.dataclose= self.datas[0].close
            #self.dataclose = self.datas[0].close[0]
                
                
        def notify_order(self, order):
           if order.status in [order.Submitted, order.Accepted]:
               # Buy/Sell order submitted/accepted to/by broker - Nothing to do
               return 
    
           # Check if an order has been completed
           # Attention: broker could reject order if not enougth cash
           if order.status in [order.Completed]:
               if order.isbuy():
                   self.log(
                       'BUY EXECUTED, Price: %.2f, Cost: %.2f' %
                       (order.executed.price,
                        order.executed.value))
    
                   self.buyprice = order.executed.price
                   
               else:  # Sell
                   self.log('SELL EXECUTED, Price: %.2f, Cost: %.2f' %
                            (order.executed.price,
                             order.executed.value))
    
               self.bar_executed = len(self)
           elif order.status in [order.Canceled, order.Margin, order.Rejected]:
               self.log('Order Canceled/Margin/Rejected')
    
           self.order = None
           
        def notify_trade(self, trade):
            if not trade.isclosed:
                return
    
            self.log('OPERATION PROFIT, GROSS %.2f, NET %.2f' %
                     (trade.pnl, trade.pnlcomm))
                
    
        def next(self):
            dtstr = self.datas[0].datetime.datetime().isoformat()
            txt = '%4d: %s - Bid %.4f - %.4f Ask, CLOSE %.4f ' % (
                (len(self), dtstr, self.datas[0].bid[0], self.datas[0].ask[0],
                 self.datas[0].close[0]))
    
            print(txt)
            
            position = self.getposition(data=self.datas[0])
            rand = random.random()
            print('-------------')
            print ('Random:', rand)
            print('-------------')
            if rand < 0.2:
                if position:
                    # SELL (with all possible default parameters)
                    self.log('SELL CREATE '.format(self.data.bid[0]))
    
                    # Keep track of the created order to avoid a 2nd order
                    self.order = self.sell(data=self.datas[0])
                    print ('Bid', self.data.bid[0])
            elif rand > 0.8:
                if not position:
                    # BUY (with all possible default parameters)
                    self.log('BUY CREATE {}'.format(self.data.ask[0]))
    
                    # Keep track of the created order to avoid a 2nd order
                    self.order = self.buy(data=self.datas[0])
                    self.log('Executed Price:{}'.format(self.order.executed.price))
                    print ('Executed Price:', self.order.executed.price)
                    print ('ASK', self.data.ask[0])
                    
    

    and finale code:

    def parse_args():
        parser = argparse.ArgumentParser(
            description='Bid/Ask Line Hierarchy',
            formatter_class=argparse.ArgumentDefaultsHelpFormatter,
        )
    
        parser.add_argument('--data', '-d', action='store',
                            required=False, default='bidask.csv',
                            help='data to add to the system')
    
        parser.add_argument('--dtformat', '-dt',
                            required=False, default='%m/%d/%Y %H:%M:%S',
                            help='Format of datetime in input')
    
        return parser.parse_args()
    
    
    def runstrategy():
        args = parse_args()
    
        cerebro = bt.Cerebro()  # Create a cerebro
           
        data = BidAskCSV(dataname=args.data, dtformat=args.dtformat)
        cerebro.adddata(data)  # Add the 1st data to cerebro
        # Set our desired cash start
        cerebro.broker.setcash(100000.0)
        
        # Add the strategy to cerebro
        cerebro.addstrategy(St)    
        
            # Add a FixedSize sizer according to the stake
        cerebro.addsizer(bt.sizers.FixedSize, stake=10)
        
        # Print out the starting conditions
        print('Starting Portfolio Value: %.2f' % cerebro.broker.getvalue())
    
        # Run over everything
        cerebro.run()
    
        # Print out the final result
        print('Final Portfolio Value: %.2f' % cerebro.broker.getvalue())
      
    
    if __name__ == '__main__':
        runstrategy()
    

    This is what I get as output:

    Starting Portfolio Value: 100000.00
       1: 2010-02-03T23:59:59.999989 - Bid 0.5346 - 0.5347 Ask, CLOSE 0.5347 
    -------------
    Random: 0.329840335116921
    -------------
       2: 2010-02-03T23:59:59.999989 - Bid 0.5343 - 0.5347 Ask, CLOSE 0.5347 
    -------------
    Random: 0.680459956622627
    -------------
       3: 2010-02-03T23:59:59.999989 - Bid 0.5543 - 0.5545 Ask, CLOSE 0.5545 
    -------------
    Random: 0.04838233726017549
    -------------
       4: 2010-02-03T23:59:59.999989 - Bid 0.5342 - 0.5344 Ask, CLOSE 0.5344 
    -------------
    Random: 0.7977225266469058
    -------------
       5: 2010-02-03T23:59:59.999989 - Bid 0.5245 - 0.5464 Ask, CLOSE 0.5464 
    -------------
    Random: 0.8661165246756014
    -------------
    2010-02-03T23:59:59.999989, BUY CREATE 0.5464
    2010-02-03T23:59:59.999989, Executed Price:0.0
    Executed Price: 0.0
    ASK 0.5464
       6: 2010-02-03T23:59:59.999989 - Bid 0.5460 - 0.5470 Ask, CLOSE 0.5470 
    -------------
    Random: 0.7797905933237207
    -------------
       7: 2010-02-03T23:59:59.999989 - Bid 0.5824 - 0.5826 Ask, CLOSE 0.5826 
    -------------
    Random: 0.585687043100518
    -------------
       8: 2010-02-03T23:59:59.999989 - Bid 0.5371 - 0.5374 Ask, CLOSE 0.5374 
    -------------
    Random: 0.2171674683269379
    -------------
       9: 2010-02-03T23:59:59.999989 - Bid 0.5793 - 0.5794 Ask, CLOSE 0.5794 
    -------------
    Random: 0.6576887524911412
    -------------
      10: 2010-02-03T23:59:59.999989 - Bid 0.5684 - 0.5688 Ask, CLOSE 0.5688 
    -------------
    Random: 0.406599019686264
    -------------
    Final Portfolio Value: 100000.00
    

    I don't understand what I am doing wrong, and why the orders are not being executed. Can I get some help here? Thanks!


  • administrators

    @roch_02 said in Execute on bid/ask:

                # BUY (with all possible default parameters)
                self.log('BUY CREATE {}'.format(self.data.ask[0]))
    
                # Keep track of the created order to avoid a 2nd order
                self.order = self.buy(data=self.datas[0])
                self.log('Executed Price:{}'.format(self.order.executed.price))
                print ('Executed Price:', self.order.executed.price)
                print ('ASK', self.data.ask[0])
    

    The order has not been executed yet. Only issued, hence the 0.0 price.

    Furthermore, see the resolution of this: Community - Getting executed twice on closing orders, because your data isn't obviously 1-day based



  • @backtrader

    Thank you for the reply.

    I changed the code and added: bt.TimeFrame.Ticks, compression, cerebro.broker.set_coc(True) and self.order = self.sell(data=self.datas[0], coc=False).

    The code is here:

    from __future__ import (absolute_import, division, print_function,
                            unicode_literals)
    
    import argparse
    
    import backtrader as bt
    import backtrader.feeds as btfeeds
    #import backtrader.indicators as btind
    import random
    
    
    class BidAskCSV(btfeeds.GenericCSVData):
        linesoverride = True  # discard usual OHLC structure
        # datetime must be present and last
        lines = ('bid', 'ask', 'close', 'open','low', 'high', 'datetime')
        # datetime (always 1st) and then the desired order for
        params = (
            # (datetime, 0), # inherited from parent class
            ('bid', 1),  # default field pos 1
            ('ask', 2),  # default field pos 2
            ('close', 2), ('open', 1), ('low', 1), ('high',2),
            ('timeframe', bt.TimeFrame.Ticks))
        
        def __init__(self):
            super(BidAskCSV, self).__init__()
        
    
    
    class St(bt.Strategy):
    
        def log(self, txt, dt= None):
            """Logging function """
            dt= dt or self.datas[0].datetime[0]
            if isinstance(dt, float):
                dt = bt.num2date(dt)
            print('%s, %s' % (dt.isoformat(), txt))
    
    
        def __init__(self):
            self.order = None
            self.dataclose= self.datas[0].close
            #self.dataclose = self.datas[0].close[0]
                
                
        def notify_order(self, order):
           if order.status in [order.Submitted, order.Accepted]:
               # Buy/Sell order submitted/accepted to/by broker - Nothing to do
               return 
    
           # Check if an order has been completed
           # Attention: broker could reject order if not enougth cash
           if order.status in [order.Completed]:
               if order.isbuy():
                   self.log(
                       'BUY EXECUTED, Price: %.4f, Cost: %.4f, bid: %.4f, ask: %.4f' %
                       (order.executed.price,
                        order.executed.value,
                        self.data.bid[0],
                        self.data.ask[0]))
    
                   self.buyprice = order.executed.price
                   
               else:  # Sell
                   self.log('SELL EXECUTED, Price: %.4f, Cost: %.4f' %
                            (order.executed.price,
                             order.executed.value))
    
               self.bar_executed = len(self)
           elif order.status in [order.Canceled, order.Margin, order.Rejected]:
               self.log('Order Canceled/Margin/Rejected')
    
           self.order = None
           
        def notify_trade(self, trade):
            if not trade.isclosed:
                return
    
            self.log('OPERATION PROFIT, GROSS %.2f, NET %.2f' %
                     (trade.pnl, trade.pnlcomm))
                
    
        def next(self):
            dtstr = self.datas[0].datetime.datetime().isoformat()
            txt = '%4d: %s - Bid %.4f - %.4f Ask, CLOSE %.4f ' % (
                (len(self), dtstr, self.datas[0].bid[0], self.datas[0].ask[0],
                 self.datas[0].close[0]))
            
            print (self.broker.getvalue())
    
            print(txt)
            
            position = self.getposition(data=self.datas[0])
            rand = random.random()
            print('-------------')
            print ('Random:', rand)
            print('-------------')          
            if rand < 0.2:
                if position:
                    # SELL (with all possible default parameters)
                    self.log('SELL CREATE '.format(self.data.bid[0]))
    
                    # Keep track of the created order to avoid a 2nd order
                    self.order = self.sell(data=self.datas[0], coc=False)
                    print ('Bid', self.data.bid[0])
            elif rand > 0.5:
                if not position:
                    # BUY (with all possible default parameters)
                    self.log('BUY CREATE {}'.format(self.data.ask[0]))
    
                    # Keep track of the created order to avoid a 2nd order
                    self.order = self.buy(data=self.datas[0])
                    #self.log('Executed Price:{}'.format(self.order.executed.price))
                    #print ('Executed Price:', self.order.executed.price)
                    print ('ASK', self.data.ask[0])
                    
    def parse_args():
        parser = argparse.ArgumentParser(
            description='Bid/Ask Line Hierarchy',
            formatter_class=argparse.ArgumentDefaultsHelpFormatter,
        )
    
        parser.add_argument('--data', '-d', action='store',
                            required=False, default='bidask.csv',
                            help='data to add to the system')
    
        parser.add_argument('--dtformat', '-dt',
                            required=False, default='%m/%d/%Y %H:%M:%S',
                            help='Format of datetime in input')
        
        parser.add_argument('--compression', required=False, default=2, type=int,
                            help='How much to compress the bars')
    
        return parser.parse_args()
    
    
    def runstrategy():
        args = parse_args()
    
        cerebro = bt.Cerebro()  # Create a cerebro
               
        data = BidAskCSV(dataname=args.data, dtformat=args.dtformat)
        cerebro.adddata(data)  # Add the 1st data to cerebro
        # Set our desired cash start
        cerebro.broker.setcash(100000.0)
        cerebro.broker.set_coc(True)
        
        cerebro.resampledata(data,
                             timeframe=bt.TimeFrame.Ticks,
                             compression = args.compression)
        
        # Add the strategy to cerebro
        cerebro.addstrategy(St)    
        
            # Add a FixedSize sizer according to the stake
        cerebro.addsizer(bt.sizers.FixedSize, stake=10)
        
        # Print out the starting conditions
        print('Starting Portfolio Value: %.2f' % cerebro.broker.getvalue())
    
        # Run over everything
        cerebro.run()
    
        # Print out the final result
        print('Final Portfolio Value: %.2f' % cerebro.broker.getvalue())
      
    
    if __name__ == '__main__':
        runstrategy()
    

    For sell order it executes with the bid price from the SELL EXECUTED time but for the buy order it executes with the ask price from the BUY CREATE time and not with the ask price from the BUY EXECUTED time.

    the output is here:

    Starting Portfolio Value: 100000.00
    100000.0
       2: 2010-02-03T16:53:51 - Bid 0.5343 - 0.5347 Ask, CLOSE 0.5347 
    -------------
    Random: 0.7585227506318774
    -------------
    2010-02-03T16:53:51, BUY CREATE 0.5347
    ASK 0.5347
    2010-02-03T16:53:52, BUY EXECUTED, Price: 0.5347, Cost: 5.3470, bid: 0.5543, ask: 0.5545
    100000.198
       3: 2010-02-03T16:53:52 - Bid 0.5543 - 0.5545 Ask, CLOSE 0.5545 
    -------------
    Random: 0.7866456795035222
    -------------
    99999.997
       4: 2010-02-03T16:53:53 - Bid 0.5342 - 0.5344 Ask, CLOSE 0.5344 
    -------------
    Random: 0.5313830132332052
    -------------
    100000.11700000001
       5: 2010-02-03T16:53:54 - Bid 0.5245 - 0.5464 Ask, CLOSE 0.5464 
    -------------
    Random: 0.409841893004643
    -------------
    100000.123
       6: 2010-02-03T16:53:54 - Bid 0.5460 - 0.5470 Ask, CLOSE 0.5470 
    -------------
    Random: 0.009108455433875506
    -------------
    2010-02-03T16:53:54, SELL CREATE 
    Bid 0.546
    2010-02-03T16:53:56, SELL EXECUTED, Price: 0.5824, Cost: 5.3470
    2010-02-03T16:53:56, OPERATION PROFIT, GROSS 0.48, NET 0.48
    100000.477
       7: 2010-02-03T16:53:56 - Bid 0.5824 - 0.5826 Ask, CLOSE 0.5826 
    -------------
    Random: 0.4250130075545223
    -------------
    100000.477
       8: 2010-02-03T16:53:57 - Bid 0.5371 - 0.5374 Ask, CLOSE 0.5374 
    -------------
    Random: 0.9467974453210086
    -------------
    2010-02-03T16:53:57, BUY CREATE 0.5374
    ASK 0.5374
    2010-02-03T16:53:58, BUY EXECUTED, Price: 0.5374, Cost: 5.3740, bid: 0.5793, ask: 0.5794
    100000.897
       9: 2010-02-03T16:53:58 - Bid 0.5793 - 0.5794 Ask, CLOSE 0.5794 
    -------------
    Random: 0.4769590069462516
    -------------
    100000.791
      10: 2010-02-03T16:53:59 - Bid 0.5684 - 0.5688 Ask, CLOSE 0.5688 
    -------------
    Random: 0.21570733606386583
    -------------
    Final Portfolio Value: 100000.79
    

    How can I correct the ask price?

    Another question, why is cost different than price? Is that the correct value?

    (I am using the bidask.csv file from backtrader data)


  • administrators

    @roch_02 said in Execute on bid/ask:

    the buy order it executes with the ask price from the BUY CREATE time and not with the ask price from the BUY EXECUTED time.

    Because you have cheat-on-close active and you are effectively asking for that price.

    @roch_02 said in Execute on bid/ask:

    Another question, why is cost different than price?

    The price is per unit of an asset. It cannot be the same as the total cost of the operation.



  • @backtrader

    Thanks for the answer but I still did not get part of it.

    So here is the issue:
    Before I choose to put cerebro.broker.set_coc(True)the price for the Buy executed and Sell executed would always be from the bid from the execution time. (correct price for the sell order)

    Once I set cerebro.broker.set_coc(True), the Buy executed and Sell executed would always be from the ask price from the created time (not from executed time).

    Setting self.sell(data=self.datas[0], coc=False) would correct the problem for the sell order and the price used is now the bid from the execution time (correct price for sell order). However, for the buy order, the ask from the created time is the price used. My question is, if it is possible to use the ask price from the execution time for the buy order?

    Thanks for your time in helping!


  • administrators

    Again: no. By setting cheat-on-close you get the price at creation time, unless the flag coc in buy/sell is set.



  • Ok, thanks once again.


Log in to reply
 

Looks like your connection to Backtrader Community was lost, please wait while we try to reconnect.