Order target percent filling up order book without orders being executed

[Christoffer Fløe Jensen]

Hi,

I am having difficulties with orders being stuck in pending in the order book, eventually resulting in not being able to trade any of them.

I have searched for ways to cancel orders if they stay in pending too long, or simply clear out all orders, but I have not found a way to make it work.

What can I do to prevent the orders from getting stuck in pending?

class XGBoost1(bt.Strategy):

    
    params = dict(Hold = 20, mute = True, max_stocks = 20, barrier = 0.6)
    
    def __init__(self):
        
        self.d_with_len = []
        self.savedPositions = dict()
        self.myOrders = dict()
        self.inds = defaultdict(dict)
            
    def notify_order(self, order):
        
        if order.status in [order.Completed]:
            
            if order.isbuy():
                
                if order.data._name not in self.savedPositions.keys():
                    self.savedPositions[order.data._name] = bt.num2date(order.executed.dt)
                
                if self.p.mute != True:
                        
                    print(['BUY',order.data._name,
                           'Size', '{}'.format(order.executed.size),
                           'Price', '{:.4f}'.format(order.executed.price),
                           'Cost', '{:.4f}'.format(order.executed.value),
                           'Time', '{}'.format(bt.num2date(order.executed.dt))])             
                
            if order.issell():
                
                if order.data._name in self.savedPositions.keys():
                    self.savedPositions.pop(order.data._name)
            
                if self.p.mute != True:
                    
                    print(['SELL',order.data._name,
                           'Size', '{}'.format(order.executed.size),
                           'Price', '{:.4f}'.format(order.executed.price),
                           'Cost', '{:.4f}'.format(order.executed.value),
                           'Time', '{}'.format(bt.num2date(order.executed.dt))])
                    
            # Save orders
            self.myOrders[order.data] = None

    def prenext(self):
        
        # Populate d_with_len with acutal data
        self.d_with_len = [d for d in self.datas if len(d)]
        
        # call next() even when data is not available for all tickers
        self.next()

    def nextstart(self):
        
        # This is called exactly ONCE, when next is 1st called and then defaults to call next
        # Now all data fulfill the criteria
        self.d_with_len = self.datas
        
        # delegate the work to next
        self.next()
        
    def next(self):
        # Portfolio rebalance every 5 days, positions rebalance every 10 days
        l = len(self)
        if l % 5 == 0:
            self.rebalance_portfolio()
        #if l % 5 == 0:
        #    self.rebalance_positions()

    def rebalance_portfolio(self):
        # XGBoost ranking
        self.rankings = list(filter(lambda d: len(d) > 5, self.datas))
        self.rankings.sort(key=lambda d: d.volume[0], reverse = True)
        
        self.rankings_r = list(filter(lambda d: len(d) > 5, self.datas))
        self.rankings_r.sort(key=lambda d: d.volume[0])
        
        #print(len(self.rankings))
        
        # get portfolio content
        date = None
        portfolio = []
        orderbook = []
        
        for i, d in enumerate(self.rankings):
            if self.myOrders.get(d, None) is not None:
                self.broker.cancel(self.myOrders[d])
                #self.myOrders[d] = self.cancel()
                #self.cancel()
        
        
        
        for i, d in enumerate(self.rankings):
            if date is None:
                date = d.datetime.datetime()
            pos = self.getposition(d).size
            
            if self.myOrders.get(d, None) is not None:
                orderbook.append(d._name)
            
            # we have a buy order due
            if pos == 0 and self.myOrders.get(d, None) is not None:
                portfolio.append(d._name)
            
            # we are waiting for something to leave our portfolio
            if pos > 0:
                portfolio.append(d._name)
        
        
        
        
        # get top 20 stuff
        top_20 = []
        top_20_new = []
        for i, d in enumerate(self.rankings):
            pos = self.getposition(d).size
            
            # it has to be something that is not being transfered
            if self.myOrders.get(d, None) is None:
                if d.volume[0] > self.p.barrier:
                    
                    # update holding period
                    if pos > 0:
                        self.savedPositions[d._name] = d.datetime.datetime()
                    
                    top_20.append(d._name)
                    if pos == 0:
                        top_20_new.append(d._name)
                        
            if len(top_20) == self.p.max_stocks: break
                        
                        
        # remove expired stuff
        expired = []
        for i, d in enumerate(self.rankings):
            pos = self.getposition(d).size
            
            # it is in our portfolio and not being sold atm
            if pos > 0 and self.myOrders.get(d, None) is None:
                
                # violates hold condition
                if (d.datetime.datetime() - self.savedPositions.get(d._name, d.datetime.datetime())).days > self.p.Hold:
                    self.myOrders[d] = self.close(d, exectype=bt.Order.Close, cheat_on_open=True)
                    expired.append(d._name)
        
        print(" ")
        print(" ----------------- ")
        print("date         = ", date)
        print("nr of stocks = ", len(self.rankings))
        print("portfolio    = ", len(portfolio))
        print("top 20       = ", len(top_20))
        print("top 20 new   =", len(top_20_new))
        print("orderbook    =", len(orderbook))
        print("expired      =", len(expired))
        rfn = min(self.p.max_stocks - len(portfolio) + len(expired), self.p.max_stocks)
        print("room for new = ", rfn)
        print(" ----------------- ")
        print(" ")
        
        # we sell to make room
        for i, d in enumerate(self.rankings_r):
            pos = self.getposition(d).size
            if rfn >= len(top_20_new):
                break
            else:
                if pos > 0 and self.myOrders.get(d, None) is None:
                    self.myOrders[d] = self.close(d, exectype=bt.Order.Close, cheat_on_open=True, price = d.close)
                    print("sell")
                    rfn += 1
        
        # we buy
        for i, d in enumerate(self.rankings):
            pos = self.getposition(d).size
            
            if d._name in top_20_new:
                print("buy")
                size = 1 / (self.p.max_stocks)
                self.myOrders[d] = self.order_target_percent(data=d, target=size, exectype=bt.Order.Close)

[ab_trader]

@Christoffer-Fløe-Jensen said in Order target percent filling up order book without orders being executed:

I am having difficulties with orders being stuck in pending in the order book, eventually resulting in not being able to trade any of them.

How do you know that the orders are pending somewhere? Seems you use market orders and for this type of order there are two conditions only: executed if it is enough funds, or rejected if it is not enough funds. If yours are not executed, than they are rejected.

[Christoffer Fløe Jensen]

@ab_trader Hi, thanks for the response.

Basically what I have done thus far is use the dictionairy myOrders[], and whenever I make an order the order information is fed to the dictionairy. Then when the order is completed the dictionairy value is set to none. Utilizing this implementation, I assume an order is stuck in pending, if the dictionairy value is different from None.

[ab_trader]

Answering your question

What can I do to prevent the orders from getting stuck in pending?

Implement better order processing. If I am correct, that the only orders you remove from the myOrders[] are complete orders, orders with other status such as rejected or cancelled are hanging there.