For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

Limit order with a day valid never gets executed? #220


  • administrators

    From Issue #220


    Hi,

    Thanks for providing such a great platform. I'm trying to run my first test strategy on this platform. I simply create two limit orders per day and make them to expire if they were not executed. The following is my code:

    class DayPercentStrategy(bt.Strategy):
    
        params = dict(
            stake=1,
            printout=True,
        )
    
        def start(self):
            pass
    
        def stop(self):
            pass
    
        def log(self, txt, dt=None):
            if self.p.printout:
                dt = dt or self.data.datetime[0]
                dt = bt.num2date(dt)
                print('%s, %s' % (dt.isoformat(), txt))
    
        def __init__(self):
            # To control operation entries
            self.buyOrder = None
            self.sellOrder = None
    
        def next(self):
            if self.buyOrder and self.sellOrder:
                return
    
            self.buy(exectype=bt.Order.Limit,
                     size=self.p.stake,
                     price=self.data.close[0] * 0.99,
                     valid=bt.Order.DAY)
    
            self.log('BUY CREATED , %.2f' % (self.data.close[0] * 0.99))
    
            self.sell(exectype=bt.Order.Limit,
                      size=self.p.stake,
                      price=self.data.close[0] * 1.01,
                      valid=bt.Order.DAY)
    
            self.log('SELL CREATED , %.2f' % (self.data.close[0] * 1.01))
    
        def notify_order(self, order):
            if order.status in [order.Submitted, order.Accepted]:
                if order.isbuy():
                    self.buyOrder = order
                else:
                    self.sellOrder = order
                return  # Await further notifications
    
            if order.status in [order.Completed]:
                if order.isbuy():
                    self.buyOrder = None
                    buytxt = 'BUY EXECUTED, %.2f' % order.executed.price
                    self.log(buytxt, order.executed.dt)
                else:
                    self.sellOrder = None
                    selltxt = 'SELL EXECUTED, %.2f' % order.executed.price
                    self.log(selltxt, order.executed.dt)
            elif order.status in [order.Expired]:
                if order.isbuy():
                    self.buyOrder = None
                    self.log('%s , BUY ORDER' % order.Status[order.status])
                else:
                    self.sellOrder = None
                    self.log('%s , SELL ORDER' % order.Status[order.status])
                if self.position:
                    self.log('CLOSE POSITION , %.2f' % self.data.close[0])
                    self.close(exectype=bt.Order.Close)
            elif order.status in [order.Canceled, order.Margin]:
                if order.isbuy():
                    self.log('%s , BUY ORDER' % order.Status[order.status])
                    self.buyOrder = None
                else:
                    self.log('%s , SELL ORDER' % order.Status[order.status])
                    self.sellOrder = None
                pass
    
        def notify_trade(self, trade):
            if trade.isclosed:
                self.log('TRADE PROFIT, GROSS %.2f, NET %.2f' %
                         (trade.pnl, trade.pnlcomm))
    
            elif trade.justopened:
                self.log('TRADE OPENED, SIZE %2d' % trade.size)
    

    The data is as following:

    Date,Open,High,Low,Close,Volume,Adj Close
    2016-12-06,106.50,107.00,105.50,106.00,7009000,106.00
    2016-12-05,105.50,106.50,105.00,106.00,9880000,106.00
    2016-12-02,106.00,106.50,105.50,105.50,5768000,105.50
    2016-12-01,107.50,108.00,105.50,105.50,9164000,105.50
    2016-11-30,107.50,108.50,107.50,107.50,6189000,107.50
    2016-11-29,108.00,108.50,107.50,107.50,5087000,107.50
    2016-11-28,107.50,108.50,107.00,108.00,2538000,108.00
    2016-11-25,107.00,108.00,106.50,107.50,1548000,107.50
    2016-11-24,107.00,108.00,106.50,107.00,5051000,107.00
    2016-11-23,106.00,108.50,106.00,107.50,4707000,107.50
    2016-11-22,106.00,106.50,106.00,106.00,5118000,106.00
    2016-11-21,106.00,107.00,105.00,106.00,4666000,106.00
    2016-11-18,105.50,107.00,105.00,106.00,4436000,106.00
    2016-11-17,104.50,106.50,104.50,106.00,6610000,106.00
    2016-11-16,104.00,105.50,104.00,105.00,10224000,105.00
    2016-11-15,105.00,105.50,104.00,104.00,10726000,104.00
    2016-11-14,105.50,106.50,105.00,105.50,9403000,105.50
    2016-11-11,108.50,109.00,105.50,105.50,10184000,105.50
    2016-11-10,110.00,110.00,108.50,108.50,4189000,108.50
    2016-11-09,110.50,110.50,108.50,108.50,5016000,108.50
    2016-11-08,110.00,110.00,108.50,110.00,4459000,110.00
    2016-11-07,108.50,110.50,108.50,109.50,2975000,109.50
    2016-11-04,109.00,109.50,108.00,108.00,3706000,108.00
    2016-11-03,108.50,110.50,108.50,109.00,4493000,109.00
    2016-11-02,108.00,109.50,108.00,109.00,3472000,109.00
    2016-11-01,109.50,110.00,108.50,109.00,4471000,109.00
    

    and the result as follows:

    2016-11-01T23:59:59, BUY CREATED , 107.91
    2016-11-01T23:59:59, SELL CREATED , 110.09
    2016-11-02T23:59:59, Expired , BUY ORDER
    2016-11-02T23:59:59, Expired , SELL ORDER
    2016-11-02T23:59:59, BUY CREATED , 107.91
    2016-11-02T23:59:59, SELL CREATED , 110.09
    2016-11-03T23:59:59, Expired , BUY ORDER
    2016-11-03T23:59:59, Expired , SELL ORDER
    2016-11-03T23:59:59, BUY CREATED , 107.91
    2016-11-03T23:59:59, SELL CREATED , 110.09
    2016-11-04T23:59:59, Expired , BUY ORDER
    2016-11-04T23:59:59, Expired , SELL ORDER
    2016-11-04T23:59:59, BUY CREATED , 106.92
    2016-11-04T23:59:59, SELL CREATED , 109.08
    2016-11-07T23:59:59, Expired , BUY ORDER
    2016-11-07T23:59:59, Expired , SELL ORDER
    2016-11-07T23:59:59, BUY CREATED , 108.41
    2016-11-07T23:59:59, SELL CREATED , 110.59
    2016-11-08T23:59:59, Expired , BUY ORDER
    2016-11-08T23:59:59, Expired , SELL ORDER
    2016-11-08T23:59:59, BUY CREATED , 108.90
    2016-11-08T23:59:59, SELL CREATED , 111.10
    2016-11-09T23:59:59, Expired , BUY ORDER
    2016-11-09T23:59:59, Expired , SELL ORDER
    2016-11-09T23:59:59, BUY CREATED , 107.41
    2016-11-09T23:59:59, SELL CREATED , 109.59
    2016-11-10T23:59:59, Expired , BUY ORDER
    2016-11-10T23:59:59, Expired , SELL ORDER
    2016-11-10T23:59:59, BUY CREATED , 107.41
    2016-11-10T23:59:59, SELL CREATED , 109.59
    2016-11-11T23:59:59, Expired , BUY ORDER
    2016-11-11T23:59:59, Expired , SELL ORDER
    2016-11-11T23:59:59, BUY CREATED , 104.44
    2016-11-11T23:59:59, SELL CREATED , 106.56
    2016-11-14T23:59:59, Expired , BUY ORDER
    2016-11-14T23:59:59, Expired , SELL ORDER
    2016-11-14T23:59:59, BUY CREATED , 104.44
    2016-11-14T23:59:59, SELL CREATED , 106.56
    2016-11-15T23:59:59, Expired , BUY ORDER
    2016-11-15T23:59:59, Expired , SELL ORDER
    2016-11-15T23:59:59, BUY CREATED , 102.96
    2016-11-15T23:59:59, SELL CREATED , 105.04
    2016-11-16T23:59:59, Expired , BUY ORDER
    2016-11-16T23:59:59, Expired , SELL ORDER
    2016-11-16T23:59:59, BUY CREATED , 103.95
    2016-11-16T23:59:59, SELL CREATED , 106.05
    2016-11-17T23:59:59, Expired , BUY ORDER
    2016-11-17T23:59:59, Expired , SELL ORDER
    2016-11-17T23:59:59, BUY CREATED , 104.94
    2016-11-17T23:59:59, SELL CREATED , 107.06
    2016-11-18T23:59:59, Expired , BUY ORDER
    2016-11-18T23:59:59, Expired , SELL ORDER
    2016-11-18T23:59:59, BUY CREATED , 104.94
    2016-11-18T23:59:59, SELL CREATED , 107.06
    2016-11-21T23:59:59, Expired , BUY ORDER
    2016-11-21T23:59:59, Expired , SELL ORDER
    2016-11-21T23:59:59, BUY CREATED , 104.94
    2016-11-21T23:59:59, SELL CREATED , 107.06
    2016-11-22T23:59:59, Expired , BUY ORDER
    2016-11-22T23:59:59, Expired , SELL ORDER
    2016-11-22T23:59:59, BUY CREATED , 104.94
    2016-11-22T23:59:59, SELL CREATED , 107.06
    2016-11-23T23:59:59, Expired , BUY ORDER
    2016-11-23T23:59:59, Expired , SELL ORDER
    2016-11-23T23:59:59, BUY CREATED , 106.42
    2016-11-23T23:59:59, SELL CREATED , 108.58
    2016-11-24T23:59:59, Expired , BUY ORDER
    2016-11-24T23:59:59, Expired , SELL ORDER
    2016-11-24T23:59:59, BUY CREATED , 105.93
    2016-11-24T23:59:59, SELL CREATED , 108.07
    2016-11-25T23:59:59, Expired , BUY ORDER
    2016-11-25T23:59:59, Expired , SELL ORDER
    2016-11-25T23:59:59, BUY CREATED , 106.42
    2016-11-25T23:59:59, SELL CREATED , 108.58
    2016-11-28T23:59:59, Expired , BUY ORDER
    2016-11-28T23:59:59, Expired , SELL ORDER
    2016-11-28T23:59:59, BUY CREATED , 106.92
    2016-11-28T23:59:59, SELL CREATED , 109.08
    2016-11-29T23:59:59, Expired , BUY ORDER
    2016-11-29T23:59:59, Expired , SELL ORDER
    2016-11-29T23:59:59, BUY CREATED , 106.42
    2016-11-29T23:59:59, SELL CREATED , 108.58
    2016-11-30T23:59:59, Expired , BUY ORDER
    2016-11-30T23:59:59, Expired , SELL ORDER
    2016-11-30T23:59:59, BUY CREATED , 106.42
    2016-11-30T23:59:59, SELL CREATED , 108.58
    2016-12-01T23:59:59, Expired , BUY ORDER
    2016-12-01T23:59:59, Expired , SELL ORDER
    2016-12-01T23:59:59, BUY CREATED , 104.44
    2016-12-01T23:59:59, SELL CREATED , 106.56
    2016-12-02T23:59:59, Expired , BUY ORDER
    2016-12-02T23:59:59, Expired , SELL ORDER
    2016-12-02T23:59:59, BUY CREATED , 104.44
    2016-12-02T23:59:59, SELL CREATED , 106.56
    2016-12-05T23:59:59, Expired , BUY ORDER
    2016-12-05T23:59:59, Expired , SELL ORDER
    2016-12-05T23:59:59, BUY CREATED , 104.94
    2016-12-05T23:59:59, SELL CREATED , 107.06
    2016-12-06T23:59:59, Expired , BUY ORDER
    2016-12-06T23:59:59, Expired , SELL ORDER
    2016-12-06T23:59:59, BUY CREATED , 104.94
    2016-12-06T23:59:59, SELL CREATED , 107.06
    

    You can see there is no order got executed. However, when you check the bar data and the result, you can see on 2016-11-08 create buy limit order at 108.90 and it should be get execute on next day (2016-11-09) since the low on 2016-11-09 is 108.5 which is low than buy price. But the order still expired at the end on 2016-11-09. Is there something wrong that I did in my code? Thank you.


  • administrators

    Hi,

    This is a classic example of trying to do intraday things with daily data.

    You issue an order on 2016-11-08 with DAY validity. It should be no surprise that no order is executed on 2016-11-09, because it is a different day and the order was only for the day in which it was issued.

    If you would do the same thing during an intraday run, the order would be alive until the end of the session and would have many bars (for example with a 1-minute resolution run) to get executed.



  • Hi,

    Thank you for explaining. I only have daily data, but I need to perform intraday trades. I read the day steps filter method you posted on blog to simulate the daily data to intraday data. However, when I create orders in strategy next routine (below the line print('- I could issue a buy order during the Opening')), it showed the following error:

    0001,0001,0001,2016-11-01T23:59:59,109.50,109.50,109.50,109.50,0.00,0.00
    - I could issue a buy order during the Opening
    2016-11-01T23:59:59, BUY CREATED , 108.41
    2016-11-01T23:59:59, SELL CREATED , 110.59
    Traceback (most recent call last):
      File "daysteps.py", line 190, in <module>
        runstrat()
      File "daysteps.py", line 157, in runstrat
        cerebro.run(**(eval('dict(' + args.cerebro + ')')))
      File "C:\Users\yp\Anaconda2\lib\site-packages\backtrader\cerebro.py", line 788, in run
        runstrat = self.runstrategies(iterstrat)
      File "C:\Users\yp\Anaconda2\lib\site-packages\backtrader\cerebro.py", line 907, in runstrategies
        self._runnext(runstrats)
      File "C:\Users\yp\Anaconda2\lib\site-packages\backtrader\cerebro.py", line 1193, in _runnext
        self._brokernotify()
      File "C:\Users\yp\Anaconda2\lib\site-packages\backtrader\cerebro.py", line 969, in _brokernotify
        self._broker.next()
      File "C:\Users\yp\Anaconda2\lib\site-packages\backtrader\brokers\bbroker.py", line 821, in next
        self._try_exec(order)
      File "C:\Users\yp\Anaconda2\lib\site-packages\backtrader\brokers\bbroker.py", line 767, in _try_exec
        popen = data.tick_open or data.open[0]
      File "C:\Users\yp\Anaconda2\lib\site-packages\backtrader\lineseries.py", line 426, in __getattr__
        return getattr(self.lines, name)
    AttributeError: 'Lines_LineSeries_DataSeries_OHLC_OHLCDateTime_Abst' object has no attribute 'tick_open'
    

    How to solve the problem? Thank you so much.



  • Besides, I tried to run pinkfish-challenge sample which contain order creation logic, but I got an error:

    C:\Users\yp\Projects\backtrader-1.9.17.105\samples\pinkfish-challenge>python pinkfish-challenge.py --no-replay --market
    Calls,Len Strat,Len Data,Datetime,Open,High,Low,Close,Volume,OpenInterest
    Traceback (most recent call last):
      File "pinkfish-challenge.py", line 345, in <module>
        runstrat()
      File "pinkfish-challenge.py", line 280, in runstrat
        cerebro.run(runonce=False, preload=False, oldbuysell=args.oldbuysell)
      File "C:\Users\yp\Anaconda2\lib\site-packages\backtrader\cerebro.py", line 809, in run
        runstrat = self.runstrategies(iterstrat)
      File "C:\Users\yp\Anaconda2\lib\site-packages\backtrader\cerebro.py", line 931, in runstrategies
        self._runnext(runstrats)
      File "C:\Users\yp\Anaconda2\lib\site-packages\backtrader\cerebro.py", line 1158, in _runnext
        dts.append(datas[i].datetime[0] if ret else None)
      File "C:\Users\yp\Anaconda2\lib\site-packages\backtrader\linebuffer.py", line 162, in __getitem__
        return self.array[self.idx + ago]
    IndexError: array index out of range
    

    Any suggestion?


  • administrators

    For the daysteps, please refer to the source code. The sample was corrected some time ago:

    The pinkfish-challenge sample runs best without --no-replay.

    The reason is that the sample was created before the new data synchronization mechanism was created and the no-replay filter inside the sample apparently broke with the change (remove preload=False from cerebro.run(...) and it will work again)

    From a result point of view both act the same, and you get 2 heartbeats for each data. The major difference is that with replay (which is the default for the sample) you actually have intraday behavior for the daily bar which your code sees reconstructed in 2 steps.

    What I cannot understand is: "I only have daily data, but I need to perform intraday trades". Because this departs from the expectation that you issue an order on 2016-11-18 and gets executed on 2016-11-19. Those are two different days and the original problem is the validity of the order, because you want to get it executed during the next session, which doesn't happen because the validity of the order has been given as DAY



  • Hi,

    Thanks for the detail explanation. My test strategy is simple, just to issue orders according to previous day's information (for example previous close price) everyday and close any position before the end of day because I don't want to keep any position overnight. The orders I issue every day should be expired at the end of the day if they don't get executed.

    The pinkfish-challenge sample without --no-replay works correctly. I will try it to implement my test strategy.

    The daysteps sample I ran correctly without any modification, however, when I added order creation codes in next routine the error occurred. The next routine is as follows:

        def next(self):
            self.callcounter += 1
    
            txtfields = list()
            txtfields.append('%04d' % self.callcounter)
            txtfields.append('%04d' % len(self))
            txtfields.append('%04d' % len(self.data0))
            txtfields.append(self.data.datetime.datetime(0).isoformat())
            txtfields.append('%.2f' % self.data0.open[0])
            txtfields.append('%.2f' % self.data0.high[0])
            txtfields.append('%.2f' % self.data0.low[0])
            txtfields.append('%.2f' % self.data0.close[0])
            txtfields.append('%.2f' % self.data0.volume[0])
            txtfields.append('%.2f' % self.data0.openinterest[0])
            print(','.join(txtfields))
    
            if self.buyOrder and self.sellOrder:
                return
    
            if len(self.data) > self.lcontrol:
                print('- I could issue a buy order during the Opening')
                self.buy(exectype=bt.Order.Limit,
                         size=self.p.stake,
                         price=self.data.close[0] * 0.99,
                         valid=bt.Order.DAY)
    
                self.log('BUY CREATED , %.2f' % (self.data.close[0] * 0.99))
    
                self.sell(exectype=bt.Order.Limit,
                          size=self.p.stake,
                          price=self.data.close[0] * 1.01,
                          valid=bt.Order.DAY)
    
                self.log('SELL CREATED , %.2f' % (self.data.close[0] * 1.01))
    
            self.lcontrol = len(self.data)
    

    The added part is:

            if self.buyOrder and self.sellOrder:
                return
    
            if len(self.data) > self.lcontrol:
                print('- I could issue a buy order during the Opening')
                self.buy(exectype=bt.Order.Limit,
                         size=self.p.stake,
                         price=self.data.close[0] * 0.99,
                         valid=bt.Order.DAY)
    
                self.log('BUY CREATED , %.2f' % (self.data.close[0] * 0.99))
    
                self.sell(exectype=bt.Order.Limit,
                          size=self.p.stake,
                          price=self.data.close[0] * 1.01,
                          valid=bt.Order.DAY)
    
                self.log('SELL CREATED , %.2f' % (self.data.close[0] * 1.01))
    


  • Besides, I tried a simpler approach just to modify the orders validity using below code:

    valid=self.data.datetime.date(0) + datetime.timedelta(days=1)
    

    But I got an error:

    2016-11-01T23:59:59, BUY CREATED , 108.41
    2016-11-01T23:59:59, SELL CREATED , 110.59
    Traceback (most recent call last):
      File "day-percent.py", line 227, in <module>
        runstrategy()
      File "day-percent.py", line 169, in runstrategy
        cerebro.run()
      File "C:\Users\yp\Anaconda2\lib\site-packages\backtrader\cerebro.py", line 809, in run
        runstrat = self.runstrategies(iterstrat)
      File "C:\Users\yp\Anaconda2\lib\site-packages\backtrader\cerebro.py", line 931, in runstrategies
        self._runnext(runstrats)
      File "C:\Users\yp\Anaconda2\lib\site-packages\backtrader\cerebro.py", line 1217, in _runnext
        self._brokernotify()
      File "C:\Users\yp\Anaconda2\lib\site-packages\backtrader\cerebro.py", line 993, in _brokernotify
        self._broker.next()
      File "C:\Users\yp\Anaconda2\lib\site-packages\backtrader\brokers\bbroker.py", line 821, in next
        if order.expire():
      File "C:\Users\yp\Anaconda2\lib\site-packages\backtrader\order.py", line 522, in expire
        if self.valid and self.data.datetime[0] > self.valid:
    TypeError: can't compare datetime.date to float
    

    It is strange because I just copied and pasted the order validity code (above line) from the document. Should it work correctly?


  • administrators

    Problem seen. Because the filter is breaking down the data, the ticks don't get filled in the platform. And the broker tries to use the best possible information to fill orders.

    If your approach is to issue an order but have it expired at the end of the next trading day, I believe the best approach would be to set the right validity for the order, rather than try to experiment with replaying.

    Setting a validity of today + 24 hours would seem the most straightforward approach, but weekends and bank holidays would not be accounted for.

    The best approach, imho, would be a trading calendar which says 2 things:

    • Which days are not valid for trading

    By asking the calendar, you would quickly know which one is the next valid trading day and you can set the order validity accordingly.



  • Hello,

    Thanks for suggestion. I tried order-execution.py sample without any modification as follows:

    python order-execution.py --exectype Limit --perc1 1 --valid 4
    

    But got the same error that I posted above:

    C:\Users\yp\Projects\backtrader-1.9.17.105\samples\order-execution>python order-execution.py --exectype Limit --perc1 1 --valid 4
    2006-01-26T23:59:59, BUY CREATE, exectype Limit, price 3605.01, valid: 2006-01-30
    Traceback (most recent call last):
      File "order-execution.py", line 269, in <module>
        runstrat()
      File "order-execution.py", line 180, in runstrat
        cerebro.run()
      File "C:\Users\yp\Anaconda2\lib\site-packages\backtrader\cerebro.py", line 809, in run
        runstrat = self.runstrategies(iterstrat)
      File "C:\Users\yp\Anaconda2\lib\site-packages\backtrader\cerebro.py", line 926, in runstrategies
        self._runonce(runstrats)
      File "C:\Users\yp\Anaconda2\lib\site-packages\backtrader\cerebro.py", line 1271, in _runonce
        self._brokernotify()
      File "C:\Users\yp\Anaconda2\lib\site-packages\backtrader\cerebro.py", line 993, in _brokernotify
        self._broker.next()
      File "C:\Users\yp\Anaconda2\lib\site-packages\backtrader\brokers\bbroker.py", line 821, in next
        if order.expire():
      File "C:\Users\yp\Anaconda2\lib\site-packages\backtrader\order.py", line 522, in expire
        if self.valid and self.data.datetime[0] > self.valid:
    TypeError: can't compare datetime.date to float
    

    Any idea?

    Besides, how do you think to add a bar count in order valid parameter, that is, the order will be expired in next n bar? By the way, users won't worry about the weekend and holiday issues since historical data only contain traded data.

    Thank you.


  • administrators

    The sample and the order validity inputs got out of sync (yes, code evolves and not all samples are re-run each and every time)

    Fixing it in the sample is rather simple, but I will also add extra input check in the order.

    In the sample to support your scenario. Change at line 104:

                    valid = self.data.datetime.date(0) + \
                            datetime.timedelta(days=self.p.valid)
    

    to

                    valid = self.data.datetime.datetime(0) + \
                            datetime.timedelta(days=self.p.valid)
                    valid = valid.replace(hour=0, minute=0, second=0)
    

    Indeed, the data contains only trading data. But the bars between a Friday and a Monday are 3 days apart. If you tell the platform to expire after 24 hours, any order issued on a Friday or right before a bank holiday, will always expire before being considered.



  • Hello,

    I followed your suggestion to set order to expire after 24 hours, but the result is the same with I set order validity to Order.DAY, that is no order will get executed.

    2016-11-01T23:59:59, BUY CREATED , 107.91
    2016-11-01T23:59:59, SELL CREATED , 110.09
    2016-11-02T23:59:59, Expired , BUY ORDER
    2016-11-02T23:59:59, Expired , SELL ORDER
    2016-11-02T23:59:59, BUY CREATED , 107.91
    2016-11-02T23:59:59, SELL CREATED , 110.09
    2016-11-03T23:59:59, Expired , BUY ORDER
    2016-11-03T23:59:59, Expired , SELL ORDER
    2016-11-03T23:59:59, BUY CREATED , 107.91
    2016-11-03T23:59:59, SELL CREATED , 110.09
    2016-11-04T23:59:59, Expired , BUY ORDER
    2016-11-04T23:59:59, Expired , SELL ORDER
    2016-11-04T23:59:59, BUY CREATED , 106.92
    2016-11-04T23:59:59, SELL CREATED , 109.08
    2016-11-07T23:59:59, Expired , BUY ORDER
    2016-11-07T23:59:59, Expired , SELL ORDER
    2016-11-07T23:59:59, BUY CREATED , 108.41
    2016-11-07T23:59:59, SELL CREATED , 110.59
    2016-11-08T23:59:59, Expired , BUY ORDER
    2016-11-08T23:59:59, Expired , SELL ORDER
    2016-11-08T23:59:59, BUY CREATED , 108.90
    2016-11-08T23:59:59, SELL CREATED , 111.10
    2016-11-09T23:59:59, Expired , BUY ORDER
    2016-11-09T23:59:59, Expired , SELL ORDER
    2016-11-09T23:59:59, BUY CREATED , 107.41
    2016-11-09T23:59:59, SELL CREATED , 109.59
    

    This is because the order execute logic seems to expire first and then execute if these two events occur at the same time (23:59:59).

    If I set order to expire after more than 24 hours (for example, 24 hours plus 1 second), the order will expire on third day. That is not the result what I expected. For example:

    2016-11-01T23:59:59, BUY CREATED , 107.91
    2016-11-01T23:59:59, SELL CREATED , 110.09
    2016-11-02T23:59:59, BUY CREATED , 107.91
    2016-11-02T23:59:59, SELL CREATED , 110.09
    2016-11-03T23:59:59, Expired , BUY ORDER
    2016-11-03T23:59:59, CLOSE POSITION , 109.00
    2016-11-03T23:59:59, Expired , SELL ORDER
    2016-11-03T23:59:59, CLOSE POSITION , 109.00
    2016-11-03T23:59:59, SELL EXECUTED, 110.09
    2016-11-03T23:59:59, TRADE OPENED, SIZE -1
    2016-11-03T23:59:59, BUY CREATED , 107.91
    2016-11-03T23:59:59, SELL CREATED , 110.09
    2016-11-04T23:59:59, Expired , BUY ORDER
    2016-11-04T23:59:59, CLOSE POSITION , 108.00
    2016-11-04T23:59:59, BUY EXECUTED, 108.00
    2016-11-04T23:59:59, BUY EXECUTED, 108.00
    2016-11-04T23:59:59, TRADE PROFIT, GROSS 2.09, NET 0.78
    

    You can see the first buy order will expire at 2016-11-03T23:59:59.

    Is it possible to change the order execution logic to execute first and then expire (if order not got executed) when these two events occur at the same time?


  • administrators

    There is a much easier approach, imho. Set the time at which the session ends.

    During the creation of the data feed, for example

    data = bt.YourFeed(dataname=yourname, sessionend=datetime.time(17, 30))
    


  • Hello,

    It seems still can't produce expecting result. The result is the same, that is still no order got executed, because these two events occur at 17:30 at the same time, and order expired before got executed. The result is as follows:

    2016-11-01T17:30:00, BUY CREATED , 107.91
    2016-11-01T17:30:00, SELL CREATED , 110.09
    2016-11-02T17:30:00, Expired , BUY ORDER
    2016-11-02T17:30:00, Expired , SELL ORDER
    2016-11-02T17:30:00, BUY CREATED , 107.91
    2016-11-02T17:30:00, SELL CREATED , 110.09
    2016-11-03T17:30:00, Expired , BUY ORDER
    2016-11-03T17:30:00, Expired , SELL ORDER
    2016-11-03T17:30:00, BUY CREATED , 107.91
    2016-11-03T17:30:00, SELL CREATED , 110.09
    2016-11-04T17:30:00, Expired , BUY ORDER
    2016-11-04T17:30:00, Expired , SELL ORDER
    2016-11-04T17:30:00, BUY CREATED , 106.92
    2016-11-04T17:30:00, SELL CREATED , 109.08
    2016-11-07T17:30:00, Expired , BUY ORDER
    2016-11-07T17:30:00, Expired , SELL ORDER
    2016-11-07T17:30:00, BUY CREATED , 108.41
    2016-11-07T17:30:00, SELL CREATED , 110.59
    2016-11-08T17:30:00, Expired , BUY ORDER
    2016-11-08T17:30:00, Expired , SELL ORDER
    2016-11-08T17:30:00, BUY CREATED , 108.90
    2016-11-08T17:30:00, SELL CREATED , 111.10
    2016-11-09T17:30:00, Expired , BUY ORDER
    2016-11-09T17:30:00, Expired , SELL ORDER
    2016-11-09T17:30:00, BUY CREATED , 107.41
    2016-11-09T17:30:00, SELL CREATED , 109.59
    2016-11-10T17:30:00, Expired , BUY ORDER
    2016-11-10T17:30:00, Expired , SELL ORDER
    2016-11-10T17:30:00, BUY CREATED , 107.41
    2016-11-10T17:30:00, SELL CREATED , 109.59
    2016-11-11T17:30:00, Expired , BUY ORDER
    2016-11-11T17:30:00, Expired , SELL ORDER
    2016-11-11T17:30:00, BUY CREATED , 104.44
    2016-11-11T17:30:00, SELL CREATED , 106.56
    2016-11-14T17:30:00, Expired , BUY ORDER
    2016-11-14T17:30:00, Expired , SELL ORDER
    2016-11-14T17:30:00, BUY CREATED , 104.44
    2016-11-14T17:30:00, SELL CREATED , 106.56
    2016-11-15T17:30:00, Expired , BUY ORDER
    2016-11-15T17:30:00, Expired , SELL ORDER
    2016-11-15T17:30:00, BUY CREATED , 102.96
    2016-11-15T17:30:00, SELL CREATED , 105.04
    2016-11-16T17:30:00, Expired , BUY ORDER
    2016-11-16T17:30:00, Expired , SELL ORDER
    2016-11-16T17:30:00, BUY CREATED , 103.95
    2016-11-16T17:30:00, SELL CREATED , 106.05
    2016-11-17T17:30:00, Expired , BUY ORDER
    2016-11-17T17:30:00, Expired , SELL ORDER
    2016-11-17T17:30:00, BUY CREATED , 104.94
    2016-11-17T17:30:00, SELL CREATED , 107.06
    2016-11-18T17:30:00, Expired , BUY ORDER
    2016-11-18T17:30:00, Expired , SELL ORDER
    2016-11-18T17:30:00, BUY CREATED , 104.94
    2016-11-18T17:30:00, SELL CREATED , 107.06
    2016-11-21T17:30:00, Expired , BUY ORDER
    2016-11-21T17:30:00, Expired , SELL ORDER
    2016-11-21T17:30:00, BUY CREATED , 104.94
    2016-11-21T17:30:00, SELL CREATED , 107.06
    2016-11-22T17:30:00, Expired , BUY ORDER
    2016-11-22T17:30:00, Expired , SELL ORDER
    2016-11-22T17:30:00, BUY CREATED , 104.94
    2016-11-22T17:30:00, SELL CREATED , 107.06
    2016-11-23T17:30:00, Expired , BUY ORDER
    2016-11-23T17:30:00, Expired , SELL ORDER
    2016-11-23T17:30:00, BUY CREATED , 106.42
    2016-11-23T17:30:00, SELL CREATED , 108.58
    2016-11-24T17:30:00, Expired , BUY ORDER
    2016-11-24T17:30:00, Expired , SELL ORDER
    2016-11-24T17:30:00, BUY CREATED , 105.93
    2016-11-24T17:30:00, SELL CREATED , 108.07
    2016-11-25T17:30:00, Expired , BUY ORDER
    2016-11-25T17:30:00, Expired , SELL ORDER
    2016-11-25T17:30:00, BUY CREATED , 106.42
    2016-11-25T17:30:00, SELL CREATED , 108.58
    2016-11-28T17:30:00, Expired , BUY ORDER
    2016-11-28T17:30:00, Expired , SELL ORDER
    2016-11-28T17:30:00, BUY CREATED , 106.92
    2016-11-28T17:30:00, SELL CREATED , 109.08
    2016-11-29T17:30:00, Expired , BUY ORDER
    2016-11-29T17:30:00, Expired , SELL ORDER
    2016-11-29T17:30:00, BUY CREATED , 106.42
    2016-11-29T17:30:00, SELL CREATED , 108.58
    2016-11-30T17:30:00, Expired , BUY ORDER
    2016-11-30T17:30:00, Expired , SELL ORDER
    2016-11-30T17:30:00, BUY CREATED , 106.42
    2016-11-30T17:30:00, SELL CREATED , 108.58
    2016-12-01T17:30:00, Expired , BUY ORDER
    2016-12-01T17:30:00, Expired , SELL ORDER
    2016-12-01T17:30:00, BUY CREATED , 104.44
    2016-12-01T17:30:00, SELL CREATED , 106.56
    2016-12-02T17:30:00, Expired , BUY ORDER
    2016-12-02T17:30:00, Expired , SELL ORDER
    2016-12-02T17:30:00, BUY CREATED , 104.44
    2016-12-02T17:30:00, SELL CREATED , 106.56
    2016-12-05T17:30:00, Expired , BUY ORDER
    2016-12-05T17:30:00, Expired , SELL ORDER
    2016-12-05T17:30:00, BUY CREATED , 104.94
    2016-12-05T17:30:00, SELL CREATED , 107.06
    2016-12-06T17:30:00, Expired , BUY ORDER
    2016-12-06T17:30:00, Expired , SELL ORDER
    2016-12-06T17:30:00, BUY CREATED , 104.94
    2016-12-06T17:30:00, SELL CREATED , 107.06
    

    Since the order created at 17:30, I've tried to set order to expire after more than 24 hours (i.e. 25 hours), the result is the same as above (no order got executed). Order still gets expired at the time of end of session, not the valid parameter value that I had set.


  • administrators

    The formulation above wasn't precise enough ... the implied meaning with 24 hours was:

    • Put the expiration order at the end of next day

    Sample snippet in the strategy:

    
    import datetime
    
    ....
    
    # Get next day
    eofthisday = datetime.datetime.combine(self.data.datetime.date(), datetime.time(23, 59, 59, 999)
    eofnextday = eofthisday + datetime.timedelta(days=1)
    self.buy(size=x, price=y, valid=eofnextday)
    

    The reason to use datetime.time(23, 59, 59, 999) rather than datetime.time.max is to avoid falling prey to floating point precision errors. The platform uses the matplolib convention to store time in a float and this may sometimes overflow.

    Some extra checks will also go into the next release to improve order validity checking.



  • Thanks for explanation. Hope next release will resolve the problem. However, this is a great platform anyway.


  • administrators

    The problem is that there is no actual problem. A DAY validity is meant for the current day and not for the next and it is useful for intraday operations.

    Your use case operates on daily data and you need an expiration which is the end of next day. And the end of the next day is a datetime value which has to be input by the user.

    Because the next trading day must no necessarily be the actual next day. It can well be 4 days apart if you put a weekend a 1 bank holiday together.

    The usual broker validities are:

    • DAY: will expire at the end of the day/session
    • GTD: good til date
    • GTC: good until canceled (for practical purposes the broker usually sets a far away date, so this is a long term GTD)

    There isn't a: NEXTDAY type. GTD is what you pass with the end of the next day as the validity is the way for your use case.



  • Hello,

    Thanks. I used your code snippet in my test strategy to put the expiration order at the end of next day. However, orders still got expired at the end of day on the third day and that is not I expect.

    It seems too complicated for a simple strategy to incorporate with a trading calendar to determine an order's validity. I will try your filter to simulate the daily data to intraday data to see if it works in my use case. Thanks again.


  • administrators

    The snippet above is meant to have the order expiring at the end of the next day, which was your use case above. As such the order will obviously be expired for the third day. To have the order still active, you would need to add extra days to the datetime.timedelta.

    If your operational plan for live trading includes having the orders expire after x days, you are going to have to deal with trading calendars yourself.

    There may be brokers which offer something like: VALID FOR THE NEXT 3 TRADING SESSIONS, which in theory would skip bank holidays and weekends. If you know of such a broker, just post it here.



  • Hi,

    I've tried pinkfish example and it works so far except an issue regarding order Margin status. I used pinkfish example to split the daily data to two bars per day and issue buy and sell orders as above at first bar. The result is as follows:

    2016-11-01T00:00:00,109.50,110.00,108.50,109.33
    2016-11-01T00:00:00, BUY CREATED , 108.24
    2016-11-01T00:00:00, SELL CREATED , 110.43
    2016-11-01T13:30:00, Margin , BUY ORDER
    2016-11-01T13:30:00,109.50,110.00,108.50,109.00
    2016-11-02T00:00:00, SELL Order Expired
    2016-11-02T00:00:00,108.00,109.50,108.00,108.50
    2016-11-02T00:00:00, BUY CREATED , 107.91
    2016-11-02T00:00:00, SELL CREATED , 110.09
    2016-11-02T13:30:00, Margin , BUY ORDER
    2016-11-02T13:30:00,108.00,109.50,108.00,109.00
    2016-11-03T00:00:00, SELL Order Expired
    2016-11-03T00:00:00,108.50,110.50,108.50,109.17
    2016-11-03T00:00:00, BUY CREATED , 107.91
    2016-11-03T00:00:00, SELL CREATED , 110.09
    2016-11-03T13:30:00, Margin , BUY ORDER
    2016-11-03T13:30:00, SELL Executed, 110.09
    2016-11-03T13:30:00,108.50,110.50,108.50,109.00
    2016-11-04T00:00:00,109.00,109.50,108.00,108.83
    2016-11-04T00:00:00, BUY CREATED , 107.91
    2016-11-04T00:00:00, SELL CREATED , 110.09
    2016-11-04T13:30:00,109.00,109.50,108.00,108.00
    2016-11-07T00:00:00, BUY Order Expired
    2016-11-07T00:00:00, CLOSE POSITION , 109.17
    2016-11-07T00:00:00, SELL Order Expired
    2016-11-07T00:00:00, CLOSE POSITION , 109.17
    

    Sell orders work nicely. However, buy orders always have Margin status and are taken off from the system. Why this could happen? I've read Order.Margin description but don't really understand what is means? How can I do to let buy orders work as the sell orders? Thanks.


  • administrators

    The default mode of the broker unless you say otherwise with a defined comission scheme is to work with stock-like products.

    In this scenario when you sell something you get cash from the action of selling that something. It is difficult to get a margin call if you are getting cash injected.

    When you buy cash is detracted from your account. Your description doesn't indicate what your stake is and how much cash you have configured the broker with (the default is 10000). Without code and details one cannot say what's going wrong.