Thanks for the detail explanation, after I set cash value, it works perfectly. Thanks again.

Hi,

I've tried pinkfish example and it works so far except an issue regarding order Margin status. I used pinkfish example to split the daily data to two bars per day and issue buy and sell orders as above at first bar. The result is as follows:

2016-11-01T00:00:00,109.50,110.00,108.50,109.33
2016-11-01T00:00:00, BUY CREATED , 108.24
2016-11-01T00:00:00, SELL CREATED , 110.43
2016-11-01T13:30:00, Margin , BUY ORDER
2016-11-01T13:30:00,109.50,110.00,108.50,109.00
2016-11-02T00:00:00, SELL Order Expired
2016-11-02T00:00:00,108.00,109.50,108.00,108.50
2016-11-02T00:00:00, BUY CREATED , 107.91
2016-11-02T00:00:00, SELL CREATED , 110.09
2016-11-02T13:30:00, Margin , BUY ORDER
2016-11-02T13:30:00,108.00,109.50,108.00,109.00
2016-11-03T00:00:00, SELL Order Expired
2016-11-03T00:00:00,108.50,110.50,108.50,109.17
2016-11-03T00:00:00, BUY CREATED , 107.91
2016-11-03T00:00:00, SELL CREATED , 110.09
2016-11-03T13:30:00, Margin , BUY ORDER
2016-11-03T13:30:00, SELL Executed, 110.09
2016-11-03T13:30:00,108.50,110.50,108.50,109.00
2016-11-04T00:00:00,109.00,109.50,108.00,108.83
2016-11-04T00:00:00, BUY CREATED , 107.91
2016-11-04T00:00:00, SELL CREATED , 110.09
2016-11-04T13:30:00,109.00,109.50,108.00,108.00
2016-11-07T00:00:00, BUY Order Expired
2016-11-07T00:00:00, CLOSE POSITION , 109.17
2016-11-07T00:00:00, SELL Order Expired
2016-11-07T00:00:00, CLOSE POSITION , 109.17

Sell orders work nicely. However, buy orders always have Margin status and are taken off from the system. Why this could happen? I've read Order.Margin description but don't really understand what is means? How can I do to let buy orders work as the sell orders? Thanks.

Hello,

Thanks. I used your code snippet in my test strategy to put the expiration order at the end of next day. However, orders still got expired at the end of day on the third day and that is not I expect.

It seems too complicated for a simple strategy to incorporate with a trading calendar to determine an order's validity. I will try your filter to simulate the daily data to intraday data to see if it works in my use case. Thanks again.

Thanks for explanation. Hope next release will resolve the problem. However, this is a great platform anyway.

Hello,

It seems still can't produce expecting result. The result is the same, that is still no order got executed, because these two events occur at 17:30 at the same time, and order expired before got executed. The result is as follows:

2016-11-01T17:30:00, BUY CREATED , 107.91
2016-11-01T17:30:00, SELL CREATED , 110.09
2016-11-02T17:30:00, Expired , BUY ORDER
2016-11-02T17:30:00, Expired , SELL ORDER
2016-11-02T17:30:00, BUY CREATED , 107.91
2016-11-02T17:30:00, SELL CREATED , 110.09
2016-11-03T17:30:00, Expired , BUY ORDER
2016-11-03T17:30:00, Expired , SELL ORDER
2016-11-03T17:30:00, BUY CREATED , 107.91
2016-11-03T17:30:00, SELL CREATED , 110.09
2016-11-04T17:30:00, Expired , BUY ORDER
2016-11-04T17:30:00, Expired , SELL ORDER
2016-11-04T17:30:00, BUY CREATED , 106.92
2016-11-04T17:30:00, SELL CREATED , 109.08
2016-11-07T17:30:00, Expired , BUY ORDER
2016-11-07T17:30:00, Expired , SELL ORDER
2016-11-07T17:30:00, BUY CREATED , 108.41
2016-11-07T17:30:00, SELL CREATED , 110.59
2016-11-08T17:30:00, Expired , BUY ORDER
2016-11-08T17:30:00, Expired , SELL ORDER
2016-11-08T17:30:00, BUY CREATED , 108.90
2016-11-08T17:30:00, SELL CREATED , 111.10
2016-11-09T17:30:00, Expired , BUY ORDER
2016-11-09T17:30:00, Expired , SELL ORDER
2016-11-09T17:30:00, BUY CREATED , 107.41
2016-11-09T17:30:00, SELL CREATED , 109.59
2016-11-10T17:30:00, Expired , BUY ORDER
2016-11-10T17:30:00, Expired , SELL ORDER
2016-11-10T17:30:00, BUY CREATED , 107.41
2016-11-10T17:30:00, SELL CREATED , 109.59
2016-11-11T17:30:00, Expired , BUY ORDER
2016-11-11T17:30:00, Expired , SELL ORDER
2016-11-11T17:30:00, BUY CREATED , 104.44
2016-11-11T17:30:00, SELL CREATED , 106.56
2016-11-14T17:30:00, Expired , BUY ORDER
2016-11-14T17:30:00, Expired , SELL ORDER
2016-11-14T17:30:00, BUY CREATED , 104.44
2016-11-14T17:30:00, SELL CREATED , 106.56
2016-11-15T17:30:00, Expired , BUY ORDER
2016-11-15T17:30:00, Expired , SELL ORDER
2016-11-15T17:30:00, BUY CREATED , 102.96
2016-11-15T17:30:00, SELL CREATED , 105.04
2016-11-16T17:30:00, Expired , BUY ORDER
2016-11-16T17:30:00, Expired , SELL ORDER
2016-11-16T17:30:00, BUY CREATED , 103.95
2016-11-16T17:30:00, SELL CREATED , 106.05
2016-11-17T17:30:00, Expired , BUY ORDER
2016-11-17T17:30:00, Expired , SELL ORDER
2016-11-17T17:30:00, BUY CREATED , 104.94
2016-11-17T17:30:00, SELL CREATED , 107.06
2016-11-18T17:30:00, Expired , BUY ORDER
2016-11-18T17:30:00, Expired , SELL ORDER
2016-11-18T17:30:00, BUY CREATED , 104.94
2016-11-18T17:30:00, SELL CREATED , 107.06
2016-11-21T17:30:00, Expired , BUY ORDER
2016-11-21T17:30:00, Expired , SELL ORDER
2016-11-21T17:30:00, BUY CREATED , 104.94
2016-11-21T17:30:00, SELL CREATED , 107.06
2016-11-22T17:30:00, Expired , BUY ORDER
2016-11-22T17:30:00, Expired , SELL ORDER
2016-11-22T17:30:00, BUY CREATED , 104.94
2016-11-22T17:30:00, SELL CREATED , 107.06
2016-11-23T17:30:00, Expired , BUY ORDER
2016-11-23T17:30:00, Expired , SELL ORDER
2016-11-23T17:30:00, BUY CREATED , 106.42
2016-11-23T17:30:00, SELL CREATED , 108.58
2016-11-24T17:30:00, Expired , BUY ORDER
2016-11-24T17:30:00, Expired , SELL ORDER
2016-11-24T17:30:00, BUY CREATED , 105.93
2016-11-24T17:30:00, SELL CREATED , 108.07
2016-11-25T17:30:00, Expired , BUY ORDER
2016-11-25T17:30:00, Expired , SELL ORDER
2016-11-25T17:30:00, BUY CREATED , 106.42
2016-11-25T17:30:00, SELL CREATED , 108.58
2016-11-28T17:30:00, Expired , BUY ORDER
2016-11-28T17:30:00, Expired , SELL ORDER
2016-11-28T17:30:00, BUY CREATED , 106.92
2016-11-28T17:30:00, SELL CREATED , 109.08
2016-11-29T17:30:00, Expired , BUY ORDER
2016-11-29T17:30:00, Expired , SELL ORDER
2016-11-29T17:30:00, BUY CREATED , 106.42
2016-11-29T17:30:00, SELL CREATED , 108.58
2016-11-30T17:30:00, Expired , BUY ORDER
2016-11-30T17:30:00, Expired , SELL ORDER
2016-11-30T17:30:00, BUY CREATED , 106.42
2016-11-30T17:30:00, SELL CREATED , 108.58
2016-12-01T17:30:00, Expired , BUY ORDER
2016-12-01T17:30:00, Expired , SELL ORDER
2016-12-01T17:30:00, BUY CREATED , 104.44
2016-12-01T17:30:00, SELL CREATED , 106.56
2016-12-02T17:30:00, Expired , BUY ORDER
2016-12-02T17:30:00, Expired , SELL ORDER
2016-12-02T17:30:00, BUY CREATED , 104.44
2016-12-02T17:30:00, SELL CREATED , 106.56
2016-12-05T17:30:00, Expired , BUY ORDER
2016-12-05T17:30:00, Expired , SELL ORDER
2016-12-05T17:30:00, BUY CREATED , 104.94
2016-12-05T17:30:00, SELL CREATED , 107.06
2016-12-06T17:30:00, Expired , BUY ORDER
2016-12-06T17:30:00, Expired , SELL ORDER
2016-12-06T17:30:00, BUY CREATED , 104.94
2016-12-06T17:30:00, SELL CREATED , 107.06

Since the order created at 17:30, I've tried to set order to expire after more than 24 hours (i.e. 25 hours), the result is the same as above (no order got executed). Order still gets expired at the time of end of session, not the valid parameter value that I had set.

Hello,

I followed your suggestion to set order to expire after 24 hours, but the result is the same with I set order validity to Order.DAY, that is no order will get executed.

2016-11-01T23:59:59, BUY CREATED , 107.91
2016-11-01T23:59:59, SELL CREATED , 110.09
2016-11-02T23:59:59, Expired , BUY ORDER
2016-11-02T23:59:59, Expired , SELL ORDER
2016-11-02T23:59:59, BUY CREATED , 107.91
2016-11-02T23:59:59, SELL CREATED , 110.09
2016-11-03T23:59:59, Expired , BUY ORDER
2016-11-03T23:59:59, Expired , SELL ORDER
2016-11-03T23:59:59, BUY CREATED , 107.91
2016-11-03T23:59:59, SELL CREATED , 110.09
2016-11-04T23:59:59, Expired , BUY ORDER
2016-11-04T23:59:59, Expired , SELL ORDER
2016-11-04T23:59:59, BUY CREATED , 106.92
2016-11-04T23:59:59, SELL CREATED , 109.08
2016-11-07T23:59:59, Expired , BUY ORDER
2016-11-07T23:59:59, Expired , SELL ORDER
2016-11-07T23:59:59, BUY CREATED , 108.41
2016-11-07T23:59:59, SELL CREATED , 110.59
2016-11-08T23:59:59, Expired , BUY ORDER
2016-11-08T23:59:59, Expired , SELL ORDER
2016-11-08T23:59:59, BUY CREATED , 108.90
2016-11-08T23:59:59, SELL CREATED , 111.10
2016-11-09T23:59:59, Expired , BUY ORDER
2016-11-09T23:59:59, Expired , SELL ORDER
2016-11-09T23:59:59, BUY CREATED , 107.41
2016-11-09T23:59:59, SELL CREATED , 109.59

This is because the order execute logic seems to expire first and then execute if these two events occur at the same time (23:59:59).

If I set order to expire after more than 24 hours (for example, 24 hours plus 1 second), the order will expire on third day. That is not the result what I expected. For example:

2016-11-01T23:59:59, BUY CREATED , 107.91
2016-11-01T23:59:59, SELL CREATED , 110.09
2016-11-02T23:59:59, BUY CREATED , 107.91
2016-11-02T23:59:59, SELL CREATED , 110.09
2016-11-03T23:59:59, Expired , BUY ORDER
2016-11-03T23:59:59, CLOSE POSITION , 109.00
2016-11-03T23:59:59, Expired , SELL ORDER
2016-11-03T23:59:59, CLOSE POSITION , 109.00
2016-11-03T23:59:59, SELL EXECUTED, 110.09
2016-11-03T23:59:59, TRADE OPENED, SIZE -1
2016-11-03T23:59:59, BUY CREATED , 107.91
2016-11-03T23:59:59, SELL CREATED , 110.09
2016-11-04T23:59:59, Expired , BUY ORDER
2016-11-04T23:59:59, CLOSE POSITION , 108.00
2016-11-04T23:59:59, BUY EXECUTED, 108.00
2016-11-04T23:59:59, BUY EXECUTED, 108.00
2016-11-04T23:59:59, TRADE PROFIT, GROSS 2.09, NET 0.78

You can see the first buy order will expire at 2016-11-03T23:59:59.

Is it possible to change the order execution logic to execute first and then expire (if order not got executed) when these two events occur at the same time?

Hello,

Thanks for suggestion. I tried order-execution.py sample without any modification as follows:

python order-execution.py --exectype Limit --perc1 1 --valid 4

But got the same error that I posted above:

C:\Users\yp\Projects\backtrader-1.9.17.105\samples\order-execution>python order-execution.py --exectype Limit --perc1 1 --valid 4
2006-01-26T23:59:59, BUY CREATE, exectype Limit, price 3605.01, valid: 2006-01-30
Traceback (most recent call last):
  File "order-execution.py", line 269, in <module>
    runstrat()
  File "order-execution.py", line 180, in runstrat
    cerebro.run()
  File "C:\Users\yp\Anaconda2\lib\site-packages\backtrader\cerebro.py", line 809, in run
    runstrat = self.runstrategies(iterstrat)
  File "C:\Users\yp\Anaconda2\lib\site-packages\backtrader\cerebro.py", line 926, in runstrategies
    self._runonce(runstrats)
  File "C:\Users\yp\Anaconda2\lib\site-packages\backtrader\cerebro.py", line 1271, in _runonce
    self._brokernotify()
  File "C:\Users\yp\Anaconda2\lib\site-packages\backtrader\cerebro.py", line 993, in _brokernotify
    self._broker.next()
  File "C:\Users\yp\Anaconda2\lib\site-packages\backtrader\brokers\bbroker.py", line 821, in next
    if order.expire():
  File "C:\Users\yp\Anaconda2\lib\site-packages\backtrader\order.py", line 522, in expire
    if self.valid and self.data.datetime[0] > self.valid:
TypeError: can't compare datetime.date to float

Any idea?

Besides, how do you think to add a bar count in order valid parameter, that is, the order will be expired in next n bar? By the way, users won't worry about the weekend and holiday issues since historical data only contain traded data.

Thank you.

Besides, I tried a simpler approach just to modify the orders validity using below code:

valid=self.data.datetime.date(0) + datetime.timedelta(days=1)

But I got an error:

2016-11-01T23:59:59, BUY CREATED , 108.41
2016-11-01T23:59:59, SELL CREATED , 110.59
Traceback (most recent call last):
  File "day-percent.py", line 227, in <module>
    runstrategy()
  File "day-percent.py", line 169, in runstrategy
    cerebro.run()
  File "C:\Users\yp\Anaconda2\lib\site-packages\backtrader\cerebro.py", line 809, in run
    runstrat = self.runstrategies(iterstrat)
  File "C:\Users\yp\Anaconda2\lib\site-packages\backtrader\cerebro.py", line 931, in runstrategies
    self._runnext(runstrats)
  File "C:\Users\yp\Anaconda2\lib\site-packages\backtrader\cerebro.py", line 1217, in _runnext
    self._brokernotify()
  File "C:\Users\yp\Anaconda2\lib\site-packages\backtrader\cerebro.py", line 993, in _brokernotify
    self._broker.next()
  File "C:\Users\yp\Anaconda2\lib\site-packages\backtrader\brokers\bbroker.py", line 821, in next
    if order.expire():
  File "C:\Users\yp\Anaconda2\lib\site-packages\backtrader\order.py", line 522, in expire
    if self.valid and self.data.datetime[0] > self.valid:
TypeError: can't compare datetime.date to float

It is strange because I just copied and pasted the order validity code (above line) from the document. Should it work correctly?

Hi,

Thanks for the detail explanation. My test strategy is simple, just to issue orders according to previous day's information (for example previous close price) everyday and close any position before the end of day because I don't want to keep any position overnight. The orders I issue every day should be expired at the end of the day if they don't get executed.

The pinkfish-challenge sample without --no-replay works correctly. I will try it to implement my test strategy.

The daysteps sample I ran correctly without any modification, however, when I added order creation codes in next routine the error occurred. The next routine is as follows:

    def next(self):
        self.callcounter += 1

        txtfields = list()
        txtfields.append('%04d' % self.callcounter)
        txtfields.append('%04d' % len(self))
        txtfields.append('%04d' % len(self.data0))
        txtfields.append(self.data.datetime.datetime(0).isoformat())
        txtfields.append('%.2f' % self.data0.open[0])
        txtfields.append('%.2f' % self.data0.high[0])
        txtfields.append('%.2f' % self.data0.low[0])
        txtfields.append('%.2f' % self.data0.close[0])
        txtfields.append('%.2f' % self.data0.volume[0])
        txtfields.append('%.2f' % self.data0.openinterest[0])
        print(','.join(txtfields))

        if self.buyOrder and self.sellOrder:
            return

        if len(self.data) > self.lcontrol:
            print('- I could issue a buy order during the Opening')
            self.buy(exectype=bt.Order.Limit,
                     size=self.p.stake,
                     price=self.data.close[0] * 0.99,
                     valid=bt.Order.DAY)

            self.log('BUY CREATED , %.2f' % (self.data.close[0] * 0.99))

            self.sell(exectype=bt.Order.Limit,
                      size=self.p.stake,
                      price=self.data.close[0] * 1.01,
                      valid=bt.Order.DAY)

            self.log('SELL CREATED , %.2f' % (self.data.close[0] * 1.01))

        self.lcontrol = len(self.data)

The added part is:

        if self.buyOrder and self.sellOrder:
            return

        if len(self.data) > self.lcontrol:
            print('- I could issue a buy order during the Opening')
            self.buy(exectype=bt.Order.Limit,
                     size=self.p.stake,
                     price=self.data.close[0] * 0.99,
                     valid=bt.Order.DAY)

            self.log('BUY CREATED , %.2f' % (self.data.close[0] * 0.99))

            self.sell(exectype=bt.Order.Limit,
                      size=self.p.stake,
                      price=self.data.close[0] * 1.01,
                      valid=bt.Order.DAY)

            self.log('SELL CREATED , %.2f' % (self.data.close[0] * 1.01))

Besides, I tried to run pinkfish-challenge sample which contain order creation logic, but I got an error:

C:\Users\yp\Projects\backtrader-1.9.17.105\samples\pinkfish-challenge>python pinkfish-challenge.py --no-replay --market
Calls,Len Strat,Len Data,Datetime,Open,High,Low,Close,Volume,OpenInterest
Traceback (most recent call last):
  File "pinkfish-challenge.py", line 345, in <module>
    runstrat()
  File "pinkfish-challenge.py", line 280, in runstrat
    cerebro.run(runonce=False, preload=False, oldbuysell=args.oldbuysell)
  File "C:\Users\yp\Anaconda2\lib\site-packages\backtrader\cerebro.py", line 809, in run
    runstrat = self.runstrategies(iterstrat)
  File "C:\Users\yp\Anaconda2\lib\site-packages\backtrader\cerebro.py", line 931, in runstrategies
    self._runnext(runstrats)
  File "C:\Users\yp\Anaconda2\lib\site-packages\backtrader\cerebro.py", line 1158, in _runnext
    dts.append(datas[i].datetime[0] if ret else None)
  File "C:\Users\yp\Anaconda2\lib\site-packages\backtrader\linebuffer.py", line 162, in __getitem__
    return self.array[self.idx + ago]
IndexError: array index out of range

Any suggestion?