For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

Orders placed with `valid=` behavior strange



  • When placing an order like so:

    self.buy_bracket(limitprice=1.1, price=1.0, stopprice=0.99, valid=self.datas[0].datetime.date(0) + timedelta(hours=8))
    

    No orders are ever placed. When I remove the valid code it works again. I've tried a few different versions of valid, including:

    valid=now() + timedelta(hours=8)

    valid=timedelta(hours=8)

    valid=datetime.timedelta(hours=8)


  • administrators

    If you work, for example, with a timeframe of Weeks, how are 8 hours supposed to make any sense?



  • @backtrader

    Python datetime objects do not care about timeframe.

    0_1530640932056_Screen Shot 2018-07-03 at 7.01.34 PM.png

    I don't think I'm setting a timeframe anywhere in backtrader that I know of.

    I don't understand your reply.


  • administrators

    You post a single line of code and say something is not happening.

    The answer points out that you could be working with a given timeframe (Weeks) for which your timedelta (8 hours), makes no sense because the order would be immediately invalidated.

    I.e.: a single line of code with no context means nothing. If you thing you have uncovered a problem (and not a problem of your own code), you can share a complete snippet which reproduces the problem.



  • @backtrader said in Orders placed with `valid=` never executed:

    The answer points out that you could be working with a given timeframe (Weeks) for which your timedelta (8 hours), makes no sense because the order would be immediately invalidated.

    So, my data is a 5M timeframe. With code like this valid=self.datas[0].datetime.date(0) + timedelta(minutes=960), all of my orders get canceled.

    I'm asking if the syntax above is right for what I'm saying: I want all orders which are not entered to expire after 16 hours (960) minutes.


  • administrators

    @tw00000 said in Orders placed with `valid=` never executed:

    I don't think I'm setting a timeframe anywhere in backtrader that I know of.

    @tw00000 said in Orders placed with `valid=` never executed:

    So, my data is a 5M timeframe

    The two statements contradict each other. Either you are not setting a timeframe or you are using a 5-minutes timeframe. A sample of some working code (and not an isolated line and then the isolated line again) could help.

    @tw00000 said in Orders placed with `valid=` never executed:

    I'm asking if the syntax above is right for what I'm saying: I want all orders which are not entered to expire after 16 hours (960) minutes.

    The syntax is right or else you would have gotten an error, or? But using the right syntax doesn't help if it's used in the wrong context.



  • @tw00000 said in Orders placed with `valid=` never executed:

    I don't think I'm setting a timeframe anywhere in backtrader that I know of.

    When you add data to the system, you may want to specify your data's timeframe and compression to notify the system.

    From the FAQ

    data = bt.feeds.MyChosenFeed(
        dataname='myname',
        timeframe=bt.TimeFrame.Minutes, compression=5,
        sessionstart=datetime.time(9, 0), sessionend=datetime.time(16, 0)
    )
    


  • @backtrader said in Orders placed with `valid=` never executed:

    @tw00000 said in Orders placed with `valid=` never executed:

    I don't think I'm setting a timeframe anywhere in backtrader that I know of.

    @tw00000 said in Orders placed with `valid=` never executed:

    So, my data is a 5M timeframe

    The two statements contradict each other. Either you are not setting a timeframe or you are using a 5-minutes timeframe. A sample of some working code (and not an isolated line and then the isolated line again) could help.

    Ok, so backtrader is automatically detecting the timeframe of my data then? I'm not setting a timeframe anywhere in my code.


    @ab_trader thank you for explaining this:

    When you add data to the system, you may want to specify your data's timeframe and compression to notify the system.
    From the FAQ

    data = bt.feeds.MyChosenFeed(
        dataname='myname',
        timeframe=bt.TimeFrame.Minutes, compression=5,
        sessionstart=datetime.time(9, 0), sessionend=datetime.time(16, 0)
    )
    

    This is interesting, my code appears to run without having set the timeframe, compression, or session start in my data feed, which is really simple:

    # Add the data
    df = pd.read_csv('data/{}_5m.csv'.format(instrument_name))
    df.columns = ['Date', 'Open', 'High', 'Low', 'Close', 'Volume']
    df.index = pd.to_datetime(df['Date'], unit='ms')
    df['Date'] = pd.to_datetime(df['Date'], unit='ms')
    df = df.drop('Date', axis=1)
    data = bt.feeds.PandasData(dataname=df)
    
    cerebro.adddata(data)
    

    I will experiment with explicitly setting the timeframe and see if that changes how the valid = arg works!


  • administrators

    @tw00000 said in Orders placed with `valid=` never executed:

    Ok, so backtrader is automatically detecting the timeframe of my data then?

    No. How do you expect it do be done automatically?

    @tw00000 said in Orders placed with `valid=` never executed:

    This is interesting, my code appears to run without having set the timeframe, compression, or session start in my data feed, which is really simple:

    Of course your code will work. The platform will move through your prices. But things which depend on timeframe/timestamps will probably not match your expectations. The platform cannot know that the bars are not daily bars, which is the default.



  • @tw00000 remember I proposed you to read docs first? :) Read at least FAQ, can save you time in the future.



  • Ok, so it still appears after extensive testing that I cannot get consistent behavior out of using valid =. Now, I am having the opposite problem - no orders ever get canceled!

    I'm just trying to cancel any order that's been "open" but "not entered" for 30 minutes. However, no orders ever get canceled.

    I'm confused that I can't find anyone else having this issue, because I've studied the docs and I think I'm following everything appropriately...

    Anyway, here's the full relevant code @backtrader ;):

    if __name__ == '__main__':
    
        class StrategyTest(bt.Strategy): 
    
            params = (
            ('take_profit_pct',0.02),
            ('stop_loss_pct',0.02),
            ('run_name', run_name),
            ('resample_rule', resample_rule),
            ('instrument_name', instrument_name)
            )
    
            def __init__(self):
                self.startcash = self.broker.getvalue()
                self.iteration_progress = tqdm(desc='Total runs', total=(self.datas[0].close.buflen()*len(self.datas)) - window, position=0)
    
            def next(self):
    
                if len(self.broker.get_orders_open()) > 0:
                        print(self.broker.get_orders_open()) #this is always empty no matter what, not sure what `get_orders_open()` actually does
    
                for index, data in enumerate(self.datas):
                    datetime, dataname = self.datetime.date(), data._name
    
                    if len(data.close) > window:
                        self.iteration_progress.update()
    
                        if len(self._orders) > 0:
                            open_orders = 1
                            for order in self._orders:
                                print(order) # This always returns a ton of orders, long after they should have been canceled in the backtest
                        else: 
                            open_orders = None
    
                        pos = self.getposition(data).size
                        if not pos:  # no market / no orders
                            try:
                                df_with_signal, _ = SIGNAL_LOGIC(
                                                    data=[data.open.get(size=window),
                                                        data.high.get(size=window),
                                                        data.low.get(size=window),
                                                        data.close.get(size=window),
                                                        [data.num2date(x) for x in data.datetime.get(size=window)]
                                                         ],
                                                    params={
                                                            'take_profit_pct': self.params.take_profit_pct,
                                                            'stop_loss_pct': self.params.stop_loss_pct,
                                                            'run_name':self.params.run_name,
                                                            'instrument_name':self.params.instrument_name,
                                                            'resample_rule':self.params.resample_rule
                                                           },
                                                   backtrader=True
                                                    )
    
                            except Exception as err:
                                print(traceback.format_exc())
                                print(sys.exc_info()[0])
                                df_with_signal = pd.DataFrame()
    
                            ### PLACING THE TRADE ###    
                            if 'SIGNAL' in df_with_signal.columns:
                                df = df_with_signal[df_with_signal['SIGNAL'].isnull() == False]
    
                                if df.shape[0] > 0:
                                    try:
    
                                        if df.iloc[-1]['SIGNAL'] == 'SELL':   
    
                                            entry_price = df_with_signal.iloc[-1]['signal_line']
                                            date_of_entry = df_with_signal.iloc[-1]['date']
                                            print(date_of_entry)
                                            print(date_of_entry+timedelta(minutes=30))
    
                                            mainside = self.buy(data=dataname, price=entry_price, exectype=bt.Order.Limit, transmit=False, valid=date_of_entry + timedelta(minutes=30), info=dataname+' SHORT Limit Entry')
    
                                            target_price = entry_price*(1-self.params.take_profit_pct)
                                            stop_loss = entry_price*(1+self.params.stop_loss_pct)
    
                                            lowside  = self.sell(data=dataname, price=target_price, size=mainside.size, exectype=bt.Order.Limit,
                                                                 transmit=False, parent=mainside, 
                                                                 valid=date_of_entry + timedelta(minutes=30),
                                                                  info=dataname+' SHORT Stop Loss')
    
                                            highside = self.sell(data=dataname, price=stop_loss, size=mainside.size, exectype=bt.Order.Stop,
                                                                 transmit=True, parent=mainside, 
                                                                 valid=date_of_entry + timedelta(minutes=30),
                                                                  info=dataname+' SHORT Take Profit')
    
                                            self.iteration_progress.write("Placing a short sell order, hopefully")
    
    
                                        if df.iloc[-1]['SIGNAL'] == 'BUY':      
    
                                            entry_price = df_with_signal.iloc[-1]['signal_line']
                                            date_of_entry = df_with_signal.iloc[-1]['date']
    
                                            mainside = self.buy(data=dataname, price=entry_price, exectype=bt.Order.Limit, transmit=False, valid=date_of_entry + timedelta(minutes=30), info=dataname+' LONG Limit Entry')
    
                                            target_price = entry_price*(1+self.params.take_profit_pct)
                                            stop_loss = entry_price*(1-self.params.stop_loss_pct)
    
                                            lowside  = self.sell(data=dataname, price=stop_loss, size=mainside.size, exectype=bt.Order.Stop,
                                                                 transmit=False, parent=mainside, 
                                                                 valid=date_of_entry + timedelta(minutes=30),
                                                                  info=dataname+' LONG Stop Loss')
                                            highside = self.sell(data=dataname, price=target_price, size=mainside.size, exectype=bt.Order.Limit,
                                                                 transmit=True, parent=mainside, 
                                                                 valid=date_of_entry + timedelta(minutes=30),
                                                                  info=dataname+' LONG Take Profit')
    
                                            self.iteration_progress.write("Placing a buy order hopefully")
    
                                    except Exception as err:
                                        print(traceback.format_exc())
                                        print(sys.exc_info()[0])
                                        print("df is like: ", df.head(), df.shape)
    
            def notify_order(self, order): #this never gets triggered no matter what, which means orders are never expiring. 
                date = self.data.datetime.datetime().date()
                print(order)
                if order.status == order.Expired:
                    print('-'*32,' NOTIFY ORDER ','-'*32)
                    print('{} Order Canceled'.format(order.info['name']))
                    print('{}, {}, Status {}: Ref: {}, Size: {}, Price: {}'.format(
                                                                date,
                                                                self.data._name,
                                                                order.status,
                                                                order.ref,
                                                                order.size,
                                                                'NA' if not order.price else round(order.price,rounding)
                                                                ))
    
            def notify_trade(self, trade):
                date = trade.data.datetime.datetime()
                if trade.isclosed:
                    print('-'*32,' NOTIFY TRADE ','-'*32)
                    print('{} at {}, Close Price: {}, Profit, Gross {}, Net {}'.format(
                                                        trade.data._name,
                                                        date,
                                                        trade.price,
                                                        round(trade.pnl,rounding),
                                                        round(trade.pnlcomm,rounding)))
                    print('-'*80)
    
    cerebro = Cerebro()
    cerebro.broker.setcash(start_cash)
    cerebro.broker.setcommission(commission=0.000005)
    cerebro.addsizer(bt.sizers.AllInSizer)
    cerebro.addstrategy(StrategyTest)
    df = pd.read_csv('binance-crypto-data/binance_ETHBTC_5m.csv'.format(instrument_name))
    
    data = bt.feeds.PandasData(dataname=df, timeframe=bt.TimeFrame.Minutes, compression=5)
    cerebro.adddata(data, name=instrument_name)
    
    strategy = cerebro.run()
    


  • It appears that somehow I fixed this in the code, and this behavior stopped happening.

    I'm not exactly sure what I did, unfortunately, but here's some stuff I added that seemed to help get clarity and prevent orders from getting placed on top of each other

    if len(self._orders) > 1:
        if datetime - bt.num2date(self._orders[-1].created.dt) < timedelta(minutes=90):
            print("Orders placed too recently, let's not place more ")
            self.recent_or_open_orders = 1
        else:
            self.recent_or_open_orders = None
    
    elif len(self._orderspending) > 0:
        self.recent_or_open_orders = 1
    
        print("Current time: {}".format(datetime))  
        for order in self._orderspending:
            print("Order {} created at {}".format(order.ref, bt.num2date(order.created.dt)))
    
    else: 
        self.recent_or_open_orders = None
    
    pos = self.getposition(data).size
    
    print(self.recent_or_open_orders)
    print(pos)
    
    if self.recent_or_open_orders == None and not pos:  # no market / no orders / no recent orders