Navigation

    Backtrader Community

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

    Why skip order.Margin status?

    General Code/Help
    2
    12
    6084
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • T
      Taewoo Kim last edited by Taewoo Kim

      In the quickstart tutorial, what is the logic behind why you skip margin orders?

          elif order.status in [order.Canceled, order.Margin, order.Rejected]:
              self.log('Order Canceled/Margin/Rejected')
      

      I can understand why cancelled / rejected orders should set self.order to none, but not why margin order.

      According to the docs:

      Order.Margin: the order execution would imply a margin call and the previously accepted order has been taken off the system
      

      However, I'm placing a tiny orders (account $100k value, value of position < $1k).

      Is there a quick way to debug this?

      1 Reply Last reply Reply Quote 0
      • B
        backtrader administrators last edited by

        It seems you misunderstand what's being done with self.order = None (which is mentioned but not shown): self.order is a sentinel in the strategy to avoid order accumulation, i.e.: an order will not be sent to the broker if another order has not yet been fully processed (Completed, Rejected, ...)

        @Taewoo-Kim said in Why skip order.Margin status?:

        Order.Margin: the order execution would imply a margin call and the previously accepted order has been taken off the system

        As you may see in the documentation you quote: the order is not being skipped. The order is no longer in the system and the strategy sets the sentinel to None in the example to allow new orders to be produced and sent.

        @Taewoo-Kim said in Why skip order.Margin status?:

        However, I'm placing a tiny orders (account $100k value, value of position < $1k).
        Is there a quick way to debug this?

        This seems to imply that you are getting Order.Margin, but without some actual code it is impossible to say what may be wrong. It could be as simple as having 100 orders in the broker each for around $1k ... order 101 wouldn't have cash enough.

        T B 2 Replies Last reply Reply Quote 0
        • T
          Taewoo Kim @backtrader last edited by Taewoo Kim

          This seems to imply that you are getting Order.Margin, but without some actual code it is impossible to say what may be wrong. It could be as simple as having 100 orders in the broker each for around $1k ... order 101 wouldn't have cash enough.

          Ok, so I went back and noticed that most buy orders (not closing short positions, but conversion of cash to long position) were ending up w/o order.Margin status in notify_order.

          I am following Oanda's unit size calculator to limit my position size to under 1% of the account balance.

          Assuming 100k account and 10:1 leverage, this yields in 8900'ish unit size:

          alt text

          My strategy calculates this correctly :

          [INFO] 23:39:42 {'volume': 722.0, 'datetime': datetime.datetime(2017, 5, 29, 0, 0), 'close': 1.1166399999999999}
          [INFO] 23:39:42 [NEXT 1] Signals: Buy True      Sell False
          Order size 8948.505823240164
          

          However, even after setting 100k

          cerebro.broker.setcash(cash)
          

          I still keep getting order.Margin.

          Any ideas?

          1 Reply Last reply Reply Quote 0
          • T
            Taewoo Kim last edited by

            strategy code is simple..

            class TemplateSt(bt.Strategy):
            	params = (
            		( 'timeframe_0_tf', '1Min'),
            	)
            
            	def __init__(self):
            		self.sma = bt.ind.MovingAverageSimple(period=5)
            		self.buysig = bt.ind.CrossDown(self.datas[0].close[0], self.sma)
            		self.sellsig = bt.ind.CrossUp(self.datas[0].close[0], self.sma)
            		
            
            	def next(self):
            		
            		if(self.buysig is None or self.sellsig is None):
            			return
            
            
            		order_size = get_units()
            
            		if(self.buysig):
            			# print("Buying")
            			if(self.position.size <0):
            				
            				if(self.last_bar_ordered != 0 and  not_much_price_diff is False):
            					logging.info("[NEXT {}] Closing short position".format(len(self)))
            					self.close()
            				else:
            					logging.info("[NEXT {}] Price diff small to close short position. Skipping".format(len(self)))
            					
            			elif(self.position.size==0):
            
            				logging.info("[NEXT {}] Creating buy order".format(len(self)))
            				logging.info("[NEXT {}] Order Size: {}".format(len(self), order_size))
            		
            				self.order = self.buy(size=order_size)
            
            		elif (self.sellsig):
            			# print("Selling")
            			if(self.position.size > 0):
            				if(self.last_bar_ordered != 0 and not_much_price_diff is False):
            					logging.info("[NEXT {}] Closing long position".format(len(self)))
            					self.close()
            				else:
            					logging.info("[NEXT {}] Price diff small to close long position. Skipping".format(len(self)))
            
            			elif(self.position.size==0):
            				logging.info("[NEXT {}] Creating sell order".format(len(self)))
            				logging.info("[NEXT {}] Order Size: {}".format(len(self), order_size))
            	
            				self.order = self.sell(size=order_size)
            
            
            	def notify_order(self, order):
            
            
            		logging.info('[NOTIFY_ORDER {}] ORDER {} - {}'.format(len(self), order.Status[order.status], order.tradeid))
            
            		if order.status in [order.Submitted]:
            			# Buy/Sell order submitted/accepted to/by broker - Nothing to do
            			return
            
            
            		if order.status in [order.Accepted]:
            			# Buy/Sell order submitted/accepted to/by broker - Nothing to do			
            			return
            
            		if order.status in [order.Expired, order.Cancelled, order.Rejected]:
            			
            			pass
            
            		if order.status in [order.Completed]:
            
            			order_type  = "buy" if( order.isbuy() ) else "sell"
            
            			logging.info('[NOTIFY_ORDER {}] {} ORDER EXECUTED [{}], Size: {}, Price: {}, Cost: {}, Comm {}'.format(
            				len(self),
            				order_type.upper(), 
            				order.ref,
            				order.executed.size,
            				order.executed.price,
            				order.executed.value,
            				order.executed.comm))
            
            		self.order = None
            
            
            	def notify_trade(self, trade):
            
            		self.order = None
            
            		if not trade.isclosed:
            			return
            
            		trade_info = {
            			"tradeid"  : trade.tradeid,
            			"traded_price" : trade.price,
            			"trade_value"   : trade.value,
            			"trade_gross" : trade.pnl,
            			"trade_net" : trade.pnlcomm,
            			"bars"  :len(self)
            		}
            
            		logging.info('[NOTIFY_TRADE] OPERATION PROFIT, GROSS %.2f, NET %.2f' % (trade.pnl, trade.pnlcomm))
            
            	def stop(self):
            
            		 
            		final_analysis = [
            			("strategy"		, os.path.basename(__file__)),
            			("timeframe_0_tf", self.p.timeframe_0_tf), 
            			("value"     , self.broker.get_value()),
            			("cash"      , self.broker.get_cash()),
            			# ("drawdown"  , dict_recursively(self.analyzers.myTimeDrawDown.get_analysis())),
            		]
            
            		final_analysis = collections.OrderedDict(final_analysis)
            
            
            		str_to_format = "{}," * len(final_analysis.keys())            
            		print(str_to_format.format(*final_analysis.values()) ,end="")
            
            1 Reply Last reply Reply Quote 0
            • B
              backtrader administrators @backtrader last edited by

              Even if it obvious to you, the fragments above don't show how many orders you send to the market or even the cash being set to 100k and many other things and not even Order.Margin being notified (and which is not even considered in notify_order)

              Is every order being met with Order.Margin? Are some orders going through? The usual chart with the evolution of the cash and account value can already reveal a lot (it usually includes also all the buy and sell points)

              You were asking:

              @Taewoo-Kim said in Why skip order.Margin status?:

              Is there a quick way to debug this?

              Yes. Add some simple print statements just before and after this:

              @Taewoo-Kim said in Why skip order.Margin status?:

              		if(self.buysig is None or self.sellsig is None):
              			return
              

              And you will see for example why that statement means nothing.

              These statements are also meaningless:
              @Taewoo-Kim said in Why skip order.Margin status?:

              		self.buysig = bt.ind.CrossDown(self.datas[0].close[0], self.sma)
              		self.sellsig = bt.ind.CrossUp(self.datas[0].close[0], self.sma)
              

              Using [x] (on close) during the init phase is only giving you a fake value because you have preloaded the data. Disable preloading with cerebro.run(preload=False) and you will see it explode, because there will be no data there.

              That means you are generating absolutely random buy and sell signals which are probably kicking in very often given the use of a fixed value in the comparison against the moving average.

              T 1 Reply Last reply Reply Quote -1
              • T
                Taewoo Kim @backtrader last edited by

                1. Setting up the cerebro

                  config_tf = conf_ini["timeframes"]["timeframe_0_tf"]
                  timeframe, compression = pandas_tf_to_backtrader_tf(config_tf)

                  midpoint, _, csv = get_data(
                  currency=conf_ini["general"]["currency"],
                  num_days_to_lookback=args.num_days_to_lookback,
                  resample=config_tf
                  )

                  timeframes = []
                  cerebro = bt.Cerebro(maxcpus=(int(conf_ini["general"]["maxcpus"]) if conf_ini["general"]["maxcpus"] is not "None" else None))

                  bt_feed = bt.feeds.GenericCSVData(
                  dataname=csv,
                  timeframe=timeframe,
                  datetime=0,
                  open=1,
                  high=2,
                  low=3,
                  close=4,
                  volume=5,
                  openinterest=-1
                  )

                  cerebro.adddata(bt_feed)

                  cerebro.addobserver(bt.observers.Broker)
                  cerebro.addobserver(bt.observers.Trades)
                  cerebro.addobserver(bt.observers.BuySell)

                  cerebro.addstrategy(St,
                  timeframe_0_tf = conf_ini["timeframes"]["timeframe_0_tf"],
                  currency=conf_ini["general"]["currency"]
                  )

                  cash = convert_data_type(conf_ini["broker"]["cash"])
                  cerebro.broker.setcash(cash)

                  stakesize = convert_data_type(conf_ini["broker"]["size"])
                  cerebro.addsizer(bt.sizers.FixedSize, stake=stakesize)

                  commission = convert_data_type(conf_ini["broker"]["commission"])
                  leverage = convert_data_type(conf_ini["broker"]["leverage"])

                  cerebro.broker.setcommission(commission=commission, leverage=leverage)

                  cerebro.addanalyzer(btanalyzers.SharpeRatio, _name='mySharpeRatio', timeframe=bt.TimeFrame.Years, compression=1, riskfreerate=0.0)
                  cerebro.addanalyzer(btanalyzers.DrawDown, _name='myDrawDown')
                  cerebro.addanalyzer(btanalyzers.AnnualReturn, _name='myAnnualReturn')
                  cerebro.addanalyzer(btanalyzers.TimeDrawDown, _name='myTimeDrawDown')

                  cerebro.addanalyzer(btanalyzers.TradeAnalyzer, _name='myTradeAnalyzer')

                  cerebro.addanalyzer(btanalyzers.GrossLeverage, _name='myGrossLeverage')

                  from datetime import datetime
                  timestarted = datetime.now()
                  thestrats = cerebro.run()
                  #strats = [s[0] for s in thestrats] # flatten the result

                  for i, strat in enumerate(thestrats):

                   final_stats = {}
                   parts=[]
                  
                   drawdown = dict_recursively(strat.analyzers.myTimeDrawDown.get_analysis())
                   sharpe = dict_recursively(strat.analyzers.mySharpeRatio.get_analysis())
                   annual = dict_recursively(strat.analyzers.myAnnualReturn.get_analysis())
                   annual_return =  next (iter (annual.values())) #
                   my_analyis = [
                       ("sharperatio" , sharpe["sharperatio"]),
                       ("maxdrawdown" , drawdown["maxdrawdown"]),
                       ("maxdrawdownperiod" , drawdown["maxdrawdownperiod"]),
                       ("annual_return" , annual_return),
                       ("time_taken"    , int((datetime.now() - timestarted).total_seconds()))
                   ]
                   my_analyis = collections.OrderedDict(my_analyis)
                  
                   string_to_format="{}," * len(my_analyis.values())
                   print(string_to_format.format(*my_analyis.values()))
                  

                  #cerebro.plot(style='bar')
                  cerebro.plot()

                2. Yes, every LONG order is being met with Order.Margin.

                3. How many orders:

                fragments above don't show how many orders you send to the market

                What I wanted to convey in the code was that I wanted to have only ONE order and wait for it until it is accepted. Is this the wrong logic?

                1. Missing Order.Margin

                not even Order.Margin being notified (and which is not even considered in notify_order)

                My apologies... i copy/pasted an older version of notify_order. It should be this (but doesn't matter b/c it just passes):

                def notify_order(self, order):
                
                
                	logging.info('[NOTIFY_ORDER {}] ORDER {} - {}'.format(len(self), order.Status[order.status], order.tradeid))
                
                	if order.status in [order.Submitted]:
                		# Buy/Sell order submitted/accepted to/by broker - Nothing to do
                		return
                
                
                	if order.status in [order.Accepted]:
                		# Buy/Sell order submitted/accepted to/by broker - Nothing to do			
                		return
                
                	if order.status in [order.Expired, order.Cancelled, order.Rejected]:
                		
                		pass
                
                	if order.status in [order.Margin]:
                		# forgot to include this
                		pass
                	
                
                	if order.status in [order.Completed]:
                
                		order_type  = "buy" if( order.isbuy() ) else "sell"
                
                		logging.info('[NOTIFY_ORDER {}] {} ORDER EXECUTED [{}], Size: {}, Price: {}, Cost: {}, Comm {}'.format(
                			len(self),
                			order_type.upper(), 
                			order.ref,
                			order.executed.size,
                			order.executed.price,
                			order.executed.value,
                			order.executed.comm))
                
                	self.order = None
                
                1. signal on init

                That means you are generating absolutely random buy and sell signals which are probably kicking in very often given the use of a fixed value in the comparison against the moving average.

                So do i need to calculate the signals within next() ?

                T B 3 Replies Last reply Reply Quote 0
                • T
                  Taewoo Kim @Taewoo Kim last edited by Taewoo Kim

                  Here's part of the output from stdout when I added bunch of logging statements.
                  The first order (which happened to be long) gets order.Margin status

                  [INFO] 16:17:25 RUNNING LOCAL BACKTEST
                  [INFO] 16:17:25 Reading cached pickle cache/oandalabs-GBP_JPY-2017-06-28_30days_15Min.pkl
                  [INFO] 16:17:25 *       *       *       *       *       *       *       *       *       *       *       *       *       **       *       *       *       *       *       *       *       *       *       *       *       *       *       *       *
                  [INFO] 16:17:25 Running backtest - 4 15
                  [INFO] 16:17:25 MaxCPUs 8
                  [INFO] 16:17:25 Adding strategy
                  [INFO] 16:17:25 Cash 100000
                  [INFO] 16:17:25 Cash 100000
                  [INFO] 16:17:25 Stake Size 2000
                  [INFO] 16:17:25 Stake Size 2000
                  [INFO] 16:17:25 Commission 0.0
                  [INFO] 16:17:25 Commission 0.0
                  [INFO] 16:17:25 Leverage 10
                  [INFO] 16:17:25 Leverage 10
                  [INFO] 16:17:25 Running cerebro
                  
                  [INFO] 16:17:26 [NEXT 90] Creating buy order
                  [INFO] 16:17:26 [NEXT 90] Order Size: 7804.36029609743
                  [INFO] 16:17:26 [NOTIFY_ORDER 91] ORDER Submitted - 0
                  [INFO] 16:17:26 [NOTIFY_ORDER 91] ORDER Margin - 0
                  
                  1 Reply Last reply Reply Quote 0
                  • B
                    backtrader administrators @Taewoo Kim last edited by

                    @Taewoo-Kim said in Why skip order.Margin status?:

                    So do i need to calculate the signals within next() ?

                    The calculation during __init__ is not done on single point value (i.e.: [x]) of the line object, but on the line itself.

                    buysig = self.data > self.data1
                    # or
                    buysig = bt.ind.CrossOver(self.data, sma)
                    # or more specific
                    buysig = bt.ind.CrossOver(self.data.close, sma)
                    

                    Because [x] is a construct which isn't valid during __init__. See:

                    • Docs - Platform Concepts (probably the section about Operators, with Stage1 and Stage2
                    1 Reply Last reply Reply Quote 0
                    • B
                      backtrader administrators @Taewoo Kim last edited by

                      @Taewoo-Kim said in Why skip order.Margin status?:

                      Is this the wrong logic?

                      Yes. self.buysig and self.sellsig are objects which you have instantiated yourself during __init__ so they cannot be None. And given the creation of the signals is against a fixed value, they will evaluate to True at random times (random from the perspective that it is unknown how the data looks like)

                      A very simple but effective logic is that of the sentinel (self.order) shown in the Docs - Quickstart which you have already seen.

                      As to why you run into Order.Margin, it is really impossible to say. The code is incomplete and somethings have changed from the 1st post to now. There are things which seem odd like setting maxcpus (optimization?)

                      Things you may do:

                      • Set check_submit=False in the broker.

                        This may simply delay the problem to the execution moment

                      • Print the generated order (print(order)) to see the generated details

                      • Print the cash level before generating an order

                      • Simplify your case by removing all the unnecessary stuff (analyzers, custom observer configuration, loading data from conf, broker configuration) and keeping just the basic logic.

                      T 1 Reply Last reply Reply Quote 0
                      • T
                        Taewoo Kim @backtrader last edited by Taewoo Kim

                        @backtrader

                        First off, thank you for all the help you've given me (and everyone else for that matter). BT framework is definitely more powerful than I originally thought.

                        I figured out that it was the currency issue - mainly JPY ( i was using GBP_JPY). When BT multiplied size of order (calculated in USD as per Oanda's formula) by the "per share" cost of GBP_JPY unit, it exceeded the cash balance. Had to add exception to divide size by 100 if JPY is quoted currency.

                        Thanks for your help once again! Sorry for making you chase the wild goose

                        1 Reply Last reply Reply Quote 0
                        • T
                          Taewoo Kim last edited by

                          Before i close off this thread.. do you see anything wrong with this syntax? Im getting zero trades...

                          def __init__(self):
                          
                          	self.fast_ema = bt.ind.ExponentialMovingAverage(period=8, plotname="Fast EMA")
                          	self.slow_ema = bt.ind.ExponentialMovingAverage(period=26, plotname="Slow EMA")		
                          	self.stddev = bt.ind.StandardDeviation(period=8)	
                          
                          	self.regime = self.fast_ema - self.slow_ema    # Positive when bullish
                          
                          	self.buysig = bt.ind.And(
                          				self.fast_ema > self.datas[0].close, 
                          				self.datas[0].close >  self.slow_ema,
                          				bt.ind.CrossUp(self.datas[0].close, self.fast_ema),
                          				self.regime > 0
                          			)
                          
                          	self.sellsig = bt.ind.And(
                          				self.fast_ema < self.datas[0].close,
                          				self.datas[0].close <  self.slow_ema,
                          				bt.ind.CrossDown(self.datas[0].close, self.fast_ema),
                          				self.regime < 0
                          			)
                          	self.order = None
                          
                          1 Reply Last reply Reply Quote 0
                          • B
                            backtrader administrators last edited by

                            The syntax is ok. But the logic seems doomed to fail:

                            @Taewoo-Kim said in Why skip order.Margin status?:

                            	self.buysig = bt.ind.And(
                            				self.fast_ema > self.datas[0].close, 
                            				self.datas[0].close >  self.slow_ema,
                            				bt.ind.CrossUp(self.datas[0].close, self.fast_ema),
                            				self.regime > 0
                            			)
                            
                              		self.fast_ema > self.datas[0].close, 
                              		self.datas[0].close >  self.slow_ema,
                            

                            These 2 place the close in between the fast_ema and slow_ema

                              		bt.ind.CrossUp(self.datas[0].close, self.fast_ema),
                            

                            And This one says that the close should have just crossed the fast_ema to the upside. Which looking at the previous statements is not possible (not possible at the same time)

                            1 Reply Last reply Reply Quote 0
                            • 1 / 1
                            • First post
                              Last post
                            Copyright © 2016, 2017, 2018, 2019, 2020, 2021 NodeBB Forums | Contributors