Navigation

    Backtrader Community

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

    margin problem when selling then buying in next

    General Code/Help
    3
    6
    545
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • darkknight9394
      darkknight9394 last edited by

      Dear all, thanks in advance for reading this query. The Idea I'm currently working on involves the strategy discussed Stocks on the move, or as shown in this https://github.com/teddykoker/blog/blob/master/_notebooks/2019-05-19-momentum-strategy-from-stocks-on-the-move-in-python.ipynbl)https://github.com/teddykoker/blog/blob/master/_notebooks/2019-05-19-momentum-strategy-from-stocks-on-the-move-in-python.ipynb.

      The problem I'm encountering is that there's a Margin call evening though that I've arranged the sell orders before the buys. During each call to next, the selling decision is made, then depending on the momentum of individual stock, buy orders will be issued by loop through the remaining available value of the portfolio. Looking at the log, I observed that
      Even though the sell orders get executed before the buy orders, the value of the portfolio isn't updated. hence no available balance for the set of buying order.

      Ideally what i want to do is to (1) sell stocks based on criteria (then have the updated portfolio cash available) then (2) buy according to the ranked momentum (with risk parity sizing) until there's no cash left

      # https://github.com/teddykoker/blog/blob/master/_notebooks/2019-05-19-momentum-strategy-from-stocks-on-the-move-in-python.ipynb
      
      from datetime import datetime
      import pandas as pd
      import matplotlib.pyplot as plt
      import numpy as np
      import calendar
      
      plt.rcParams["figure.figsize"] = (10, 6)  # (w, h)
      plt.ioff()
      
      from scipy.stats import linregress
      import backtrader as bt
      
      
      class Momentum(bt.Indicator):
          lines = ('trend',)
          params = (('period', 90),)
      
          def __init__(self):
              self.addminperiod(self.params.period)
      
          def next(self):
              returns = np.log(self.data.get(size=self.p.period))
              x = np.arange(len(returns))
              slope, _, rvalue, _, _ = linregress(x, returns)
              annualized = (1 + slope) ** 252
              self.lines.trend[0] = annualized * (rvalue ** 2)
      
      
      
      class Strategy(bt.Strategy):
          def __init__(self):
              self.i = 0
              self.inds = {}
              self.spy = self.datas[0]
              self.stocks = self.datas[1:]
      
              self.spy_sma200 = bt.indicators.SimpleMovingAverage(self.spy.close, period=200)
      
              for d in self.stocks:
                  self.inds[d] = {}
                  self.inds[d]["momentum"] = Momentum(d.close, period=30)
                  self.inds[d]["sma100"] = bt.indicators.SimpleMovingAverage(d.close, period=100)
                  self.inds[d]["atr20"] = bt.indicators.ATR(d, period=20)
      
          def prenext(self):
              # call next() even when data is not available for all tickers
              self.next()
      
          def next(self):
              from utilites import week_number_of_month
              current_date = bt.utils.date.num2date(self.datas[0].datetime[0])
              print("rebalance portfolio on " + str(current_date))
              self.rebalance_portfolio()
      
          def notify_order(self, order):
              if order.status in [order.Submitted, order.Accepted]:
                  # Buy/Sell order submitted/accepted to/by broker - Nothing to do
                  return
      
              # Check if an order has been completed
              # Attention: broker could reject order if not enougth cash
              if order.status in [order.Completed]:
                  if order.isbuy():
                      print(order.data._name + ' BUY EXECUTED, %.0f shares at $%.2f .PORT VAL $%.2f' % (order.size, order.executed.price,self.broker.get_value()))
                  elif order.issell():
                      print(order.data._name + ' SELL EXECUTED, %.0f shares at %.2f .PORT VAL $%.2f' % (order.size, order.executed.price, self.broker.get_value()))
      
                  self.bar_executed = len(self)
      
              elif order.status in [order.Canceled, order.Margin, order.Rejected]:
                  #print(order.data._name + ' Order Canceled/Margin/Rejected, %.0f shares' % (order.size))  # , order.price))
                  print('---statust below---')
                  print(order.status)
                  print(order.data._name + ' Order Canceled/Margin/Rejected, %.0f shares, price %.2f .PORT VAL $%.2f' % (order.size , order.executed.price, self.broker.get_value()))
      
              # Write down: no pending order
              self.order = None
      
          def rebalance_portfolio(self):
              print('============================')
              # only look at data that we can have indicators for
              self.rankings = list(filter(lambda d: len(d) > 100, self.stocks))
              self.rankings.sort(key=lambda d: self.inds[d]["momentum"][0], reverse=True)
      
              num_stocks = len(self.rankings)
      
              # sell stocks based on criteria
              for i, d in enumerate(self.rankings):
                  if self.getposition(d).size != 0:
                      is_close_position = False
      
                      if i > num_stocks * 0.2 or d < self.inds[d]["sma100"]:
                          print("exiting position for " + d._name + " due to exit conditions" + str(
                              bt.utils.date.num2date(d.datetime[0])) + " " + str(self.getposition(d).size))
                          is_close_position = True
      
                      if is_close_position:
                          self.close(d,coc=True)
      
              #if self.spy < self.spy_sma200:
              #    return
      
              """
              print('------')
              for i, d in enumerate(self.rankings):
                  if self.getposition(d).size != 0:
                      print(d._name)
              print('-----')
              """
              # buy stocks with remaining cash
              value_available = self.broker.get_value()
              for i, d in enumerate(self.rankings[:int(num_stocks * 0.2)]):
      
                  if self.getposition(d).size == 0:
      
                      size = value_available * 0.001 / self.inds[d]["atr20"]
      
                      if value_available <= 0:
                          print("rebalance_portfolio: value_avalaible " + str(value_available))
                          break
      
                      self.buy(data=d, size=size)
      
                      value_available -= size * d.close[0]
      
                      print("rebalance_portfolio " + d._name + " size = " + str(size) + " value remaining=" + str(
                          value_available) + " invested=" + str(size * d.close[0]) + " size=" + str(
                          size) + " NUMSTOCK INVEST" + str(i))
                  else:
                      print("rebalance_portfolio:: EXISTING " + d._name + " size= " + str(self.getposition(d).size))
      
              print("rebalance_portfolio:: remaining value " + str(value_available))
      
      
      
      def parse_iqfeed_csv_to_df(iq_feed_file, start_date_YYYYMMDD=None, end_date_YYYYMMDD=None):
          datetime_parser_ignore_tz = lambda x: pd.datetime.strptime(x.rsplit('-', 1)[0], "%Y-%m-%d %H:%M:%S")
          df = pd.read_csv(iq_feed_file, parse_dates=[0], date_parser=datetime_parser_ignore_tz)
          df.set_index('datetime', inplace=True)
          df = df.sort_index(axis=1)
      
          if start_date_YYYYMMDD is not None and end_date_YYYYMMDD is not None:
              df = df.loc[(df.index > start_date_YYYYMMDD) & (df.index < end_date_YYYYMMDD)]
      
          return df
      
      
      if __name__ == "__main__":
      
          casename = "test_snp_margin"
      
          # loop to find all the tickers
          import os
          import time
      
          t0 = time.time()
          tickers = []
          max_number_of_stocks = 500 # only use this when you want to limit the stock universe (for testing only)
      
          start_date = '2008-01-01'
          end_date = '2010-06-26'
      
          # As we can see in the code, the strategy looks for stocks it needs to sell every week in the rebalance_portfolio method and rebalances
          # all of its positions every other week in the rebalance_positions method. Now let's run a backtest!
          cerebro = bt.Cerebro(stdstats=False)
          #cerebro.broker.set_coc(True)
          #cerebro.broker.set_checksubmit(False)
      
          index_file = "SPY_20080101_20200824_23400.csv"
          benchmark_data = parse_iqfeed_csv_to_df(index_file, start_date_YYYYMMDD=start_date, end_date_YYYYMMDD=end_date)
      
          cerebro.adddata(bt.feeds.PandasData(dataname=benchmark_data, name="SPY", plot=True))
      
          cerebro.addobserver(bt.observers.Benchmark)
          cerebro.addobserver(bt.observers.TimeReturn, timeframe=bt.TimeFrame.NoTimeFrame)
      
          src_data_dir = r"C:\Users\thomas\PycharmProjects\pyiqfeed_dataDownloadOnly\MARKETDATA_iShares-Core-SP-500-ETF_fund"
          #src_data_dir = r"C:\Users\thomas\PycharmProjects\pyiqfeed_dataDownloadOnly\MARKETDATA_iShares-NASDAQ-100-Index-ETF-CAD-Hedged_fund"
          # src_data_dir = r"C:\Users\thomas\PycharmProjects\pyiqfeed_dataDownloadOnly\MARKETDATA_iShares-Russell-2000-ETF_fund"
      
          count=0
          for file in os.listdir(src_data_dir):
              if count < max_number_of_stocks:
                  # df = pd.read_csv(os.path.join(src_data_dir,file), parse_dates=True, index_col=0)
                  ticker = file.split('_')[0]
                  df = parse_iqfeed_csv_to_df(os.path.join(src_data_dir, file), start_date_YYYYMMDD=start_date,
                                              end_date_YYYYMMDD=end_date)
      
                  if len(df) > 100:  # data must be long enough to compute 100 day SMA
                      cerebro.adddata(bt.feeds.PandasData(dataname=df, name=ticker, plot=False))
                      # print(df)
                      print("added " + ticker)
      
                  tickers.append(ticker)
              count += 1
      
          # Set our desired cash start
          cerebro.broker.setcash(10000.0)
          cerebro.addstrategy(Strategy)
          results = cerebro.run()
      
      
      
      Sumit Pandey 1 Reply Last reply Reply Quote 1
      • darkknight9394
        darkknight9394 last edited by

        Here's a log print out that show after the sell order execution, the portfolio values/ cash remains unchanged. I'd want to know how is it possible for the strategy to update the cash after sell, before buying

        ODFL SELL EXECUTED, -5 shares at 8.85 .PORT VAL $9449.18 .PORT CASH $2472.54
        NI SELL EXECUTED, -18 shares at 6.08 .PORT VAL $9449.18 .PORT CASH $2472.54
        AEE SELL EXECUTED, -4 shares at 27.61 .PORT VAL $9449.18 .PORT CASH $2472.54
        CMCSA SELL EXECUTED, -6 shares at 8.39 .PORT VAL $9449.18 .PORT CASH $2472.54
        ES SELL EXECUTED, -4 shares at 25.77 .PORT VAL $9449.18 .PORT CASH $2472.54
        DTE SELL EXECUTED, -3 shares at 43.05 .PORT VAL $9449.18 .PORT CASH $2472.54
        MNST BUY EXECUTED, 17 shares at $6.59 .PORT VAL $9449.18 .PORT CASH $2472.54
        PWR BUY EXECUTED, 4 shares at $21.36 .PORT VAL $9449.18 .PORT CASH $2472.54
        VRSN BUY EXECUTED, 4 shares at $24.90 .PORT VAL $9449.18 .PORT CASH $2472.54
        SIVB BUY EXECUTED, 2 shares at $42.58 .PORT VAL $9449.18 .PORT CASH $2472.54
        CTXS BUY EXECUTED, 3 shares at $34.03 .PORT VAL $9449.18 .PORT CASH $2472.54

        1 Reply Last reply Reply Quote 0
        • Sumit Pandey
          Sumit Pandey last edited by

          @darkknight9394 said in margin problem when selling then buying in next:

          order.data._name

          I am also having the same confusion. Seems like the BT takes in all orders in the loop at once and processes it later.
          How can we update the cash position in the loop itself?

          1 Reply Last reply Reply Quote 0
          • Sumit Pandey
            Sumit Pandey @darkknight9394 last edited by

            @ab_trader Can you please help regarding this issue.

            1 Reply Last reply Reply Quote 0
            • A
              ab_trader last edited by

              @darkknight9394 said in margin problem when selling then buying in next:

              Even though the sell orders get executed before the buy orders, the value of the portfolio isn't updated. hence no available balance for the set of buying order.

              Cash is updated internally after each executed sell/buy order, but the broker cash amount is delivered to the strategy level only at the next price available. On your issue my guess (since your logs don't show any issues, than guess only) would be is that you affected by differences in close and open prices. You rebalance the portfolio based on the previous bar equity and previous bar close prices, but orders are executed based on the coming bar open prices. Open prices are usually different from previous close prices, therefore it maybe not enough money from sales to execute buy order, especially last one. Use 95-99% of the portfolio equity to mitigate this effect.

              To have earlier cash change notifications try to set quicknotify=True , might help to see the changes in cash, but I don't think that the issue is here - Docs - Cerebro - Reference

              @Sumit-Pandey said in margin problem when selling then buying in next:

              I am also having the same confusion. Seems like the BT takes in all orders in the loop at once and processes it later.

              Reading docs will help to avoid confusion and guessing Docs - Cerebro - Backtesting Logic

              • If my answer helped, hit reputation up arrow at lower right corner of the post.
              • Python Debugging With Pdb
              • New to python and bt - check this out
              1 Reply Last reply Reply Quote 1
              • darkknight9394
                darkknight9394 last edited by

                Thanks for replying to my post @Sumit-Pandey @ab_trader. I gave the 95-99% of the portfolio value a try and it still didn't solve the problem. If I understand correctly, this 95-99% is to reserve the amount of cash so that when the next open price is greater than the current close price, we'll still have sufficient funds for the purchase. There is however no guarantee that it will solve the problem. There might be 2 related issue here 1) is that I'm using the current bar valuation for funds allocation so I've to sell the stock first before purchasing (but same bar sell then buy doesn't work) 2) the changes in price that @ab_trader mentioned. Will continue investing this. I think in the worst case I can always force the selling first, then do all the buying in the bar afterwards, but since I'm testing this on daily timeframe, it would be best not to doing so.

                1 Reply Last reply Reply Quote 1
                • 1 / 1
                • First post
                  Last post
                Copyright © 2016, 2017, 2018, 2019, 2020, 2021 NodeBB Forums | Contributors