For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

margin problem when selling then buying in next



  • Dear all, thanks in advance for reading this query. The Idea I'm currently working on involves the strategy discussed Stocks on the move, or as shown in this https://github.com/teddykoker/blog/blob/master/_notebooks/2019-05-19-momentum-strategy-from-stocks-on-the-move-in-python.ipynbl)https://github.com/teddykoker/blog/blob/master/_notebooks/2019-05-19-momentum-strategy-from-stocks-on-the-move-in-python.ipynb.

    The problem I'm encountering is that there's a Margin call evening though that I've arranged the sell orders before the buys. During each call to next, the selling decision is made, then depending on the momentum of individual stock, buy orders will be issued by loop through the remaining available value of the portfolio. Looking at the log, I observed that
    Even though the sell orders get executed before the buy orders, the value of the portfolio isn't updated. hence no available balance for the set of buying order.

    Ideally what i want to do is to (1) sell stocks based on criteria (then have the updated portfolio cash available) then (2) buy according to the ranked momentum (with risk parity sizing) until there's no cash left

    # https://github.com/teddykoker/blog/blob/master/_notebooks/2019-05-19-momentum-strategy-from-stocks-on-the-move-in-python.ipynb
    
    from datetime import datetime
    import pandas as pd
    import matplotlib.pyplot as plt
    import numpy as np
    import calendar
    
    plt.rcParams["figure.figsize"] = (10, 6)  # (w, h)
    plt.ioff()
    
    from scipy.stats import linregress
    import backtrader as bt
    
    
    class Momentum(bt.Indicator):
        lines = ('trend',)
        params = (('period', 90),)
    
        def __init__(self):
            self.addminperiod(self.params.period)
    
        def next(self):
            returns = np.log(self.data.get(size=self.p.period))
            x = np.arange(len(returns))
            slope, _, rvalue, _, _ = linregress(x, returns)
            annualized = (1 + slope) ** 252
            self.lines.trend[0] = annualized * (rvalue ** 2)
    
    
    
    class Strategy(bt.Strategy):
        def __init__(self):
            self.i = 0
            self.inds = {}
            self.spy = self.datas[0]
            self.stocks = self.datas[1:]
    
            self.spy_sma200 = bt.indicators.SimpleMovingAverage(self.spy.close, period=200)
    
            for d in self.stocks:
                self.inds[d] = {}
                self.inds[d]["momentum"] = Momentum(d.close, period=30)
                self.inds[d]["sma100"] = bt.indicators.SimpleMovingAverage(d.close, period=100)
                self.inds[d]["atr20"] = bt.indicators.ATR(d, period=20)
    
        def prenext(self):
            # call next() even when data is not available for all tickers
            self.next()
    
        def next(self):
            from utilites import week_number_of_month
            current_date = bt.utils.date.num2date(self.datas[0].datetime[0])
            print("rebalance portfolio on " + str(current_date))
            self.rebalance_portfolio()
    
        def notify_order(self, order):
            if order.status in [order.Submitted, order.Accepted]:
                # Buy/Sell order submitted/accepted to/by broker - Nothing to do
                return
    
            # Check if an order has been completed
            # Attention: broker could reject order if not enougth cash
            if order.status in [order.Completed]:
                if order.isbuy():
                    print(order.data._name + ' BUY EXECUTED, %.0f shares at $%.2f .PORT VAL $%.2f' % (order.size, order.executed.price,self.broker.get_value()))
                elif order.issell():
                    print(order.data._name + ' SELL EXECUTED, %.0f shares at %.2f .PORT VAL $%.2f' % (order.size, order.executed.price, self.broker.get_value()))
    
                self.bar_executed = len(self)
    
            elif order.status in [order.Canceled, order.Margin, order.Rejected]:
                #print(order.data._name + ' Order Canceled/Margin/Rejected, %.0f shares' % (order.size))  # , order.price))
                print('---statust below---')
                print(order.status)
                print(order.data._name + ' Order Canceled/Margin/Rejected, %.0f shares, price %.2f .PORT VAL $%.2f' % (order.size , order.executed.price, self.broker.get_value()))
    
            # Write down: no pending order
            self.order = None
    
        def rebalance_portfolio(self):
            print('============================')
            # only look at data that we can have indicators for
            self.rankings = list(filter(lambda d: len(d) > 100, self.stocks))
            self.rankings.sort(key=lambda d: self.inds[d]["momentum"][0], reverse=True)
    
            num_stocks = len(self.rankings)
    
            # sell stocks based on criteria
            for i, d in enumerate(self.rankings):
                if self.getposition(d).size != 0:
                    is_close_position = False
    
                    if i > num_stocks * 0.2 or d < self.inds[d]["sma100"]:
                        print("exiting position for " + d._name + " due to exit conditions" + str(
                            bt.utils.date.num2date(d.datetime[0])) + " " + str(self.getposition(d).size))
                        is_close_position = True
    
                    if is_close_position:
                        self.close(d,coc=True)
    
            #if self.spy < self.spy_sma200:
            #    return
    
            """
            print('------')
            for i, d in enumerate(self.rankings):
                if self.getposition(d).size != 0:
                    print(d._name)
            print('-----')
            """
            # buy stocks with remaining cash
            value_available = self.broker.get_value()
            for i, d in enumerate(self.rankings[:int(num_stocks * 0.2)]):
    
                if self.getposition(d).size == 0:
    
                    size = value_available * 0.001 / self.inds[d]["atr20"]
    
                    if value_available <= 0:
                        print("rebalance_portfolio: value_avalaible " + str(value_available))
                        break
    
                    self.buy(data=d, size=size)
    
                    value_available -= size * d.close[0]
    
                    print("rebalance_portfolio " + d._name + " size = " + str(size) + " value remaining=" + str(
                        value_available) + " invested=" + str(size * d.close[0]) + " size=" + str(
                        size) + " NUMSTOCK INVEST" + str(i))
                else:
                    print("rebalance_portfolio:: EXISTING " + d._name + " size= " + str(self.getposition(d).size))
    
            print("rebalance_portfolio:: remaining value " + str(value_available))
    
    
    
    def parse_iqfeed_csv_to_df(iq_feed_file, start_date_YYYYMMDD=None, end_date_YYYYMMDD=None):
        datetime_parser_ignore_tz = lambda x: pd.datetime.strptime(x.rsplit('-', 1)[0], "%Y-%m-%d %H:%M:%S")
        df = pd.read_csv(iq_feed_file, parse_dates=[0], date_parser=datetime_parser_ignore_tz)
        df.set_index('datetime', inplace=True)
        df = df.sort_index(axis=1)
    
        if start_date_YYYYMMDD is not None and end_date_YYYYMMDD is not None:
            df = df.loc[(df.index > start_date_YYYYMMDD) & (df.index < end_date_YYYYMMDD)]
    
        return df
    
    
    if __name__ == "__main__":
    
        casename = "test_snp_margin"
    
        # loop to find all the tickers
        import os
        import time
    
        t0 = time.time()
        tickers = []
        max_number_of_stocks = 500 # only use this when you want to limit the stock universe (for testing only)
    
        start_date = '2008-01-01'
        end_date = '2010-06-26'
    
        # As we can see in the code, the strategy looks for stocks it needs to sell every week in the rebalance_portfolio method and rebalances
        # all of its positions every other week in the rebalance_positions method. Now let's run a backtest!
        cerebro = bt.Cerebro(stdstats=False)
        #cerebro.broker.set_coc(True)
        #cerebro.broker.set_checksubmit(False)
    
        index_file = "SPY_20080101_20200824_23400.csv"
        benchmark_data = parse_iqfeed_csv_to_df(index_file, start_date_YYYYMMDD=start_date, end_date_YYYYMMDD=end_date)
    
        cerebro.adddata(bt.feeds.PandasData(dataname=benchmark_data, name="SPY", plot=True))
    
        cerebro.addobserver(bt.observers.Benchmark)
        cerebro.addobserver(bt.observers.TimeReturn, timeframe=bt.TimeFrame.NoTimeFrame)
    
        src_data_dir = r"C:\Users\thomas\PycharmProjects\pyiqfeed_dataDownloadOnly\MARKETDATA_iShares-Core-SP-500-ETF_fund"
        #src_data_dir = r"C:\Users\thomas\PycharmProjects\pyiqfeed_dataDownloadOnly\MARKETDATA_iShares-NASDAQ-100-Index-ETF-CAD-Hedged_fund"
        # src_data_dir = r"C:\Users\thomas\PycharmProjects\pyiqfeed_dataDownloadOnly\MARKETDATA_iShares-Russell-2000-ETF_fund"
    
        count=0
        for file in os.listdir(src_data_dir):
            if count < max_number_of_stocks:
                # df = pd.read_csv(os.path.join(src_data_dir,file), parse_dates=True, index_col=0)
                ticker = file.split('_')[0]
                df = parse_iqfeed_csv_to_df(os.path.join(src_data_dir, file), start_date_YYYYMMDD=start_date,
                                            end_date_YYYYMMDD=end_date)
    
                if len(df) > 100:  # data must be long enough to compute 100 day SMA
                    cerebro.adddata(bt.feeds.PandasData(dataname=df, name=ticker, plot=False))
                    # print(df)
                    print("added " + ticker)
    
                tickers.append(ticker)
            count += 1
    
        # Set our desired cash start
        cerebro.broker.setcash(10000.0)
        cerebro.addstrategy(Strategy)
        results = cerebro.run()
    
    
    


  • Here's a log print out that show after the sell order execution, the portfolio values/ cash remains unchanged. I'd want to know how is it possible for the strategy to update the cash after sell, before buying

    ODFL SELL EXECUTED, -5 shares at 8.85 .PORT VAL $9449.18 .PORT CASH $2472.54
    NI SELL EXECUTED, -18 shares at 6.08 .PORT VAL $9449.18 .PORT CASH $2472.54
    AEE SELL EXECUTED, -4 shares at 27.61 .PORT VAL $9449.18 .PORT CASH $2472.54
    CMCSA SELL EXECUTED, -6 shares at 8.39 .PORT VAL $9449.18 .PORT CASH $2472.54
    ES SELL EXECUTED, -4 shares at 25.77 .PORT VAL $9449.18 .PORT CASH $2472.54
    DTE SELL EXECUTED, -3 shares at 43.05 .PORT VAL $9449.18 .PORT CASH $2472.54
    MNST BUY EXECUTED, 17 shares at $6.59 .PORT VAL $9449.18 .PORT CASH $2472.54
    PWR BUY EXECUTED, 4 shares at $21.36 .PORT VAL $9449.18 .PORT CASH $2472.54
    VRSN BUY EXECUTED, 4 shares at $24.90 .PORT VAL $9449.18 .PORT CASH $2472.54
    SIVB BUY EXECUTED, 2 shares at $42.58 .PORT VAL $9449.18 .PORT CASH $2472.54
    CTXS BUY EXECUTED, 3 shares at $34.03 .PORT VAL $9449.18 .PORT CASH $2472.54



  • @darkknight9394 said in margin problem when selling then buying in next:

    order.data._name

    I am also having the same confusion. Seems like the BT takes in all orders in the loop at once and processes it later.
    How can we update the cash position in the loop itself?



  • @ab_trader Can you please help regarding this issue.



  • @darkknight9394 said in margin problem when selling then buying in next:

    Even though the sell orders get executed before the buy orders, the value of the portfolio isn't updated. hence no available balance for the set of buying order.

    Cash is updated internally after each executed sell/buy order, but the broker cash amount is delivered to the strategy level only at the next price available. On your issue my guess (since your logs don't show any issues, than guess only) would be is that you affected by differences in close and open prices. You rebalance the portfolio based on the previous bar equity and previous bar close prices, but orders are executed based on the coming bar open prices. Open prices are usually different from previous close prices, therefore it maybe not enough money from sales to execute buy order, especially last one. Use 95-99% of the portfolio equity to mitigate this effect.

    To have earlier cash change notifications try to set quicknotify=True , might help to see the changes in cash, but I don't think that the issue is here - Docs - Cerebro - Reference

    @Sumit-Pandey said in margin problem when selling then buying in next:

    I am also having the same confusion. Seems like the BT takes in all orders in the loop at once and processes it later.

    Reading docs will help to avoid confusion and guessing Docs - Cerebro - Backtesting Logic



  • Thanks for replying to my post @Sumit-Pandey @ab_trader. I gave the 95-99% of the portfolio value a try and it still didn't solve the problem. If I understand correctly, this 95-99% is to reserve the amount of cash so that when the next open price is greater than the current close price, we'll still have sufficient funds for the purchase. There is however no guarantee that it will solve the problem. There might be 2 related issue here 1) is that I'm using the current bar valuation for funds allocation so I've to sell the stock first before purchasing (but same bar sell then buy doesn't work) 2) the changes in price that @ab_trader mentioned. Will continue investing this. I think in the worst case I can always force the selling first, then do all the buying in the bar afterwards, but since I'm testing this on daily timeframe, it would be best not to doing so.


Log in to reply
 

});