margin problem when selling then buying in next
-
Dear all, thanks in advance for reading this query. The Idea I'm currently working on involves the strategy discussed Stocks on the move, or as shown in this https://github.com/teddykoker/blog/blob/master/_notebooks/2019-05-19-momentum-strategy-from-stocks-on-the-move-in-python.ipynbl)https://github.com/teddykoker/blog/blob/master/_notebooks/2019-05-19-momentum-strategy-from-stocks-on-the-move-in-python.ipynb.
The problem I'm encountering is that there's a Margin call evening though that I've arranged the sell orders before the buys. During each call to next, the selling decision is made, then depending on the momentum of individual stock, buy orders will be issued by loop through the remaining available value of the portfolio. Looking at the log, I observed that
Even though the sell orders get executed before the buy orders, the value of the portfolio isn't updated. hence no available balance for the set of buying order.Ideally what i want to do is to (1) sell stocks based on criteria (then have the updated portfolio cash available) then (2) buy according to the ranked momentum (with risk parity sizing) until there's no cash left
# https://github.com/teddykoker/blog/blob/master/_notebooks/2019-05-19-momentum-strategy-from-stocks-on-the-move-in-python.ipynb from datetime import datetime import pandas as pd import matplotlib.pyplot as plt import numpy as np import calendar plt.rcParams["figure.figsize"] = (10, 6) # (w, h) plt.ioff() from scipy.stats import linregress import backtrader as bt class Momentum(bt.Indicator): lines = ('trend',) params = (('period', 90),) def __init__(self): self.addminperiod(self.params.period) def next(self): returns = np.log(self.data.get(size=self.p.period)) x = np.arange(len(returns)) slope, _, rvalue, _, _ = linregress(x, returns) annualized = (1 + slope) ** 252 self.lines.trend[0] = annualized * (rvalue ** 2) class Strategy(bt.Strategy): def __init__(self): self.i = 0 self.inds = {} self.spy = self.datas[0] self.stocks = self.datas[1:] self.spy_sma200 = bt.indicators.SimpleMovingAverage(self.spy.close, period=200) for d in self.stocks: self.inds[d] = {} self.inds[d]["momentum"] = Momentum(d.close, period=30) self.inds[d]["sma100"] = bt.indicators.SimpleMovingAverage(d.close, period=100) self.inds[d]["atr20"] = bt.indicators.ATR(d, period=20) def prenext(self): # call next() even when data is not available for all tickers self.next() def next(self): from utilites import week_number_of_month current_date = bt.utils.date.num2date(self.datas[0].datetime[0]) print("rebalance portfolio on " + str(current_date)) self.rebalance_portfolio() def notify_order(self, order): if order.status in [order.Submitted, order.Accepted]: # Buy/Sell order submitted/accepted to/by broker - Nothing to do return # Check if an order has been completed # Attention: broker could reject order if not enougth cash if order.status in [order.Completed]: if order.isbuy(): print(order.data._name + ' BUY EXECUTED, %.0f shares at $%.2f .PORT VAL $%.2f' % (order.size, order.executed.price,self.broker.get_value())) elif order.issell(): print(order.data._name + ' SELL EXECUTED, %.0f shares at %.2f .PORT VAL $%.2f' % (order.size, order.executed.price, self.broker.get_value())) self.bar_executed = len(self) elif order.status in [order.Canceled, order.Margin, order.Rejected]: #print(order.data._name + ' Order Canceled/Margin/Rejected, %.0f shares' % (order.size)) # , order.price)) print('---statust below---') print(order.status) print(order.data._name + ' Order Canceled/Margin/Rejected, %.0f shares, price %.2f .PORT VAL $%.2f' % (order.size , order.executed.price, self.broker.get_value())) # Write down: no pending order self.order = None def rebalance_portfolio(self): print('============================') # only look at data that we can have indicators for self.rankings = list(filter(lambda d: len(d) > 100, self.stocks)) self.rankings.sort(key=lambda d: self.inds[d]["momentum"][0], reverse=True) num_stocks = len(self.rankings) # sell stocks based on criteria for i, d in enumerate(self.rankings): if self.getposition(d).size != 0: is_close_position = False if i > num_stocks * 0.2 or d < self.inds[d]["sma100"]: print("exiting position for " + d._name + " due to exit conditions" + str( bt.utils.date.num2date(d.datetime[0])) + " " + str(self.getposition(d).size)) is_close_position = True if is_close_position: self.close(d,coc=True) #if self.spy < self.spy_sma200: # return """ print('------') for i, d in enumerate(self.rankings): if self.getposition(d).size != 0: print(d._name) print('-----') """ # buy stocks with remaining cash value_available = self.broker.get_value() for i, d in enumerate(self.rankings[:int(num_stocks * 0.2)]): if self.getposition(d).size == 0: size = value_available * 0.001 / self.inds[d]["atr20"] if value_available <= 0: print("rebalance_portfolio: value_avalaible " + str(value_available)) break self.buy(data=d, size=size) value_available -= size * d.close[0] print("rebalance_portfolio " + d._name + " size = " + str(size) + " value remaining=" + str( value_available) + " invested=" + str(size * d.close[0]) + " size=" + str( size) + " NUMSTOCK INVEST" + str(i)) else: print("rebalance_portfolio:: EXISTING " + d._name + " size= " + str(self.getposition(d).size)) print("rebalance_portfolio:: remaining value " + str(value_available)) def parse_iqfeed_csv_to_df(iq_feed_file, start_date_YYYYMMDD=None, end_date_YYYYMMDD=None): datetime_parser_ignore_tz = lambda x: pd.datetime.strptime(x.rsplit('-', 1)[0], "%Y-%m-%d %H:%M:%S") df = pd.read_csv(iq_feed_file, parse_dates=[0], date_parser=datetime_parser_ignore_tz) df.set_index('datetime', inplace=True) df = df.sort_index(axis=1) if start_date_YYYYMMDD is not None and end_date_YYYYMMDD is not None: df = df.loc[(df.index > start_date_YYYYMMDD) & (df.index < end_date_YYYYMMDD)] return df if __name__ == "__main__": casename = "test_snp_margin" # loop to find all the tickers import os import time t0 = time.time() tickers = [] max_number_of_stocks = 500 # only use this when you want to limit the stock universe (for testing only) start_date = '2008-01-01' end_date = '2010-06-26' # As we can see in the code, the strategy looks for stocks it needs to sell every week in the rebalance_portfolio method and rebalances # all of its positions every other week in the rebalance_positions method. Now let's run a backtest! cerebro = bt.Cerebro(stdstats=False) #cerebro.broker.set_coc(True) #cerebro.broker.set_checksubmit(False) index_file = "SPY_20080101_20200824_23400.csv" benchmark_data = parse_iqfeed_csv_to_df(index_file, start_date_YYYYMMDD=start_date, end_date_YYYYMMDD=end_date) cerebro.adddata(bt.feeds.PandasData(dataname=benchmark_data, name="SPY", plot=True)) cerebro.addobserver(bt.observers.Benchmark) cerebro.addobserver(bt.observers.TimeReturn, timeframe=bt.TimeFrame.NoTimeFrame) src_data_dir = r"C:\Users\thomas\PycharmProjects\pyiqfeed_dataDownloadOnly\MARKETDATA_iShares-Core-SP-500-ETF_fund" #src_data_dir = r"C:\Users\thomas\PycharmProjects\pyiqfeed_dataDownloadOnly\MARKETDATA_iShares-NASDAQ-100-Index-ETF-CAD-Hedged_fund" # src_data_dir = r"C:\Users\thomas\PycharmProjects\pyiqfeed_dataDownloadOnly\MARKETDATA_iShares-Russell-2000-ETF_fund" count=0 for file in os.listdir(src_data_dir): if count < max_number_of_stocks: # df = pd.read_csv(os.path.join(src_data_dir,file), parse_dates=True, index_col=0) ticker = file.split('_')[0] df = parse_iqfeed_csv_to_df(os.path.join(src_data_dir, file), start_date_YYYYMMDD=start_date, end_date_YYYYMMDD=end_date) if len(df) > 100: # data must be long enough to compute 100 day SMA cerebro.adddata(bt.feeds.PandasData(dataname=df, name=ticker, plot=False)) # print(df) print("added " + ticker) tickers.append(ticker) count += 1 # Set our desired cash start cerebro.broker.setcash(10000.0) cerebro.addstrategy(Strategy) results = cerebro.run() -
Here's a log print out that show after the sell order execution, the portfolio values/ cash remains unchanged. I'd want to know how is it possible for the strategy to update the cash after sell, before buying
ODFL SELL EXECUTED, -5 shares at 8.85 .PORT VAL $9449.18 .PORT CASH $2472.54
NI SELL EXECUTED, -18 shares at 6.08 .PORT VAL $9449.18 .PORT CASH $2472.54
AEE SELL EXECUTED, -4 shares at 27.61 .PORT VAL $9449.18 .PORT CASH $2472.54
CMCSA SELL EXECUTED, -6 shares at 8.39 .PORT VAL $9449.18 .PORT CASH $2472.54
ES SELL EXECUTED, -4 shares at 25.77 .PORT VAL $9449.18 .PORT CASH $2472.54
DTE SELL EXECUTED, -3 shares at 43.05 .PORT VAL $9449.18 .PORT CASH $2472.54
MNST BUY EXECUTED, 17 shares at $6.59 .PORT VAL $9449.18 .PORT CASH $2472.54
PWR BUY EXECUTED, 4 shares at $21.36 .PORT VAL $9449.18 .PORT CASH $2472.54
VRSN BUY EXECUTED, 4 shares at $24.90 .PORT VAL $9449.18 .PORT CASH $2472.54
SIVB BUY EXECUTED, 2 shares at $42.58 .PORT VAL $9449.18 .PORT CASH $2472.54
CTXS BUY EXECUTED, 3 shares at $34.03 .PORT VAL $9449.18 .PORT CASH $2472.54 -
@darkknight9394 said in margin problem when selling then buying in next:
order.data._name
I am also having the same confusion. Seems like the BT takes in all orders in the loop at once and processes it later.
How can we update the cash position in the loop itself? -
@ab_trader Can you please help regarding this issue.
-
@darkknight9394 said in margin problem when selling then buying in next:
Even though the sell orders get executed before the buy orders, the value of the portfolio isn't updated. hence no available balance for the set of buying order.
Cash is updated internally after each executed sell/buy order, but the broker cash amount is delivered to the strategy level only at the next price available. On your issue my guess (since your logs don't show any issues, than guess only) would be is that you affected by differences in close and open prices. You rebalance the portfolio based on the previous bar equity and previous bar close prices, but orders are executed based on the coming bar open prices. Open prices are usually different from previous close prices, therefore it maybe not enough money from sales to execute buy order, especially last one. Use 95-99% of the portfolio equity to mitigate this effect.
To have earlier cash change notifications try to set
quicknotify=True, might help to see the changes in cash, but I don't think that the issue is here - Docs - Cerebro - Reference@Sumit-Pandey said in margin problem when selling then buying in next:
I am also having the same confusion. Seems like the BT takes in all orders in the loop at once and processes it later.
Reading docs will help to avoid confusion and guessing Docs - Cerebro - Backtesting Logic
-
Thanks for replying to my post @Sumit-Pandey @ab_trader. I gave the 95-99% of the portfolio value a try and it still didn't solve the problem. If I understand correctly, this 95-99% is to reserve the amount of cash so that when the next open price is greater than the current close price, we'll still have sufficient funds for the purchase. There is however no guarantee that it will solve the problem. There might be 2 related issue here 1) is that I'm using the current bar valuation for funds allocation so I've to sell the stock first before purchasing (but same bar sell then buy doesn't work) 2) the changes in price that @ab_trader mentioned. Will continue investing this. I think in the worst case I can always force the selling first, then do all the buying in the bar afterwards, but since I'm testing this on daily timeframe, it would be best not to doing so.
