Thank you for your answers!
To be sure we are on the same page, I'd like to specify how I would write the process if I had to code it from scratch not using BackTrader and dealing with realtime.
Let’s add a simple moving average system to complicate the things a little bit.
For every day in test and for real time:
- Ask the data provider (I use Bloomberg professional but I can also download data into CSV/Excel from it) the composition of index "X" at the date. For Eurostoxx 50 it would give me a list 50 tickers. Obviously this list will change in time, usually it is reviewed every three months. I could optimize the request to be made only at the end of the quarter but to be sure let’s ask Bloomberg the membership data every single day.
- Ask the data provider the needed n historical data (PX_LAST) of all the tickers in the members list and for every Series compute a moving average on it. Let’s say a 50 day moving average.
- If a ticker is in this list but not in my portfolio, BUY it at next open if the last close is above it's moving average.
- If a ticker is in my portfolio but not in the list (so it's not anymore a member of the index), SELL it at next open. I don’t won’t it to be part of my portfolio.
- If a ticker is in my portfolio and it's also in the index list but the last close is below the moving average sell it at next open.
- Compute the size of each new position according to some formula (equal weight to be simple but could be some more complex algorithm like minimum variance, risk parity…)
Now, If I only want to backtest and not to deal with realtime trading I have a big advantage.
I could ask Bloomberg (or any other data provider that can give me this information) the historical composition of the index I'm tracking for each day in the test. In the end, at the very last day, with a simple loop I can get all the tickers that have been members of the index in the period of time between test_start_date and test_end_date. This is why I thought I could simply add a column “membership” with a Y/N flag.
You’re right about the fact not all the historical series would start the same day. We have two options here:
- Find a way to have BackTrader asking the CSV data provider, at each heartbeat, the composition of the index, the same as I’d do with Bloomberg in real time if I had to write the whole process from scratch. Is it possible with BackTrader?
- Cheat a little bit with the benefit of insight. Align, before importing the data, all the series to start at the same day with pandas. This could mean having to add anther column to be sure I won’t buy/sell a ticker on fake data. Sure, I would have to pre-process all the data with pandas before feeding BackTrader with it but it wouldn't be a big deal. And if done with care it won't damage the validity of the test.
I'd like to know if there are better ways to handle this.
Thanks, I am well aware of all the problems to compute the size using the last close, the only data we know for sure, and the possibile slippage I would get at next open (positive or negative). I guess the best option is, as you pointed out, to keep some free cash (5%?) and to prioritise the sells orders to be sure to get all the buy orders filled. I didn't mention it in my posto only to keep the things the simplest as I could