Multi-asset ranking and rebalancing

timzhang

First of all, I want to thank you for building and maintaining this great project. I only started learning backtrader a few days ago, but I've already liked it comparing to a few other systems I've used in the past.

Just a quick question on backtesting an asset allocation strategy. My idea is to rank several assets based on prices and some fundamental data, and then rebalance my portfolio each month. What's the best way to approach this using backtrader? Is it matter of just loading multiple datasets, do a ranking calculation within 'next' and then work out the trade position for rebalancing? I saw you mentioned something about multi timeframe in a post on multi instrument. I've got a bit confused here.

Thanks

backtrader

Dealing with a multi-asset scheme is a lot easier than dealing with a multi-timeframe scheme. Assuming that in the former, your use case, all assets carry the same timeframe/compression signature.

Rebalancing would ideally be achieved by means of order_target_percent.

The family of order_target_xxx is documented here: https://www.backtrader.com/docu/order_target/order_target.html

order_target_percent will use the current total value of your portfolio + cash and then try to adjust the position of the asset in the order to match the target percent.

When working with large percentages it would be good to disable checksubmit in the broker (See Docs - Broker). The rationale being that if the current total portofolio is large, the broker might reject some orders if no cash is left.

You could order the orders in such a way that this doesn't happen, but disabling checksubmitwould be easier.

This could happen as follows:

Imagine that your current total position is close to 100%
The first order to do the rebalancing of several assets increases the position
This makes the total value go over 100% and cash enters negative territory
A subsequent order for another asset would reduce the position and therefore replenish the cash, but unfortunately the 1st order has already been rejected.

timzhang

@backtrader Thank you, I'll try it out!

timzhang

@backtrader
In terms of use order targer to rebalance, how does backtesting broker execute the orders? Do I set_coc to True and set_checksubmit to False? I loaded 10 different monthly return series and try to trade top 5 equal weight based on last 3 month returns. But my below strategy gave me some really extreme large final portfolio value. I think probably something wrong with the way I execute my orders?

Anything is about how I use ranking in bt, do you reckon it is the correct approach or should I implement everything under 'next'?

class TestStrategy(bt.Strategy):

    def __init__(self):
        # calculate last 3 month return
        for i in range(0, 10):
            self.datas[i].rtn = (self.datas[i].close(0) / self.datas[i].close(-3)) - 1


    def next(self):
        rtn_ls = np.array([self.datas[i].rtn[0] for i in range(0, 10)])
        rtn_target = np.percentile(rtn_ls, 50)
        for i in range(0, 10):
            if self.datas[i].rtn[0] > rtn_target:
                self.order_target_percent(target=0.2)
            else:
                self.order_target_percent(target=0.0)

Thank you

backtrader

@timzhang said in Multi-asset ranking and rebalancing:

In terms of use order targer to rebalance, how does backtesting broker execute the orders?

The order_target_family does simply translate a requested unit (target_size, target_value, target_percentage) into an order with the needed size to achieve (or come as close as possible) the requested target_xxxx.

As such the execution type can be specified to order_target_xxx and will be passed to the generated order. If not specified and as in the case of buy and sell, the default execution type is Market.

Do I set_coc to True and set_checksubmit to False?

That's a decision only you can make depending on your scenario, considering whether you need the closing price of the bar under evalution or the next incoming price and if you want the order getting in regardless of cash requests.

Anything is about how I use ranking in bt, do you reckon it is the correct approach or should I implement everything under 'next'?

No experience with that ranking methodology, so nothing can be really said.

timzhang

@backtrader
Thank you, I got the order_target working. My previous error is due to not specifying the particular data series, as multiple assets are traded in this strategy.

But still having some problems with set_checksubmit. Even I set it to False and set_coc to True, my pending buy order still not get executed. It fell off due to Margin. I can see there's enough Cash once I execute the sell orders. But the buy order was dropped first before the sell orders get executed. Any ideas on work around this issue?

backtrader

Without knowing the actual specifics the only good way to overcome your approach is:

Don't go all in

or

Increase the amount of cash and restrict yourself to a well-known value

The most probable case is that you are exceeding the cash limit even if it's only by some cents.

timzhang

Thank you @backtrader
would it make more sense to execute the sell trades first then buy within bbroker's 'next' when backtesting strategy? After all, you have to sell first to fund the buy trades.

backtrader

Also. Orders are executed in the order in which they are given to the system.

timzhang

@backtrader
Thank you for mentioning that the trades are executed in the order been given to the system. I think if I want to change how it is filled, I can override the self.broker.pending variable at the end of next function under my strategy. I can convert deque to a list, sort it and then convert it back to deque. The only problem I have now is to sort a list of broker.order objects. What's the best way to do that?

backtrader

Docs - Orders

There is the reference for the order objects. You probably want ref.

Orders are not actually executed as given. Execution is attempted as given. For Market orders both will be the same, but this won't hold true for other things like Limit

timzhang

@backtrader
cool, got order_target_percent working for my long-only portfolio. Just wondering if I'd like to implement a long short strategy, do I use negative portfolio weight as target in order_target_percent?

backtrader

Uncharted territory. Give it a try!

cnimativ

@timzhang @backtrader
Is there anyway to reference data by their name/ticker native in backtrader? Debugging data0, data1, etc gets confusing at times especially after we work with more than a dozen tickers!

RandyT

@cnimativ This is not ideal, and I have yet to do this for managing more than a handful of symbols, but you can do the following in strategy.init().

    self.data_ES = self.getdatabyname('ES')
    self.data_SPY = self.getdatabyname('SPY')
    self.data_GLD = self.getdatabyname('GLD')
    ...

Of course the 'names" must be assigned when creating the data feeds but it does help.

    cerebro.resampledata(data0, name='SPY', timeframe=..., compression=..)
    ...

backtrader

The rationale behind working with either data0... dataX or datas[x] is that the developed algorithm doesn't actually depend on specific products, which is usually the case if one is using names.

The example from @randyt, is a good case: data_ES, which seems to indicate that something is being specifically tailored for the ES (Mini probably). Of course this also a perfectly valid use case, because someone may be willing to only develop against a specific set of things. Or because the names are set generically as could be with Bullish vs Bearish (one would be always working with 2 instruments, each having a behavior matching those names)

Some ways to address this:

Create references using getdatabyname as done by @RandyT
Have something else like
- self.datanames.x and/or self.datanames["x"] (or a shorter version like dnames)

Any opinions?

RandyT

Just to expand briefly on my current use case.. @backtrader presents a good reminder that tailoring to specific instruments can be a fools path. :-)

I am currently working on a strategy that trades on index futures, but uses other data feeds as signalling data feeds. It is helpful to to easily see the name in the reference to know you are passing the correct dataset to the correct indicator, etc.

What might be helpful from my perspective, is the ability to put certain datasets in classes of data. So a class created for the actually instruments you are executing trades against, vs. a class for signal generation or feeding to indicators in my use case. That might also enable us to ignore certain classes of data as it ticks through strategy.next(). I've written some ugly code to attempt to only look at the specific data values I care about in that part of the strategy and ignore others.

Regarding your suggestions @backtrader, I like shorter names like dnames and the ability to use dotted notation to reference them opposed to indexes.

As I start looking more at pairs trading, there also seems to be a use case to be able to easily identify which of the pair members you are operating on.

My .02

cnimativ

@backtrader @RandyT

My use case is that I am using backtrader to backtest asset allocation strategies with ETF inputs in the typical OHLCV data and inputs with single point daily data, e.g., Treasury rates and daily indices. One plan for the future is running/applying deep q learning models but that's obviously down the line.

I prefer the second option with shortened name like dname. Would be nice if dname support pulling multiple data with wildcards into a pandas df.

Currently for backtrader, we reference to different tickers as self.datas[0], self.data0, etc. It makes life hard when a strategy involves in several instruments and harder when a whole bunch of other non-tradeable-data is loaded. IMO, it is intuitive reference data as, self.data.<dataname/ticker> or self.data["<dataname/ticker>"] when we add data via cerebro.adddata(data, name=<dataname/ticker>).

@RandyT Wouldn't reference data by their name allow us to construct signals already? What do you mean specifically by ignoring certain classes of data?

backtrader

@cnimativ said in Multi-asset ranking and rebalancing:

self.data.<dataname/ticker>

The major problem with using self.data.xxxx is that self.data is already taken. The rationale for choosing it as an alias for self.data0 and therefore for self.data[0] is that many use cases involve a single data feed and users shouldn't have to thing about indexing (not even with self.data0)

Hence, for example, the proposal for something like self.dnames.xxxx

RandyT

@cnimativ
What I am describing I believe is the same thing you are doing.

I have a number of different data sources that I am pulling into a strategy. Some of them in different timeframes than the instrument I am actually executing trades on. Anything that is not a data source for an instrument I want to send buy/sell orders on is "reference" data by my definition. I don't really care about any data I see in strategy.next() other than the indicator values I have defined and the current time, which I am getting from the data source I am trading.

In my case (in this first system I have been porting to BT), I am looking for the closing bar for the NYSE instrument and the values of the indicators calculated at the close of that bar to trigger an order on the futures contract which I am seeing a bar for every minute. All other data sources I have configured (which are all ticking through next()) are noise that I need to filter out to all me to focus on the few things of importance.

Hope that makes sense. Sounds like you are doing something similar with Treasury rates, etc. being the "reference" data.