Multi-asset ranking and rebalancing

backtrader

@backtrader said in Multi-asset ranking and rebalancing:

Hence, for example, the proposal for something like self.dnames.xxxx

Further considerations on this:

Some versions ago self.datas in Cerebro was decoupled from the actual internal array used for looping over the data, to automatically sort the data feeds according to timeframe / compression and better synchronize timestamps.

This opens a window to use this pattern in strategies

self.datas[i] where i is an integer

and

self.datas[name] where name is a string

Some additional options, following conventions found for example in pandas

self.datas.iloc[i]
self.datas.loc[name]

Note

The rationale to name the data feeds array as datas was a conscious decision even if datas is not correct (Latin: datum -> data, English: data both for singular and plural) to differentiate it from self.data which points to the 1st of the data feeds (and in many cases the only one) and to make clear with the artificial plural that self.datas is an array (or collection of items)

backtrader

A first approach is implemented with this commit in the development branch:

https://github.com/mementum/backtrader/commit/a403a8e5a6f2783ab808501f2e952b2ed23b2a7a

Data instances can be reached as:

print('Closing YHOO price is:', self.dnames.YHOO.close[0])

or

print('Closing YHOO price is:', self.dnames.['YHOO'].close[0])

Of course the name YHOO has to be assigned during data addition to cerebro

cerebro.adddata(myadta, name='YHOO')

cnimativ

@RandyT
Yes, we have similar methods, though I am not 'trading' per se, but rebalancing/allocation on a monthly basis.

I am attempting to implement my vectorized backtesting in Python/Excel to BT to automate the whole process. Definitely some kinks to work out!

cnimativ

@backtrader Is there a way reference orders w/ tickers? Right now it seems that the orders are placed based on data series, not by name of the ticker/security.

It is very hard to figure out the fill and share count for each ticker especially during rebalancing.

backtrader

@backtrader said in Get name of data inside the indicator:

An Order instance (let's call it order) carries an attribute data which is the asset on which the order has been issued. The ticker name should be available as: order.data._name (where _name is the name you have assigned when using adddata)

Names are actually a late addition to backtrader, because one of the underlying design concepts is that the development of a trading idea should, ideally, not be bound to a specific asset.

cnimativ

@backtrader Thank you for the tip. I think the underlying design concept of a 'trading idea' is true for technical indicators but is not applicable to other quantitative strategies.

How about for backtesting securities with long date ranges where Adj Close is materially different than Close? For example, At the end of the series, Adj Close == Close, but at the beginning of the series, Close is much higher than Adj Close for companies offering dividends.

Correct me if I am wrong, Backtrader simulates using unadjusted Open and Close data, not Adj Close (or adjusted open when trading at the open), so the long-term results will differ materially. I am having a hard time reconciling between Excel and Backtrader, where my Excel is the ground Truth for simple strategies.

Maybe I should backwardly adjust my pricing data before feeding it into Backtrader.

ab_trader

For Yahoo data feeds default behavior is to use adjusted close. Use adjclose params to change it.

https://www.backtrader.com/docu/dataautoref.html, check out YahooFinanceCSVData.

backtrader

@cnimativ said in Multi-asset ranking and rebalancing:

Correct me if I am wrong, Backtrader simulates using unadjusted Open and Close data, not Adj Close (or adjusted open when trading at the open), so the long-term results will differ materially. I am having a hard time reconciling between Excel and Backtrader, where my Excel is the ground Truth for simple strategies.

backtrader makes absolutely no assumptions about the prices. Data feeds provide the prices.

As mentioned by @ab_trader the YahooFinanceXXXData feeds use the adjclose price offered by the Yahoo delivered data to adjust the 4 price components (OHLC) before passing them into the system. And as also stated by @ab_trader, you can disable that.

cnimativ

@ab_trader @backtrader Thanks! I will try that.

mac0

Hi,

I am new to backtrader but it looks promising!

I'm trying to replicate some tactical asset allocation strategies. Hence, I need to rebalance between assets (every month in this case).

As I understand, I would need to use the "order_target_percent" but when doing so, many trades are being rejected due to margin constraints. I found that you could "cheat on open" to use the next open to compute the trade size. However, it doesn't seem to work in conjunction with "order_target_percent". My orders are still rejected, the size of the order being the same in both cases.

To make things simple, I just run the example "cheat-on-open.py" and I replace the line "self.order(buy) = self.buy()" by "self.order_target_percent(target=0.999)".

Any idea?

backtrader

First (although it's assumed it's already being done)

The sample must be executed with --cerebro cheat_on_open=True.

Second:

oder_target_percent needs a price to translate the percentage and it has therefore to default to something (actually done by its relative order_target_value) which is data.close. This is so, because it has to create an order with a given size to get you to the wished %. Whereas buy(size=x) already knows which the size is.
Invoke oder_target_percent with price=data.open[0]

mac0

Working like a charm! Thank you!

mac0

@timzhang said in Multi-asset ranking and rebalancing:

@backtrader
cool, got order_target_percent working for my long-only portfolio. Just wondering if I'd like to implement a long short strategy, do I use negative portfolio weight as target in order_target_percent?

How did you get it to force the broker to execute sell orders first and then buy orders? Is it possible to do it after the orders have been submitted?

Thanks!

backtrader

Execution is attempted following order insertion in the system. If all the orders are of type Market, they will obviously be executed sequentially.

Limit and other execution types which have conditions, will of course be executed when the condition(s) are met.

mac0

@backtrader

Not sure how changing order from Market to Limit will change the order of my orders as all are assumed to be filled on next open (cheat-on-open True).

How would you force backtrader to execute first sell orders to avoid margin? Is it possible to do it after the orders have been submitted?

backtrader

If you set an order to Limit and the limit execution price to a value which is not in the range of the high - low range, the order will not be executed. If that Limit order was sent 1st and a Market order was sent afterwards, the Market order will be executed and the Limit order will remain in the system awaiting the next execution attempt (when a new set of prices are delivered)

my orders as all are assumed to be filled on next open (cheat-on-open True)

That's wrong. You get a chance to do something before the open. That doesn't mean your orders are assume to be filled. If you set unrealistic execution conditions, your orders will for sure not be executed.

How would you force backtrader to execute first sell orders to avoid margin? Is it possible to do it after the orders have been submitted?

This was answered above. Execution order is attempted following the order in which the orders were sent to the system. You decide what will be executed first (if all orders are Market)

Not it cannot be done afterwards. You cannot do it in a real broker/exchange either.

mac0

thanks.

Would it make sense to have an intermediary step between next_open() and when order are submitted in order to be able to order all orders that are to be sent for execution at the same time? It would avoid margin rejection in backtests. I understand that in real market condition you wouldn't know which of the orders will be executed first but one usually doesn't mind being on margin for a few seconds.

backtrader

@mac0 said in Multi-asset ranking and rebalancing:

Would it make sense to have an intermediary step between next_open() ...

Not it wouldn't. One of the goals is to try to be as close to reality as possible. If you send Market orders to the market, they will get also executed and you will meet the same rejection policy from the broker because you don't have the cash.

@mac0 said in Multi-asset ranking and rebalancing:

to order all orders that are to be sent for execution at the same time

The platform (and the broker within it) doesn't know what your intention is and if all orders have to be executed at the same time. The policy is the same as you have when you submit the order to a broker (and is finally sent to an exchange): First-Come-First-Served.

Obviously and as explained above, orders with conditions may remain in the queue if the condition is not met.

mac0

@backtrader said in Multi-asset ranking and rebalancing:

Not it wouldn't. One of the goals is to try to be as close to reality as possible. If you send Market orders to the market, they will get also executed and you will meet the same rejection policy from the broker because you don't have the cash.

I agree with that but there is no reason why sell orders can't be sent first in real life as well. Anyways, I could manage to force the order to start with the sell orders first but I am not yet out of my mess...

getvalue() is not equal to the sum of its components (cf snippet below) + cash ... How is it possible? both are using the last close even with coo, right?

My apologies for all these stupid questions...

print cerebro.broker.getvalue()
print cerebro.broker.cash
for s in self.getdatanames():
         print self.broker.getvalue(datas=[self.getdatabyname(s)])

backtrader

@mac0 said in Multi-asset ranking and rebalancing:

I agree with that but there is no reason why sell orders can't be sent first in real life as well

You are actually making the decision on which orders get sent first. If you want to send sell orders first, and without knowing what your algorithm/logic does, could be as easy as sorting the target % according to the change. The % being reduced (something will be sold) will be the first ones to be realized through an order_target_percent

@mac0 said in Multi-asset ranking and rebalancing:

getvalue() is not equal to the sum of its components (cf snippet below) + cash

The snippet doesn't help much without the actual values the snippet prints. But since you are using cheat_on_open the following may apply:

The global value is calculated after the close and cached (the value is used by several components)
The individual values are calculated on the spot and use the new close price. The broker doesn't know you are cheating and individual data value calculation is not used everywhere

You can cache the individual data values in next and access them later in the new cycle during next_open