Multiple asset with missing data

Aris L

I'm trying to load multiple assets, however only last day's data is coming out. I assume when there're missing data in one asset on a day, the system will skip that day for all assets. It's quite likely on a particular day that some stock is not trading, so collectively a large number of days are being skipped.

Would it make sense to change this behavior? I guess if the asset doesn't trade on one day, it's fine to skip for that asset, but other assets with data should still run.

backtrader

@aris-l said in Multiple asset with missing data:

the system will skip that day for all assets

No.

Unless you provide more information, nothing can be really stated.

Aris L

@backtrader said in Multiple asset with missing data:

No.

Thanks, seems working for me now. Turns out prenext gets called, and next doesn't. That means the logic should probably be put in both prenext and next.

For information, I tried
stock A: 2018-11-30 to 2018-12-10.
stock B: 2018-12-05 to 2018-12-10.

The result is:

prenext. A 2018-11-30
prenext. B 2018-11-30
prenext. A 2018-12-03
prenext. B 2018-12-03
prenext. A 2018-12-04
prenext. B 2018-12-04
next. A 2018-12-05
next. B 2018-12-05
next. A 2018-12-06
next. B 2018-12-06
next. A 2018-12-07
next. B 2018-12-07
next. A 2018-12-10
next. B 2018-12-10

The strategy code is:

class LogStrategy(bt.Strategy):

    def prenext(self):
        for data in self.datas:
            curr_date = self.datas[0].datetime.date().strftime('%Y-%m-%d')
            logger.info(f'prenext. {data.AssetId[0]:.0f}, {curr_date}')

    def next(self):
        for data in self.datas:
            curr_date = self.datas[0].datetime.date().strftime('%Y-%m-%d')
            logger.info(f'next. {data.AssetId[0]:.0f}, {curr_date}')

Aris L

Initially I thought 'next' for stock A should get called with all the days, but instead it gets truncated by the lesser stock B.

backtrader

@aris-l said in Multiple asset with missing data:

Turns out prenext gets called, and next doesn't.

You probably want to read this:

And concentrate on the sections that describe prenext, next and nextstart. It seems you are trying to do something akin to survirvorship bias, so you may want to see the discussions in threads like:

Community - Backtesting multiple symbols with different date ranges - backtest won't start until all data feeds have data

You should also be aware that in prenext not all guarantees for the buffers can be met.

@aris-l said in Multiple asset with missing data:

Initially I thought 'next' for stock A should get called with all the days, but instead it gets truncated by the lesser stock B.

This statement is simply wrong.

Aris L

@backtrader said in Multiple asset with missing data:

This statement is simply wrong.

From the above example, first 'next' call for A is on 2018-12-05, even though A has data 2018-11-30. From method calls perspective, A's 'next' calls gets truncated.

I don't mean by actual data, actual data can't be truncated, because it still gets called in 'prenext'.

backtrader

You fail to understand what's happening. There is no truncation. The framework will only call your next when all the buffers can be guaranteed to have data available (i.e.: you can always read the value at [0])

Before next you have prenext in which such guarantee doesn't exist. You can do many things in prenext but you need to make many manual checks.

You probably want to read again the documents linked above.

Aris L

Indeed, I'm still following through quick start and exploring from there. Will read document links at further steps.

Though prenext is probably a good idea to put in the start, as it's natural concept if multi datafeed is considered. Or even better, it'll be great to have a multi-asset version of quickstart by itself.

backtrader

@aris-l said in Multiple asset with missing data:

as it's natural concept

It is not even if you think it is. You are confusing multiple assets with assets that fit the survivorship bias.

And backtrader is not, with the automatic provisions, suited for the survivorship bias. It is suited for multiple assets but not for the scenario you want.

Aris L

That's why a multi-asset version of quickstart would be very helpful for starters to clarify things, as it would touch upon things that single-asset wouldn't.

So you mean backtrader's multiple asset assume all assets have the same length (therefore, no survivorship bias present). It doesn't support the multiple assets with different length.

backtrader

No. Different lengths are perfectly supported. You can simultaneously trade the SPX and the DAX, which have different trading calendars and different lengths.

Your concrete problem is the warm-up or minimum period which is needed to ensure all buffers can produce something at [0]. If you have things starting at different dates, the buffers cannot be guaranteed to deliver until the latest date has been reached.