HTTP Rest Datafeed

polr

Hello,

First, I really love this framework. I've learned a lot just reading the documentation and source code and I find it great to work with. It's really a thoughtful and well executed project.

I have a question on general architecture and more specifically datafeeds.

I would like to apply a single strategy on several commodities markets and get performance data on this strategy at a portfolio level. I've seen one discussion on this:

https://community.backtrader.com/topic/9/portfolio-inventory-management/2

That's fairly clear (at least before working on it directly I understand the concept), but in relation to data feeds and contract rolling I was wondering how best to handle the situation.

From what I understand from the documentation rolling futures involves entering all contracts as independent data feeds and these data feeds are given as input to bt.feeds.RollOver (for quick reference: https://www.backtrader.com/docu/data-rollover/rolling-futures-over.html).

My question is this -- if I want to do a test across WTI for example from 1995 to 2010 I have 180 data feeds to enter (12 * 15). Then if I want to test the portfolio across 25 products I get into the thousands of feeds quite quickly. Am I correct in my understanding?

Another option is completing the rolling operations externally, storing this data in a csv and using a rolled csv as a data feed. This is how I currently use the system but it becomes problematic when I do operations on a look back window, for example. I need to do operations externally of backtrader. I would prefer to keep the strategy implementation fully inside back trader.

I have a function that takes a datetime object as a parameter and returns a list of the front, first roll, second roll etc for the given date, among other information. I was wondering if it was possible to provide a function as input to a data feed such that the data feed provided to the Strategy object is in essence a symbol and a rolling function (in other words, it doesn't yet hold data but knows what to do when the strategy iteratively provides it a date during the implementation of the strategy.

Hopefully this makes sense -- any thoughts or ideas would be greatly appreciated.

backtrader

@polr said in HTTP Rest Datafeed:

My question is this -- if I want to do a test across WTI for example from 1995 to 2010 I have 180 data feeds to enter (12 * 15). Then if I want to test the portfolio across 25 products I get into the thousands of feeds quite quickly. Am I correct in my understanding?

12 * 15 seems quite right. If you have 25 different assets and you want to backtest of all them at the same time, yes, you will have 12 * 15 * 25

If

@polr said in HTTP Rest Datafeed:

I have a function that takes a datetime object as a parameter and returns a list of the front, first roll, second roll etc for the given date, among other information. I was wondering if it was possible to provide a function as input to a data feed such that the data feed provided to the Strategy object is in essence a symbol and a rolling function (in other words, it doesn't yet hold data but knows what to do when the strategy iteratively provides it a date during the implementation of the strategy.

Such a data feed would be possible, but it doesn't exist at the moment. You simply have to override _load, and when called provide the data. From inside _load you can decide where the next batch of data is coming from.

In any case you end with the same problem as above: you need to be able to fetch 12 * 15 data feeds. Whereas above you do it in a loop/list comprehension (quite easily if the naming is consistent), you pass the complexity into a different beast if you make a data feed.

Example with a list comprehension

basename = `myfeed-{}.txt`
dfeeds = [bt.feeds.GenericCSVData(dataname=basename.format(i) for i in range(1995, 2011)]
cerebro.rollover(*dfeeds)

That should do the trick (you may obviously do extra things, like setting the format of the datetime field or others and it may be better to unroll the comprehension and have a loop with a couple extra lines)