Store Development

ThatBlokeDave

Hello,
I really would like to have a crack at adding a store for live trading. I understand my preference in store might not be aligned with everyone else so I think the only way to satisfy my personal requirements is to get my hands dirty.

My problem.... The inner workings of backtrader are a complex (at least to me!) I took a look at the source code and I am a bit lost as to where to start.

Would it be possible to document or do a blog post for a dummies guide to integrating 3rd party stores? Things like a check list of integration points, how to provide the data stream, expected formats etc..

An even better dream for me would be to have a standard API that we can use. Then just massage the providers data responses / streams and plug it into the platform.

Maybe I am trying to run before I can walk but it would be great to learn more.

backtrader

Let's try to summarize:

Non-Live Data feeds have well defined interfaces and it is documented

You only have to override _load(self), which will be in charge of loading the values into the lines of the data series (in most cases these will be: datetime, open, high, low, close, volume and for futures openinterest

It returns True if it has been able to fill the next set of values or False to indicate the end of the data stream.

What's left for the implementer: the decision as to how _load(self) receives the values from the feed. Examples:
- From a file: for each call to _load you can simply read the next line, process the data, fill the lines and return, until EOF is met.
  
  There is even documentation of how to do it for a binary file Docs - Binary Datafeed Development
  
  This has been for example generalized for CSV-based sources by adding a _loadline method, which receives the line broken down into tokens. In this case only overriding of _loadline is needed. See Docs - CSV Data Feed Development
Live Data Feeds could have a couple of things added
- Methods islive and tzoffset, the chosen pattern for data reception and the extra return value for _load which is None to indicate that the data feed has currently nothing to deliver but could have it later (the data stream is active but has not come to an end)
  
  The problem with the rest ... it is provider dependent. And the hackhaton (aka BacktraderCon 2017) last weekend has proven it. The initial implementation the guys had followed the guidelines from the Oanda implementation, but because the provider (Kraken) only offers polling and has low limits for its Rate Limiting policy, everything is a historical download at the end of the day. Suddenly instead of 2 queues, both queues are the same but the usage is different.
Broker have a well defined interface

Here it is really a matter of work in the store, which offers a private interface to the broker. Through this interface for example the store will convert a order from backtrader to an order which is palatable by the real broker. On the way back, broker notifications have to be adapted, to change the status of orders appropriately.

A kind of paper to give guidelines can be considered (will be done so to say), but at the end of the day and the guys from the BacktraderCon could tell a lot about it, it's about the very small details from each broker.

ThatBlokeDave

Thanks! I will take a look when I get chance and have a go when I get the chance.

Quỳnh H Nguyễn

I want to continue this thread, although it is so old, because I have a simple relative question.

@backtrader : In case I have a custom live data feed, and I want to use backtrader in the middle level - generate trade signal only - no live trade. So no need to build a custom store, only develop a live data feed right?

Thank you very much!

Quỳnh H Nguyễn

I check on the backtrader source code. There is a Chainer data feed, but I don't see any example to use this type of data feed (Chainer).

It looks like a simple live data feed example that could be extended to any new data source live by provide background threads to manage online source. Right?