What's the right way to use data from API in a strategy?

brettelliot

Hi everyone,

Thanks again for the great platform! Having a lot of fun working with it.

I've built an indicator that reads a csv file of events and my backtest can use those events to make trades. But for live trading, I'd like to hit a JSON api for the event data.

I could call it in the strategy's next() method. Or I can do that in the indicator and use the indicator in the strategy. Or maybe I should make a feed (cause that's the only place i've seen live connections being used).

What's the preferred way to do this?

Thanks!

backtrader

@brettelliot said in What's the right way to use data from API in a strategy?:

I could call it in the strategy's next() method

That would probably be fatal. But it depends on the speed at which you have to make decisions. You'd be blocking the main thread.

@brettelliot said in What's the right way to use data from API in a strategy?:

What's the preferred way to do this?

There is no preferred way. My personal opinion:

It is a light processing stream of events: put it in a background thread and put the incoming events in a queue.Queue, which is thread safe. When entering next, you can check if an event is waiting in the queue (pushed by the background thread) or not and do something else.

Even with the GIL and because processing is light, using threads seems sensible.
The stream of events needs some serious processing before being passed to next.

Use multiprocessing to really use multiple cores and avoid blocking the main thread. The communication channels to pass things back to the main thread have the same (or very similar) APIs, although they are a bit slower because data has to be copied from process to process (this time should be marginal compared with the actual processing time to make it worthwhile)
Using a data feed

It's also a sensible opinion. You have in this case to provide a timestamp in the datetime line. You may simply use it as a timeframe Ticks data feed, without resampling/replaying to get every event. Remember that your real data feeds (which will probably have larger timeframes) will be available, but in most cases won't move forward (the len remains the same) when your event is seen in next