Quandl Data Feed - Futures Data
jxm262 last edited by
I'm trying to import cme futures data using the Quandl Data feed I see in the docs - https://www.backtrader.com/docu/dataautoref.html?highlight=quandl , but getting an error.
bt.feeds.Quandl( dataset='CME', fromdate=datetime.datetime(2017, 1, 1), todate=datetime.datetime(2017, 12, 30), dataname='RBH2017', apikey='...' ) # error File "/Users/jmaat/anaconda3/envs/algo/lib/python3.5/site-packages/backtrader/feeds/quandl.py", line 108, in _loadline h = float(linetokens[next(i)]) IndexError: list index out of range
It appears the headers on the incoming data (cme futures) don't correspond to the accepted column names used in the source code (wiki stock data works fine).
For example, the provided example getting YHOO data under WIKI
req: https://www.quandl.com/api/v3/datasets/WIKI/YHOO.csv resp: headers in csv Date | Open | High | Low | Close | Volume | ...
But data from CME futures look like this
req: https://www.quandl.com/api/v3/datasets/CME/RBH2017.csv?order=asc&start_date=2017-01-01&end_date=2017-12-30 resp: csv headers Date | Open | High | Low | Last | Change | Settle | Volume
Note, the the change from
Lastbetween the 2 calls. I think it looks like the quandl feed is implementation specific to handle a specific csv header/column types? (ie it must match the first example).
Here's the code I'm tracing through - https://github.com/mementum/backtrader/blob/3e9c6b256e37724769e937a9a0c33ee93c4a09b4/backtrader/feeds/quandl.py#L107
I was wondering if it's possible to customize the naming/positioning of the headers coming in from the datafeed, similar to GenericCSVData
ie. something along these lines rb17 = bt.feeds.Quandl( dataset='CME', fromdate=datetime.datetime(2017, 1, 1), todate=datetime.datetime(2017, 12, 30), dataname='RBH2017', datetime=0, open=1, high=2, low=3, close=4, volume=7, openinterest=-1, apikey='xxx' )
Does anyone know an easy way to import quandl futures data from CME? Alternatively, would anyone like me to make a PR to generalize this somehow so users can add some sort of custom converter? (or at least handle futures and stock data in this same class)
Side note - this library is awesome :) I'm liking it much more than zipline at the moment.
ThatBlokeDave last edited by
Great timing. I just had a similar issue with this using London Stock Exchange data.
One workaround is to use the official Quandl api to download the data first. Then import it as a pandas data feed.
ab_trader last edited by
I believe I had the same issues. And decided to download data from Quandl separately from backtrader and then use simple generic CSV datafeed. More useful for me cause I am going to create continuous futures by myself.
jxm262 last edited by
For now, I just extended out the existing feeds.Quandl class and rewrote this part. Obviously doesn't handle
adjcloseand could probably break in the future, if anything changed on the quandl side.
# from feeds.quandl.QuandlCSV def _loadline(self, linetokens): i = itertools.count(0) dttxt = linetokens[next(i)] # YYYY-MM-DD dt = date(int(dttxt[0:4]), int(dttxt[5:7]), int(dttxt[8:10])) dtnum = date2num(datetime.combine(dt, self.p.sessionend)) self.lines.datetime = dtnum # todo add this back in the future # if self.p.adjclose: # for _ in range(7): # next(i) # skip ohlcv, ex-dividend, split ratio o = float(linetokens) h = float(linetokens) l = float(linetokens) c = float(linetokens) v = float(linetokens) # o = float(linetokens[next(i)]) # h = float(linetokens[next(i)]) # l = float(linetokens[next(i)]) # c = float(linetokens[next(i)]) # v = float(linetokens[next(i)]) self.lines.openinterest = 0.0
I'll try to make something more reusable/formal and add in a PR. Would be nice to give back to the community :)