Adding a Quandl Data Feed

shawndaniel

Just started using backtrader today and ran into trouble trying to plot a quandl data feed. I keep getting this error "ValueError: posx and posy should be finite values"

The data is just a Pandas Dataframe which only contains 2 columns, Date and Values. I.E;

2009-01-31       0.000000
2009-02-01    1100.000000
2017-02-01  227771.619795
...                   ...

So far this section in the docs seems like the likely solution for this https://www.backtrader.com/docu/datafeed-develop-csv.html but before I dive into this, I would like to know ahead of time if there are any caveats I should know about when working with this kind of data. I just want to add some MA's to it as part of a strategy. PyAlgoTrade seems to already include support for quandl data, fyi.

backtrader

A brief code sample to understand how you actually load the Dataframe as a data feed would be key to understand where posx and posy play a role (for sure not inside backtrader ... probably inside pandas)

The best way to load such a particular feed would be to follow this: Community - How to Feed Backtrader Alternative Data

The summary:

A datetime field is always needed
You can put your other column into any other field like open, high, low, close, volumen or openinterest, which are already predefined, by passing the appropriate parameter during the creation of the data feed (if the name of the column matches any of those, it will be autodetected, else do something like PandasData(dataname=df, open=1), where 1 indicates that the column with index 1 contains data to put into open

As a plus you could even override the lines hierarchy with linesoverride. See this community post Community - Execute on bid/ask and/or this Blog - Escape from OHLC Land A datetime field is always needed.

shawndaniel

@backtrader First I grab the dataframe using quandl.get(), then If I set the data's 'value' column as any of the params, i.e; volume. I get a pandas error:

File "pandas/index.pyx", line 65, in pandas.index.get_value_at (pandas/index.c:2759)
File "pandas/src/util.pxd", line 69, in util.get_value_at (pandas/index.c:16931)
IndexError: index out of bounds:

def strategy(datafeed):
    cerebro = bt.Cerebro()
    data = bt.feeds.PandasData(dataname=datafeed,
                           datetime=None,
                           high=None,
                           low=None,
                           open=None,
                           close=None,
                           volume=1,
                           openinterest=None)
    cerebro.adddata(data)
    cerebro.run()
    cerebro.plot()

If I try to import the same data as CSV format the following code produces this error in matplotlib:

"ValueError: posx and posy should be finite values"

data = bt.feeds.GenericCSVData(dataname=datafeed,
                               datetime=0,
                               dtformat='%Y-%m-%d',
                               tmformat=-1,
                               high=-1,
                               low=-1,
                               open=-1,
                               close=-1,
                               volume=1,
                               openinterest=-1)

backtrader

@shawndaniel said in Adding a Quandl Data Feed:

datetime=None,
...
volume=1

If datetime is None is because the dataframe has the timestamps in the index. If you only have 1 column (seems so according to your data) it cannot for sure be 1

How your csv data looks is unknown. How you load it may be right or wrong.

shawndaniel

@backtrader The data is shown in the original post.

backtrader

The original post contains an excerpt of the printout of a pandas.Dataframe, not the csv format.

But in any case the problem with matplotlib is due to having only the presence of volume and the absence of any other component (open, high, close, low). The volume is usually meant to be plotted as an overlay and if the other components aren't there, there is actually no axis (because there are no x values)

The best and quick option is to put the value in the close field and let if be plotted as a line on close (which is the default)

If a plotting format like bars (like in volume) were needed, some extra work would be needed without developing something (untested approach)

Load the data
Disable plotting of the data with data.plotinfo.plot = False
Create a Simple Moving Average on it with period=1 as in ma = bt.ind.SMA(data, period=1)
Tell the moving average to plot on its own: ma.plotinfo.subplot = True
The the line of the moving average to plot as bars: ma.plotlines.sma = dict(_method='bar', width=1.0) (from MACD)

shawndaniel

@backtrader Thank you :) using it as the close instead of volume worked with GenericCSV method. I already tried setting the value as any of the params for the pandas method before but it didn't work (IndexError: index out of bounds) so it didn't cross my mind as a solution for the other method..Thanks again.

Taewoo Kim

@shawndaniel

There's also the pandas method that should solve it.

FYI I couldn'get GenericCSV method to work either. So I had to feed data through pandas and use the pandas method

backtrader

There is a native Quandl data feed (for the WIKI Data) starting with 1.9.48.116

Maxim Korobov

A little patch:

    if self.p.apikey is not None:
        urlargs.append('api_key=%s' % self.p.apikey)

There was a typo previously.