Indicator based on other indicators with 2 data feeds

Søren Pallesen

Getting an error when base an indicator on a calculation of two other indicators from a second data feed:

import backtrader as bt
import backtrader.indicators as btind

class myStrategy(bt.SignalStrategy):        
    def __init__(self):
        self.ema_day = btind.EMA(self.data1, period=15)
        self.kama_day = btind.KAMA(self.data1, period=25, fast=3, slow=30)
        differ = self.ema_day - self.kama_day
        self.roc = btind.ROC(differ, period=6)

    def next(self):
        pass

data = bt.feeds.GenericCSVData(
    dataname='./data/VXX-15m-2017-10-01us.csv',
    dtformat='%Y-%m-%d %H:%M:%S',
    openinterest=-1, 
    timeframe=bt.TimeFrame.Minutes, 
    compression=15
)    
    
data_day = bt.feeds.GenericCSVData(
    dataname='./data/VXX-1d-2017-10-01us.csv',
    dtformat='%Y-%m-%d',
    openinterest=-1, 
    timeframe=bt.TimeFrame.Days, 
    compression=1
)

cerebro = bt.Cerebro()
cerebro.addstrategy(myStrategy)
cerebro.adddata(data)
cerebro.adddata(data_day)

cerebro.run()
cerebro.plot()

And get this error: ValueError: x and y must have same first dimension, but have shapes (7542,) and (300,)

Here is the full message:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-1-6c0f97016110> in <module>
     34 
     35 cerebro.run()
---> 36 cerebro.plot()

~\Anaconda3\envs\IBbt\lib\site-packages\backtrader\cerebro.py in plot(self, plotter, numfigs, iplot, start, end, width, height, dpi, tight, use, **kwargs)
    989                 rfig = plotter.plot(strat, figid=si * 100,
    990                                     numfigs=numfigs, iplot=iplot,
--> 991                                     start=start, end=end, use=use)
    992                 # pfillers=pfillers2)
    993 

~\Anaconda3\envs\IBbt\lib\site-packages\backtrader\plot\plot.py in plot(self, strategy, figid, numfigs, iplot, start, end, **kwargs)
    226                         subinds=self.dplotsover[ind],
    227                         upinds=self.dplotsup[ind],
--> 228                         downinds=self.dplotsdown[ind])
    229 
    230             cursor = MultiCursor(

~\Anaconda3\envs\IBbt\lib\site-packages\backtrader\plot\plot.py in plotind(self, iref, ind, subinds, upinds, downinds, masterax)
    450                 xdata = np.array(xdata)[lplotmask]
    451 
--> 452             plottedline = pltmethod(xdata, lplotarray, **plotkwargs)
    453             try:
    454                 plottedline = plottedline[0]

~\Anaconda3\envs\IBbt\lib\site-packages\matplotlib\__init__.py in inner(ax, *args, **kwargs)
   1715                     warnings.warn(msg % (label_namer, func.__name__),
   1716                                   RuntimeWarning, stacklevel=2)
-> 1717             return func(ax, *args, **kwargs)
   1718         pre_doc = inner.__doc__
   1719         if pre_doc is None:

~\Anaconda3\envs\IBbt\lib\site-packages\matplotlib\axes\_axes.py in plot(self, *args, **kwargs)
   1370         kwargs = cbook.normalize_kwargs(kwargs, _alias_map)
   1371 
-> 1372         for line in self._get_lines(*args, **kwargs):
   1373             self.add_line(line)
   1374             lines.append(line)

~\Anaconda3\envs\IBbt\lib\site-packages\matplotlib\axes\_base.py in _grab_next_args(self, *args, **kwargs)
    402                 this += args[0],
    403                 args = args[1:]
--> 404             for seg in self._plot_args(this, kwargs):
    405                 yield seg
    406 

~\Anaconda3\envs\IBbt\lib\site-packages\matplotlib\axes\_base.py in _plot_args(self, tup, kwargs)
    382             x, y = index_of(tup[-1])
    383 
--> 384         x, y = self._xy_from_xy(x, y)
    385 
    386         if self.command == 'plot':

~\Anaconda3\envs\IBbt\lib\site-packages\matplotlib\axes\_base.py in _xy_from_xy(self, x, y)
    241         if x.shape[0] != y.shape[0]:
    242             raise ValueError("x and y must have same first dimension, but "
--> 243                              "have shapes {} and {}".format(x.shape, y.shape))
    244         if x.ndim > 2 or y.ndim > 2:
    245             raise ValueError("x and y can be no greater than 2-D, but have "

ValueError: x and y must have same first dimension, but have shapes (7542,) and (300,)

Its a resent install of BT (v1.9.67.122). I'm properly missing something simple ...Any ideas?

backtrader

You are getting a plotting error due to the usage of multiple data feeds with different timeframes. The calculation has run fine. Not all plotting scenarios can be handled.

Søren Pallesen

@søren-pallesen said in Indicator based on other indicators with 2 data feeds:

@backtrader thanks for the response. Just to be clear there are no problems plotting with multiple data feeds with different timeframes. It breaks when I add calculated indicators on the 2nd feed like:

        differ = self.ema_day - self.kama_day
        self.roc = btind.ROC(differ, period=6)

Adding the same calculated indicator on the first feed works and plots fine.

backtrader

It was already clear. It is simply that putting all things together in a matplotlib chart with multiple timeframes and multiple different lengths (due to indicators) cannot always be done. There were several iterations of the code to try to achieve it.

James Benson Bacque

At the risk of embarrassing myself in my very first post in this community by creating a zombie in replying to an old thread, i would like to say I found this thread useful, but runonce=False not so much. I bumped into:
"""
ValueError: x and y must have same first dimension, but have shapes (1796,) and (1795,)"
"""
when first adding just now resampling to daily data on a 1795-long pre-existing intra-day strategy with a bunch of my own indicators and observers being plotted.

I poked around a bit in the code before I found another work-around (and then, because I was hoping to do better, this thread...) to keep running back-tests seeing most of the plot with:
"""
cerebro.plot(end = end-datetime.timedelta(hours = 8, minutes = 0, seconds = 0.01)) # hack to get around resample extra bar plot or whatever...
"""
We'll see how long that lasts as I continue to develop with mixed timescales now. I have no idea why stopping the plot 8+epsilon hours early avoids the issue in my particular code-case (for now). But maybe the same idea might help someone else, so I bother to resurrect the zombie.

As an aside, I am somewhat stunned, as a total py newbie, that in a fairly short time I have been able to pull together a system that is downloading from IB, running my own custom strategy and indicators and observers on their live or my captured CSVs, producing plots that show what is going on...

Very nice job on the infrastructure. Kudos @backtrader.
PS: I also noticed that a prenext in one of my observers failed to execute after adding:
"""
cerebro.resampledata(data0, name = 'data2', timeframe=bt.TimeFrame.Days,
compression=1)
"""
which I thought odd, but not odd enough in itself to trouble anyone about. Now. On to inter-day, with almost all my plot, I hope.