IndexError from Multiple Data Feed

[vensaiten]

I have coded following strategy:

import backtrader as bt
from datetime import datetime  # For datetime objects

#Oanda Account Info
api_key = "XXXX"
account_number = "0000"

# Create a Stratey
class RenkoMA(bt.Strategy):

    params = (
        ('period',10),
    )

    def __init__(self):
        self.startcash = self.broker.getvalue()
        # Keep a reference to the "close" line in the data[0] dataseries
        self.dataclose = self.datas[0].close
        # Keep a reference to the "open" line in the data[0] dataseries
        self.dataopen = self.datas[0].open
        # Add a ExponentialMovingAverage indicator
        self.ma = bt.indicators.ExponentialMovingAverage(period = self.params.period)

        # To keep track of pending orders
        self.order = None

    def notify_order(self, order):
        if order.status in [order.Submitted, order.Accepted]:
            # Buy/Sell order submitted/accepted to/by broker - Nothing to do
            return

        # Write down: no pending order
        self.order = None

    def next(self):
        # Check if an order is pending ... if yes, we cannot send a 2nd one
        if self.order:
            return

        # current and previous bars are bullish
        if self.dataclose[0] > self.ma[0] and self.dataclose[0] > self.dataopen[0] and self.dataclose[-1] > self.dataopen[-1]:
            

            # BUY with all possible default parameters
            self.order = self.order_target_percent(target = 0.05)

        # current and previous bars are bearish
        elif self.dataclose[0] < self.ma[0] and self.dataclose[0] < self.dataopen[0] and self.dataclose[-1] < self.dataopen[-1]:
            

            # SELL with all possible default parameters
            self.order = self.order_target_percent(target = -0.05)

        else:
            pass



if __name__ == '__main__':

    cerebro = bt.Cerebro()

    cerebro.addstrategy(RenkoMA)

    oandastore = bt.stores.OandaStore(token=api_key, account=account_number, practice=True)

    data = oandastore.getdata(
        dataname = "USD_JPY",
        timeframe = bt.TimeFrame.Minutes,
        compression = 60,
        fromdate = datetime(2017,5,1),
        todate=datetime(2017,12,31)
        )

    # Convert to Renko
    data.addfilter(bt.filters.Renko, size=0.05)

    # Add data
    cerebro.adddata(data)

    startcash = 5000.0
    cerebro.broker.setcash(startcash)

    print('Starting Portfolio Value: %.2f' % cerebro.broker.getvalue())

    cerebro.run()

    print('Final Portfolio Value: %.2f' % cerebro.broker.getvalue())

    PL = cerebro.broker.getvalue() - startcash

    print("P/L: $%.2f" %PL)

    # Finally plot the end results
    cerebro.plot(style="candle")

I want to use this strategy with multiple data feed, so I read the Multi Example, I tried my best to modify the code, and got this:

import backtrader as bt
from datetime import datetime  # For datetime objects

#Oanda Account Info
api_key = "XXXX"
account_number = "0000"

# Create a Stratey
class RenkoMA(bt.Strategy):

    params = (
        ('period', 10),
    )

    def __init__(self):
        self.inds = dict()
        for i,d in enumerate(self.datas):
            self.inds[d]=dict()
            self.inds[d]['dataclose'] = self.datas[0].close
            self.inds[d]['dataopen'] = self.datas[0].open
            self.inds[d]['ma'] = bt.indicators.ExponentialMovingAverage(period = self.params.period)


        # To keep track of pending orders
        self.order = None

    def next(self):
        for i,d in enumerate(self.datas):
            dt, dn = self.datetime.date(), d._name
            # Check if an order is pending, we cannot send a 2nd one
            if self.order:
                return

            if self.inds[d]['dataclose'][0] > self.ma[0] and self.inds[d]['dataclose'][0] > self.inds[d]['dataopen'][0] and self.inds[d]['dataclose'][-1] > self.inds[d]['dataopen'][-1]:
                # current and previous bars are bullish

                # BUY with all possible default parameters
                self.order = self.order_target_percent(data=d,target = 0.05)

            elif self.inds[d]['dataclose'][0] < self.ma[0] and self.inds[d]['dataclose'][0] < self.inds[d]['dataopen'][0] and self.inds[d]['dataclose'][-1] < self.inds[d]['dataopen'][-1]:
                # current and previous bars are bearish

                # SELL with all possible default parameters
                self.order = self.order_target_percent(data=d,target = -0.05)

            else:
                pass



if __name__ == '__main__':

    cerebro = bt.Cerebro()

    cerebro.addstrategy(RenkoMA)

    oandastore = bt.stores.OandaStore(token=api_key, account=account_number, practice=True)

    # Create data list
    datalist = [
        'USD_JPY',
        'EUR_JPY',
        'GBP_JPY',
    ]

    # Loop through the data list adding to cerebro
    for i in range(len(datalist)):
        data = oandastore.getdata(
            dataname = datalist[i],
            timeframe = bt.TimeFrame.Minutes,
            compression = 60,
            fromdate = datetime(2017,5,1),
            todate=datetime(2017,12,31)
            )
        # Convert to Renko
        data.addfilter(bt.filters.Renko, size=0.05)

        # Add data
        cerebro.adddata(data, name=datalist[i])



    startcash = 5000.0
    cerebro.broker.setcash(startcash)

    print('Starting Value: $%.2f' % cerebro.broker.getvalue())


    cerebro.run()

    print('Final Value: $%.2f' % cerebro.broker.getvalue())

    PL = cerebro.broker.getvalue() - startcash

    print('P/L: $%.2f' % PL)


    # Finally plot the end results
    cerebro.plot(style="candle")

But I get the error of IndexError: array index out of range. What am I doing wrong?

[ab_trader]

It is really interesting. In your self.inds dictionary you use data feed object as a dictionary key. I didn't know it is possible, but maybe this is the source of error. Also your self.inds will have same values for all records, since these values are calculated based on the 1st data feed self.datas[0]. Probably you need to rewrite it as

        self.inds = dict()
        for i, d in enumerate(self.datas):
            self.inds[d._name] = dict()
            self.inds[d._name]['dataclose'] = d.close
            self.inds[d._name]['dataopen'] = d.open
            self.inds[d._name]['ma'] = bt.indicators.ExponentialMovingAverage(data=d, period = self.params.period)

Also would be nice to have extended error message to understand where the error was thrown.

[backtrader]

@ab_trader said in IndexError from Multiple Data Feed:

self.inds[d._name]['ma'] = bt.indicators.ExponentialMovingAverage(data=d, period = self.params.period)

this should be

self.inds[d._name]['ma'] = bt.indicators.ExponentialMovingAverage(d, period=self.params.period)

In any case I find it difficult to believe that the code shown in the 2nd part (multi-feed) is actually generating the quoted error. During __init__

@vensaiten said in IndexError from Multiple Data Feed:

    def __init__(self):
        self.inds = dict()
        for i,d in enumerate(self.datas):
            self.inds[d]=dict()
            self.inds[d]['dataclose'] = self.datas[0].close
            self.inds[d]['dataopen'] = self.datas[0].open
            self.inds[d]['ma'] = bt.indicators.ExponentialMovingAverage(period = self.params.period)

Later during next

@vensaiten said in IndexError from Multiple Data Feed:

if self.inds[d]['dataclose'][0] > self.ma[0] and self.inds[d]['dataclose'][0] > self.inds[d]['dataopen'][0] and self.inds[d]['dataclose'][-1] > self.inds[d]['dataopen'][-1]:

Where is self.ma defined during __init__? This seems like a mixture of copy/cut & paste from the single-feed script and something which is being run privately.

With the code shown in the snippet the error would for sure not be IndexError but AttributeError

[vensaiten]

Thanks for the tip. I will definitely post the entire error message from now on.

I have made the change and replaced with d._name, but still got the same error:

Traceback (most recent call last):
  File "renko_ema_oanda_multipleData.py", line 89, in <module>
    cerebro.run()
  File "/Users/XXX/anaconda3/lib/python3.6/site-packages/backtrader/cerebro.py", line 1127, in run
    runstrat = self.runstrategies(iterstrat)
  File "/Users/XXX/anaconda3/lib/python3.6/site-packages/backtrader/cerebro.py", line 1295, in runstrategies
    self._runnext(runstrats)
  File "/Users/XXX/anaconda3/lib/python3.6/site-packages/backtrader/cerebro.py", line 1547, in _runnext
    dts.append(datas[i].datetime[0] if ret else None)
  File "/Users/XXX/anaconda3/lib/python3.6/site-packages/backtrader/linebuffer.py", line 163, in __getitem__
    return self.array[self.idx + ago]
IndexError: array index out of range

[backtrader]

@vensaiten said in IndexError from Multiple Data Feed:

        # Convert to Renko
        data.addfilter(bt.filters.Renko, size=0.05)

        # Add data
        cerebro.adddata(data, name=datalist[i])

This is probably the potential cause. The Renko filter delays the delivery of the data, but the system tries to synchronize the multiple feeds, because some data is/was available.

Would need deep debugging.

[vensaiten]

@backtrader said in IndexError from Multiple Data Feed:

The Renko filter delays the delivery of the data, but the system tries to synchronize the multiple feeds, because some data is/was available.

You must be right. The strategy works with multiple data feed when there is no Renko filter. I guess I will just run this strategy separately for each data feed until there is a solution.