Strategy - Error IndexError: array index out of range

Eff

Hey, guys!

Please, I am receiving the error stating below:

I believe that the problem is that the data is insufficient, because when performing the backtest in 1h, it works, but in 4h and 1d this logic does not work.

def next(self):
# Simply log the closing price of the series from the reference
self.log('Close, %.2f' % self.dataclose[0])
# Check if an order is pending ... if yes, we cannot send a 2nd one
if self.order:
return

    # Check if we are in the market
    if not self.position:

        # Not yet ... we MIGHT BUY if ..
        if self.histo.lines.histo[1] < self.histo.lines.histo[2] and ((self.stoch.lines.percK[1] > self.stoch.lines.percD[1] and self.stoch.lines.percK[2] < self.stoch.lines.percD[2]) or self.stoch.lines.percK[1] > self.stoch.lines.percD[1]) and self.histo.lines.histo[1] < 0 and self.stoch.lines.percD[1] < 40:
            # BUY, BUY, BUY!!! (with default parameters)
            self.log('BUY CREATE, %.2f' % self.dataclose[0])

            # Keep track of the created order to avoid a 2nd order
            self.order = self.buy()

    else:

        # Already in the market ... we might sell
        # self.macd.lines.signal[1] > self.macd.lines.macd[1] and self.macd.lines.signal[2] < self.macd.lines.macd[1]:
        
        if self.macd.lines.signal[1] > self.macd.lines.macd[1] and self.macd.lines.signal[2] < self.macd.lines.macd[2]: #Falta Profit e Stop
            # SELL, SELL, SELL!!! (with all possible default parameters)
            self.log('SELL CREATE, %.2f' % self.dataclose[0])

            # Keep track of the created order to avoid a 2nd order
            self.order = self.sell()

the problem probably happens when the data is not enough to close the last opening (purchase), is there any attribute that can keep the operation open without generating the error?

Eff

@eff said in Strategy - Error IndexError: array index out of range:

Hey, guys!

Please, I am receiving the error stating below:

I believe that the problem is that the data is insufficient, because when performing the backtest in 1h, it works, but in 4h and 1d this logic does not work.

def next(self):
# Simply log the closing price of the series from the reference
self.log('Close, %.2f' % self.dataclose[0])
# Check if an order is pending ... if yes, we cannot send a 2nd one
if self.order:
return
    # Check if we are in the market
    if not self.position:

        # Not yet ... we MIGHT BUY if ..
        if self.histo.lines.histo[1] < self.histo.lines.histo[2] and ((self.stoch.lines.percK[1] > self.stoch.lines.percD[1] and self.stoch.lines.percK[2] < self.stoch.lines.percD[2]) or self.stoch.lines.percK[1] > self.stoch.lines.percD[1]) and self.histo.lines.histo[1] < 0 and self.stoch.lines.percD[1] < 40:
            # BUY, BUY, BUY!!! (with default parameters)
            self.log('BUY CREATE, %.2f' % self.dataclose[0])

            # Keep track of the created order to avoid a 2nd order
            self.order = self.buy()

    else:

        # Already in the market ... we might sell
        # self.macd.lines.signal[1] > self.macd.lines.macd[1] and self.macd.lines.signal[2] < self.macd.lines.macd[1]:
        
        if self.macd.lines.signal[1] > self.macd.lines.macd[1] and self.macd.lines.signal[2] < self.macd.lines.macd[2]: #Falta Profit e Stop
            # SELL, SELL, SELL!!! (with all possible default parameters)
            self.log('SELL CREATE, %.2f' % self.dataclose[0])

            # Keep track of the created order to avoid a 2nd order
            self.order = self.sell()
the problem probably happens when the data is not enough to close the last opening (purchase), is there any attribute that can keep the operation open without generating the error?

run-out

@eff You are using get somewhere but I cannot see where as you only included limited code.

get looks backward to get an array of size n. You must ensure there is a minimum period so that you can get n numbers. Use addminperiod

vladisld

@eff said in Strategy - Error IndexError: array index out of range:

self.macd.lines.signal[1] > self.macd.lines.macd[1] and self.macd.lines.signal[2] < self.macd.lines.macd[2]

It seems you are trying to access the data from the future ( using positive indexes for indicators' lines) - probably not a good idea unless your indexes are built for such functionality ( which I doubt, particularly for macd ).

more about indexing here: https://www.backtrader.com/docu/concepts/#indexing-0-and-1

Eff

@run-out

Follow the full code, thanks

from future import (absolute_import, division, print_function,
unicode_literals)

import datetime # For datetime objects
import os.path # To manage paths
import sys # To find out the script name (in argv[0])

Import the backtrader platform

import backtrader as bt
import datetime

import pandas as pd
import quantstats

Create a Stratey

class TestStrategy(bt.Strategy):
params = (
('maperiod', 15),
('printlog', False),
('stop_loss',0.02),
('trail', False),
)

def log(self, txt, dt=None):
    ''' Logging function fot this strategy'''
    dt = dt or self.datas[0].datetime.date(0)
    print('%s, %s' % (dt.isoformat(), txt))

def __init__(self):
    # Keep a reference to the "close" line in the data[0] dataseries
    self.dataclose = self.datas[0].close

    # To keep track of pending orders and buy price/commission
    self.order = None
    self.buyprice = None
    self.buycomm = None
    self.mcross = None

    self.histo = bt.indicators.MACDHisto()
    self.macd = bt.indicators.MACD()
    self.stoch = bt.indicators.Stochastic()

    self.result_long = False
    self.result_short = False

def notify_order(self, order):
    if order.status in [order.Submitted, order.Accepted]:
        # Buy/Sell order submitted/accepted to/by broker - Nothing to do
        return

    # Check if an order has been completed
    # Attention: broker could reject order if not enough cash
    if order.status in [order.Completed]:
        if order.isbuy():
            self.log(
                'BUY EXECUTED, Price: %.2f, Cost: %.2f, Comm %.2f' %
                (order.executed.price,
                 order.executed.value,
                 order.executed.comm))

            self.buyprice = order.executed.price
            self.buycomm = order.executed.comm
        else:  # Sell
            self.log('SELL EXECUTED, Price: %.2f, Cost: %.2f, Comm %.2f' %
                     (order.executed.price,
                      order.executed.value,
                      order.executed.comm))

        self.bar_executed = len(self)

    elif order.status in [order.Canceled, order.Margin, order.Rejected]:
        self.log('Order Canceled/Margin/Rejected')

    self.order = None

def notify_trade(self, trade):
    if not trade.isclosed:
        return

    self.log('OPERATION PROFIT, GROSS %.2f, NET %.2f' %
             (trade.pnl, trade.pnlcomm))

def next(self):
    # Simply log the closing price of the series from the reference
    self.log('Close, %.2f' % self.dataclose[0])
    # Check if an order is pending ... if yes, we cannot send a 2nd one
    if self.order:
        return
    
    # Check if we are in the market
    if not self.position:

        # Not yet ... we MIGHT BUY if ..
        if self.histo.lines.histo[1] < self.histo.lines.histo[2] and ((self.stoch.lines.percK[1] > self.stoch.lines.percD[1] and self.stoch.lines.percK[2] < self.stoch.lines.percD[2]) or self.stoch.lines.percK[1] > self.stoch.lines.percD[1]) and self.histo.lines.histo[1] < 0 and self.stoch.lines.percD[1] < 40:
            # BUY, BUY, BUY!!! (with default parameters)
            self.log('BUY CREATE, %.2f' % self.dataclose[0])

            # Keep track of the created order to avoid a 2nd order
            self.order = self.buy()

    else:

        # Already in the market ... we might sell
        # self.macd.lines.signal[1] > self.macd.lines.macd[1] and self.macd.lines.signal[2] < self.macd.lines.macd[1]:
        
        if self.macd.lines.signal[1] > self.macd.lines.macd[1] and self.macd.lines.signal[2] < self.macd.lines.macd[2]: #Falta Profit e Stop
            # SELL, SELL, SELL!!! (with all possible default parameters)
            self.log('SELL CREATE, %.2f' % self.dataclose[0])

            # Keep track of the created order to avoid a 2nd order
            self.order = self.sell()

Create a cerebro entity

cerebro = bt.Cerebro()

Add a strategy

cerebro.addstrategy(TestStrategy)

modpath = os.path.dirname(os.path.abspath(sys.argv[0]))
datapath = os.path.join(modpath, 'btc_bars1h.csv')

data = bt.feeds.GenericCSVData(
dataname=datapath,
fromdate=datetime.datetime(2018, 1, 1),
todate=datetime.datetime(2020, 12, 31),
nullvalue=0.0,
dtformat=('%Y-%m-%d %H:%M:%S'),
datetime=0,
high=2,
low=3,
open=1,
close=4,
volume=-1,
openinterest=-1
)

cerebro.adddata(data)
cerebro.broker.setcash(100000.0)
cerebro.broker.setcommission(commission=0.001)

Add analyzers

cerebro.addanalyzer(bt.analyzers.PyFolio, _name='PyFolio')

if name == 'main':

start_portfolio_value = cerebro.broker.getvalue()
results = cerebro.run()
start = results[0]
end_portfolio_value = cerebro.broker.getvalue()
pnl = end_portfolio_value - start_portfolio_value

print(f'Starting Portfolio Value: {start_portfolio_value:2f}')
print(f'Final Portfolio Value: {end_portfolio_value:2f}')
print(f'PnL: {pnl:.2f}')

portfolio_stats = start.analyzers.getbyname('PyFolio')
returns, positions, transactions, gross_lev = portfolio_stats.get_pf_items()
returns.index = returns.index.tz_convert(None)

quantstats.reports.html(returns, output='stats1h.html', title='BTC Sentiment')

cerebro.plot()

Eff

@vladisld

Thanks, I'll take a look

davidavr

@eff I think that the issue might be in next where you are referencing past values such as self.histo.lines.histo[1] and self.stock.lines.percD[2] which may not exist yet in the first couple of calls to next.

I tried adding a line in your strategy's init method to force warming up long enough so those historical points would be valid and I no longer get an exception:

self.highest = bt.indicators.Highest(30)

A cleaner way would be to check the length of those indicators at the entry point to next and returning if they aren't ready.