[FIXED] Ichimoku: Unable to get Future cloud

cgi1

From what I understand backtrader can be operated pseudo-vectorized using runonce=True option of Cerebro. As a result indicators are getting pre-calculated before strategy execution starts.

Keeping that in mind accessing the current cloud using

self.ikh = bt.indicators.Ichimoku()
self.ikh.senkou_span_a[0]
self.ikh.senkou_span_b[0]

works as expected. Current cloud means: The cloud which is on position [0] and has been calculated -26 bars in the past (because the cloud calculation forward-pushs 26 bars).

In a second step I would assume to access the future cloud using

self.ikh = bt.indicators.Ichimoku()
self.ikh.senkou_span_a[26]
self.ikh.senkou_span_b[26]

which fails with {IndexError} array index out of range.

So is it possible to access one of the future senkou_span_a or future senkou_span_b lines which has been calculated on the [0, -1, -2 ... -26] bars?

cgi1

Oh... Just reverted from 1.9.54.122 to 1.9.50.117 and it works like expected.

pip3 install 'backtrader==1.9.50.117' --force-reinstall
Collecting backtrader==1.9.50.117
  Downloading backtrader-1.9.50.117-py2.py3-none-any.whl (391kB)
    100% |████████████████████████████████| 399kB 1.3MB/s 
Installing collected packages: backtrader
  Found existing installation: backtrader 1.9.54.122
    Uninstalling backtrader-1.9.54.122:
      Successfully uninstalled backtrader-1.9.54.122
Successfully installed backtrader-1.9.50.117

backtrader

There is a new feature for indicators introduced with 1.9.54.x:

Commit Hash: 27c406ccdb37a88eb115e132ddbdbd2407b41fa1
Comment: Allow indicators to disable runonce

Because HeikinAshi due to the recursive nature cannot work properly in runonce=True mode. And the testing code remained in forcing runonce=False

Looking into the future is not possible when runonce=False.

Corrected and pushed to the development branch.

cgi1

@backtrader Will check it to provide feedback shortly.

cgi1

@backtrader: Now works again as expected. Thanks!

André

Is this really solved? Following simple test does not work:

class IchimokuStrategy(StrategyBase.StrategyBase):
    def __init__(self):
        self.Ichimoku = bt.indicators.Ichimoku()

    def next(self):
        print(self.Ichimoku.senkou_span_a[26])

if __name__ == '__main__':
    cerebro = bt.Cerebro()
    cerebro.adddata(data)
    cerebro.addstrategy(IchimokuStrategy)
    cerebro.run(runonce=True)

    cerebro.plot()

It throws following error:

File "/Users/BackTrader/IchimokuStrategy.py", line 25, in next
  print(self.Ichimoku.senkou_span_a[26])
File "/usr/local/lib/python3.7/site-packages/backtrader/linebuffer.py", line 163, in __getitem__
  return self.array[self.idx + ago]

Why can't I get the values from the future?

Euler Sousa

Hey guys,

I'm still facing the same situation reported by @André. Does anyone was able to access ichimoku indicators from the "future"?

ps. The platform is awesome. Congratulations for the amazing job.

Regards,

Euler

run-out

@André said in [FIXED] Ichimoku: Unable to get Future cloud:

Why can't I get the values from the future?

Backtrader is a backtesting engine designed to simulate real life. It does not allow access to future values. [0] is the current bar, and [-1]...[-n] are previous values. 'Future' values, like in real life, don't exist in the ecosystem.

Please let me know if I misunderstood your question.

Euler Sousa

It is clear for me the goal of backtesting engine and the fact that during the next method you have access to the current bar [0], the indicators pre-calculated on the init method, and all their previous values [-1] ...[-n] .

The particular situation mentioned here is considering Ichimoku indicators, in special senkou_span_a and senkou_span_b, which are calculated by using past bars information (i.e. high, low of 52 and 9 periods before [0]). These two indicators are then projected 26 periods ahead of the current moment (i.e. [26]). For some strategies, it might be useful to have these "future" values to enhance the decision-making process at the current time [0].

One workaround that I was able to do is getting the past values needed to calculate those two indicators using get() method, for example:

self.data0.high.get(size=52)

and then calculating the indicators again:

import pandas as pd  

def tenkan_sen(high,low):
    # Tenkan-sen (Conversion Line): (9-period high + 9-period low)/2))
    nine_period_high = pd.Series(high).rolling(window= 9).max()
    nine_period_low = pd.Series(low).rolling(window= 9).min()
    
    return (nine_period_high + nine_period_low) /2


def kijun_sen(high,low):
    # Kijun-sen (Base Line): (26-period high + 26-period low)/2))
    period26_high = pd.Series(high).rolling(window=26).max()
    period26_low = pd.Series(low).rolling(window=26).min()
    return (period26_high + period26_low) / 2


def senkou_span_a(tenkan_sen,kijun_sen):
    # Senkou Span A (Leading Span A): (Conversion Line + Base Line)/2))
    return ((pd.Series(tenkan_sen) + pd.Series(kijun_sen)) / 2).shift(26)


def senkou_span_b(high,low):
    # Senkou Span B (Leading Span B): (52-period high + 52-period low)/2))
    period52_high = pd.Series(high).rolling(window=52).max()
    period52_low = pd.Series(low).rolling(window=52).min()
    return ((period52_high + period52_low) / 2).shift(26)

This nice piece of code was taken from this article.

I believe that even though this approach is not performatic, it can be useful in cases, for instance, you have some machine learning model for predicting some useful information used in the decision-making process.

Euler Sousa

@André, here it goes:

class IchimokuStrategy(bt.Strategy):
    def __init__(self):
        self.ichi= bt.indicators.Ichimoku()

    def next(self):
        senkou_a = self._senkou_span_a(tenkan_sen=self.ichi.l.tenkan_sen.get(size=26),
                                       kijun_sen=self.ichi.l.kijun_sen.get(size=26))
        print(f"{senkou_a[25]}")  # pandas.Series are 0 indexed based
    
    @staticmethod
    def _senkou_span_a(tenkan_sen,kijun_sen):        
        # Senkou Span A (Leading Span A): (Conversion Line + Base Line)/2))
        return ((pd.Series(tenkan_sen) + pd.Series(kijun_sen)) / 2)
    
if __name__ == '__main__':
    cerebro = bt.Cerebro()
    cerebro.adddata(data)
    cerebro.addstrategy(IchimokuStrategy)
    cerebro.run(runonce=True)

    cerebro.plot()

vladisld

@André said in [FIXED] Ichimoku: Unable to get Future cloud:

Why can't I get the values from the future?

From a technical perspective, it is possible to get the future indicator values in case the data was preloaded. For example, the following code works for me:

import os
import backtrader as bt

class IchimokuStrategy(bt.Strategy):
    def __init__(self):
        self.Ichimoku = bt.indicators.Ichimoku(self.datas[0])

    def next(self):
        try:
            if len(self.datas[0]) < self.datas[0].buflen() - 26:
                print(f'{self.datas[0].close[0]}:{self.Ichimoku.senkou_span_a[26]}')
            else:
                print(f'{self.datas[0].close[0]}:nan')
        except:
            raise

if __name__ == '__main__':
    cerebro = bt.Cerebro()
    data_path = os.path.join(bt.__file__, '../../datas/yhoo-1996-2014.txt')
    data = bt.feeds.YahooFinanceCSVData(dataname=data_path)
    cerebro.adddata(data)
    cerebro.addstrategy(IchimokuStrategy)
    cerebro.run(runonce=True)
    cerebro.plot()

The exception you've reported is raised only if the senkou_span_a[26] bar is requested for the last 25 datas[0] bars - those bars are not existing and so can't be requested.

Euler Sousa

@vladisld, thanks for the reply. It worked here also !

Euler Sousa

another quick question is there a way of getting an array of all 26 values? like self.Ichimoku.senkou_span_a[0:26] ?

Euler Sousa

I made another workaround to achieve it haha ... using Python compreehensive lists :

print(f"{[self.Ichimoku.senkou_span_a[i] for i in range(26)]}")

please let me know if there is a better way to do it.

One more time, thanks for the contributions.

vladisld

probably the following will work:

print(f'{self.datas[0].close[0]}:{self.Ichimoku.senkou_span_a.get(0, 26)}')

this will produce the output like:

9.01:array('d', [9.6925, 9.535, 9.5225, 9.5225, 9.7925, 10.04, 10.0825, 10.0825, 10.09, 10.1525, 10.135, 10.135, 10.135, 10.135, 9.985, 9.945, 9.92, 9.78, 9.774999999999999, 9.73, 9.73, 9.5025, 9.4175, 9.307500000000001, 9.125, 9.125])
8.9:array('d', [9.535, 9.5225, 9.5225, 9.7925, 10.04, 10.0825, 10.0825, 10.09, 10.1525, 10.135, 10.135, 10.135, 10.135, 9.985, 9.945, 9.92, 9.78, 9.774999999999999, 9.73, 9.73, 9.5025, 9.4175, 9.307500000000001, 9.125, 9.125, 9.0125])

Euler Sousa

I ran some tests and for this case, we should use ago parameters as the starting point in "future" to look backwards, for example :

print(f"{list(self.Ichimoku.senkou_span_a.get(ago= 3, size=4))}")

which will yield the same result as :

print(f"{[self.Ichimoku.senkou_span_a[i] for i in range(4)]}")

However, If we pass ago=0 it will get the last 3 values, [0],[-1] ,[-2], which for this particular case is not in our interest.

I checked the documentation in the Getting slice part and it makes sense.

Thanks for the help.

Bo Vargas

@Euler-Sousa Is this the only solution you have found while using live data (no preloaded)? Are you just placing this inside the next() function and computing the values each time around?

sapi

Hello,

Today I have had the same problem trying to access future values from senkou_span_(a|b) and correctly drawing the cloud

A quick solution that I found is adding 26 more nan rows in my pandas dataframe, before giving it to cerebro

data_in = original_data_c.copy()
delta = data_in.index[-1] - data_in.index[-2]
for i in range(26):
    data_in.loc[data_in.index.max() + delta] = None

data = bt.feeds.PandasData(dataname=data_in,
                           fromdate=data_in.index[0],
                           todate=data_in.index[-1])
cerebro.adddata(data)

Then I just need to check that I'm not working with a price value that is nan

class Ichimoku_strategy(bt.Strategy):

    def __init__(self):
        self.ichimoku = Ichimoku()

    def next(self):
        if not math.isnan(data.open[0]):
            print(self.ichimoku.l.senkou_span_a[25])

Then I did a small hack to avoid drawing tenkan_sen and kijun_sen beyond the price values