For the indicator bt.talib.CORREL, if I pass in two indicators with missing values in the middle, no correlation will be calculated after the first missing value, even the values are there afterward, as a demonstration(a and b are both line object, i only show them as array here):

import numpy as np

a = np.array([1,1,1,1     ,1,1,1,1,1])     #space on 4th element is for readability
b = np.array([1,1,1,np.nan,1,1,1,1,1])

print(bt.talib.CORREL(a,b,timepriod=3))
>>  [nan,nan,1,nan,nan,nan,nan,nan,nan]

The intended result should be:

[nan,nan,1,nan,nan,nan,1,1,1]

In my actual case, both a and b are about 300 days long, and the missing values is about 30 days long in the middle of one data, and I tried to calculate correlation for a 6 days window, not sure if this has to do with issue. All the value before missing ones are correct.

@ab_trader

Thank you for your reply, originally I thought indicator carries the information of which data it was calculated upon. Even if it was another indicator, the original data info should be pass by. Guess I was wrong

@momentum

Hi, how do you access the smas in your self.smas list with reference to data name?
Like how to you know which data was used to calculate smas[0] in later codes?

The backtrader.indicators.sma.MovingAverageSimple object doesnt have a _name.

@nicola-prada

I think this can be done by creating subclass of existing indicator, effectively 'extending' an indicator

@backtrader

Well elaborated example.

@backtrader

My goal is described in my old post:

https://community.backtrader.com/topic/1076/daily-upper-and-lower-bound-of-price/3

Only here I am trying to take into account daily upper and lower bound of price for target order.

target order or not, I did try to follow your suggestion in old post, but:

1. Having a custom Sizer

This wont work for me since my order are with specific size, a global custom Sizer will be overwritten.

2.Having a commission scheme

I read the custom commission scheme, dont see how this will work. The

def _getcommission(self, size, price, pseudoexec)

only pass in the execution price for orders, not previous bar's (when the order is created) close price, I wont be able to compare these two prices thus impose an inf commission accordingly.

Official name for such trading restriction is called daily maximum price fluctuation I think, and it is common in commodity futures as well, might be a good idea to implement such feature natively.

@nicola-prada

I think pre-processing is requied if additional information are required for stock selection.

What I did was including all additional infomation into volume line instead of adding new lines, since default BT behaviour is to ignore volume anyway(I hope in the future more complicated logic can be included), and it may seem intuitive to only trade when volume is not 0. So volume value is set to 1 if particular stock is in the index in your case. Of course you may wanna add additional infomation into volume line, like tradability etc., but yeah always look out for look ahead bias.

In my case I need to add about 3600 stocks into BT and running a 5 years daily bar backtest takes about 18 mins. I preprocess the data with pandas and create a custom pandas feed. I have tested that having less lines actually speed the process up a bit.

Did you try to code it the way you shown or you are waiting for an answer? Can be quicker to code it by yourself.

I did, currently I am working around the issue by manipulating my data in some way. Still stuck with strategy level code though.
The snippet shown was merely a demonstration of what I hoped to achieve, not working code.

@backtrader

after reading ..\samples\timers\scheduled.py, i noticed:

def prenext(self):
    self.next()

is this the same as copying all code in next() into prenext() so prenext() and next() are exactly the same?

@nicola-prada @backtrader

Hi, here is my solution to not aligned data:

def __init__(self):
    self.stklist = [i for i in self.getdatanames()]
    for d in self.stklist:
        self.dnames[d].lengthlist = [0]

def prenext(self):
    for d in self.stklist:
        datad = self.dnames[d]
        datad.lengthlist.append(len(datad))
            if len(datad) > 0:
              if datad.lengthlist[-1] > datad.lengthlist[-2]:
                  .....

def next(self):
    for d in self.stklist:
        datad = self.dnames[d]
        datad.lengthlist.append(len(datad))
            if len(datad) > 0:
              if datad.lengthlist[-1] > datad.lengthlist[-2]:
                  .....

So in __init__ I create a list of all stocks and an attribute called lengthlist for every stock. Then on every bar, the length of each stock is checked and appended, if the length starts to grow, it means data start to be meaningful and might be used for later calculations. Everything in next() is just a repetition of prenext(), so your strategy will run even not all data comes into play.
This way your wont need to manipute the original data or add extra lines.

Please let me know if there is a better or intuitive way, as I think prenext() were originally there for indicator like SMA calculation before the required period were met. If there is a switch to call next() even not all data lenght is met might be very handy.