Porting a pandas dataframe dependent indicator
Working to port an indicator that is currently implemented using pandas dataframes. Not clear to me if there is function in the standard Indicator tools that would replace the pandas capability that I currently depend on.
Here is a bit of code to show what I am trying to do in backtrader. Any suggestions here would be appreciated.
class IND(bt.Indicator): lines = ('ind', 'dval1', 'dval2', 'dval3', 'dval4', 'dval5', 'dval6', 'dval7', 'dval8',) def __init__(self): super(IND, self).__init__() def next(self): self.dval1 = (self.data.close / (self.data.high + self.data.low)) self.dval2 = (self.dval1 * 0.90) + (self.dval1[-1] * 0.5) self.dval3 = (self.dval2 * 0.90) + (self.dval2[-1] * 0.5) self.dval4 = (self.dval3 * 0.90) + (self.dval3[-1] * 0.5) self.dval5 = (self.dval4 * 0.90) + (self.dval4[-1] * 0.5) self.dval6 = (self.dval5 * 0.90) + (self.dval5[-1] * 0.5) self.dval7 = (self.dval6 + self.dval6[-1]) / 2 self.dval8 = (self.dval7 * 0.60) + (self.dval7[-1] * 0.25) self.lines.ind = self.dval8
And here is the relevant pandas dataframe code. Is there a way to do this without pandas or should I just implement as it was using pandas? Comments in the pandas datafeed section of the docs suggests to me that pandas is not required when doing these types of things in backtrader.
def update_indicators(self): df = self.hist_ohlc.append(self.last_prices.resample('D').ohlc()) df['value1'] = df['close'] / (df['high'] + df['low']) df['value2'] = (df['value1'] * 0.90) + (df['value1'].shift(1) * 0.5) df['value3'] = (df['value2'] * 0.90) + (df['value2'].shift(1) * 0.5) df['value4'] = (df['value3'] * 0.90) + (df['value3'].shift(1) * 0.5) df['value5'] = (df['value4'] * 0.90) + (df['value4'].shift(1) * 0.5) df['value6'] = (df['value5'] * 0.90) + (df['value5'].shift(1) * 0.5) df['value7'] = (df['value6'] + df['value6'].shift(1)) / 2 df['value8'] = (df['value7'] * 0.60) + (df['value7'].shift(1) * 0.25) ind = df['value8']
backtrader tries to give you what Pandas does (one-shot operations), whilst at the same time being built from the ground up to support step-by-step operations, which is of course needed to connect to a live trading and to do the best possible backtesting.
Let's try to use the arsenal provided by backtrader to port the code. It is actually not that different.
class IND(bt.Indicator): lines = ('ind',) def __init__(self): # Indicator has no __init__, no need to use super here dval1 = self.data.close / (self.data.high + self.data.low) ... dval8 = dval7 * 0.6 + dval7(-1) * 0.25 self.lines.ind = dval8
Et voilá!. Pass your pandas DataFrame using the
backtrader.feeds.PandasDataand if your column names are
openinterest) and it will work for this and any other indicator.
As you see, everything has been defined in
__init__. Let's look at the 1st line:
dval1 = self.data.close / (self.data.high + self.data.low)
It seems a regular arithmetic operation but there are no indices to
self.data.closeand the others. This is because the operation generates an object which is assigned to
dval1. This object will later deliver values during the backtesting/trading phase and the values will be addressable (if needed to) with the
Looking at the last calculation:
dval8 = dval7 * 0.6 + dval7(-1) * 0.25
dval7has been calculated finally calculated from the operations that start with
dval7is also an object and this operation says:
dval8is calculated, please use the current value of
0.6and the previous value of
(-1)notation, multiplied by
The final action is:
self.lines.ind = dval8
Which translates to:
- During backtesting/trading, put the currently calculated value of
dval8in the current index of the output line
As you see, there is no need to declare the different
dvalXas lines, because these are intermediate calculations and not output lines. These intermediate operations don't even need to be kept in instance attributes, because the final assingment to
self.lines.indensures python keeps all objects alive.
Thanks for the thorough explanation. Seems the tools are well thought out.
And a follow-on question, how would I get a slice of values held as dval8? I would typically do something as follows with the DataFrame. My attempts with dval8.get() is not yet giving me the expected result.
The design of the platform decided against slicing even if this may sound non-pythonic. The rationale behind this:
The choice to use
for the current moment (values which are being produced/calculated in this iteration) and
[-2]... to address the previous values
As such ... a slicing to get the current moment and the previous 4 would be something like:
[-4:0], which would be mostly confusing.
Now and straight to the question. It depends where you want to apply
If you are trying to do that in
__init__: it won't work.
dval8is an object still to be filled during the iterations of the backtesting/trading process. During
__init__, it is empty, awaiting to be triggered during each iteration for calculations and adding values to its own internal buffer
If you are trying to do that in
my_values = dval8.get(size=x, ago=y)
In this context:
sizeshould be straightforward: how many values must be returned
agooffset to look (backwards, because the future is not yet known)
0starts with the current moment (this is the default in the call)
-1starts with the previous value and the more negative the greater the offset backwards
If the current values in
dval8are for example
[1, 2, 3, 4, 5, 6 , 7, 8, 9]:
[7, 8, 9]
[4, 5, 6, 7]
Hopefully this helps
Helpful. Thank you. Off and running here....