@backtrader said in Multi-Data Indicator implementation help.:
Before I read the messages. See please the article below and why that isn't a good idea.
- Articles - Momentum Strategy - Next and PreNext
Will do, thank you!
If you could respond to the questions as well, that would be amazing! :)
Sorry for the relentless stream of replies, haha. I have solved the problem, with multiple data (assets).
import backtrader as bt
import numpy as np
class MQN(bt.Indicator):
lines = ('mqn', 'ma',)
params = (
('len', 35),
('ma_len', 20),
)
def __init__(self):
self.addminperiod(self.p.len+1)
def next(self):
for i, d in enumerate(self.datas):
data = d.get(size=self.p.len+1)
changes = []
for i in range(1, len(data)):
changes.append(data[-i] - data[-i-1])
if len(changes) == self.p.len:
avg_chng = np.average(changes)
stdev = np.std(changes)
sqroot = np.sqrt(self.p.len)
mqn = sqroot * (avg_chng/stdev)
self.l.mqn[0] = mqn
ma = self.l.mqn.get(size=self.p.ma_len)
self.l.ma[0] = np.average(ma)
Calling this in my strategy, and passing in the specified data (asset) does the trick and plots each separate indicator, with its respective SMA.
I hope my documentation of progress here has/will be useful to some.
My final questions are:
Is the above code the most efficient/cleanest/best way to go about handling the operations I'm doing? As mentioned earlier, I have several other indicators, similar in nature to this one, so I would like to make sure I really nail the right way before making my way through programming all of them.
Is there a way/hack to eliminate the "instantiation lag" that occurs with indicators in BT? Is it possible to pass in data from before the start of the backtest so that any indicators are already fully instantiated at t=0?
Managed to get something running!
import backtrader as bt
import numpy as np
class MQN4(bt.Indicator):
lines = ('mqn', 'ma',)
params = (
('len', 35),
('ma_len', 20),
)
def __init__(self):
self.addminperiod(self.p.len+1)
def next(self):
d = self.data.get(size=self.p.len+1)
changes = []
for i in range(1, len(d)):
changes.append(d[-i] - d[-i-1])
avg_chng = sum(changes) / self.p.len
stdev = np.std(changes)
sqroot = (self.p.len ** 0.5)
mqn = sqroot * (avg_chng/stdev)
self.l.mqn[0] = mqn
# ma = bt.ind.SMA(self.l.mqn, period=self.p.ma_len)
# self.l.ma[0] = ma
This gives me what I would like! For one asset. How could I modify (or utilise!) 'lines' so that I can have a separate output on each asset? i.e. N separately computed indicators for N assets.
Furthermore, is there any way to remove the "gap" at the beginning, i.e. remove the buffering of the indicator? Is it possible to feed through (t-x) data but begin the backtest at (t)? Or any other way?
Yet another iteration. Getting closer I feel, but this one hangs on:
self.l.mqn[0] = mqn
with:
TypeError: must be real number, not LinesOperation
import backtrader as bt
import numpy as np
class MQN3(bt.Indicator):
lines = ('mqn',)
params = (
('len', 35),
('ma_len', 20),
)
def __init__(self):
self.addminperiod(self.p.len)
def next(self):
d = self.data.get(size=self.p.len)
print(len(d))
print(d)
avg_chng = bt.ind.SMA(d, period=self.p.len)
stdev = bt.ind.StdDev(d, period=self.p.len)
sqroot = (self.p.len ** 0.5)
print("{} {} {}".format(avg_chng, stdev, sqroot))
mqn = (sqroot * (avg_chng/stdev))
print(mqn)
# ma = bt.indicators.MovingAverageSimple(mqn, period=self.p.ma_len)
self.l.mqn[0] = mqn
# self.lines.ma[0] = ma
'd' is collecting the most recent 35 closes, as it should.
'sqroot' is giving the correct multiplier.
With the print string, I can't see the values for 'avg_chng' and 'stdev', just the objects themselves. So potentially this is where i'm getting something wrong?
I'm also unsure of how to turn 'd' into a list of 'changes' (d[i] - d[i-1]) and pass this to SMA and StdDev.
Can anyone help me understand where i'm going wrong? :)
I've also now tried this approach, still hanging an error.
import backtrader as bt
class MQN2(bt.Indicator):
lines = ('mqn', 'ma')
params = (
('len', 35),
('ma_len', 20),
)
def __init__(self):
self.cur = self.data.close[0]
self.prev = self.data.close[-1]
self.change = self.cur - self.prev
self.avg_chng = bt.indicators.MovingAverageSimple(self.change, period=self.p.len)
self.stdev = bt.indicators.StandardDeviation(self.change, period=self.p.len)
self.sqroot = self.p.len ** 0.5
def next(self):
mqn = self.sqroot[0] * (self.avg_chng[0]/self.stdev[0])
# ma = bt.indicators.MovingAverageSimple(mqn, period=self.p.ma_len)
self.lines.mqn[0] = mqn
# self.lines.ma[0] = maHi there,
As I've slowly been piecing together my bt set-up, the time has come to try get some indicators working. I'm having some trouble understanding how the indicator ecosystem works, and just how I can achieve what I would like.
Here are the requirements (for each asset):
Here is the current implementation of my code, note, it does not work. I'm a fairly new programmer, so perhaps something glaringly obvious is wrong. My main suspect is that it may have something to do with the fact I am doing computations for each asset in self.datas (in next()), but only having one lines output? Also, I realise I'm doing 'nothing' with the dict() entries, I was trying that as a way to handle the problem, but it yielded the same error as the current iteration.
import backtrader as bt
import numpy as np
class MQN(bt.Indicator):
lines = ('mqn', 'ma')
params = (
('len', 35),
('ma_len', 20),
)
def __init__(self):
self.changes = dict()
self.avg_chng = dict()
def next(self):
for i, d in enumerate(self.datas):
cur = d.close[0]
prev = d.close[-1]
change = cur - prev
self.changes[d] = change
average = bt.indicators.MovingAverageSimple(change, period=self.p.len)
self.avg_chng[d] = average
stdev = bt.indicators.StandardDeviation(change, period=self.p.len)
sqroot = np.sqrt(self.p.len)
mqn = sqroot * (average/stdev)
ma = bt.indicators.MovingAverageSimple(mqn, period=self.p.ma_len)
self.lines.mqn[0] = mqn
self.lines.ma[0] = ma
If someone could help me understand what can be done in the bt.indicator ecosystem (as I have other indicators that do similar things to this), and help me to achieve the indicator outcome, that would be simply amazing!
Much appreciated in advance for any help/answers!
ab_trader, you are awesome! Thank you so, so much for helping me get this up and running. Now that I know the structure of how to do this, I feel confident I can now pull in any custom Pandas data frame I would like! Now the fun "work" of statistical analysis and optimisation can begin!
Now that I have this simple measure working, perhaps you could help with a future issue. If the custom Pandas DataFrame contained the same columns as we have been working with, but multiple "assets" instead of just one (as in the example we've been working with) - how would everything work? Similar to how we've done it above, or something a bit different to deal with the dual-indexing?
Much appreciated :)
@ab_trader said in Struggling to implement custom Pandas data feed:
Looks like you've created your new CustomDataLoader data class, but still use bt built-in Pandas data class:
Gah! Thank you so much for that!
Big thanks to anyone trying to help so far, it really is much appreciated as I am somewhat of a new coder, with a discretionary strategy that I have been using for years that I would like to turn into an algo. I have taken some python classes at Uni, but alas, I am still learning.
SO, even after pointing to the newly created class as you suggested, I still hit an error.
I have greatly simplified the code into one to allow someone to hopefully help me get to the bottom of this. Here it is:
from __future__ import (absolute_import, division, print_function,
unicode_literals)
import argparse
import backtrader as bt
import backtrader.feeds as btfeeds
from make_df import get_df
class StratData(btfeeds.DataBase):
lines = ('datetime', 'Open', 'High', 'Low', 'Close', 'TSL_2', 'TSL_2_L/S', 'Fast_ATR', 'Slow_ATR', 'Impulse')
params = (
('datetime', None),
('open', 'Open'),
('high', 'High'),
('low', 'Low'),
('close', 'Close'),
('volume', None),
('openinterest', None),
('TSL_2', 4),
('TSL_2_L/S', 5),
('Fast_ATR', 6),
('Slow_ATR', 7),
('Impulse', 8)
)
# if False:
# # No longer needed with version 1.9.62.122
# datafields = btfeeds.PandasData.datafields + (
# ['optix_close', 'optix_pess', 'optix_opt'])
class StrategyPrint(bt.Strategy):
def next(self):
print('%03d %f %f, %f' % (
len(self),
self.data.l.Close[0],
self.data.l.TSL_2[0],
self.data.l.Impulse[0],))
def runstrat():
args = parse_args()
# Create a cerebro entity
cerebro = bt.Cerebro(stdstats=False)
# Add a strategy
cerebro.addstrategy(StrategyPrint)
dataframe = get_df()
if not args.noprint:
print('--------------------------------------------------')
print(dataframe)
print('--------------------------------------------------')
# Pass it to the backtrader datafeed and add it to the cerebro
data = StratData(dataname=dataframe)
cerebro.adddata(data)
# Run over everything
cerebro.run()
# Plot the result
if not args.noplot:
cerebro.plot(style='bar')
def parse_args():
parser = argparse.ArgumentParser(
description='Pandas test script')
parser.add_argument('--noheaders', action='store_true', default=False,
required=False,
help='Do not use header rows')
parser.add_argument('--noprint', action='store_true', default=False,
help='Print the dataframe')
parser.add_argument('--noplot', action='store_true', default=False,
help='Do not plot the chart')
return parser.parse_args()
if __name__ == '__main__':
runstrat()
Running this prints my Pandas df, here's a head():
--------------------------------------------------
Open High ... Slow_ATR Impulse
2015-01-01 22:00:00+00:00 0.81827 0.81856 ... 0.006917 green
2015-01-04 22:00:00+00:00 0.80734 0.81074 ... 0.006949 green
2015-01-05 22:00:00+00:00 0.80855 0.81574 ... 0.007015 green
2015-01-06 22:00:00+00:00 0.80828 0.80902 ... 0.006871 grey
2015-01-07 22:00:00+00:00 0.80756 0.81308 ... 0.006773 grey
2015-01-08 22:00:00+00:00 0.81198 0.82089 ... 0.007171 green
But instead of plotting/logging, it ultimately results in this error:
Traceback (most recent call last):
File "/Users/kayne/Desktop/PyCharm Algos/pandas_test.py", line 92, in <module>
runstrat()
File "/Users/kayne/Desktop/PyCharm Algos/pandas_test.py", line 71, in runstrat
cerebro.plot(style='bar')
File "/anaconda3/lib/python3.7/site-packages/backtrader/cerebro.py", line 996, in plot
plotter.show()
File "/anaconda3/lib/python3.7/site-packages/backtrader/plot/plot.py", line 795, in show
self.mpyplot.show()
AttributeError: 'Plot_OldSync' object has no attribute 'mpyplot'
I really do apologise if this is really basic stuff, I have looked at any backtrader Pandas related topic and docs, github etc. but still can't seem to figure this out - I know it is most likely very easy!
Thanks in advance!
Ok, so I was wrong, and it does matplotlib is implemented. However, I fixed that error by adding this to my code:
data = bt.feeds.PandasData(dataname=dataframe, volume=False, openinterest=False)
That gets rid of that ugly long error, but now I am getting this error:
ValueError: Location based indexing can only have [integer, integer slice (START point is INCLUDED, END point is EXCLUDED), listlike of integers, boolean array] types
Again, i'm struggling on this error - google points (logically) to pandas indexing, but this is weird as i'm not indexing anything.
Can anyone point me in the right direction?
Hi, I did google it and did see the matplotlib specific error. However, I thought backtrader implements bokeh for graphing, not matplotlib? Perhaps I am way off, forgive my ignorance.
Under that assumption, are you saying that there looks nothing wrong with the code in the second post?
Thanks for getting back so quickly :)