Custom Indicator - Mixing Timeframes - different scenarios
-
Hey all,
First of all, thank you for the amazing backtesting library! I've been trying to use a custom indicator that uses multiple timeframes and I have found some things I dont understand. I've read a lot on the forum and still don't understand some behaviour that occurs with multiple timeframes. Below I'll try to as accurately possible explain the different code I've tried and results I've gotten. I on purpose do not use the cerebro.resample function as I read in another post that if you want custom lines to your datafeed, you should not use the resample function. In my simplified example I have not added any extra lines, am just printing out data, and have NOT used the multiple timeframes in calculations, just loaded the timeframes in the indicator. My goal is to understand why I am getting different results when using different code in this simplified example.
Data used (first 4 rows):
1 minute dataframe
date_of_close open high low close
2018-01-01 00:01:00 13715.65 13715.65 13681.00 13707.92
2018-01-01 00:02:00 13707.91 13707.91 13666.11 13694.92
2018-01-01 00:03:00 13682.00 13694.94 13680.00 13680.00
2018-01-01 00:04:00 13679.98 13679.98 13601.00 13645.9960 minute dataframe
date_of_close open high low close
2018-01-01 01:00:00 13715.65 13715.65 13400.01 13529.01
2018-01-01 02:00:00 13528.99 13595.89 13155.38 13203.06
2018-01-01 03:00:00 13203.00 13418.43 13200.00 13330.18
2018-01-01 04:00:00 13330.26 13611.27 13290.00 13410.03Scenario 1 code:
- runonce=True
- 1 datafeed passed to indicator
class TestCrossIndicator(bt.Indicator): lines = ('crossline',) params = dict(period=25) def __init__(self): self.ema = btind.ExponentialMovingAverage(self.datas[0].close, period=self.params.period) self.lines.crossline = self.datas[0].close > self.ema
class TestCrossStrategy(bt.Strategy): def __init__(self): self.indicatorline = TestCrossIndicator(self.datas[1]) self.ema = btind.ExponentialMovingAverage(self.datas[1].close, period=25) self.strategyline = self.datas[1].close > self.ema def next(self): print('LTF Strategy: ', len(self.datas[0])) print('HTF Strategy: ', len(self.datas[1])) print('Indicator line', self.indicatorline[0]) print('Strategy line', self.strategyline[0]) print(self.indicatorline[0] == self.strategyline[0])
cerebro = bt.Cerebro() ltf_data = PandasData(dataname=ltf_train, timeframe=bt.TimeFrame.Minutes, compression=1, plot=True) htf_data = PandasData(dataname=htf_train, timeframe=bt.TimeFrame.Minutes, compression=60, plot=True) data = cerebro.adddata(ltf_data) data2 = cerebro.adddata(htf_data) cerebro.addstrategy(TestCrossStrategy) cerebro.broker.set_cash(10000) print('Starting Portfolio Value: %.2f' % cerebro.broker.getvalue()) cerebro.run(runonce=True) print('Final Portfolio Value: %.2f' % cerebro.broker.getvalue())
Output Scenario 1:
LTF Strategy: 1500 HTF Strategy: 25 Indicator line 1.0 Strategy line 1.0 True LTF Strategy: 1501 HTF Strategy: 25 Indicator line 1.0 Strategy line 1.0 True LTF Strategy: 1502 HTF Strategy: 25 Indicator line 1.0 Strategy line 1.0 True
Code runs as I would expect. Minimum period of 25 based on datas[1], indicator line and strategy line are equivalent.
Scenario 2 code:
- runonce=True
- 2 datafeeds passed to indicator
Code adjusted for scenario 2:
A. Add a datafeed to the indicator.self.indicatorline = TestCrossIndicator(self.datas[0], self.datas[1])
B. Adjust indicator to take into account second datafeed
self.ema = btind.ExponentialMovingAverage(self.datas[1].close, period=self.params.period) self.lines.crossline = self.datas[1].close > self.ema
Output scenario 2:
LTF Strategy: 1500 HTF Strategy: 25 Indicator line nan Strategy line 1.0 False LTF Strategy: 1501 HTF Strategy: 25 Indicator line nan Strategy line 1.0 False LTF Strategy: 1502 HTF Strategy: 25 Indicator line nan Strategy line 1.0 False
The indicator line is NaN as opposed to the expected 1. I do not understand why it is is NaN. I read https://www.backtrader.com/docu/mixing-timeframes/indicators-mixing-timeframes/ and based upon that reading I thought I only needed the runonce=False argument in celebro.run() if I am doing certain calculations based on multiple datafeeds (such as making comparisons between data feeds). However, I have just loaded the 2 datafeeds in the indicator and am using only 1 datafeed for calculations.
I had a closer look at the above and added the following code to TestCrossIndicator.
def next(self): print('LTF Indicator:', len(self.datas[0])) print('HTF Indicator:', len(self.datas[1]))
Output:
LTF Indicator: 25 HTF Indicator: 25 LTF Indicator: 26 HTF Indicator: 26 LTF Indicator: 27 HTF Indicator: 27 LTF Indicator: 28 HTF Indicator: 28 LTF Indicator: 29 HTF Indicator: 29 .......... .......... LTF Indicator: 7074 HTF Indicator: 7074 LTF Indicator: 7075 HTF Indicator: 7075 LTF Strategy: 1500 HTF Strategy: 25 Indicator line nan Strategy line 1.0 False LTF Strategy: 1501 HTF Strategy: 25 Indicator line nan Strategy line 1.0 False LTF Strategy: 1502 HTF Strategy: 25 Indicator line nan Strategy line 1.0 False
The above output was somewhat unexpected to me that the indicator output was printed first and that the LEN of the HTF datafeed is equal to the LEN of the LTF datafeed. I assume this has something to do with the indicator being pre-processed due to runonce=True, but I expected the datafeeds LEN in Indicator to still be as in Strategy. Also not sure why the next() method was ran before the strategy next() method, as I thought only the init_ method would be pre-processed of the custom indicator.
Scenario 3:
- Runonce=False
- 2 datafeeds passed to indicator
LTF Indicator: 25 HTF Indicator: 0 LTF Indicator: 26 HTF Indicator: 0 LTF Indicator: 27 HTF Indicator: 0 LTF Indicator: 28 HTF Indicator: 0 LTF Indicator: 29 HTF Indicator: 0 ....... ....... ....... LTF Strategy: 1500 HTF Strategy: 25 Indicator line 1.0 Strategy line 1.0 True LTF Indicator: 1501 HTF Indicator: 25 LTF Strategy: 1501 HTF Strategy: 25 Indicator line 1.0 Strategy line 1.0 True LTF Indicator: 1502 HTF Indicator: 25 LTF Strategy: 1502 HTF Strategy: 25 Indicator line 1.0 Strategy line 1.0 True
Output is as I expected, with next() method of Indicator running along with next() method of Strategy. LEN of HTF datafeed is correct in custom indicator. Indicator line equals Strategy line.
Summary
Summarized based on the different scenarios above it seems that I should always use runonce=False when using a custom indicator with multiple data feeds. Is this true? I would love to better understand why the behaviour in scenario 2 occurs.In the Strategy class, the strategyline seems to work fine even with runonce=True. As runonce=True is faster based on the docs, should one implement indicators in the Strategy class and not use the seperate Indicator class if highest speed is preferred?
For completeness sake: the above behaviour in scenario 2 does not occur when I am using the celebro.resample method to load the second datafeed. In such case, the indicator works fine even if not using runonce=False. I do have to manually set the minimum period though in the indicator when using the resample method. As mentioned in the introduction, I am on purpose not using the resample method as my production code will have custom lines (e.g. see https://community.backtrader.com/topic/266/line-info-lost-during-resampling)
Im still a beginner in programming, so I hope I posted all the relevant information. If there is any additional information you need, please let me know :) Thanks a lot in advance and once again thanks for the platform!
Kind regards,
Monstrar
-
It seems that the behaviour is for me unexpected in scenario 3 (runonce=False, multiple timeframes) if I do not have the strategyline defined (see in code below the part I commented out). Used a 15 minute data feed as opposed to 1hour data feed here, as I found this error while working on 15 minute data feed.
Code used (Scenario 4):
class TestCrossIndicator(bt.Indicator): lines = ('crossline',) params = dict(period=25) def __init__(self): self.ema = btind.ExponentialMovingAverage(self.datas[1].close, period=self.params.period) self.lines.crossline = self.datas[1].close > self.ema def next(self): print('LTF Indicator:', len(self.datas[0])) print('HTF Indicator:', len(self.datas[1])) class TestCrossStrategy(bt.Strategy): def __init__(self): self.indicatorline = TestCrossIndicator(self.datas[0], self.datas[1]) # self.ema = btind.ExponentialMovingAverage(self.datas[1].close, period=25) # self.strategyline = self.datas[1].close > self.ema def next(self): print('LTF Strategy: ', len(self.datas[0])) print('HTF Strategy: ', len(self.datas[1])) print('Indicator line', self.indicatorline[0]) # print('Strategy line', self.strategyline[0]) # print(self.indicatorline[0] == self.strategyline[0]) cerebro = bt.Cerebro() ltf_data = PandasData(dataname=ltf_train, timeframe=bt.TimeFrame.Minutes, compression=1, plot=True) htf_data = PandasData(dataname=htf_train, timeframe=bt.TimeFrame.Minutes, compression=15, plot=True) data = cerebro.adddata(ltf_data) data2 = cerebro.adddata(htf_data) cerebro.addstrategy(TestCrossStrategy) cerebro.broker.set_cash(10000) print('Starting Portfolio Value: %.2f' % cerebro.broker.getvalue()) cerebro.run(runonce=False) print('Final Portfolio Value: %.2f' % cerebro.broker.getvalue())
Output Scenario 4:
LTF Indicator: 25 HTF Indicator: 1 LTF Strategy: 25 HTF Strategy: 1 Indicator line nan LTF Indicator: 26 HTF Indicator: 1 LTF Strategy: 26 HTF Strategy: 1 Indicator line nan LTF Indicator: 27 HTF Indicator: 1 LTF Strategy: 27 HTF Strategy: 1 Indicator line nan LTF Indicator: 28 HTF Indicator: 1 LTF Strategy: 28 HTF Strategy: 1 Indicator line nan LTF Indicator: 29 HTF Indicator: 1 LTF Strategy: 29 HTF Strategy: 1 Indicator line nan ..... ..... LTF Indicator: 374 HTF Indicator: 24 LTF Strategy: 374 HTF Strategy: 24 Indicator line nan LTF Indicator: 375 HTF Indicator: 25 LTF Strategy: 375 HTF Strategy: 25 Indicator line 1.0 LTF Indicator: 376 HTF Indicator: 25 LTF Strategy: 376 HTF Strategy: 25 Indicator line 1.0
In this case, the indicator seems to work as expected, providing NaN for the first 24 bars of the HTF data feed. However, the indicator and strategy next method seem to use the minimum period based on the LTF data feed (LTF starts at 25) and not on the HTF data feed. I don't understand why this happens.
Once again thanks!