For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/
(**NOT**) Bug: Indicator on higher timeframe is called at every cycle of lower timeframe
-
Hello,
As reported previously [here] (https://community.backtrader.com/topic/441/psar-indicator-and-multiple-timeframes/7) there seem to be an issue with indicators on resampled data being called too often ?
Example code below:
1)load data on smaller timeframe (5min)
2)resample data on daily
3)call a strategy with an indicator on daily data
4)the indicator "next" method is called at every cycle of 5min instead of every day.import backtrader as bt import logging from backtrader.feeds import PandasData import pandas as pd logging.basicConfig(level=logging.DEBUG, format='%(message)s',filemode='w') class MyIndicator(bt.Indicator): lines = ('et',) params = (("period", 100),) def __init__(self): self.prevdate = 0 def next(self): #if self.data.datetime[0] == self.prevdate: #print("returning as still the same date") # return #self.prevdate = self.data.datetime[0] self.log("-----new indicator cycle-----------") #df = bt_to_pandas(self.data, self.p.period) #self.log("rows extracted:{}".format(len(df))) def log(self, txt, dt=None): dt = dt or self.data.datetime[0] if isinstance(dt, float): dt = bt.num2date(dt) logging.info('%s, %s' % (dt.isoformat(), txt)) class MyStrategy(bt.Strategy): def __init__(self): self.myindic = MyIndicator(self.data1, period=50) def next(self): self.log('strategy next pos {}'.format( self.position.size)) def log(self, txt, dt=None): dt = dt or self.data.datetime[0] if isinstance(dt, float): dt = bt.num2date(dt) logging.debug('%s, %s' % (dt.isoformat(), txt)) def main(): df = pd.read_pickle("bitmex_BTCUSD_5m_100days.pkl") datasmalltf = PandasData(dataname=df, timeframe=bt.TimeFrame.Minutes, compression=5) cerebro = bt.Cerebro() cerebro.adddata(datasmalltf) datadaily = cerebro.resampledata(datasmalltf, timeframe=bt.TimeFrame.Days, compression=1, name='daily') cerebro.addstrategy(MyStrategy) cerebro.run(runonce=True, stdstats=False) if __name__ == "__main__": print('starting main') main()2019-01-01T23:59:59.999989, -----new indicator cycle----------- 2019-01-02T14:00:00, strategy next pos 0 2019-01-01T23:59:59.999989, -----new indicator cycle----------- 2019-01-02T14:05:00, strategy next pos 0 2019-01-01T23:59:59.999989, -----new indicator cycle----------- 2019-01-02T14:10:00, strategy next pos 0 2019-01-01T23:59:59.999989, -----new indicator cycle----------- 2019-01-02T14:15:00, strategy next pos 0 2019-01-01T23:59:59.999989, -----new indicator cycle----------- 2019-01-02T14:20:00, strategy next pos 0 2019-01-01T23:59:59.999989, -----new indicator cycle----------- 2019-01-02T14:25:00, strategy next pos 0 2019-01-01T23:59:59.999989, -----new indicator cycle----------- 2019-01-02T14:30:00, strategy next pos 0 2019-01-01T23:59:59.999989, -----new indicator cycle----------- 2019-01-02T14:35:00, strategy next pos 0 2019-01-01T23:59:59.999989, -----new indicator cycle----------- 2019-01-02T14:40:00, strategy next pos 0 2019-01-01T23:59:59.999989, -----new indicator cycle----------- 2019-01-02T14:45:00, strategy next pos 0 2019-01-01T23:59:59.999989, -----new indicator cycle----------- 2019-01-02T14:50:00, strategy next pos 0 2019-01-01T23:59:59.999989, -----new indicator cycle----------- 2019-01-02T14:55:00, strategy next pos 0 2019-01-01T23:59:59.999989, -----new indicator cycle----------- 2019-01-02T15:00:00, strategy next pos 0 2019-01-01T23:59:59.999989, -----new indicator cycle----------- 2019-01-02T15:05:00, strategy next pos 0 2019-01-01T23:59:59.999989, -----new indicator cycle----------- 2019-01-02T15:10:00, strategy next pos 0 2019-01-01T23:59:59.999989, -----new indicator cycle----------- 2019-01-02T15:15:00, strategy next pos 0 2019-01-01T23:59:59.999989, -----new indicator cycle----------- 2019-01-02T15:20:00, strategy next pos 0 2019-01-01T23:59:59.999989, -----new indicator cycle----------- 2019-01-02T15:25:00, strategy next pos 0 2019-01-01T23:59:59.999989, -----new indicator cycle----------- 2019-01-02T15:30:00, strategy next pos 0 2019-01-01T23:59:59.999989, -----new indicator cycle----------- 2019-01-02T15:35:00, strategy next pos 0 2019-01-01T23:59:59.999989, -----new indicator cycle----------- 2019-01-02T15:40:00, strategy next pos 0 2019-01-01T23:59:59.999989, -----new indicator cycle----------- 2019-01-02T15:45:00, strategy next pos 0 2019-01-01T23:59:59.999989, -----new indicator cycle-----------I have a workaround for this by storing the date in a variable of the indicator and comparing the date at the beginning of the "next" method, but this is dirty
def __init__(self): self.prevdate = 0 def next(self): if self.data.datetime[0] == self.prevdate: print("returning as still the same date") return self.prevdate = self.data.datetime[0] self.log("-----new indicator cycle-----------")2019-01-13T23:59:59.999989, returning as still the same date 2019-01-14T13:50:00, strategy next pos 0 2019-01-13T23:59:59.999989, returning as still the same date 2019-01-14T13:55:00, strategy next pos 0 2019-01-13T23:59:59.999989, returning as still the same date 2019-01-14T14:00:00, strategy next pos 0 2019-01-13T23:59:59.999989, returning as still the same date 2019-01-14T14:05:00, strategy next pos 0 2019-01-13T23:59:59.999989, returning as still the same date 2019-01-14T14:10:00, strategy next pos 0 2019-01-13T23:59:59.999989, returning as still the same date 2019-01-14T14:15:00, strategy next pos 0 2019-01-13T23:59:59.999989, returning as still the same date 2019-01-14T14:20:00, strategy next pos 0 2019-01-13T23:59:59.999989, returning as still the same date 2019-01-14T14:25:00, strategy next pos 0 2019-01-13T23:59:59.999989, returning as still the same date -
Now I have been using the previous workaround for a while, but found another issue that could be linked to my code but not sure.
Now same code as above but extracting data at every cycle of the indicator for a period of 20 (last 20 candles)
The data is extracted correctly for the first 20 cycles but after that the amount of data extracted diminish instead of staying at 20
At the first cycle there is only one candle to extract, then 2, etc up to 20, but after this we should extract always 20 candles until the end. Instead we have 20,19,18 and so on
This works correctly with the same code with only 1 timeframe (not using the resampled tf)
import backtrader as bt import logging from backtrader.feeds import PandasData import pandas as pd logging.basicConfig(level=logging.DEBUG, format='%(message)s',filemode='w') class MyIndicator(bt.Indicator): lines = ('et',) params = (("period", 100),) def __init__(self): self.prevdate = 0 def next(self): if self.data.datetime[0] == self.prevdate: #self.log("returning as still the same date") return self.prevdate = self.data.datetime[0] self.log("-----new indicator cycle-----------") df = bt_to_pandas(self.data, self.p.period) self.log("rows extracted:{}".format(len(df))) def log(self, txt, dt=None): dt = dt or self.data.datetime[0] if isinstance(dt, float): dt = bt.num2date(dt) logging.info('%s, %s' % (dt.isoformat(), txt)) class MyStrategy(bt.Strategy): def __init__(self): self.myindic = MyIndicator(self.data1, period=20) def next(self): pass #self.log('strategy next pos {}'.format( self.position.size)) def log(self, txt, dt=None): dt = dt or self.data.datetime[0] if isinstance(dt, float): dt = bt.num2date(dt) logging.debug('%s, %s' % (dt.isoformat(), txt)) def bt_to_pandas(btdata, period): logging.debug("getting bt data for period {}".format(period)) get = lambda mydata: mydata.get(ago=0, size=period) fields = { 'Open': get(btdata.open), 'High': get(btdata.high), 'Low': get(btdata.low), 'Close': get(btdata.close), 'Volume': get(btdata.volume) } time = [btdata.num2date(x) for x in get(btdata.datetime)] df = pd.DataFrame(data=fields, index=time) logging.debug("data collected for period {}".format(len(df))) return df def main(): df = pd.read_pickle("bitmex_BTCUSD_5m_100days.pkl") datasmalltf = PandasData(dataname=df, timeframe=bt.TimeFrame.Minutes, compression=5) cerebro = bt.Cerebro() cerebro.adddata(datasmalltf) datadaily = cerebro.resampledata(datasmalltf, timeframe=bt.TimeFrame.Days, compression=1, name='daily') cerebro.addstrategy(MyStrategy) cerebro.run(runonce=True, stdstats=False) if __name__ == "__main__": print('starting main') main()2019-01-01T23:59:59.999989, -----new indicator cycle----------- getting bt data for period 20 data collected for period 1 2019-01-01T23:59:59.999989, rows extracted:1 2019-01-02T23:59:59.999989, -----new indicator cycle----------- getting bt data for period 20 data collected for period 2 2019-01-02T23:59:59.999989, rows extracted:2 2019-01-03T23:59:59.999989, -----new indicator cycle----------- getting bt data for period 20 data collected for period 3 2019-01-03T23:59:59.999989, rows extracted:3 2019-01-04T23:59:59.999989, -----new indicator cycle----------- getting bt data for period 20 data collected for period 4 2019-01-04T23:59:59.999989, rows extracted:4 2019-01-05T23:59:59.999989, -----new indicator cycle----------- getting bt data for period 20 data collected for period 5 2019-01-05T23:59:59.999989, rows extracted:5 2019-01-06T23:59:59.999989, -----new indicator cycle----------- getting bt data for period 20 data collected for period 6 2019-01-06T23:59:59.999989, rows extracted:6 2019-01-07T23:59:59.999989, -----new indicator cycle----------- getting bt data for period 20 data collected for period 7 2019-01-07T23:59:59.999989, rows extracted:7 2019-01-08T23:59:59.999989, -----new indicator cycle----------- getting bt data for period 20 data collected for period 8 2019-01-08T23:59:59.999989, rows extracted:8 2019-01-09T23:59:59.999989, -----new indicator cycle----------- getting bt data for period 20 data collected for period 9 2019-01-09T23:59:59.999989, rows extracted:9 2019-01-10T23:59:59.999989, -----new indicator cycle----------- getting bt data for period 20 data collected for period 10 2019-01-10T23:59:59.999989, rows extracted:10 2019-01-11T23:59:59.999989, -----new indicator cycle----------- getting bt data for period 20 data collected for period 9 2019-01-11T23:59:59.999989, rows extracted:9 2019-01-12T23:59:59.999989, -----new indicator cycle----------- getting bt data for period 20 data collected for period 8 2019-01-12T23:59:59.999989, rows extracted:8 2019-01-13T23:59:59.999989, -----new indicator cycle----------- getting bt data for period 20 data collected for period 7 2019-01-13T23:59:59.999989, rows extracted:7 2019-01-14T23:59:59.999989, -----new indicator cycle----------- getting bt data for period 20 data collected for period 6 2019-01-14T23:59:59.999989, rows extracted:6 2019-01-15T23:59:59.999989, -----new indicator cycle----------- getting bt data for period 20 data collected for period 5 2019-01-15T23:59:59.999989, rows extracted:5 2019-01-16T23:59:59.999989, -----new indicator cycle----------- getting bt data for period 20 data collected for period 4 2019-01-16T23:59:59.999989, rows extracted:4 2019-01-17T23:59:59.999989, -----new indicator cycle----------- getting bt data for period 20 data collected for period 3 2019-01-17T23:59:59.999989, rows extracted:3 2019-01-18T23:59:59.999989, -----new indicator cycle----------- getting bt data for period 20 data collected for period 2 2019-01-18T23:59:59.999989, rows extracted:2 2019-01-19T23:59:59.999989, -----new indicator cycle----------- getting bt data for period 20 data collected for period 1 2019-01-19T23:59:59.999989, rows extracted:1 2019-01-20T23:59:59.999989, -----new indicator cycle----------- getting bt data for period 20 data collected for period 20 2019-01-20T23:59:59.999989, rows extracted:20 2019-01-21T23:59:59.999989, -----new indicator cycle----------- getting bt data for period 20 data collected for period 20 2019-01-21T23:59:59.999989, rows extracted:20 2019-01-22T23:59:59.999989, -----new indicator cycle----------- getting bt data for period 20 data collected for period 20 2019-01-22T23:59:59.999989, rows extracted:20 2019-01-23T23:59:59.999989, -----new indicator cycle----------- getting bt data for period 20 data collected for period 20 2019-01-23T23:59:59.999989, rows extracted:20 2019-01-24T23:59:59.999989, -----new indicator cycle----------- getting bt data for period 20 data collected for period 20 2019-01-24T23:59:59.999989, rows extracted:20 2019-01-25T23:59:59.999989, -----new indicator cycle----------- getting bt data for period 20 data collected for period 20 2019-01-25T23:59:59.999989, rows extracted:20 2019-01-26T23:59:59.999989, -----new indicator cycle----------- getting bt data for period 20 data collected for period 20 2019-01-26T23:59:59.999989, rows extracted:20 2019-01-27T23:59:59.999989, -----new indicator cycle----------- getting bt data for period 20 data collected for period 20 -
@pepelepew25 said in Bug: Indicator on higher timeframe is called at every cycle of lower timeframe:
4)the indicator "next" method is called at every cycle of 5min instead of every day.
Not a bug. I suggest you read the documentation, you can start at Docs - Platform Concepts and read about
len.
