3. (**NOT**) Bug: Indicator on higher timeframe is called at every cycle of lower timeframe
For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

(**NOT**) Bug: Indicator on higher timeframe is called at every cycle of lower timeframe


  • Hello,

    As reported previously [here] (https://community.backtrader.com/topic/441/psar-indicator-and-multiple-timeframes/7) there seem to be an issue with indicators on resampled data being called too often ?

    Example code below:
    1)load data on smaller timeframe (5min)
    2)resample data on daily
    3)call a strategy with an indicator on daily data
    4)the indicator "next" method is called at every cycle of 5min instead of every day.

    import backtrader as bt
import logging
from backtrader.feeds import PandasData
import pandas as pd
logging.basicConfig(level=logging.DEBUG, format='%(message)s',filemode='w')

class MyIndicator(bt.Indicator):
    lines = ('et',)
    params = (("period", 100),)

    def __init__(self):
        self.prevdate = 0

    def next(self):

        #if self.data.datetime[0] == self.prevdate:
            #print("returning as still the same date")
        #    return
        #self.prevdate = self.data.datetime[0]
        self.log("-----new indicator cycle-----------")

        #df = bt_to_pandas(self.data, self.p.period)
        #self.log("rows extracted:{}".format(len(df)))

    def log(self, txt, dt=None):
        dt = dt or self.data.datetime[0]
        if isinstance(dt, float):
            dt = bt.num2date(dt)
        logging.info('%s, %s' % (dt.isoformat(), txt))

class MyStrategy(bt.Strategy):
    def __init__(self):
        self.myindic = MyIndicator(self.data1, period=50)

    def next(self):
        self.log('strategy next pos {}'.format( self.position.size))

    def log(self, txt, dt=None):
        dt = dt or self.data.datetime[0]
        if isinstance(dt, float):
            dt = bt.num2date(dt)
        logging.debug('%s, %s' % (dt.isoformat(), txt))

def main():
    df = pd.read_pickle("bitmex_BTCUSD_5m_100days.pkl")
    datasmalltf = PandasData(dataname=df, timeframe=bt.TimeFrame.Minutes, compression=5)

    cerebro = bt.Cerebro()

    cerebro.adddata(datasmalltf)
    datadaily = cerebro.resampledata(datasmalltf, timeframe=bt.TimeFrame.Days, compression=1, name='daily')

    cerebro.addstrategy(MyStrategy)
    cerebro.run(runonce=True, stdstats=False)


if __name__ == "__main__":
    print('starting main')
    main()

    2019-01-01T23:59:59.999989, -----new indicator cycle-----------
2019-01-02T14:00:00, strategy next pos 0
2019-01-01T23:59:59.999989, -----new indicator cycle-----------
2019-01-02T14:05:00, strategy next pos 0
2019-01-01T23:59:59.999989, -----new indicator cycle-----------
2019-01-02T14:10:00, strategy next pos 0
2019-01-01T23:59:59.999989, -----new indicator cycle-----------
2019-01-02T14:15:00, strategy next pos 0
2019-01-01T23:59:59.999989, -----new indicator cycle-----------
2019-01-02T14:20:00, strategy next pos 0
2019-01-01T23:59:59.999989, -----new indicator cycle-----------
2019-01-02T14:25:00, strategy next pos 0
2019-01-01T23:59:59.999989, -----new indicator cycle-----------
2019-01-02T14:30:00, strategy next pos 0
2019-01-01T23:59:59.999989, -----new indicator cycle-----------
2019-01-02T14:35:00, strategy next pos 0
2019-01-01T23:59:59.999989, -----new indicator cycle-----------
2019-01-02T14:40:00, strategy next pos 0
2019-01-01T23:59:59.999989, -----new indicator cycle-----------
2019-01-02T14:45:00, strategy next pos 0
2019-01-01T23:59:59.999989, -----new indicator cycle-----------
2019-01-02T14:50:00, strategy next pos 0
2019-01-01T23:59:59.999989, -----new indicator cycle-----------
2019-01-02T14:55:00, strategy next pos 0
2019-01-01T23:59:59.999989, -----new indicator cycle-----------
2019-01-02T15:00:00, strategy next pos 0
2019-01-01T23:59:59.999989, -----new indicator cycle-----------
2019-01-02T15:05:00, strategy next pos 0
2019-01-01T23:59:59.999989, -----new indicator cycle-----------
2019-01-02T15:10:00, strategy next pos 0
2019-01-01T23:59:59.999989, -----new indicator cycle-----------
2019-01-02T15:15:00, strategy next pos 0
2019-01-01T23:59:59.999989, -----new indicator cycle-----------
2019-01-02T15:20:00, strategy next pos 0
2019-01-01T23:59:59.999989, -----new indicator cycle-----------
2019-01-02T15:25:00, strategy next pos 0
2019-01-01T23:59:59.999989, -----new indicator cycle-----------
2019-01-02T15:30:00, strategy next pos 0
2019-01-01T23:59:59.999989, -----new indicator cycle-----------
2019-01-02T15:35:00, strategy next pos 0
2019-01-01T23:59:59.999989, -----new indicator cycle-----------
2019-01-02T15:40:00, strategy next pos 0
2019-01-01T23:59:59.999989, -----new indicator cycle-----------
2019-01-02T15:45:00, strategy next pos 0
2019-01-01T23:59:59.999989, -----new indicator cycle-----------

    I have a workaround for this by storing the date in a variable of the indicator and comparing the date at the beginning of the "next" method, but this is dirty

        def __init__(self):
        self.prevdate = 0

    def next(self):
        if self.data.datetime[0] == self.prevdate:
            print("returning as still the same date")
            return
        self.prevdate = self.data.datetime[0]
        self.log("-----new indicator cycle-----------")

    2019-01-13T23:59:59.999989, returning as still the same date
2019-01-14T13:50:00, strategy next pos 0
2019-01-13T23:59:59.999989, returning as still the same date
2019-01-14T13:55:00, strategy next pos 0
2019-01-13T23:59:59.999989, returning as still the same date
2019-01-14T14:00:00, strategy next pos 0
2019-01-13T23:59:59.999989, returning as still the same date
2019-01-14T14:05:00, strategy next pos 0
2019-01-13T23:59:59.999989, returning as still the same date
2019-01-14T14:10:00, strategy next pos 0
2019-01-13T23:59:59.999989, returning as still the same date
2019-01-14T14:15:00, strategy next pos 0
2019-01-13T23:59:59.999989, returning as still the same date
2019-01-14T14:20:00, strategy next pos 0
2019-01-13T23:59:59.999989, returning as still the same date
2019-01-14T14:25:00, strategy next pos 0
2019-01-13T23:59:59.999989, returning as still the same date
    0
    B
     1 Reply

  • Now I have been using the previous workaround for a while, but found another issue that could be linked to my code but not sure.

    Now same code as above but extracting data at every cycle of the indicator for a period of 20 (last 20 candles)

    The data is extracted correctly for the first 20 cycles but after that the amount of data extracted diminish instead of staying at 20

    At the first cycle there is only one candle to extract, then 2, etc up to 20, but after this we should extract always 20 candles until the end. Instead we have 20,19,18 and so on

    This works correctly with the same code with only 1 timeframe (not using the resampled tf)

    import backtrader as bt
import logging
from backtrader.feeds import PandasData
import pandas as pd
logging.basicConfig(level=logging.DEBUG, format='%(message)s',filemode='w')


class MyIndicator(bt.Indicator):
    lines = ('et',)
    params = (("period", 100),)

    def __init__(self):
        self.prevdate = 0

    def next(self):
        if self.data.datetime[0] == self.prevdate:
            #self.log("returning as still the same date")
            return
        self.prevdate = self.data.datetime[0]
        self.log("-----new indicator cycle-----------")

        df = bt_to_pandas(self.data, self.p.period)
        self.log("rows extracted:{}".format(len(df)))

    def log(self, txt, dt=None):
        dt = dt or self.data.datetime[0]
        if isinstance(dt, float):
            dt = bt.num2date(dt)
        logging.info('%s, %s' % (dt.isoformat(), txt))


class MyStrategy(bt.Strategy):
    def __init__(self):
        self.myindic = MyIndicator(self.data1, period=20)

    def next(self):
        pass
        #self.log('strategy next pos {}'.format( self.position.size))

    def log(self, txt, dt=None):
        dt = dt or self.data.datetime[0]
        if isinstance(dt, float):
            dt = bt.num2date(dt)
        logging.debug('%s, %s' % (dt.isoformat(), txt))


def bt_to_pandas(btdata, period):
    logging.debug("getting bt data for period {}".format(period))
    get = lambda mydata: mydata.get(ago=0, size=period)

    fields = {
        'Open': get(btdata.open),
        'High': get(btdata.high),
        'Low': get(btdata.low),
        'Close': get(btdata.close),
        'Volume': get(btdata.volume)
    }
    time = [btdata.num2date(x) for x in get(btdata.datetime)]

    df = pd.DataFrame(data=fields, index=time)
    logging.debug("data collected for period {}".format(len(df)))
    return df


def main():
    df = pd.read_pickle("bitmex_BTCUSD_5m_100days.pkl")
    datasmalltf = PandasData(dataname=df, timeframe=bt.TimeFrame.Minutes, compression=5)

    cerebro = bt.Cerebro()

    cerebro.adddata(datasmalltf)
    datadaily = cerebro.resampledata(datasmalltf, timeframe=bt.TimeFrame.Days, compression=1, name='daily')

    cerebro.addstrategy(MyStrategy)
    cerebro.run(runonce=True, stdstats=False)


if __name__ == "__main__":
    print('starting main')
    main()

    2019-01-01T23:59:59.999989, -----new indicator cycle-----------
getting bt data for period 20
data collected for period 1
2019-01-01T23:59:59.999989, rows extracted:1
2019-01-02T23:59:59.999989, -----new indicator cycle-----------
getting bt data for period 20
data collected for period 2
2019-01-02T23:59:59.999989, rows extracted:2
2019-01-03T23:59:59.999989, -----new indicator cycle-----------
getting bt data for period 20
data collected for period 3
2019-01-03T23:59:59.999989, rows extracted:3
2019-01-04T23:59:59.999989, -----new indicator cycle-----------
getting bt data for period 20
data collected for period 4
2019-01-04T23:59:59.999989, rows extracted:4
2019-01-05T23:59:59.999989, -----new indicator cycle-----------
getting bt data for period 20
data collected for period 5
2019-01-05T23:59:59.999989, rows extracted:5
2019-01-06T23:59:59.999989, -----new indicator cycle-----------
getting bt data for period 20
data collected for period 6
2019-01-06T23:59:59.999989, rows extracted:6
2019-01-07T23:59:59.999989, -----new indicator cycle-----------
getting bt data for period 20
data collected for period 7
2019-01-07T23:59:59.999989, rows extracted:7
2019-01-08T23:59:59.999989, -----new indicator cycle-----------
getting bt data for period 20
data collected for period 8
2019-01-08T23:59:59.999989, rows extracted:8
2019-01-09T23:59:59.999989, -----new indicator cycle-----------
getting bt data for period 20
data collected for period 9
2019-01-09T23:59:59.999989, rows extracted:9
2019-01-10T23:59:59.999989, -----new indicator cycle-----------
getting bt data for period 20
data collected for period 10
2019-01-10T23:59:59.999989, rows extracted:10
2019-01-11T23:59:59.999989, -----new indicator cycle-----------
getting bt data for period 20
data collected for period 9
2019-01-11T23:59:59.999989, rows extracted:9
2019-01-12T23:59:59.999989, -----new indicator cycle-----------
getting bt data for period 20
data collected for period 8
2019-01-12T23:59:59.999989, rows extracted:8
2019-01-13T23:59:59.999989, -----new indicator cycle-----------
getting bt data for period 20
data collected for period 7
2019-01-13T23:59:59.999989, rows extracted:7
2019-01-14T23:59:59.999989, -----new indicator cycle-----------
getting bt data for period 20
data collected for period 6
2019-01-14T23:59:59.999989, rows extracted:6
2019-01-15T23:59:59.999989, -----new indicator cycle-----------
getting bt data for period 20
data collected for period 5
2019-01-15T23:59:59.999989, rows extracted:5
2019-01-16T23:59:59.999989, -----new indicator cycle-----------
getting bt data for period 20
data collected for period 4
2019-01-16T23:59:59.999989, rows extracted:4
2019-01-17T23:59:59.999989, -----new indicator cycle-----------
getting bt data for period 20
data collected for period 3
2019-01-17T23:59:59.999989, rows extracted:3
2019-01-18T23:59:59.999989, -----new indicator cycle-----------
getting bt data for period 20
data collected for period 2
2019-01-18T23:59:59.999989, rows extracted:2
2019-01-19T23:59:59.999989, -----new indicator cycle-----------
getting bt data for period 20
data collected for period 1
2019-01-19T23:59:59.999989, rows extracted:1
2019-01-20T23:59:59.999989, -----new indicator cycle-----------
getting bt data for period 20
data collected for period 20
2019-01-20T23:59:59.999989, rows extracted:20
2019-01-21T23:59:59.999989, -----new indicator cycle-----------
getting bt data for period 20
data collected for period 20
2019-01-21T23:59:59.999989, rows extracted:20
2019-01-22T23:59:59.999989, -----new indicator cycle-----------
getting bt data for period 20
data collected for period 20
2019-01-22T23:59:59.999989, rows extracted:20
2019-01-23T23:59:59.999989, -----new indicator cycle-----------
getting bt data for period 20
data collected for period 20
2019-01-23T23:59:59.999989, rows extracted:20
2019-01-24T23:59:59.999989, -----new indicator cycle-----------
getting bt data for period 20
data collected for period 20
2019-01-24T23:59:59.999989, rows extracted:20
2019-01-25T23:59:59.999989, -----new indicator cycle-----------
getting bt data for period 20
data collected for period 20
2019-01-25T23:59:59.999989, rows extracted:20
2019-01-26T23:59:59.999989, -----new indicator cycle-----------
getting bt data for period 20
data collected for period 20
2019-01-26T23:59:59.999989, rows extracted:20
2019-01-27T23:59:59.999989, -----new indicator cycle-----------
getting bt data for period 20
data collected for period 20
    0
  • B
    administrators

    @pepelepew25 said in Bug: Indicator on higher timeframe is called at every cycle of lower timeframe:

    4)the indicator "next" method is called at every cycle of 5min instead of every day.

    Not a bug. I suggest you read the documentation, you can start at Docs - Platform Concepts and read about len.

    1
Log in to reply
 
});