Trouble with daily replay

Kurru

I am trying to use Backtrader to perform live trades using daily indicators, but I am having problems with using replay. My broker is Interactive Broker.

Unexpected bars/ticks
When I request daily replay, backtrader merges multiple days into a single bar during the backfill phase. The logs show this when len(self.data) is between 110-115.

I was expecting the backfill phase would result in 1 bar per day and another 1 bar for today. With multiple historical days being merged together, this is causing my indicators to be wrong (as multiple days are merged into 1).

Mysterious trading days
I seem to be receiving data for days that shouldnt be open. Such as Jan 1st (New Year) and Jan 9th (Saturday). Is this expected?

Higher than expected next calls when live
Since I requested 5s data from IB, I was expecting the replay to trigger about every ~5s, but it seems to be happening every 1-3seconds instead. Why?

Have I configured something incorrectly or am I misusing Backtrader?

My code:

import backtrader as bt

class LoggingStrategy(bt.Strategy):
    def __init__(self):
        self.moving_average = bt.indicators.SMA(
            self.data.close, period=5, plotname='5 day moving average'
        )

    def next(self):
        txt = list()
        txt.append('{}'.format(len(self.data)))
        txt.append('%s' % self.data.datetime.datetime(0).strftime('%Y-%m-%dT%H:%M:%S'))
        txt.append('O{:.2f}'.format(self.data.open[0]))
        txt.append('H{:.2f}'.format(self.data.high[0]))
        txt.append('L{:.2f}'.format(self.data.low[0]))
        txt.append('C{:.2f}'.format(self.data.close[0]))
        txt.append('V{:.0f}'.format(self.data.volume[0]))
        txt.append('MA{:.2f}'.format(self.moving_average[0]))
        print(' '.join(txt))

    def notify_data(self, data, status, *args, **kwargs):
        print('*' * 5, 'DATA NOTIF:', data._getstatusname(status), *args)


def main():
    cerebro = bt.Cerebro()

    cerebro.addstrategy(LoggingStrategy)

    ibstore = bt.stores.IBStore(host='127.0.0.1', port=7497, clientId=35)
    broker = ibstore.getbroker()
    cerebro.setbroker(broker)

    data0 = ibstore.getdata(dataname='TSLA-STK-SMART-USD',
                            timeframe=bt.TimeFrame.Seconds, compression=5)
    cerebro.replaydata(data0, timeframe=bt.TimeFrame.Days, compression=1)

    cerebro.run()


if __name__ == '__main__':
    main()

Output from my terminal:

109 2021-01-01T00:00:00 O665.38 H718.72 L664.00 C704.50 V643563 MA662.00
110 2021-01-05T00:00:00 O709.00 H744.49 L709.00 C733.60 V334298 MA684.60
110 2021-01-06T00:00:00 O709.00 H754.40 L709.00 C752.90 V547322 MA688.46
110 2021-01-07T00:00:00 O709.00 H774.00 L709.00 C763.60 V854628 MA690.60
111 2021-01-08T00:00:00 O769.00 H834.72 L767.20 C829.50 V342071 MA721.10
111 2021-01-09T00:00:00 O769.00 H884.89 L767.20 C867.50 V842450 MA728.70
112 2021-01-12T00:00:00 O866.38 H870.50 L803.62 C818.50 V386430 MA763.90
112 2021-01-13T00:00:00 O866.38 H870.50 L803.62 C852.00 V701243 MA770.60
112 2021-01-14T00:00:00 O866.38 H870.50 L803.62 C843.10 V922337 MA768.82
113 2021-01-15T00:00:00 O840.00 H863.00 L837.66 C847.95 V210184 MA805.33
113 2021-01-16T00:00:00 O840.00 H863.00 L819.10 C823.77 V472942 MA800.49
114 2021-01-20T00:00:00 O837.00 H850.00 L833.00 C843.25 V156179 MA828.24
114 2021-01-21T00:00:00 O837.00 H860.18 L833.00 C851.87 V320048 MA829.97
114 2021-01-22T00:00:00 O837.00 H860.18 L833.00 C841.99 V449676 MA827.99
115 2021-01-23T00:00:00 O840.82 H848.00 L828.62 C847.50 V125938 MA844.77
115 2021-01-26T00:00:00 O840.82 H900.40 L828.62 C874.50 V380851 MA850.17
115 2021-01-27T00:00:00 O840.82 H900.40 L828.62 C887.91 V527896 MA852.85
116 2021-01-28T00:00:00 O888.33 H891.51 L798.43 C820.39 V149899 MA843.43
***** DATA NOTIF: LIVE
116 2021-01-28T11:32:12 O888.33 H891.51 L798.43 C836.49 V149900 MA846.65
116 2021-01-28T11:32:13 O888.33 H891.51 L798.43 C836.22 V149904 MA846.60
116 2021-01-28T11:32:13 O888.33 H891.51 L798.43 C836.24 V149905 MA846.60
116 2021-01-28T11:32:13 O888.33 H891.51 L798.43 C836.23 V149912 MA846.60
116 2021-01-28T11:32:14 O888.33 H891.51 L798.43 C836.04 V149913 MA846.56
116 2021-01-28T11:32:14 O888.33 H891.51 L798.43 C835.94 V149914 MA846.54
116 2021-01-28T11:32:16 O888.33 H891.51 L798.43 C835.83 V149919 MA846.52

vladisld

@kurru said in Trouble with daily replay:

I requested 5s data from IB

AFAIK IB could only be requested to provide either ticks (default) or real-time 5 sec bars ( if rtbar parameter is set to True in getdata method). In case of ticks, backtrader is transforming it to look like bars with open/high/low/close prices set to the tick price.

In your case the ticks are requested ( rtbar is false by default) so you may get the 'tick' bar every time it is available ( could be much faster than 1 second depending on what your are trading). The fact that both timeframe and compression arguments were provided to the getdata method is irrelevant for live feed (they only used for getting the historical data AFAIR).

As for replay anomaly, I'm not sure here - probably others may help.