IB Live Data Feed: differences between resampled data and backfilled data



  • Hi all,

    I'm doing some tests with backtrader + IB + live data feeds and I'm getting some differences between resampled data and backfilled data.

    Initially my goal is simple: A log with exactly the same bars/data than the TWS chart shows.

    Given the following code:

    class Test(bt.Strategy):
    
        ######################
        #
        ######################
        def notify_data(self, data, status, * args, ** kwargs):
            print('*' * 5, 'DATA NOTIF:', data._getstatusname(status), * args)
    
    
        ######################
        #
        ######################
        def notify_store(self, msg, * args, ** kwargs):
            print('*' * 5, 'STORE NOTIF:', msg)
    
        ######################
        #
        ######################
        def __init__(self):
            ''''''
        ######################
        #
        ######################
        def prenext(self):
            ''''''
            #self.next(frompre=True)
    
    
    
        ######################
        #
        ######################
        def log(self, txt):
            now = datetime.datetime.now()
            print('%s:  %s' % (
                  now.strftime(dtfmt), txt))
    
        def next (self, frompre=False):
    
            self.log("Frompre: %d data0 %s open %f high %f low %f close %f vol %d" %
                     (frompre,
                     self.data0.datetime.datetime(0).strftime(dtfmt),
                     self.data0.open[0],
                     self.data0.high[0],
                     self.data0.low[0],
                     self.data0.close[0],
                     self.data0.volume[0]))
    
    def parse_args():
        parser = argparse.ArgumentParser(formatter_class=argparse.ArgumentDefaultsHelpFormatter)
    
        parser.add_argument('--notifyall',
                            required=False, action='store_true',
                            help='Notify all messages to strategy as store notifs')
    
        parser.add_argument('--debug',
                            required=False, action='store_true',
                            help='Display all info received form IB')
    
    
        parser.add_argument('--host', default='127.0.0.1',
                            required=False, action='store',
                            help='Host for the Interactive Brokers TWS Connection')
    
        parser.add_argument('--port', default=4002, type=int,
                            required=False, action='store',
                            help='Port for the Interactive Brokers TWS Connection')
    
    
        parser.add_argument('--clientId', default=None, type=int,
                            required=False, action='store',
                            help='Client Id to connect to TWS (default: random)')
    
        parser.add_argument('--no-timeoffset',
                            required=False, action='store_true',
                            help=('Do not Use TWS/System time offset for non '
                            'timestamped prices and to align resampling'))
    
        parser.add_argument('--reconnect', default=3, type=int,
                            required=False, action='store',
                            help='Number of recconnection attempts to TWS')
    
        parser.add_argument('--timeout', default=3.0, type=float,
                            required=False, action='store',
                            help='Timeout between reconnection attempts to TWS')
    
        parser.add_argument('--data0', default=None,
                            required=False, action='store',
                            help='data 0 into the system')
    
        return parser.parse_args()
    
    if __name__ == '__main__':
    
        dtfmt = '%Y-%m-%dT%H:%M:%S.%f'
        now = datetime.datetime.now();
        print ("Now is %s" % (now.strftime(dtfmt)))
    
    
        args = parse_args()
    
        # Create a cerebro entity
        cerebro = bt.Cerebro(exactbars=True, runonce=False, preload=False)
    
    
        ibstore = bt.stores.IBStore(
                                    host=args.host, port=args.port,
                                    clientId=args.clientId, timeoffset=not args.no_timeoffset,
                                    reconnect=args.reconnect, timeout=args.timeout,
                                    notifyall=args.notifyall, _debug=args.debug)
    
    
        cerebro.setbroker(ibstore.getbroker())
    
    
    
        data0 = ibstore.getdata(dataname=args.data0,
                                backfill_start=True,
                                timeframe=bt.TimeFrame.Minutes,
                                compression=1,
                                name='d0',
                                qcheck=20)
    
        # New resampler
        cerebro.resampledata(
                             data0,
                             timeframe=bt.TimeFrame.Minutes,
                             compression=1)
    
        # Add a strategy
    
        cerebro.addstrategy(Test)
    
        cerebro.run()
    

    First run: backfilled data + resampled data:
    resampled_vs_backfilled.py --port 4002 --clientId 15 --data0 GBL-201709-DTB

    2017-06-16T09:48:19.251639: Frompre: 0 data0 2017-06-16T07:44:00.000000 open 164.41 high 164.41 low 164.34 close 164.35 vol 2712
    2017-06-16T09:48:19.257446: Frompre: 0 data0 2017-06-16T07:45:00.000000 open 164.35 high 164.36 low 164.33 close 164.35 vol 1530
    2017-06-16T09:48:19.259503: Frompre: 0 data0 2017-06-16T07:46:00.000000 open 164.34 high 164.35 low 164.32 close 164.34 vol 1330
    2017-06-16T09:48:19.261535: Frompre: 0 data0 2017-06-16T07:47:00.000000 open 164.34 high 164.35 low 164.34 close 164.35 vol 409
    2017-06-16T09:48:19.263397: Frompre: 0 data0 2017-06-16T07:48:00.000000 open 164.34 high 164.35 low 164.34 close 164.35 vol 165
    ***** DATA NOTIF: LIVE
    2017-06-16T09:49:02.211599: Frompre: 0 data0 2017-06-16T07:49:00.000000 open 164.35 high 164.39 low 164.35 close 164.38 vol 798
    2017-06-16T09:50:03.389798: Frompre: 0 data0 2017-06-16T07:50:00.000000 open 164.38 high 164.38 low 164.36 close 164.37 vol 538
    2017-06-16T09:51:02.751163: Frompre: 0 data0 2017-06-16T07:51:00.000000 open 164.37 high 164.38 low 164.36 close 164.37 vol 780
    2017-06-16T09:52:04.579715: Frompre: 0 data0 2017-06-16T07:52:00.000000 open 164.36 high 164.38 low 164.36 close 164.37 vol 747
    2017-06-16T09:53:02.244969: Frompre: 0 data0 2017-06-16T07:53:00.000000 open 164.37 high 164.37 low 164.36 close 164.36 vol 246
    2017-06-16T09:54:00.565664: Frompre: 0 data0 2017-06-16T07:54:00.000000 open 164.36 high 164.37 low 164.34 close 164.36 vol 896
    2017-06-16T09:55:11.430551: Frompre: 0 data0 2017-06-16T07:55:00.000000 open 164.35 high 164.36 low 164.34 close 164.34 vol 414
    2017-06-16T09:56:04.272170: Frompre: 0 data0 2017-06-16T07:56:00.000000 open 164.34 high 164.34 low 164.31 close 164.32 vol 2116
    2017-06-16T09:57:01.123514: Frompre: 0 data0 2017-06-16T07:57:00.000000 open 164.33 high 164.35 low 164.33 close 164.35 vol 931

    Second run: The previously resampled bars are now backfilled bars:

    2017-06-16T09:57:59.194470: Frompre: 0 data0 2017-06-16T07:44:00.000000 open 164.41 high 164.41 low 164.34 close 164.35 vol 2712
    2017-06-16T09:57:59.196351: Frompre: 0 data0 2017-06-16T07:45:00.000000 open 164.35 high 164.36 low 164.33 close 164.35 vol 1530
    2017-06-16T09:57:59.201949: Frompre: 0 data0 2017-06-16T07:46:00.000000 open 164.34 high 164.35 low 164.32 close 164.34 vol 1330
    2017-06-16T09:57:59.204162: Frompre: 0 data0 2017-06-16T07:47:00.000000 open 164.34 high 164.35 low 164.34 close 164.35 vol 409
    2017-06-16T09:57:59.206197: Frompre: 0 data0 2017-06-16T07:48:00.000000 open 164.34 high 164.39 low 164.34 close 164.38 vol 961
    2017-06-16T09:57:59.207977: Frompre: 0 data0 2017-06-16T07:49:00.000000 open 164.38 high 164.38 low 164.36 close 164.37 vol 540
    2017-06-16T09:57:59.210999: Frompre: 0 data0 2017-06-16T07:50:00.000000 open 164.37 high 164.38 low 164.36 close 164.37 vol 780
    2017-06-16T09:57:59.215024: Frompre: 0 data0 2017-06-16T07:51:00.000000 open 164.37 high 164.38 low 164.35 close 164.37 vol 747
    2017-06-16T09:57:59.216797: Frompre: 0 data0 2017-06-16T07:52:00.000000 open 164.37 high 164.37 low 164.36 close 164.36 vol 246
    2017-06-16T09:57:59.218651: Frompre: 0 data0 2017-06-16T07:53:00.000000 open 164.36 high 164.37 low 164.34 close 164.36 vol 890
    2017-06-16T09:57:59.220877: Frompre: 0 data0 2017-06-16T07:54:00.000000 open 164.36 high 164.36 low 164.34 close 164.34 vol 415
    2017-06-16T09:57:59.223913: Frompre: 0 data0 2017-06-16T07:55:00.000000 open 164.34 high 164.34 low 164.31 close 164.33 vol 2114
    2017-06-16T09:57:59.229085: Frompre: 0 data0 2017-06-16T07:56:00.000000 open 164.33 high 164.35 low 164.32 close 164.35 vol 934
    2017-06-16T09:57:59.230990: Frompre: 0 data0 2017-06-16T07:57:00.000000 open 164.35 high 164.35 low 164.33 close 164.35 vol 427

    Problems detected:

    1.- Some bars are different. e.g: the bar with 211X volume

    • resampled data -> close 164.32 vol 2116
    • backfilled data -> close 164.33 vol 2114

    2.- Different timestamps, e.g: the bar with volume 246:

    • resampled data time-> 07:53
    • backfilled data time -> 07:52

    I've tried increasing the qcheck up-to 20 without success

    Thanks in advance.


  • administrators

    @borodiliz said in IB Live Data Feed: differences between resampled data and backfilled data:

    I've tried increasing the qcheck up-to 20 without success

    You should decrease the qcheck parameter for more checks per second, which it's probably your intention (qcheck=0.5, the default, forces the system to wake-up every 0.5 seconds to do resampling clean-up for example, which in the absence of negotiation will allow the resampler to spit out bars by checking the real-time clock)

    Without claiming that backtrader is perfect and its resampling is perfect and ... and ... the expectation that the backfilled data and the real-time resampled data will be equal is too much to expect. Add to the equation that Interactive Brokers dismisses itself as a data provider and has disclaimers about the data.

    Reasons:

    • Ticks may be missed
    • Ticks may come in too late and be discarded (there is a parameter to control it and add them to the next resampled bar)
    • The resampler has to deliver in real-time using the real-time clock and cannot wait for ticks to show up if they are delayed. At 2017-06-16T07:57:00 it has to give you a bar, but it may not have all information from Interactive Brokers
    • The 4 times a second snapshots are not guaranteed to contain exact information.

    Some of this problems can be alleviated by switching to Real-Time Bars for which Interactive Brokers guarantees consolidated information (check the rtbar parameter in Docs - Interactive Brokers. But this is not available for all products.

    You can also face the following problem:

    • Backfill today and get bars with information X
    • Backfill tomorrow and get bars with information Y (slight changes to X), because some information has been consolidated.

    @borodiliz said in IB Live Data Feed: differences between resampled data and backfilled data:

    2.- Different timestamps, e.g: the bar with volume 246:

    resampled data time-> 07:53
    backfilled data time -> 07:52

    This is because IB is moving the data pointer to the beginning of the minute rather than to the end. It's just a convention.



  • Thanks for your quick reply @backtrader

    You can't claim backtrader is perfect, but I can claim backtrader is ... awesome!

    I can understand the complexity using tickString, so let's go with RealTimeBars data:

    Because I do not want bars as be delivered as much in real-time as possible and I perfer to "wait" a while I've set the following (qcheck=2 , timeoffset=True and rtbar=True):

        ibstore = bt.stores.IBStore(
                                    host=args.host, port=args.port,
                                    clientId=args.clientId, timeoffset=True,
                                    reconnect=args.reconnect, timeout=args.timeout,
                                    notifyall=args.notifyall, _debug=args.debug)
    
        cerebro.setbroker(ibstore.getbroker())
    
        data0 = ibstore.getdata(dataname=args.data0,
                                backfill_start=True,
                                timeframe=bt.TimeFrame.Minutes,
                                compression=1,
                                name='d0',
                                qcheck=2,
                                rtbar=True)
    

    First run: check resampled data with debug mode enabled:

    ...........
    <realtimeBar reqId=16777217, time=1497608880, open=164.35, high=164.35, low=164.35, close=164.35, volume=0, wap=164.35, count=0>
    <realtimeBar reqId=16777217, time=1497608885, open=164.35, high=164.35, low=164.35, close=164.35, volume=4, wap=164.35, count=3>
    <realtimeBar reqId=16777217, time=1497608890, open=164.35, high=164.35, low=164.35, close=164.35, volume=0, wap=164.35, count=0>
    <realtimeBar reqId=16777217, time=1497608895, open=164.35, high=164.35, low=164.34, close=164.34, volume=9, wap=164.35, count=2>
    <realtimeBar reqId=16777217, time=1497608900, open=164.34, high=164.34, low=164.34, close=164.34, volume=2, wap=164.34, count=1>
    <realtimeBar reqId=16777217, time=1497608905, open=164.34, high=164.34, low=164.34, close=164.34, volume=205, wap=164.34, count=2>
    <currentTime time=1497608915>
    <realtimeBar reqId=16777217, time=1497608910, open=164.34, high=164.34, low=164.34, close=164.34, volume=133, wap=164.34, count=9>
    <realtimeBar reqId=16777217, time=1497608915, open=164.34, high=164.34, low=164.34, close=164.34, volume=0, wap=164.34, count=0>
    <realtimeBar reqId=16777217, time=1497608920, open=164.34, high=164.34, low=164.34, close=164.34, volume=1, wap=164.34, count=1>
    <realtimeBar reqId=16777217, time=1497608925, open=164.35, high=164.35, low=164.35, close=164.35, volume=1, wap=164.35, count=1>
    <realtimeBar reqId=16777217, time=1497608930, open=164.35, high=164.35, low=164.35, close=164.35, volume=79, wap=164.35, count=1>
    <realtimeBar reqId=16777217, time=1497608935, open=164.35, high=164.35, low=164.35, close=164.35, volume=0, wap=164.35, count=0>
    2017-06-16T12:29:03.091121: Frompre: 0 data0 2017-06-16T10:29:00.000000 open 164.350000 high 164.350000 low 164.340000 close 164.350000 vol 434
    <realtimeBar reqId=16777217, time=1497608940, open=164.35, high=164.35, low=164.35, close=164.35, volume=1, wap=164.35, count=1>
    <realtimeBar reqId=16777217, time=1497608945, open=164.35, high=164.35, low=164.35, close=164.35, volume=30, wap=164.35, count=2>
    <realtimeBar reqId=16777217, time=1497608950, open=164.35, high=164.35, low=164.35, close=164.35, volume=0, wap=164.35, count=0>
    <realtimeBar reqId=16777217, time=1497608955, open=164.34, high=164.34, low=164.34, close=164.34, volume=1, wap=164.34, count=1>
    <realtimeBar reqId=16777217, time=1497608960, open=164.35, high=164.35, low=164.34, close=164.34, volume=13, wap=164.34, count=4>
    <realtimeBar reqId=16777217, time=1497608965, open=164.34, high=164.34, low=164.34, close=164.34, volume=15, wap=164.34, count=1>
    <currentTime time=1497608975>
    <realtimeBar reqId=16777217, time=1497608970, open=164.34, high=164.34, low=164.34, close=164.34, volume=13, wap=164.34, count=1>
    <realtimeBar reqId=16777217, time=1497608975, open=164.35, high=164.35, low=164.35, close=164.35, volume=1, wap=164.35, count=1>
    <realtimeBar reqId=16777217, time=1497608980, open=164.35, high=164.35, low=164.35, close=164.35, volume=13, wap=164.35, count=2>
    <realtimeBar reqId=16777217, time=1497608985, open=164.35, high=164.36, low=164.35, close=164.36, volume=48, wap=164.35, count=15>
    <realtimeBar reqId=16777217, time=1497608990, open=164.35, high=164.35, low=164.35, close=164.35, volume=129, wap=164.35, count=19>
    <realtimeBar reqId=16777217, time=1497608995, open=164.35, high=164.35, low=164.35, close=164.35, volume=26, wap=164.35, count=2>
    2017-06-16T12:30:03.100416: Frompre: 0 data0 2017-06-16T10:30:00.000000 open 164.350000 high 164.360000 low 164.340000 close 164.350000 vol 289
    <realtimeBar reqId=16777217, time=1497609000, open=164.34, high=164.34, low=164.34, close=164.34, volume=21, wap=164.34, count=1>
    <realtimeBar reqId=16777217, time=1497609005, open=164.34, high=164.34, low=164.34, close=164.34, volume=0, wap=164.34, count=0>
    <realtimeBar reqId=16777217, time=1497609010, open=164.34, high=164.34, low=164.34, close=164.34, volume=104, wap=164.34, count=9>
    <realtimeBar reqId=16777217, time=1497609015, open=164.34, high=164.34, low=164.34, close=164.34, volume=1, wap=164.34, count=1>
    <realtimeBar reqId=16777217, time=1497609020, open=164.34, high=164.34, low=164.34, close=164.34, volume=0, wap=164.34, count=0>
    <realtimeBar reqId=16777217, time=1497609025, open=164.33, high=164.33, low=164.33, close=164.33, volume=1, wap=164.33, count=1>
    <currentTime time=1497609035>
    <realtimeBar reqId=16777217, time=1497609030, open=164.34, high=164.34, low=164.34, close=164.34, volume=1, wap=164.34, count=1>
    <realtimeBar reqId=16777217, time=1497609035, open=164.34, high=164.34, low=164.34, close=164.34, volume=0, wap=164.34, count=0>
    <realtimeBar reqId=16777217, time=1497609040, open=164.34, high=164.34, low=164.34, close=164.34, volume=0, wap=164.34, count=0>
    <realtimeBar reqId=16777217, time=1497609045, open=164.33, high=164.33, low=164.33, close=164.33, volume=286, wap=164.33, count=24>
    <realtimeBar reqId=16777217, time=1497609050, open=164.33, high=164.33, low=164.33, close=164.33, volume=51, wap=164.33, count=6>
    <realtimeBar reqId=16777217, time=1497609055, open=164.33, high=164.34, low=164.33, close=164.34, volume=243, wap=164.34, count=5>
    2017-06-16T12:31:03.092756: Frompre: 0 data0 2017-06-16T10:31:00.000000 open 164.340000 high 164.340000 low 164.330000 close 164.340000 vol 687
    <realtimeBar reqId=16777217, time=1497609060, open=164.33, high=164.33, low=164.33, close=164.33, volume=160, wap=164.33, count=2>
    <realtimeBar reqId=16777217, time=1497609065, open=164.33, high=164.33, low=164.33, close=164.33, volume=0, wap=164.33, count=0>
    <realtimeBar reqId=16777217, time=1497609070, open=164.33, high=164.33, low=164.33, close=164.33, volume=86, wap=164.33, count=6>
    <realtimeBar reqId=16777217, time=1497609075, open=164.33, high=164.33, low=164.33, close=164.33, volume=0, wap=164.33, count=0>
    <realtimeBar reqId=16777217, time=1497609080, open=164.34, high=164.34, low=164.34, close=164.34, volume=33, wap=164.34, count=1>
    <realtimeBar reqId=16777217, time=1497609085, open=164.33, high=164.33, low=164.33, close=164.33, volume=48, wap=164.33, count=6>
    <currentTime time=1497609095>
    <realtimeBar reqId=16777217, time=1497609090, open=164.33, high=164.33, low=164.33, close=164.33, volume=24, wap=164.33, count=5>
    <realtimeBar reqId=16777217, time=1497609095, open=164.33, high=164.33, low=164.33, close=164.33, volume=43, wap=164.33, count=6>
    <realtimeBar reqId=16777217, time=1497609100, open=164.33, high=164.33, low=164.33, close=164.33, volume=0, wap=164.33, count=0>
    <realtimeBar reqId=16777217, time=1497609105, open=164.33, high=164.33, low=164.33, close=164.33, volume=0, wap=164.33, count=0>
    <realtimeBar reqId=16777217, time=1497609110, open=164.33, high=164.33, low=164.33, close=164.33, volume=0, wap=164.33, count=0>
    <realtimeBar reqId=16777217, time=1497609115, open=164.33, high=164.33, low=164.33, close=164.33, volume=54, wap=164.33, count=3>
    2017-06-16T12:32:03.096349: Frompre: 0 data0 2017-06-16T10:32:00.000000 open 164.330000 high 164.340000 low 164.330000 close 164.330000 vol 288

    Second run: The previously resampled bars are now backfilled bars:

    2017-06-16T12:33:44.813167: Frompre: 0 data0 2017-06-16T10:28:00.000000 open 164.350000 high 164.350000 low 164.340000 close 164.350000 vol 434
    2017-06-16T12:33:44.814699: Frompre: 0 data0 2017-06-16T10:29:00.000000 open 164.350000 high 164.360000 low 164.340000 close 164.350000 vol 290
    2017-06-16T12:33:44.816210: Frompre: 0 data0 2017-06-16T10:30:00.000000 open 164.340000 high 164.340000 low 164.330000 close 164.340000 vol 708
    2017-06-16T12:33:44.817950: Frompre: 0 data0 2017-06-16T10:31:00.000000 open 164.330000 high 164.340000 low 164.330000 close 164.330000 vol 448

    My conclusion: First rtbar is not taken into account

    After checking ibdata.py source code and debug data I think the problem arises when datetime for a rtbar is equal to the just emitted resampled bar. I seems those minor changes solves the problem.

    Test again:

    2017-06-16T13:27:02.523913: Frompre: 0 data0 2017-06-16T11:27:00.000000 open 164.340000 high 164.340000 low 164.330000 close 164.340000 vol 15
    <realtimeBar reqId=16777217, time=1497612420, open=164.34, high=164.34, low=164.34, close=164.34, volume=2, wap=164.34, count=2>
    <realtimeBar reqId=16777217, time=1497612425, open=164.34, high=164.34, low=164.34, close=164.34, volume=0, wap=164.34, count=0>
    <realtimeBar reqId=16777217, time=1497612430, open=164.34, high=164.34, low=164.34, close=164.34, volume=0, wap=164.34, count=0>
    <realtimeBar reqId=16777217, time=1497612435, open=164.34, high=164.34, low=164.34, close=164.34, volume=1, wap=164.34, count=1>
    <realtimeBar reqId=16777217, time=1497612440, open=164.34, high=164.34, low=164.34, close=164.34, volume=0, wap=164.34, count=0>
    <realtimeBar reqId=16777217, time=1497612445, open=164.34, high=164.34, low=164.34, close=164.34, volume=18, wap=164.34, count=2>
    <realtimeBar reqId=16777217, time=1497612450, open=164.33, high=164.34, low=164.33, close=164.34, volume=11, wap=164.33, count=2>
    <realtimeBar reqId=16777217, time=1497612455, open=164.34, high=164.34, low=164.34, close=164.34, volume=0, wap=164.34, count=0>
    <realtimeBar reqId=16777217, time=1497612460, open=164.34, high=164.34, low=164.34, close=164.34, volume=0, wap=164.34, count=0>
    <currentTime time=1497612466>
    <realtimeBar reqId=16777217, time=1497612465, open=164.34, high=164.34, low=164.34, close=164.34, volume=349, wap=164.34, count=42>
    <realtimeBar reqId=16777217, time=1497612470, open=164.34, high=164.34, low=164.34, close=164.34, volume=152, wap=164.34, count=17>
    <realtimeBar reqId=16777217, time=1497612475, open=164.34, high=164.34, low=164.34, close=164.34, volume=20, wap=164.34, count=5>
    2017-06-16T13:28:02.429118: Frompre: 0 data0 2017-06-16T11:28:00.000000 open 164.340000 high 164.340000 low 164.330000 close 164.340000 vol 553
    <realtimeBar reqId=16777217, time=1497612480, open=164.34, high=164.34, low=164.34, close=164.34, volume=61, wap=164.34, count=10>
    <realtimeBar reqId=16777217, time=1497612485, open=164.34, high=164.34, low=164.34, close=164.34, volume=3, wap=164.34, count=2>
    <realtimeBar reqId=16777217, time=1497612490, open=164.34, high=164.34, low=164.34, close=164.34, volume=0, wap=164.34, count=0>
    <realtimeBar reqId=16777217, time=1497612495, open=164.34, high=164.34, low=164.34, close=164.34, volume=1, wap=164.34, count=1>
    <realtimeBar reqId=16777217, time=1497612500, open=164.34, high=164.34, low=164.34, close=164.34, volume=2, wap=164.34, count=2>
    <realtimeBar reqId=16777217, time=1497612505, open=164.34, high=164.34, low=164.34, close=164.34, volume=4, wap=164.34, count=4>
    <realtimeBar reqId=16777217, time=1497612510, open=164.34, high=164.34, low=164.34, close=164.34, volume=3, wap=164.34, count=3>
    <realtimeBar reqId=16777217, time=1497612515, open=164.34, high=164.34, low=164.34, close=164.34, volume=2, wap=164.34, count=2>
    <realtimeBar reqId=16777217, time=1497612520, open=164.34, high=164.34, low=164.34, close=164.34, volume=3, wap=164.34, count=3>
    <currentTime time=1497612526>
    <realtimeBar reqId=16777217, time=1497612525, open=164.34, high=164.34, low=164.34, close=164.34, volume=3, wap=164.34, count=2>
    <realtimeBar reqId=16777217, time=1497612530, open=164.34, high=164.35, low=164.34, close=164.34, volume=256, wap=164.34, count=10>
    <realtimeBar reqId=16777217, time=1497612535, open=164.34, high=164.34, low=164.34, close=164.34, volume=0, wap=164.34, count=0>
    2017-06-16T13:29:02.331917: Frompre: 0 data0 2017-06-16T11:29:00.000000 open 164.340000 high 164.350000 low 164.340000 close 164.340000 vol 338
    <realtimeBar reqId=16777217, time=1497612540, open=164.35, high=164.35, low=164.35, close=164.35, volume=2, wap=164.35, count=2>
    <realtimeBar reqId=16777217, time=1497612545, open=164.35, high=164.35, low=164.35, close=164.35, volume=352, wap=164.35, count=25>
    <realtimeBar reqId=16777217, time=1497612550, open=164.35, high=164.35, low=164.35, close=164.35, volume=39, wap=164.35, count=6>
    <realtimeBar reqId=16777217, time=1497612555, open=164.35, high=164.35, low=164.35, close=164.35, volume=0, wap=164.35, count=0>
    <realtimeBar reqId=16777217, time=1497612560, open=164.35, high=164.35, low=164.35, close=164.35, volume=10, wap=164.35, count=5>
    <realtimeBar reqId=16777217, time=1497612565, open=164.35, high=164.35, low=164.35, close=164.35, volume=36, wap=164.35, count=4>
    <realtimeBar reqId=16777217, time=1497612570, open=164.35, high=164.35, low=164.35, close=164.35, volume=143, wap=164.35, count=12>
    <realtimeBar reqId=16777217, time=1497612575, open=164.35, high=164.35, low=164.35, close=164.35, volume=0, wap=164.35, count=0>
    <realtimeBar reqId=16777217, time=1497612580, open=164.35, high=164.35, low=164.35, close=164.35, volume=0, wap=164.35, count=0>
    <currentTime time=1497612586>
    <realtimeBar reqId=16777217, time=1497612585, open=164.35, high=164.35, low=164.35, close=164.35, volume=3, wap=164.35, count=1>
    <realtimeBar reqId=16777217, time=1497612590, open=164.34, high=164.34, low=164.34, close=164.34, volume=240, wap=164.34, count=6>
    <realtimeBar reqId=16777217, time=1497612595, open=164.34, high=164.34, low=164.34, close=164.34, volume=0, wap=164.34, count=0>
    2017-06-16T13:30:02.806358: Frompre: 0 data0 2017-06-16T11:30:00.000000 open 164.350000 high 164.350000 low 164.340000 close 164.340000 vol 825
    <realtimeBar reqId=16777217, time=1497612600, open=164.35, high=164.35, low=164.34, close=164.34, volume=15, wap=164.34, count=2>
    <realtimeBar reqId=16777217, time=1497612605, open=164.35, high=164.35, low=164.35, close=164.35, volume=182, wap=164.35, count=32>
    <realtimeBar reqId=16777217, time=1497612610, open=164.35, high=164.35, low=164.35, close=164.35, volume=203, wap=164.35, count=17>
    <realtimeBar reqId=16777217, time=1497612615, open=164.36, high=164.36, low=164.36, close=164.36, volume=8, wap=164.36, count=4>
    <realtimeBar reqId=16777217, time=1497612620, open=164.36, high=164.37, low=164.36, close=164.37, volume=372, wap=164.36, count=23>
    <realtimeBar reqId=16777217, time=1497612625, open=164.36, high=164.37, low=164.36, close=164.37, volume=347, wap=164.37, count=27>
    <realtimeBar reqId=16777217, time=1497612630, open=164.37, high=164.37, low=164.37, close=164.37, volume=27, wap=164.37, count=6>
    <realtimeBar reqId=16777217, time=1497612635, open=164.37, high=164.37, low=164.36, close=164.37, volume=88, wap=164.36, count=4>
    <realtimeBar reqId=16777217, time=1497612640, open=164.36, high=164.36, low=164.36, close=164.36, volume=7, wap=164.36, count=1>
    <currentTime time=1497612646>
    <realtimeBar reqId=16777217, time=1497612645, open=164.36, high=164.36, low=164.36, close=164.36, volume=20, wap=164.36, count=1>
    <realtimeBar reqId=16777217, time=1497612650, open=164.37, high=164.37, low=164.37, close=164.37, volume=2, wap=164.37, count=2>
    <realtimeBar reqId=16777217, time=1497612655, open=164.36, high=164.36, low=164.36, close=164.36, volume=14, wap=164.36, count=1>
    2017-06-16T13:31:03.093858: Frompre: 0 data0 2017-06-16T11:31:00.000000 open 164.350000 high 164.370000 low 164.340000 close 164.360000 vol 1285

    Second run: The previously resampled bars are now backfilled bars:

    2017-06-16T13:32:02.206769: Frompre: 0 data0 2017-06-16T11:26:00.000000 open 164.340000 high 164.340000 low 164.330000 close 164.340000 vol 15
    2017-06-16T13:32:02.208600: Frompre: 0 data0 2017-06-16T11:27:00.000000 open 164.340000 high 164.340000 low 164.330000 close 164.340000 vol 553
    2017-06-16T13:32:02.210083: Frompre: 0 data0 2017-06-16T11:28:00.000000 open 164.340000 high 164.350000 low 164.340000 close 164.340000 vol 338
    2017-06-16T13:32:02.211503: Frompre: 0 data0 2017-06-16T11:29:00.000000 open 164.350000 high 164.350000 low 164.340000 close 164.340000 vol 825
    2017-06-16T13:32:02.212988: Frompre: 0 data0 2017-06-16T11:30:00.000000 open 164.350000 high 164.370000 low 164.340000 close 164.360000 vol 1285
    2017-06-16T13:32:02.219781: Frompre: 0 data0 2017-06-16T11:31:00.000000 open 164.360000 high 164.370000 low 164.360000 close 164.370000 vol 285

    Now in my opinion data seems to be correct.

    Thanks for your time


  • administrators

    @borodiliz said in IB Live Data Feed: differences between resampled data and backfilled data:

    My conclusion: First rtbar is not taken into account

    May of course be. Will be looked into. Thank you for the analysis.


Log in to reply
 

Looks like your connection to Backtrader Community was lost, please wait while we try to reconnect.