Navigation

    Backtrader Community
    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

    IB Live Data Feed: differences between resampled data and backfilled data

    General Code/Help
    2
    4
    1720
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • B
      borodiliz last edited by borodiliz

      Hi all,

      I'm doing some tests with backtrader + IB + live data feeds and I'm getting some differences between resampled data and backfilled data.

      Initially my goal is simple: A log with exactly the same bars/data than the TWS chart shows.

      Given the following code:

      class Test(bt.Strategy):

    ######################
    #
    ######################
    def notify_data(self, data, status, * args, ** kwargs):
        print('*' * 5, 'DATA NOTIF:', data._getstatusname(status), * args)


    ######################
    #
    ######################
    def notify_store(self, msg, * args, ** kwargs):
        print('*' * 5, 'STORE NOTIF:', msg)

    ######################
    #
    ######################
    def __init__(self):
        ''''''
    ######################
    #
    ######################
    def prenext(self):
        ''''''
        #self.next(frompre=True)



    ######################
    #
    ######################
    def log(self, txt):
        now = datetime.datetime.now()
        print('%s:  %s' % (
              now.strftime(dtfmt), txt))

    def next (self, frompre=False):

        self.log("Frompre: %d data0 %s open %f high %f low %f close %f vol %d" %
                 (frompre,
                 self.data0.datetime.datetime(0).strftime(dtfmt),
                 self.data0.open[0],
                 self.data0.high[0],
                 self.data0.low[0],
                 self.data0.close[0],
                 self.data0.volume[0]))

def parse_args():
    parser = argparse.ArgumentParser(formatter_class=argparse.ArgumentDefaultsHelpFormatter)

    parser.add_argument('--notifyall',
                        required=False, action='store_true',
                        help='Notify all messages to strategy as store notifs')

    parser.add_argument('--debug',
                        required=False, action='store_true',
                        help='Display all info received form IB')


    parser.add_argument('--host', default='127.0.0.1',
                        required=False, action='store',
                        help='Host for the Interactive Brokers TWS Connection')

    parser.add_argument('--port', default=4002, type=int,
                        required=False, action='store',
                        help='Port for the Interactive Brokers TWS Connection')


    parser.add_argument('--clientId', default=None, type=int,
                        required=False, action='store',
                        help='Client Id to connect to TWS (default: random)')

    parser.add_argument('--no-timeoffset',
                        required=False, action='store_true',
                        help=('Do not Use TWS/System time offset for non '
                        'timestamped prices and to align resampling'))

    parser.add_argument('--reconnect', default=3, type=int,
                        required=False, action='store',
                        help='Number of recconnection attempts to TWS')

    parser.add_argument('--timeout', default=3.0, type=float,
                        required=False, action='store',
                        help='Timeout between reconnection attempts to TWS')

    parser.add_argument('--data0', default=None,
                        required=False, action='store',
                        help='data 0 into the system')

    return parser.parse_args()

if __name__ == '__main__':

    dtfmt = '%Y-%m-%dT%H:%M:%S.%f'
    now = datetime.datetime.now();
    print ("Now is %s" % (now.strftime(dtfmt)))


    args = parse_args()

    # Create a cerebro entity
    cerebro = bt.Cerebro(exactbars=True, runonce=False, preload=False)


    ibstore = bt.stores.IBStore(
                                host=args.host, port=args.port,
                                clientId=args.clientId, timeoffset=not args.no_timeoffset,
                                reconnect=args.reconnect, timeout=args.timeout,
                                notifyall=args.notifyall, _debug=args.debug)


    cerebro.setbroker(ibstore.getbroker())



    data0 = ibstore.getdata(dataname=args.data0,
                            backfill_start=True,
                            timeframe=bt.TimeFrame.Minutes,
                            compression=1,
                            name='d0',
                            qcheck=20)

    # New resampler
    cerebro.resampledata(
                         data0,
                         timeframe=bt.TimeFrame.Minutes,
                         compression=1)

    # Add a strategy

    cerebro.addstrategy(Test)

    cerebro.run()

      First run: backfilled data + resampled data:
      resampled_vs_backfilled.py --port 4002 --clientId 15 --data0 GBL-201709-DTB

      2017-06-16T09:48:19.251639: Frompre: 0 data0 2017-06-16T07:44:00.000000 open 164.41 high 164.41 low 164.34 close 164.35 vol 2712
      2017-06-16T09:48:19.257446: Frompre: 0 data0 2017-06-16T07:45:00.000000 open 164.35 high 164.36 low 164.33 close 164.35 vol 1530
      2017-06-16T09:48:19.259503: Frompre: 0 data0 2017-06-16T07:46:00.000000 open 164.34 high 164.35 low 164.32 close 164.34 vol 1330
      2017-06-16T09:48:19.261535: Frompre: 0 data0 2017-06-16T07:47:00.000000 open 164.34 high 164.35 low 164.34 close 164.35 vol 409
      2017-06-16T09:48:19.263397: Frompre: 0 data0 2017-06-16T07:48:00.000000 open 164.34 high 164.35 low 164.34 close 164.35 vol 165
      ***** DATA NOTIF: LIVE
      2017-06-16T09:49:02.211599: Frompre: 0 data0 2017-06-16T07:49:00.000000 open 164.35 high 164.39 low 164.35 close 164.38 vol 798
      2017-06-16T09:50:03.389798: Frompre: 0 data0 2017-06-16T07:50:00.000000 open 164.38 high 164.38 low 164.36 close 164.37 vol 538
      2017-06-16T09:51:02.751163: Frompre: 0 data0 2017-06-16T07:51:00.000000 open 164.37 high 164.38 low 164.36 close 164.37 vol 780
      2017-06-16T09:52:04.579715: Frompre: 0 data0 2017-06-16T07:52:00.000000 open 164.36 high 164.38 low 164.36 close 164.37 vol 747
      2017-06-16T09:53:02.244969: Frompre: 0 data0 2017-06-16T07:53:00.000000 open 164.37 high 164.37 low 164.36 close 164.36 vol 246
      2017-06-16T09:54:00.565664: Frompre: 0 data0 2017-06-16T07:54:00.000000 open 164.36 high 164.37 low 164.34 close 164.36 vol 896
      2017-06-16T09:55:11.430551: Frompre: 0 data0 2017-06-16T07:55:00.000000 open 164.35 high 164.36 low 164.34 close 164.34 vol 414
      2017-06-16T09:56:04.272170: Frompre: 0 data0 2017-06-16T07:56:00.000000 open 164.34 high 164.34 low 164.31 close 164.32 vol 2116
      2017-06-16T09:57:01.123514: Frompre: 0 data0 2017-06-16T07:57:00.000000 open 164.33 high 164.35 low 164.33 close 164.35 vol 931

      Second run: The previously resampled bars are now backfilled bars:

      2017-06-16T09:57:59.194470: Frompre: 0 data0 2017-06-16T07:44:00.000000 open 164.41 high 164.41 low 164.34 close 164.35 vol 2712
      2017-06-16T09:57:59.196351: Frompre: 0 data0 2017-06-16T07:45:00.000000 open 164.35 high 164.36 low 164.33 close 164.35 vol 1530
      2017-06-16T09:57:59.201949: Frompre: 0 data0 2017-06-16T07:46:00.000000 open 164.34 high 164.35 low 164.32 close 164.34 vol 1330
      2017-06-16T09:57:59.204162: Frompre: 0 data0 2017-06-16T07:47:00.000000 open 164.34 high 164.35 low 164.34 close 164.35 vol 409
      2017-06-16T09:57:59.206197: Frompre: 0 data0 2017-06-16T07:48:00.000000 open 164.34 high 164.39 low 164.34 close 164.38 vol 961
      2017-06-16T09:57:59.207977: Frompre: 0 data0 2017-06-16T07:49:00.000000 open 164.38 high 164.38 low 164.36 close 164.37 vol 540
      2017-06-16T09:57:59.210999: Frompre: 0 data0 2017-06-16T07:50:00.000000 open 164.37 high 164.38 low 164.36 close 164.37 vol 780
      2017-06-16T09:57:59.215024: Frompre: 0 data0 2017-06-16T07:51:00.000000 open 164.37 high 164.38 low 164.35 close 164.37 vol 747
      2017-06-16T09:57:59.216797: Frompre: 0 data0 2017-06-16T07:52:00.000000 open 164.37 high 164.37 low 164.36 close 164.36 vol 246
      2017-06-16T09:57:59.218651: Frompre: 0 data0 2017-06-16T07:53:00.000000 open 164.36 high 164.37 low 164.34 close 164.36 vol 890
      2017-06-16T09:57:59.220877: Frompre: 0 data0 2017-06-16T07:54:00.000000 open 164.36 high 164.36 low 164.34 close 164.34 vol 415
      2017-06-16T09:57:59.223913: Frompre: 0 data0 2017-06-16T07:55:00.000000 open 164.34 high 164.34 low 164.31 close 164.33 vol 2114
      2017-06-16T09:57:59.229085: Frompre: 0 data0 2017-06-16T07:56:00.000000 open 164.33 high 164.35 low 164.32 close 164.35 vol 934
      2017-06-16T09:57:59.230990: Frompre: 0 data0 2017-06-16T07:57:00.000000 open 164.35 high 164.35 low 164.33 close 164.35 vol 427

      Problems detected:

      1.- Some bars are different. e.g: the bar with 211X volume

      • resampled data -> close 164.32 vol 2116
      • backfilled data -> close 164.33 vol 2114

      2.- Different timestamps, e.g: the bar with volume 246:

      • resampled data time-> 07:53
      • backfilled data time -> 07:52

      I've tried increasing the qcheck up-to 20 without success

      Thanks in advance.

      1 Reply Last reply Reply Quote 0
      • B
        backtrader administrators last edited by

        @borodiliz said in IB Live Data Feed: differences between resampled data and backfilled data:

        I've tried increasing the qcheck up-to 20 without success

        You should decrease the qcheck parameter for more checks per second, which it's probably your intention (qcheck=0.5, the default, forces the system to wake-up every 0.5 seconds to do resampling clean-up for example, which in the absence of negotiation will allow the resampler to spit out bars by checking the real-time clock)

        Without claiming that backtrader is perfect and its resampling is perfect and ... and ... the expectation that the backfilled data and the real-time resampled data will be equal is too much to expect. Add to the equation that Interactive Brokers dismisses itself as a data provider and has disclaimers about the data.

        Reasons:

        • Ticks may be missed
        • Ticks may come in too late and be discarded (there is a parameter to control it and add them to the next resampled bar)
        • The resampler has to deliver in real-time using the real-time clock and cannot wait for ticks to show up if they are delayed. At 2017-06-16T07:57:00 it has to give you a bar, but it may not have all information from Interactive Brokers
        • The 4 times a second snapshots are not guaranteed to contain exact information.

        Some of this problems can be alleviated by switching to Real-Time Bars for which Interactive Brokers guarantees consolidated information (check the rtbar parameter in Docs - Interactive Brokers. But this is not available for all products.

        You can also face the following problem:

        • Backfill today and get bars with information X
        • Backfill tomorrow and get bars with information Y (slight changes to X), because some information has been consolidated.

        @borodiliz said in IB Live Data Feed: differences between resampled data and backfilled data:

        2.- Different timestamps, e.g: the bar with volume 246:

        resampled data time-> 07:53
        backfilled data time -> 07:52

        This is because IB is moving the data pointer to the beginning of the minute rather than to the end. It's just a convention.

        1 Reply Last reply Reply Quote 0
        • B
          borodiliz last edited by

          Thanks for your quick reply @backtrader

          You can't claim backtrader is perfect, but I can claim backtrader is ... awesome!

          I can understand the complexity using tickString, so let's go with RealTimeBars data:

          Because I do not want bars as be delivered as much in real-time as possible and I perfer to "wait" a while I've set the following (qcheck=2 , timeoffset=True and rtbar=True):

              ibstore = bt.stores.IBStore(
                                host=args.host, port=args.port,
                                clientId=args.clientId, timeoffset=True,
                                reconnect=args.reconnect, timeout=args.timeout,
                                notifyall=args.notifyall, _debug=args.debug)

    cerebro.setbroker(ibstore.getbroker())

    data0 = ibstore.getdata(dataname=args.data0,
                            backfill_start=True,
                            timeframe=bt.TimeFrame.Minutes,
                            compression=1,
                            name='d0',
                            qcheck=2,
                            rtbar=True)

          First run: check resampled data with debug mode enabled:

          ...........
          <realtimeBar reqId=16777217, time=1497608880, open=164.35, high=164.35, low=164.35, close=164.35, volume=0, wap=164.35, count=0>
          <realtimeBar reqId=16777217, time=1497608885, open=164.35, high=164.35, low=164.35, close=164.35, volume=4, wap=164.35, count=3>
          <realtimeBar reqId=16777217, time=1497608890, open=164.35, high=164.35, low=164.35, close=164.35, volume=0, wap=164.35, count=0>
          <realtimeBar reqId=16777217, time=1497608895, open=164.35, high=164.35, low=164.34, close=164.34, volume=9, wap=164.35, count=2>
          <realtimeBar reqId=16777217, time=1497608900, open=164.34, high=164.34, low=164.34, close=164.34, volume=2, wap=164.34, count=1>
          <realtimeBar reqId=16777217, time=1497608905, open=164.34, high=164.34, low=164.34, close=164.34, volume=205, wap=164.34, count=2>
          <currentTime time=1497608915>
          <realtimeBar reqId=16777217, time=1497608910, open=164.34, high=164.34, low=164.34, close=164.34, volume=133, wap=164.34, count=9>
          <realtimeBar reqId=16777217, time=1497608915, open=164.34, high=164.34, low=164.34, close=164.34, volume=0, wap=164.34, count=0>
          <realtimeBar reqId=16777217, time=1497608920, open=164.34, high=164.34, low=164.34, close=164.34, volume=1, wap=164.34, count=1>
          <realtimeBar reqId=16777217, time=1497608925, open=164.35, high=164.35, low=164.35, close=164.35, volume=1, wap=164.35, count=1>
          <realtimeBar reqId=16777217, time=1497608930, open=164.35, high=164.35, low=164.35, close=164.35, volume=79, wap=164.35, count=1>
          <realtimeBar reqId=16777217, time=1497608935, open=164.35, high=164.35, low=164.35, close=164.35, volume=0, wap=164.35, count=0>
          2017-06-16T12:29:03.091121: Frompre: 0 data0 2017-06-16T10:29:00.000000 open 164.350000 high 164.350000 low 164.340000 close 164.350000 vol 434
          <realtimeBar reqId=16777217, time=1497608940, open=164.35, high=164.35, low=164.35, close=164.35, volume=1, wap=164.35, count=1>
          <realtimeBar reqId=16777217, time=1497608945, open=164.35, high=164.35, low=164.35, close=164.35, volume=30, wap=164.35, count=2>
          <realtimeBar reqId=16777217, time=1497608950, open=164.35, high=164.35, low=164.35, close=164.35, volume=0, wap=164.35, count=0>
          <realtimeBar reqId=16777217, time=1497608955, open=164.34, high=164.34, low=164.34, close=164.34, volume=1, wap=164.34, count=1>
          <realtimeBar reqId=16777217, time=1497608960, open=164.35, high=164.35, low=164.34, close=164.34, volume=13, wap=164.34, count=4>
          <realtimeBar reqId=16777217, time=1497608965, open=164.34, high=164.34, low=164.34, close=164.34, volume=15, wap=164.34, count=1>
          <currentTime time=1497608975>
          <realtimeBar reqId=16777217, time=1497608970, open=164.34, high=164.34, low=164.34, close=164.34, volume=13, wap=164.34, count=1>
          <realtimeBar reqId=16777217, time=1497608975, open=164.35, high=164.35, low=164.35, close=164.35, volume=1, wap=164.35, count=1>
          <realtimeBar reqId=16777217, time=1497608980, open=164.35, high=164.35, low=164.35, close=164.35, volume=13, wap=164.35, count=2>
          <realtimeBar reqId=16777217, time=1497608985, open=164.35, high=164.36, low=164.35, close=164.36, volume=48, wap=164.35, count=15>
          <realtimeBar reqId=16777217, time=1497608990, open=164.35, high=164.35, low=164.35, close=164.35, volume=129, wap=164.35, count=19>
          <realtimeBar reqId=16777217, time=1497608995, open=164.35, high=164.35, low=164.35, close=164.35, volume=26, wap=164.35, count=2>
          2017-06-16T12:30:03.100416: Frompre: 0 data0 2017-06-16T10:30:00.000000 open 164.350000 high 164.360000 low 164.340000 close 164.350000 vol 289
          <realtimeBar reqId=16777217, time=1497609000, open=164.34, high=164.34, low=164.34, close=164.34, volume=21, wap=164.34, count=1>
          <realtimeBar reqId=16777217, time=1497609005, open=164.34, high=164.34, low=164.34, close=164.34, volume=0, wap=164.34, count=0>
          <realtimeBar reqId=16777217, time=1497609010, open=164.34, high=164.34, low=164.34, close=164.34, volume=104, wap=164.34, count=9>
          <realtimeBar reqId=16777217, time=1497609015, open=164.34, high=164.34, low=164.34, close=164.34, volume=1, wap=164.34, count=1>
          <realtimeBar reqId=16777217, time=1497609020, open=164.34, high=164.34, low=164.34, close=164.34, volume=0, wap=164.34, count=0>
          <realtimeBar reqId=16777217, time=1497609025, open=164.33, high=164.33, low=164.33, close=164.33, volume=1, wap=164.33, count=1>
          <currentTime time=1497609035>
          <realtimeBar reqId=16777217, time=1497609030, open=164.34, high=164.34, low=164.34, close=164.34, volume=1, wap=164.34, count=1>
          <realtimeBar reqId=16777217, time=1497609035, open=164.34, high=164.34, low=164.34, close=164.34, volume=0, wap=164.34, count=0>
          <realtimeBar reqId=16777217, time=1497609040, open=164.34, high=164.34, low=164.34, close=164.34, volume=0, wap=164.34, count=0>
          <realtimeBar reqId=16777217, time=1497609045, open=164.33, high=164.33, low=164.33, close=164.33, volume=286, wap=164.33, count=24>
          <realtimeBar reqId=16777217, time=1497609050, open=164.33, high=164.33, low=164.33, close=164.33, volume=51, wap=164.33, count=6>
          <realtimeBar reqId=16777217, time=1497609055, open=164.33, high=164.34, low=164.33, close=164.34, volume=243, wap=164.34, count=5>
          2017-06-16T12:31:03.092756: Frompre: 0 data0 2017-06-16T10:31:00.000000 open 164.340000 high 164.340000 low 164.330000 close 164.340000 vol 687
          <realtimeBar reqId=16777217, time=1497609060, open=164.33, high=164.33, low=164.33, close=164.33, volume=160, wap=164.33, count=2>
          <realtimeBar reqId=16777217, time=1497609065, open=164.33, high=164.33, low=164.33, close=164.33, volume=0, wap=164.33, count=0>
          <realtimeBar reqId=16777217, time=1497609070, open=164.33, high=164.33, low=164.33, close=164.33, volume=86, wap=164.33, count=6>
          <realtimeBar reqId=16777217, time=1497609075, open=164.33, high=164.33, low=164.33, close=164.33, volume=0, wap=164.33, count=0>
          <realtimeBar reqId=16777217, time=1497609080, open=164.34, high=164.34, low=164.34, close=164.34, volume=33, wap=164.34, count=1>
          <realtimeBar reqId=16777217, time=1497609085, open=164.33, high=164.33, low=164.33, close=164.33, volume=48, wap=164.33, count=6>
          <currentTime time=1497609095>
          <realtimeBar reqId=16777217, time=1497609090, open=164.33, high=164.33, low=164.33, close=164.33, volume=24, wap=164.33, count=5>
          <realtimeBar reqId=16777217, time=1497609095, open=164.33, high=164.33, low=164.33, close=164.33, volume=43, wap=164.33, count=6>
          <realtimeBar reqId=16777217, time=1497609100, open=164.33, high=164.33, low=164.33, close=164.33, volume=0, wap=164.33, count=0>
          <realtimeBar reqId=16777217, time=1497609105, open=164.33, high=164.33, low=164.33, close=164.33, volume=0, wap=164.33, count=0>
          <realtimeBar reqId=16777217, time=1497609110, open=164.33, high=164.33, low=164.33, close=164.33, volume=0, wap=164.33, count=0>
          <realtimeBar reqId=16777217, time=1497609115, open=164.33, high=164.33, low=164.33, close=164.33, volume=54, wap=164.33, count=3>
          2017-06-16T12:32:03.096349: Frompre: 0 data0 2017-06-16T10:32:00.000000 open 164.330000 high 164.340000 low 164.330000 close 164.330000 vol 288

          Second run: The previously resampled bars are now backfilled bars:

          2017-06-16T12:33:44.813167: Frompre: 0 data0 2017-06-16T10:28:00.000000 open 164.350000 high 164.350000 low 164.340000 close 164.350000 vol 434
          2017-06-16T12:33:44.814699: Frompre: 0 data0 2017-06-16T10:29:00.000000 open 164.350000 high 164.360000 low 164.340000 close 164.350000 vol 290
          2017-06-16T12:33:44.816210: Frompre: 0 data0 2017-06-16T10:30:00.000000 open 164.340000 high 164.340000 low 164.330000 close 164.340000 vol 708
          2017-06-16T12:33:44.817950: Frompre: 0 data0 2017-06-16T10:31:00.000000 open 164.330000 high 164.340000 low 164.330000 close 164.330000 vol 448

          My conclusion: First rtbar is not taken into account

          After checking ibdata.py source code and debug data I think the problem arises when datetime for a rtbar is equal to the just emitted resampled bar. I seems those minor changes solves the problem.

          Test again:

          2017-06-16T13:27:02.523913: Frompre: 0 data0 2017-06-16T11:27:00.000000 open 164.340000 high 164.340000 low 164.330000 close 164.340000 vol 15
          <realtimeBar reqId=16777217, time=1497612420, open=164.34, high=164.34, low=164.34, close=164.34, volume=2, wap=164.34, count=2>
          <realtimeBar reqId=16777217, time=1497612425, open=164.34, high=164.34, low=164.34, close=164.34, volume=0, wap=164.34, count=0>
          <realtimeBar reqId=16777217, time=1497612430, open=164.34, high=164.34, low=164.34, close=164.34, volume=0, wap=164.34, count=0>
          <realtimeBar reqId=16777217, time=1497612435, open=164.34, high=164.34, low=164.34, close=164.34, volume=1, wap=164.34, count=1>
          <realtimeBar reqId=16777217, time=1497612440, open=164.34, high=164.34, low=164.34, close=164.34, volume=0, wap=164.34, count=0>
          <realtimeBar reqId=16777217, time=1497612445, open=164.34, high=164.34, low=164.34, close=164.34, volume=18, wap=164.34, count=2>
          <realtimeBar reqId=16777217, time=1497612450, open=164.33, high=164.34, low=164.33, close=164.34, volume=11, wap=164.33, count=2>
          <realtimeBar reqId=16777217, time=1497612455, open=164.34, high=164.34, low=164.34, close=164.34, volume=0, wap=164.34, count=0>
          <realtimeBar reqId=16777217, time=1497612460, open=164.34, high=164.34, low=164.34, close=164.34, volume=0, wap=164.34, count=0>
          <currentTime time=1497612466>
          <realtimeBar reqId=16777217, time=1497612465, open=164.34, high=164.34, low=164.34, close=164.34, volume=349, wap=164.34, count=42>
          <realtimeBar reqId=16777217, time=1497612470, open=164.34, high=164.34, low=164.34, close=164.34, volume=152, wap=164.34, count=17>
          <realtimeBar reqId=16777217, time=1497612475, open=164.34, high=164.34, low=164.34, close=164.34, volume=20, wap=164.34, count=5>
          2017-06-16T13:28:02.429118: Frompre: 0 data0 2017-06-16T11:28:00.000000 open 164.340000 high 164.340000 low 164.330000 close 164.340000 vol 553
          <realtimeBar reqId=16777217, time=1497612480, open=164.34, high=164.34, low=164.34, close=164.34, volume=61, wap=164.34, count=10>
          <realtimeBar reqId=16777217, time=1497612485, open=164.34, high=164.34, low=164.34, close=164.34, volume=3, wap=164.34, count=2>
          <realtimeBar reqId=16777217, time=1497612490, open=164.34, high=164.34, low=164.34, close=164.34, volume=0, wap=164.34, count=0>
          <realtimeBar reqId=16777217, time=1497612495, open=164.34, high=164.34, low=164.34, close=164.34, volume=1, wap=164.34, count=1>
          <realtimeBar reqId=16777217, time=1497612500, open=164.34, high=164.34, low=164.34, close=164.34, volume=2, wap=164.34, count=2>
          <realtimeBar reqId=16777217, time=1497612505, open=164.34, high=164.34, low=164.34, close=164.34, volume=4, wap=164.34, count=4>
          <realtimeBar reqId=16777217, time=1497612510, open=164.34, high=164.34, low=164.34, close=164.34, volume=3, wap=164.34, count=3>
          <realtimeBar reqId=16777217, time=1497612515, open=164.34, high=164.34, low=164.34, close=164.34, volume=2, wap=164.34, count=2>
          <realtimeBar reqId=16777217, time=1497612520, open=164.34, high=164.34, low=164.34, close=164.34, volume=3, wap=164.34, count=3>
          <currentTime time=1497612526>
          <realtimeBar reqId=16777217, time=1497612525, open=164.34, high=164.34, low=164.34, close=164.34, volume=3, wap=164.34, count=2>
          <realtimeBar reqId=16777217, time=1497612530, open=164.34, high=164.35, low=164.34, close=164.34, volume=256, wap=164.34, count=10>
          <realtimeBar reqId=16777217, time=1497612535, open=164.34, high=164.34, low=164.34, close=164.34, volume=0, wap=164.34, count=0>
          2017-06-16T13:29:02.331917: Frompre: 0 data0 2017-06-16T11:29:00.000000 open 164.340000 high 164.350000 low 164.340000 close 164.340000 vol 338
          <realtimeBar reqId=16777217, time=1497612540, open=164.35, high=164.35, low=164.35, close=164.35, volume=2, wap=164.35, count=2>
          <realtimeBar reqId=16777217, time=1497612545, open=164.35, high=164.35, low=164.35, close=164.35, volume=352, wap=164.35, count=25>
          <realtimeBar reqId=16777217, time=1497612550, open=164.35, high=164.35, low=164.35, close=164.35, volume=39, wap=164.35, count=6>
          <realtimeBar reqId=16777217, time=1497612555, open=164.35, high=164.35, low=164.35, close=164.35, volume=0, wap=164.35, count=0>
          <realtimeBar reqId=16777217, time=1497612560, open=164.35, high=164.35, low=164.35, close=164.35, volume=10, wap=164.35, count=5>
          <realtimeBar reqId=16777217, time=1497612565, open=164.35, high=164.35, low=164.35, close=164.35, volume=36, wap=164.35, count=4>
          <realtimeBar reqId=16777217, time=1497612570, open=164.35, high=164.35, low=164.35, close=164.35, volume=143, wap=164.35, count=12>
          <realtimeBar reqId=16777217, time=1497612575, open=164.35, high=164.35, low=164.35, close=164.35, volume=0, wap=164.35, count=0>
          <realtimeBar reqId=16777217, time=1497612580, open=164.35, high=164.35, low=164.35, close=164.35, volume=0, wap=164.35, count=0>
          <currentTime time=1497612586>
          <realtimeBar reqId=16777217, time=1497612585, open=164.35, high=164.35, low=164.35, close=164.35, volume=3, wap=164.35, count=1>
          <realtimeBar reqId=16777217, time=1497612590, open=164.34, high=164.34, low=164.34, close=164.34, volume=240, wap=164.34, count=6>
          <realtimeBar reqId=16777217, time=1497612595, open=164.34, high=164.34, low=164.34, close=164.34, volume=0, wap=164.34, count=0>
          2017-06-16T13:30:02.806358: Frompre: 0 data0 2017-06-16T11:30:00.000000 open 164.350000 high 164.350000 low 164.340000 close 164.340000 vol 825
          <realtimeBar reqId=16777217, time=1497612600, open=164.35, high=164.35, low=164.34, close=164.34, volume=15, wap=164.34, count=2>
          <realtimeBar reqId=16777217, time=1497612605, open=164.35, high=164.35, low=164.35, close=164.35, volume=182, wap=164.35, count=32>
          <realtimeBar reqId=16777217, time=1497612610, open=164.35, high=164.35, low=164.35, close=164.35, volume=203, wap=164.35, count=17>
          <realtimeBar reqId=16777217, time=1497612615, open=164.36, high=164.36, low=164.36, close=164.36, volume=8, wap=164.36, count=4>
          <realtimeBar reqId=16777217, time=1497612620, open=164.36, high=164.37, low=164.36, close=164.37, volume=372, wap=164.36, count=23>
          <realtimeBar reqId=16777217, time=1497612625, open=164.36, high=164.37, low=164.36, close=164.37, volume=347, wap=164.37, count=27>
          <realtimeBar reqId=16777217, time=1497612630, open=164.37, high=164.37, low=164.37, close=164.37, volume=27, wap=164.37, count=6>
          <realtimeBar reqId=16777217, time=1497612635, open=164.37, high=164.37, low=164.36, close=164.37, volume=88, wap=164.36, count=4>
          <realtimeBar reqId=16777217, time=1497612640, open=164.36, high=164.36, low=164.36, close=164.36, volume=7, wap=164.36, count=1>
          <currentTime time=1497612646>
          <realtimeBar reqId=16777217, time=1497612645, open=164.36, high=164.36, low=164.36, close=164.36, volume=20, wap=164.36, count=1>
          <realtimeBar reqId=16777217, time=1497612650, open=164.37, high=164.37, low=164.37, close=164.37, volume=2, wap=164.37, count=2>
          <realtimeBar reqId=16777217, time=1497612655, open=164.36, high=164.36, low=164.36, close=164.36, volume=14, wap=164.36, count=1>
          2017-06-16T13:31:03.093858: Frompre: 0 data0 2017-06-16T11:31:00.000000 open 164.350000 high 164.370000 low 164.340000 close 164.360000 vol 1285

          Second run: The previously resampled bars are now backfilled bars:

          2017-06-16T13:32:02.206769: Frompre: 0 data0 2017-06-16T11:26:00.000000 open 164.340000 high 164.340000 low 164.330000 close 164.340000 vol 15
          2017-06-16T13:32:02.208600: Frompre: 0 data0 2017-06-16T11:27:00.000000 open 164.340000 high 164.340000 low 164.330000 close 164.340000 vol 553
          2017-06-16T13:32:02.210083: Frompre: 0 data0 2017-06-16T11:28:00.000000 open 164.340000 high 164.350000 low 164.340000 close 164.340000 vol 338
          2017-06-16T13:32:02.211503: Frompre: 0 data0 2017-06-16T11:29:00.000000 open 164.350000 high 164.350000 low 164.340000 close 164.340000 vol 825
          2017-06-16T13:32:02.212988: Frompre: 0 data0 2017-06-16T11:30:00.000000 open 164.350000 high 164.370000 low 164.340000 close 164.360000 vol 1285
          2017-06-16T13:32:02.219781: Frompre: 0 data0 2017-06-16T11:31:00.000000 open 164.360000 high 164.370000 low 164.360000 close 164.370000 vol 285

          Now in my opinion data seems to be correct.

          Thanks for your time

          B 1 Reply Last reply Reply Quote 0
          • B
            backtrader administrators @borodiliz last edited by

            @borodiliz said in IB Live Data Feed: differences between resampled data and backfilled data:

            My conclusion: First rtbar is not taken into account

            May of course be. Will be looked into. Thank you for the analysis.

            1 Reply Last reply Reply Quote 1
            • 1 / 1
            • First post
              Last post
            Copyright © 2016, 2017, 2018 NodeBB Forums | Contributors
            $(document).ready(function () { app.coldLoad(); }); }