Navigation

    Backtrader Community
    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

    IB Tick deliver latency

    General Code/Help
    2
    13
    1454
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • bb2
      bb2 last edited by

      I have been running the code in the link below to see if there is any remarkable difference between system clock and IB ticker data's datetime. Indeed there is a remarkable difference and I am trying to figure out what might be causing such a delay in IB ticker's delivery to the system.

      Code:
      https://gist.github.com/eduedix/7c985bf2d0c71f159b521733f6dc503b

      Logs:

      https://gist.github.com/eduedix/009246c3a8a91a1619a5adac4e5a967b

      Best

      1 Reply Last reply Reply Quote 1
      • B
        backtrader administrators last edited by

        Would you care to elaborate what the problem actually is?

        1 Reply Last reply Reply Quote 0
        • bb2
          bb2 last edited by

          Sure, the datetime of IB data ticks are not in sync with system clock.

          If the system is way behind the data delivery time, then it will be late to make new orders.

          I thought self.datas[i].datetime.datetime() would refer to the datetime of the delivery for a given data ( in this case tick data added via adddata ). But as seen in the logs, this is not the case, there is always a few seconds of diff between system and data datetimes. I will also paste the logs here with more instruments for you to realize the difference in system and bt's data clocks.

          log with more instruments here:
          https://gist.github.com/eduedix/b3b42f2e6f3c0d6654d48c3b05694448

          1 Reply Last reply Reply Quote 0
          • B
            backtrader administrators last edited by backtrader

            As long as you keep on posting logs and don't actually let us know where the problem is ...

            bb2 1 Reply Last reply Reply Quote 0
            • bb2
              bb2 @backtrader last edited by bb2

              Let's check this bit from the log above:

              2018-08-28 12:13:34.922744
AUD.CAD-CASH-IDEALPRO_tick 2018-08-28 12:13:09.201050 0.95024 ***
AUD.NZD-CASH-IDEALPRO_tick 2018-08-28 12:13:09.331496 1.09546 ***
AUD.USD-CASH-IDEALPRO_tick 2018-08-28 12:13:08.527748 0.73544 ***
CAD.CHF-CASH-IDEALPRO_tick 2018-08-28 12:13:09.359146 0.7551 ---
CAD.JPY-CASH-IDEALPRO_tick 2018-08-28 12:13:08.669510 85.936 ***
CHF.JPY-CASH-IDEALPRO_tick 2018-08-28 12:13:08.735663 113.805 ***
EUR.AUD-CASH-IDEALPRO_tick 2018-08-28 12:13:08.291379 1.59258 ***
EUR.CAD-CASH-IDEALPRO_tick 2018-08-28 12:13:09.347569 1.51333 ***
EUR.CHF-CASH-IDEALPRO_tick 2018-08-28 12:13:08.311837 1.14277 ***
EUR.GBP-CASH-IDEALPRO_tick 2018-08-28 12:13:09.305364 0.90687 ***
EUR.JPY-CASH-IDEALPRO_tick 2018-08-28 12:13:08.375999 130.054 ***
EUR.USD-CASH-IDEALPRO_tick 2018-08-28 12:13:09.292429 1.17127 ***
GBP.AUD-CASH-IDEALPRO_tick 2018-08-28 12:13:09.155154 1.75601 ***
GBP.CAD-CASH-IDEALPRO_tick 2018-08-28 12:13:09.286686 1.66871 ***
GBP.CHF-CASH-IDEALPRO_tick 2018-08-28 12:13:08.647251 1.26006 ***
GBP.JPY-CASH-IDEALPRO_tick 2018-08-28 12:13:08.609080 143.406 ***
GBP.NZD-CASH-IDEALPRO_tick 2018-08-28 12:13:08.719570 1.92374 ***
GBP.USD-CASH-IDEALPRO_tick 2018-08-28 12:13:09.343485 1.29151 ***
NZD.CAD-CASH-IDEALPRO_tick 2018-08-28 12:13:09.085550 0.86737 ***
NZD.USD-CASH-IDEALPRO_tick 2018-08-28 12:13:08.648840 0.67132 ***
USD.CAD-CASH-IDEALPRO_tick 2018-08-28 12:13:09.348243 1.29209 ***
USD.CHF-CASH-IDEALPRO_tick 2018-08-28 12:13:09.272795 0.97567 ***
USD.JPY-CASH-IDEALPRO_tick 2018-08-28 12:13:07.832831 111.039 ***

              system clock: 2018-08-28 12:13:34.922744, latest delivered data's datetime: CAD.CHF-CASH-IDEALPRO_tick 2018-08-28 12:13:09.359146 0.7551 ---.

              The data arrived at 12:13:09, but before 12:13:34 the system will not be able to make an order which depends on the data from 12:13:09 . In other terms, the system is way too late to make a new order.

              1 Reply Last reply Reply Quote 1
              • B
                backtrader administrators last edited by

                utcnow uses your system clock. The timestamps from IB are from IB, which obviously uses the system clock of IB (no, no the system clock of the system on which the gateway runs, but from the server)

                bb2 1 Reply Last reply Reply Quote 0
                • bb2
                  bb2 @backtrader last edited by bb2

                  @backtrader in that case I would expect the difference to be between 2 to be almost constant throughout the runtime. This is definitely not the case.

                  1 Reply Last reply Reply Quote 0
                  • B
                    backtrader administrators last edited by

                    The differences in your log are sometimes smaller and sometimes larger.

                    But you seem to suggest that it always grow, which isn't the case. Some ticks seem to catch up.

                    bb2 1 Reply Last reply Reply Quote 0
                    • bb2
                      bb2 @backtrader last edited by

                      @backtrader indeed. the difference is not constant. It ranges roughly between 2 and 20 seconds. And I am trying to figure out why there is such a lag anyway.

                      I am sure this is not because of the internet connection, because I have been running the same test with ib_insync ( no framework upon) and am experiencing a lag between 0.1 and 0.6 seconds.

                      1 Reply Last reply Reply Quote 1
                      • bb2
                        bb2 last edited by

                        @backtrader So, is there anything else I can do to resolve this issue?

                        What mostly happens is, the time difference increases up to some level (20-21 seconds) and then just like a spike the system collects a big chunk of data, which leads to reduced time difference suddenly again. This is happening in a cycle.

                        Is this the expected behaviour when backtrader is used with IB? I was hoping to get at most 1 second of latency.

                        Best

                        1 Reply Last reply Reply Quote 0
                        • bb2
                          bb2 last edited by

                          @backtrader

                          I have done many tests to figure out what might be causing such delays. backtrader itself can handle multiple contracts well for backtesting, so the issue was not backtrader itself.

                          I have tried reqMktData on multiple contracts (20-30) simultaneously with ib_insync and IbPy2 and it turned out that IbPy2 was lagging way behind and causing the delays. (lots of more than 5 seconds delays for IbPy2 and way less amount of 1-2 seconds and less delays for ib_insync).

                          I will post here the test codes for you guys if you want to.

                          Best

                          1 Reply Last reply Reply Quote 0
                          • bb2
                            bb2 last edited by

                            In addition, when a network problem or similar occurs, with stalled IbPy2 and multiple contracts, it will not be able to notice the network problem right away like ib_insync.

                            1 Reply Last reply Reply Quote 0
                            • bb2
                              bb2 last edited by

                              contacted IB live chat and it was suggested to use ib_insync instead of ibapi or IBpy to avoid lags.

                              1 Reply Last reply Reply Quote 2
                              • 1 / 1
                              • First post
                                Last post
                              Copyright © 2016, 2017, 2018, 2019, 2020, 2021 NodeBB Forums | Contributors