3. ibtest.py and vix?
For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

ibtest.py and vix?

  • R

    hello,

    i tried to get VIX from ibtest - no success.. How to get VIX in realtime?

    python ./ibtest.py --data0 VIX-IND-CBOE --timeframe Seconds --port 4001
    Server Version: 76
    TWS Time at connection:20180307 16:05:05 EET

    Strategy Created

    Timezone from ContractDetails: CST
    Datetime, Open, High, Low, Close, Volume, OpenInterest, SMA
    ***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm connection is OK:usbond>
    ***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm connection is OK:eufarm>
    ***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm connection is OK:usfuture>
    ***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm connection is OK:usfuture.us>
    ***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm connection is OK:usfarm.us>
    ***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm connection is OK:usfarm>
    ***** STORE NOTIF: <error id=-1, errorCode=2106, errorMsg=HMDS data farm connection is OK:euhmds>
    ***** STORE NOTIF: <error id=-1, errorCode=2106, errorMsg=HMDS data farm connection is OK:hkhmds>
    ***** STORE NOTIF: <error id=-1, errorCode=2106, errorMsg=HMDS data farm connection is OK:cashhmds>
    ***** STORE NOTIF: <error id=-1, errorCode=2106, errorMsg=HMDS data farm connection is OK:fundfarm>
    ***** STORE NOTIF: <error id=-1, errorCode=2106, errorMsg=HMDS data farm connection is OK:ushmds>
    ***** DATA NOTIF: DELAYED
    ***** STORE NOTIF: <error id=-1, errorCode=2108, errorMsg=Market data farm connection is inactive but should be available upon demand.usbond>
    ***** STORE NOTIF: <error id=-1, errorCode=2108, errorMsg=Market data farm connection is inactive but should be available upon demand.usbond>

    0
  • B
    administrators

    Isn't that symbol for the continuous display in TWS? If yes, that's an artifact in TWS. You cannot get continuous futures over the API.

    0
  • R

    it's index, should be no problem to get it, qtpylib getting it from same 4001 port.
    looks like there some issue.

    0
  • B
    administrators

    The symbol may need to be more specific (getting the currency added for example). I don't know the complete specification for that symbol.

    0
    R
     1 Reply
  • B
    administrators

    Another possibility is that the symbol was not yet trading real-time ... and the sample will therefore wait until it does to find out how much it can back-fill.

    Add --historical

    0
  • R

    @backtrader
    ***** DATA NOTIF: DISCONNECTED
    ***** STORE NOTIF: <error id=-1, errorCode=2106, errorMsg=HMDS data farm connection is OK:ushmds.us>
    ***** STORE NOTIF: <error id=16777217, errorCode=162, errorMsg=Historical Market Data Service error message:HMDS query returned no data: VIX@CBOE Trades>

    there no trades - it's index.

    0
  • B
    administrators

    Most indices (there may be many I am unaware of) trade (in the sense that they are calculated) between A -> B and don't have real-time ticks between B -> A.

    If VIX-IND-CBOE trades 24 hours a day non-stop, the situation may be different.

    As stated above, the symbol specification may need extra data.

    0
    R
     1 Reply
  • R

    this info - from the terminal

    0_1520439506728_Screen Shot 2018-03-07 at 18.18.06.png

    similiar to sample with GBP.USD
    0_1520439614405_Screen Shot 2018-03-07 at 18.19.59.png

    0
  • R

    @backtrader, here working sample from ezibpy

    0_1520440114907_Screen Shot 2018-03-07 at 18.26.05.png

    when i put into ibtest.py VIX-IND - getting following error:

    ***** STORE NOTIF: <error id=16777216, errorCode=200, errorMsg=No security definition has been found for the request>
    ***** STORE NOTIF: Ambiguous contract: none/multiple answers received
    ***** DATA NOTIF: DISCONNECTED

    0
  • R

    @backtrader, can you please advice how to handle such index with in backtrader (i think to move into this platform).

    0
  • R

    @backtrader as I understand there some difference between handling market data and real-time data (that not traded, like indexes) - I open bug in github - the author closed it. how can this issue be resolved?

    0
    B
     1 Reply
  • B
    administrators

    @rami-ramich said in ibtest.py and vix?:

    as I understand there some difference between handling market data and real-time data (that not traded, like indexes)

    There seems to be misunderstanding from your side, there is no such difference. The only thing that was stated above was about opening and closing times. Even if an index isn't directly traded, it is only active during trading times.

    @rami-ramich said in ibtest.py and vix?:

    I open bug in github - the author closed it. how can this issue be resolved?

    Yes, closed. Opening an issue while discussing here is not very productive.

    You are confusing issue with something you are not able to achieve. The ibtest.py is a sample with complex behavior intended to test many different behaviours and situations. It is not a one stop shop: "it is going to do exactly what I had in mind".

    @rami-ramich said in ibtest.py and vix?:

    when i put into ibtest.py VIX-IND - getting following error:

    The Interactive Brokers API doesn't specify how applications gather the needed info for a ticker. If that notation works with some applications, it may not work with backtrader and may not work with many other applications. That notation is not related to how the information is passed to the IB API.

    See here for more info, since you are not resampling and the test is probably failing due to it:

    And this which shows how to do it with your own code and not rely on something which is possibly not intended for your use case such as ibtest.py

    0
    R
     1 Reply
  • R

    @backtrader Thank you for a quick response.
    I already tested with --resample option - no index data, when the market was open.
    i will try when market will open with this sample from medium.

    I very hope i will get this index at real time.

    Thank you.

    0
  • R

    deleted...

    0
  • R

    I played with ibtest.py from sample folder.
    historical data it's able to get, but real time - unable...
    it's receive index as debug output <tickPrice tickerId=16777217, field=4, price=15.78, canAutoExecute=0>

    Could you please point me to any sample that access to any index ticker?

    python ./ibtest.py --data0 VIX-IND-CBOE --resample --timeframe Seconds --compression 1 --port 4001 --debug

    Server Version: 76
    TWS Time at connection:20180313 01:26:16 EET
    <managedAccounts accountsList=U20zzzz,U21zzzz,>
    <nextValidId orderId=1>
    <error id=-1, errorCode=2104, errorMsg=Market data farm connection is OK:eufarm>
    <error id=-1, errorCode=2104, errorMsg=Market data farm connection is OK:usfuture>
    <error id=-1, errorCode=2104, errorMsg=Market data farm connection is OK:usfuture.us>
    <error id=-1, errorCode=2104, errorMsg=Market data farm connection is OK:cashfarm>
    <error id=-1, errorCode=2104, errorMsg=Market data farm connection is OK:usfarm.us>
    <error id=-1, errorCode=2104, errorMsg=Market data farm connection is OK:usfarm>
    <error id=-1, errorCode=2106, errorMsg=HMDS data farm connection is OK:ushmds.us>
    <error id=-1, errorCode=2106, errorMsg=HMDS data farm connection is OK:euhmds>
    <error id=-1, errorCode=2106, errorMsg=HMDS data farm connection is OK:cashhmds>
    <error id=-1, errorCode=2106, errorMsg=HMDS data farm connection is OK:fundfarm>
    <error id=-1, errorCode=2106, errorMsg=HMDS data farm connection is OK:ushmds>
    <currentTime time=1520897517>
    <contractDetails reqId=16777216, contractDetails=<ib.ext.ContractDetails.ContractDetails object at 0x1100eff60>>
    <contractDetailsEnd reqId=16777216>

    Strategy Created

    Timezone from ContractDetails: CST
    Datetime, Open, High, Low, Close, Volume, OpenInterest, SMA
    ***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm connection is OK:eufarm>
    ***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm connection is OK:usfuture>
    ***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm connection is OK:usfuture.us>
    ***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm connection is OK:cashfarm>
    ***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm connection is OK:usfarm.us>
    ***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm connection is OK:usfarm>
    ***** STORE NOTIF: <error id=-1, errorCode=2106, errorMsg=HMDS data farm connection is OK:ushmds.us>
    ***** STORE NOTIF: <error id=-1, errorCode=2106, errorMsg=HMDS data farm connection is OK:euhmds>
    ***** STORE NOTIF: <error id=-1, errorCode=2106, errorMsg=HMDS data farm connection is OK:cashhmds>
    ***** STORE NOTIF: <error id=-1, errorCode=2106, errorMsg=HMDS data farm connection is OK:fundfarm>
    ***** STORE NOTIF: <error id=-1, errorCode=2106, errorMsg=HMDS data farm connection is OK:ushmds>
    ***** DATA NOTIF: DELAYED
    <tickString tickerId=16777217, tickType=45, value=1520885690>
    <tickPrice tickerId=16777217, field=4, price=15.78, canAutoExecute=0>
    <tickSize tickerId=16777217, field=5, size=0>
    <tickPrice tickerId=16777217, field=6, price=16.35, canAutoExecute=0>
    <tickPrice tickerId=16777217, field=7, price=15.18, canAutoExecute=0>
    <tickPrice tickerId=16777217, field=9, price=14.64, canAutoExecute=0>
    <tickPrice tickerId=16777217, field=1, price=-1.0, canAutoExecute=1>
    <tickSize tickerId=16777217, field=0, size=0>
    <tickPrice tickerId=16777217, field=2, price=-1.0, canAutoExecute=1>
    <tickSize tickerId=16777217, field=3, size=0>

    python ./ibtest.py --data0 VIX-IND-CBOE --resample --timeframe Seconds --compression 1 --port 4001 --historical
    Server Version: 76
    TWS Time at connection:20180313 01:36:14 EET

    Strategy Created

    Timezone from ContractDetails: CST
    Datetime, Open, High, Low, Close, Volume, OpenInterest, SMA
    ***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm connection is OK:eufarm>
    ***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm connection is OK:usfuture>
    ***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm connection is OK:usfuture.us>
    ***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm connection is OK:cashfarm>
    ***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm connection is OK:usfarm>
    ***** STORE NOTIF: <error id=-1, errorCode=2106, errorMsg=HMDS data farm connection is OK:ushmds.us>
    ***** STORE NOTIF: <error id=-1, errorCode=2106, errorMsg=HMDS data farm connection is OK:euhmds>
    ***** STORE NOTIF: <error id=-1, errorCode=2106, errorMsg=HMDS data farm connection is OK:cashhmds>
    ***** STORE NOTIF: <error id=-1, errorCode=2106, errorMsg=HMDS data farm connection is OK:fundfarm>
    ***** STORE NOTIF: <error id=-1, errorCode=2106, errorMsg=HMDS data farm connection is OK:ushmds>
    ***** DATA NOTIF: DELAYED
    Data0, 0001, 736765.8229166666, 2018-03-12T14:45:00.000000, 15.4, 15.4, 15.4, 15.4, 0.0, 0, nan
    Data0, 0002, 736765.8229282408, 2018-03-12T14:45:01.000000, 15.4, 15.4, 15.4, 15.4, 0.0, 0, nan
    Data0, 0003, 736765.8229398148, 2018-03-12T14:45:02.000000, 15.4, 15.4, 15.4, 15.4, 0.0, 0, nan
    Data0, 0004, 736765.8229513889, 2018-03-12T14:45:03.000000, 15.4, 15.4, 15.4, 15.4, 0.0, 0, nan
    Data0, 0005, 736765.8229629629, 2018-03-12T14:45:04.000000, 15.4, 15.4, 15.4, 15.4, 0.0, 0, 15.4
    Data0, 0006, 736765.822974537, 2018-03-12T14:45:05.000000, 15.4, 15.4, 15.4, 15.4, 0.0, 0, 15.4
    Data0, 0007, 736765.8229861112, 2018-03-12T14:45:06.000000, 15.4, 15.4, 15.4, 15.4, 0.0, 0, 15.4
    Data0, 0008, 736765.8229976852, 2018-03-12T14:45:07.000000, 15.4, 15.4, 15.4, 15.4, 0.0, 0, 15.4
    Data0, 0009, 736765.8230092593, 2018-03-12T14:45:08.000000, 15.4, 15.4, 15.4, 15.4, 0.0, 0, 15.4
    Data0, 0010, 736765.8230208333, 2018-03-12T14:45:09.000000, 15.4, 15.4, 15.4, 15.4, 0.0, 0, 15.4
    Data0, 0011, 736765.8230324074, 2018-03-12T14:45:10.000000, 15.4, 15.4, 15.4, 15.4, 0.0, 0, 15.4
    Data0, 0012, 736765.8230439815, 2018-03-12T14:45:11.000000, 15.4, 15.4, 15.4, 15.4, 0.0, 0, 15.4
    Data0, 0013, 736765.8230555556, 2018-03-12T14:45:12.000000, 15.4, 15.4, 15.4, 15.4, 0.0, 0, 15.4
    Data0, 0014, 736765.8230671296, 2018-03-12T14:45:13.000000, 15.4, 15.4, 15.4, 15.4, 0.0, 0, 15.4
    Data0, 0015, 736765.8230787037, 2018-03-12T14:45:14.000000, 15.4, 15.4, 15.4, 15.4, 0.0, 0, 15.4
    Data0, 0016, 736765.8230902777, 2018-03-12T14:45:15.000000, 15.4, 15.4, 15.4, 15.4, 0.0, 0, 15.4
    Data0, 0017, 736765.8231018519, 2018-03-12T14:45:16.000000, 15.4, 15.4, 15.4, 15.4, 0.0, 0, 15.4
    .
    .
    .
    Data0, 1792, 736765.8436458333, 2018-03-12T15:14:51.000000, 15.78, 15.78, 15.78, 15.78, 0.0, 0, 15.785999999999998
    Data0, 1793, 736765.8436574074, 2018-03-12T15:14:52.000000, 15.78, 15.78, 15.78, 15.78, 0.0, 0, 15.784
    Data0, 1794, 736765.8436689815, 2018-03-12T15:14:53.000000, 15.78, 15.78, 15.78, 15.78, 0.0, 0, 15.782
    Data0, 1795, 736765.8436805556, 2018-03-12T15:14:54.000000, 15.78, 15.78, 15.78, 15.78, 0.0, 0, 15.779999999999998
    Data0, 1796, 736765.8436921296, 2018-03-12T15:14:55.000000, 15.78, 15.78, 15.78, 15.78, 0.0, 0, 15.779999999999998
    Data0, 1797, 736765.8437037037, 2018-03-12T15:14:56.000000, 15.78, 15.78, 15.78, 15.78, 0.0, 0, 15.779999999999998
    Data0, 1798, 736765.8437152777, 2018-03-12T15:14:57.000000, 15.78, 15.78, 15.78, 15.78, 0.0, 0, 15.779999999999998
    Data0, 1799, 736765.8437268519, 2018-03-12T15:14:58.000000, 15.78, 15.78, 15.78, 15.78, 0.0, 0, 15.779999999999998
    Data0, 1800, 736765.8437384259, 2018-03-12T15:14:59.000000, 15.78, 15.78, 15.78, 15.78, 0.0, 0, 15.779999999999998
    ***** DATA NOTIF: DISCONNECTED

    I reviewed code a bit, i am not good in python yet, this piece of code looks responsible for this
    <tickPrice tickerId=16777217, field=1, price=-1.0, canAutoExecute=1>

    def tickPrice(self, msg):
    '''Cash Markets have no notion of "last_price"/"last_size" and the
    tracking of the price is done (industry de-facto standard at least with
    the IB API) following the BID price
    A RTVolume which will only contain a price is put into the client's
    queue to have a consistent cross-market interface
    '''
    # Used for "CASH" markets
    # The price field has been seen to be missing in some instances even if
    # "field" is 1
    tickerId = msg.tickerId
    if self.iscash[tickerId]:
        if msg.field == 1:  # Bid Price
            try:
                if msg.price == -1.0:
                    # seems to indicate the stream is halted for example in
                    # between 23:00 - 23:15 CET for FOREX
                    return
    0
  • R

    @backtrader, could you please help me with the issue?

    0
  • B
    administrators

    That only says that if the price is -1.0 the message has to be ignored. That's also for CASH (or CFD) markets (for example: Forex pairs like USD/JPY)

    For an IND the subscription is for tickstring messages and not tickprice messages. And for real-time the code is waiting for a tickstring with price information and the message apparently ... doesn't arrive.

    0
    R
     1 Reply
  • R

    @backtrader thank you for quick answer.
    Could you please advice how to fix or workaround it?
    I need this IND for the algo.

    I coded this - could you please tell if this OK?

    def tickPrice(self, msg):
    '''Cash Markets have no notion of "last_price"/"last_size" and the
    tracking of the price is done (industry de-facto standard at least with
    the IB API) following the BID price

    A RTVolume which will only contain a price is put into the client's
    queue to have a consistent cross-market interface
    '''
    # Used for "CASH" markets
    # The price field has been seen to be missing in some instances even if
    # "field" is 1
    tickerId = msg.tickerId
    if self.iscash[tickerId]:
        if msg.field == 1 :  # Bid Price
            try:
                if msg.price == -1.0:
                    # seems to indicate the stream is halted for example in
                    # between 23:00 - 23:15 CET for FOREX
                    return
            except AttributeError:
                pass

            try:
                rtvol = RTVolume(price=msg.price, tmoffset=self.tmoffset)
                # print('rtvol with datetime:', rtvol.datetime)
            except ValueError:  # price not in message ...
                pass
            else:
                self.qs[tickerId].put(rtvol)
    elif msg.field == 4:
        try:
            if msg.price == -1.0:
                    # seems to indicate the stream is halted for example in
                    # between 23:00 - 23:15 CET for FOREX
                return
        except AttributeError:
            pass

        try:
            rtvol = RTVolume(price=msg.price, tmoffset=self.tmoffset)
                # print('rtvol with datetime:', rtvol.datetime)
        except ValueError:  # price not in message ...
             pass
        else:
            self.qs[tickerId].put(rtvol)

    python ./ibtest.py --data0 VIX-IND-CBOE --data1 QQQ-STK --resample --timeframe Minutes --port 4001 --no-backfill

    strange, it's different time.... on 2 items

    Data0, 0361, 736768.8034722222, 2018-03-15T14:17:00.000000, 16.6, 16.63, 16.6, 16.62, 0.0, 0, 16.610000000000003
    Data1, 0675, 736768.8034722222, 2018-03-15T15:17:00.000000, 171.48, 171.5, 171.44, 171.46, 399.0, 0, nan
    Data0, 0362, 736768.8041666667, 2018-03-15T14:18:00.000000, 16.65, 16.79, 16.65, 16.79, 0.0, 0, 16.656
    Data1, 0676, 736768.8041666667, 2018-03-15T15:18:00.000000, 171.45, 171.45, 171.34, 171.37, 703.0, 0, nan
    Data0, 0363, 736768.8048611111, 2018-03-15T14:19:00.000000, 16.75, 16.87, 16.75, 16.87, 0.0, 0, 16.696
    Data1, 0677, 736768.8048611111, 2018-03-15T15:19:00.000000, 171.37, 171.37, 171.26, 171.28, 451.0, 0, nan
    Data0, 0364, 736768.8055555555, 2018-03-15T14:20:00.000000, 16.84, 16.84, 16.8, 16.8, 0.0, 0, 16.736
    Data1, 0678, 736768.8055555555, 2018-03-15T15:20:00.000000, 171.27, 171.34, 171.26, 171.34, 162.0, 0, nan
    Data0, 0365, 736768.80625, 2018-03-15T14:21:00.000000, 16.76, 16.81, 16.71, 16.71, 0.0, 0, 16.758000000000003
    Data1, 0679, 736768.80625, 2018-03-15T15:21:00.000000, 171.36, 171.43, 171.35, 171.41, 410.0, 0, nan

    and get error... without --resample

    Data1, 0676, 736768.8041666667, 2018-03-15T15:18:00.000000, 171.37, 171.37, 171.26, 171.28, 428, 0, nan
    Traceback (most recent call last):
    File "./ibtest.py", line 558, in <module>
    runstrategy()
    File "./ibtest.py", line 336, in runstrategy
    cerebro.run(exactbars=args.exactbars)
    File "/Users/roman/anaconda3/envs/newenv/lib/python3.6/site-packages/backtrader/cerebro.py", line 1127, in run
    runstrat = self.runstrategies(iterstrat)
    File "/Users/roman/anaconda3/envs/newenv/lib/python3.6/site-packages/backtrader/cerebro.py", line 1295, in runstrategies
    self._runnext(runstrats)
    File "/Users/roman/anaconda3/envs/newenv/lib/python3.6/site-packages/backtrader/cerebro.py", line 1626, in _runnext
    strat._next()
    File "/Users/roman/anaconda3/envs/newenv/lib/python3.6/site-packages/backtrader/strategy.py", line 325, in _next
    super(Strategy, self)._next()
    File "/Users/roman/anaconda3/envs/newenv/lib/python3.6/site-packages/backtrader/lineiterator.py", line 258, in _next
    indicator._next()
    File "/Users/roman/anaconda3/envs/newenv/lib/python3.6/site-packages/backtrader/lineiterator.py", line 258, in _next
    indicator._next()
    File "/Users/roman/anaconda3/envs/newenv/lib/python3.6/site-packages/backtrader/lineiterator.py", line 275, in _next
    self.next()
    File "/Users/roman/anaconda3/envs/newenv/lib/python3.6/site-packages/backtrader/indicators/basicops.py", line 356, in next
    math.fsum(self.data.get(size=self.p.period)) / self.p.period
    File "/Users/roman/anaconda3/envs/newenv/lib/python3.6/site-packages/backtrader/lineseries.py", line 241, in get
    return self.lines[line].get(ago, size=size)
    File "/Users/roman/anaconda3/envs/newenv/lib/python3.6/site-packages/backtrader/linebuffer.py", line 182, in get
    return list(islice(self.array, start, end))
    ValueError: Indices for islice() must be None or an integer: 0 <= x <= sys.maxsize.

    0
    B
     1 Reply
  • B
    administrators

    @rami-ramich said in ibtest.py and vix?:

    Could you please advice how to fix or workaround it?

    VIX is not an index which has ever been accessed over here.

    @rami-ramich said in ibtest.py and vix?:

    and get error... without --resample

    It's only a sample to test different behaviors. It's not a bulletproof script to test strategies.

    @rami-ramich said in ibtest.py and vix?:

    I coded this - could you please tell if this OK?
    def tickPrice(self, msg):

    As expressed above. No access to your VIX.

    0
  • R

    @backtrader my code modification is working, i have access to VIX-IND-CBOE data, but from your point of view modifications are OK? can you merge into main branch?

    the crash stack i provided happen inside of backtrader framework.

    0