Interactive brokers data feed return strange (wrong?) data with two assets
-
Hi,
I am using Interactive brokers data feeds. I use two assets in my strategy: SPY and VIX. I am trying to figure out how to make interactive brokers work with 2 two data lines. For the beginning, here is the simple code which print out 5 min data for VIX and SPY :
import backtrader as bt import datetime class MyStrategy(bt.Strategy): def log(self, txt, dt=None, doprint=False): if doprint: dt = dt or self.datas[0].datetime.datetime(0) print('%s, %s' % (dt.isoformat(), txt)) def next(self): self.log(f'VIX {self.datas[0].close[0]}', doprint=True) self.log(f'SPY {self.datas[1].close[0]}', doprint=True) cerebro = bt.Cerebro() cerebro.addstrategy(MyStrategy) store=bt.stores.IBStore(host="127.0.0.1", port=7496, clientId=77) stockkwargs = dict( timeframe=bt.TimeFrame.Minutes, historical=True, fromdate=datetime.datetime(2020, 10, 26), todate=datetime.datetime(2020, 10, 27), compression=1 ) data0 = store.getdata(dataname="SPY-STK-SMART-USD", **stockkwargs) data1 = store.getdata(dataname='VIX-IND-CBOE-USD', **stockkwargs) cerebro.resampledata(data1, timeframe=bt.TimeFrame.Minutes, compression=5) cerebro.resampledata(data0, timeframe=bt.TimeFrame.Minutes, compression=5) cerebro.run()
Here are the logs:
(base) PS C:\Users\Mislav\Documents\GitHub\trademl\trademl\algos> python .\variance_backcusum.py Server Version: 76 TWS Time at connection:20201202 11:33:20 CET 2020-10-26T03:00:00, VIX 30.0 2020-10-26T03:00:00, SPY 341.4 2020-10-26T03:05:00, VIX 29.94 2020-10-26T03:05:00, SPY 341.74 2020-10-26T03:10:00, VIX 29.93 2020-10-26T03:10:00, SPY 341.81 2020-10-26T03:15:00, VIX 29.95 2020-10-26T03:15:00, SPY 341.86 2020-10-26T03:20:00, VIX 30.03 2020-10-26T03:20:00, SPY 341.6 2020-10-26T03:25:00, VIX 30.05 2020-10-26T03:25:00, SPY 341.61 2020-10-26T03:30:00, VIX 30.08 2020-10-26T03:30:00, SPY 341.46 2020-10-26T03:35:00, VIX 30.04 2020-10-26T03:35:00, SPY 341.81 2020-10-26T03:40:00, VIX 30.03 2020-10-26T03:40:00, SPY 341.83 2020-10-26T03:45:00, VIX 29.92 2020-10-26T03:45:00, SPY 341.91 2020-10-26T03:50:00, VIX 29.91 2020-10-26T03:50:00, SPY 341.96 2020-10-26T03:55:00, VIX 29.9 2020-10-26T03:55:00, SPY 341.91 2020-10-26T04:00:00, VIX 29.89 2020-10-26T04:00:00, SPY 341.94 2020-10-26T04:05:00, VIX 29.84 2020-10-26T04:05:00, SPY 342.09 2020-10-26T04:10:00, VIX 29.85 2020-10-26T04:10:00, SPY 342.06 2020-10-26T04:15:00, VIX 29.82 2020-10-26T04:15:00, SPY 342.06 2020-10-26T04:20:00, VIX 29.78 2020-10-26T04:20:00, SPY 342.05 2020-10-26T04:25:00, VIX 29.82 2020-10-26T04:25:00, SPY 342.09 2020-10-26T04:30:00, VIX 29.83 2020-10-26T04:30:00, SPY 342.07 2020-10-26T04:35:00, VIX 29.86 2020-10-26T04:35:00, SPY 341.96 2020-10-26T04:40:00, VIX 29.92 2020-10-26T04:40:00, SPY 341.96 2020-10-26T04:45:00, VIX 29.96 2020-10-26T04:45:00, SPY 341.7 2020-10-26T04:50:00, VIX 29.98 2020-10-26T04:50:00, SPY 341.69 2020-10-26T04:55:00, VIX 29.94 2020-10-26T04:55:00, SPY 341.86 2020-10-26T05:00:00, VIX 30.03 2020-10-26T05:00:00, SPY 341.86 2020-10-26T05:05:00, VIX 30.0 2020-10-26T05:05:00, SPY 341.67 2020-10-26T05:10:00, VIX 29.93 2020-10-26T05:10:00, SPY 341.87 2020-10-26T05:15:00, VIX 29.9 2020-10-26T05:15:00, SPY 341.9 2020-10-26T05:20:00, VIX 29.84 2020-10-26T05:20:00, SPY 342.0 2020-10-26T05:25:00, VIX 29.83 2020-10-26T05:25:00, SPY 342.0 2020-10-26T05:30:00, VIX 29.84 2020-10-26T05:30:00, SPY 341.9 2020-10-26T05:35:00, VIX 29.82 2020-10-26T05:35:00, SPY 341.99 2020-10-26T05:40:00, VIX 29.83 2020-10-26T05:40:00, SPY 341.8 2020-10-26T05:45:00, VIX 29.84 2020-10-26T05:45:00, SPY 341.85 2020-10-26T05:50:00, VIX 29.81 2020-10-26T05:50:00, SPY 341.95 2020-10-26T05:55:00, VIX 29.77 2020-10-26T05:55:00, SPY 342.02 2020-10-26T06:00:00, VIX 29.69 2020-10-26T06:00:00, SPY 342.35 2020-10-26T06:05:00, VIX 29.6 2020-10-26T06:05:00, SPY 342.31 2020-10-26T06:10:00, VIX 29.62 2020-10-26T06:10:00, SPY 342.36 2020-10-26T06:15:00, VIX 29.64 2020-10-26T06:15:00, SPY 342.3 2020-10-26T06:20:00, VIX 29.71 2020-10-26T06:20:00, SPY 342.16 2020-10-26T06:25:00, VIX 29.71 2020-10-26T06:25:00, SPY 342.17 2020-10-26T06:30:00, VIX 29.72 2020-10-26T06:30:00, SPY 342.11 2020-10-26T06:35:00, VIX 29.63 2020-10-26T06:35:00, SPY 342.24 2020-10-26T06:40:00, VIX 29.66 2020-10-26T06:40:00, SPY 342.1 2020-10-26T06:45:00, VIX 29.68 2020-10-26T06:45:00, SPY 342.04 2020-10-26T06:50:00, VIX 29.77 2020-10-26T06:50:00, SPY 341.78 2020-10-26T06:55:00, VIX 29.84 2020-10-26T06:55:00, SPY 341.76 2020-10-26T07:00:00, VIX 29.84 2020-10-26T07:00:00, SPY 341.67 2020-10-26T07:05:00, VIX 29.91 2020-10-26T07:05:00, SPY 341.7 2020-10-26T07:10:00, VIX 29.98 2020-10-26T07:10:00, SPY 341.5 2020-10-26T07:15:00, VIX 29.99 2020-10-26T07:15:00, SPY 341.65 2020-10-26T07:20:00, VIX 29.93 2020-10-26T07:20:00, SPY 341.52 2020-10-26T07:25:00, VIX 29.89 2020-10-26T07:25:00, SPY 341.81 2020-10-26T07:30:00, VIX 29.84 2020-10-26T07:30:00, SPY 341.82 2020-10-26T07:35:00, VIX 29.84 2020-10-26T07:35:00, SPY 341.74 2020-10-26T07:40:00, VIX 29.9 2020-10-26T07:40:00, SPY 341.63 2020-10-26T07:45:00, VIX 29.93 2020-10-26T07:45:00, SPY 341.56 2020-10-26T07:50:00, VIX 29.96 2020-10-26T07:50:00, SPY 341.47 2020-10-26T07:55:00, VIX 29.93 2020-10-26T07:55:00, SPY 341.65 2020-10-26T08:00:00, VIX 29.9 2020-10-26T08:00:00, SPY 341.73 2020-10-26T08:05:00, VIX 29.87 2020-10-26T08:05:00, SPY 341.77 2020-10-26T08:10:00, VIX 29.74 2020-10-26T08:10:00, SPY 342.0 2020-10-26T08:10:00, VIX 29.74 2020-10-26T08:10:00, SPY 342.11 2020-10-26T08:10:00, VIX 29.74 2020-10-26T08:10:00, SPY 342.06 2020-10-26T08:10:00, VIX 29.74 2020-10-26T08:10:00, SPY 342.11 2020-10-26T08:30:00, VIX 29.35 2020-10-26T08:30:00, SPY 342.37 2020-10-26T08:35:00, VIX 29.41 2020-10-26T08:35:00, SPY 342.57 2020-10-26T08:40:00, VIX 29.54 2020-10-26T08:40:00, SPY 342.1 2020-10-26T08:45:00, VIX 29.64 2020-10-26T08:45:00, SPY 341.96 2020-10-26T08:50:00, VIX 29.47 2020-10-26T08:50:00, SPY 342.53 2020-10-26T08:55:00, VIX 29.37 2020-10-26T08:55:00, SPY 342.53 2020-10-26T09:00:00, VIX 29.29 2020-10-26T09:00:00, SPY 342.83 2020-10-26T09:05:00, VIX 29.28 2020-10-26T09:05:00, SPY 342.8 2020-10-26T09:10:00, VIX 29.32 2020-10-26T09:10:00, SPY 342.62 2020-10-26T09:15:00, VIX 29.31 2020-10-26T09:15:00, SPY 342.67 2020-10-26T09:20:00, VIX 29.45 2020-10-26T09:20:00, SPY 342.36 2020-10-26T09:25:00, VIX 29.82 2020-10-26T09:25:00, SPY 341.58 2020-10-26T09:30:00, VIX 30.17 2020-10-26T09:30:00, SPY 340.99 2020-10-26T09:35:00, VIX 30.19 2020-10-26T09:35:00, SPY 340.99 2020-10-26T09:40:00, VIX 30.19 2020-10-26T09:40:00, SPY 340.96 2020-10-26T09:45:00, VIX 30.49 2020-10-26T09:45:00, SPY 340.25 2020-10-26T09:50:00, VIX 30.63 2020-10-26T09:50:00, SPY 339.97 2020-10-26T09:55:00, VIX 30.96 2020-10-26T09:55:00, SPY 339.65 2020-10-26T10:00:00, VIX 31.42 2020-10-26T10:00:00, SPY 339.08 2020-10-26T10:05:00, VIX 31.38 2020-10-26T10:05:00, SPY 339.04 2020-10-26T10:10:00, VIX 31.09 2020-10-26T10:10:00, SPY 339.34 2020-10-26T10:15:00, VIX 31.81 2020-10-26T10:15:00, SPY 338.24 2020-10-26T10:20:00, VIX 31.43 2020-10-26T10:20:00, SPY 338.81 2020-10-26T10:25:00, VIX 31.41 2020-10-26T10:25:00, SPY 338.8 2020-10-26T10:30:00, VIX 31.74 2020-10-26T10:30:00, SPY 338.02 2020-10-26T10:35:00, VIX 31.46 2020-10-26T10:35:00, SPY 338.73 2020-10-26T10:40:00, VIX 31.54 2020-10-26T10:40:00, SPY 338.65 2020-10-26T10:45:00, VIX 31.37 2020-10-26T10:45:00, SPY 338.85 2020-10-26T10:50:00, VIX 31.33 2020-10-26T10:50:00, SPY 338.97 2020-10-26T10:55:00, VIX 31.37 2020-10-26T10:55:00, SPY 338.63 2020-10-26T11:00:00, VIX 31.52 2020-10-26T11:00:00, SPY 338.71 2020-10-26T11:05:00, VIX 31.61 2020-10-26T11:05:00, SPY 338.54 2020-10-26T11:10:00, VIX 31.59 2020-10-26T11:10:00, SPY 338.46 2020-10-26T11:15:00, VIX 31.51 2020-10-26T11:15:00, SPY 338.74 2020-10-26T11:20:00, VIX 31.48 2020-10-26T11:20:00, SPY 338.62 2020-10-26T11:25:00, VIX 31.44 2020-10-26T11:25:00, SPY 338.63 2020-10-26T11:30:00, VIX 31.94 2020-10-26T11:30:00, SPY 337.75 2020-10-26T11:35:00, VIX 31.94 2020-10-26T11:35:00, SPY 337.66 2020-10-26T11:40:00, VIX 31.9 2020-10-26T11:40:00, SPY 337.66 2020-10-26T11:45:00, VIX 32.02 2020-10-26T11:45:00, SPY 337.37 2020-10-26T11:50:00, VIX 32.14 2020-10-26T11:50:00, SPY 337.44 2020-10-26T11:55:00, VIX 32.26 2020-10-26T11:55:00, SPY 337.5 2020-10-26T12:00:00, VIX 32.3 2020-10-26T12:00:00, SPY 337.47 2020-10-26T12:05:00, VIX 32.34 2020-10-26T12:05:00, SPY 337.45 2020-10-26T12:10:00, VIX 32.25 2020-10-26T12:10:00, SPY 337.27 2020-10-26T12:15:00, VIX 32.55 2020-10-26T12:15:00, SPY 336.72 2020-10-26T12:20:00, VIX 32.76 2020-10-26T12:20:00, SPY 336.44 2020-10-26T12:25:00, VIX 32.91 2020-10-26T12:25:00, SPY 336.2 2020-10-26T12:30:00, VIX 33.4 2020-10-26T12:30:00, SPY 335.75 2020-10-26T12:35:00, VIX 32.78 2020-10-26T12:35:00, SPY 336.39 2020-10-26T12:40:00, VIX 32.91 2020-10-26T12:40:00, SPY 336.4 2020-10-26T12:45:00, VIX 32.88 2020-10-26T12:45:00, SPY 336.79 2020-10-26T12:50:00, VIX 32.48 2020-10-26T12:50:00, SPY 337.15 2020-10-26T12:55:00, VIX 32.42 2020-10-26T12:55:00, SPY 337.39 2020-10-26T13:00:00, VIX 32.43 2020-10-26T13:00:00, SPY 337.53 2020-10-26T13:05:00, VIX 32.26 2020-10-26T13:05:00, SPY 337.84 2020-10-26T13:10:00, VIX 32.4 2020-10-26T13:10:00, SPY 337.67 2020-10-26T13:15:00, VIX 32.48 2020-10-26T13:15:00, SPY 337.92 2020-10-26T13:20:00, VIX 32.56 2020-10-26T13:20:00, SPY 337.89 2020-10-26T13:25:00, VIX 32.14 2020-10-26T13:25:00, SPY 338.59 2020-10-26T13:30:00, VIX 32.13 2020-10-26T13:30:00, SPY 338.37 2020-10-26T13:35:00, VIX 32.46 2020-10-26T13:35:00, SPY 338.08 2020-10-26T13:40:00, VIX 32.55 2020-10-26T13:40:00, SPY 338.14 2020-10-26T13:45:00, VIX 32.93 2020-10-26T13:45:00, SPY 337.71 2020-10-26T13:50:00, VIX 32.83 2020-10-26T13:50:00, SPY 338.1 2020-10-26T13:55:00, VIX 32.88 2020-10-26T13:55:00, SPY 338.18 2020-10-26T14:00:00, VIX 33.08 2020-10-26T14:00:00, SPY 338.05 2020-10-26T14:05:00, VIX 33.4 2020-10-26T14:05:00, SPY 337.75 2020-10-26T14:10:00, VIX 33.65 2020-10-26T14:10:00, SPY 337.52 2020-10-26T14:15:00, VIX 33.42 2020-10-26T14:15:00, SPY 337.73 2020-10-26T14:20:00, VIX 32.9 2020-10-26T14:20:00, SPY 338.15 2020-10-26T14:25:00, VIX 32.85 2020-10-26T14:25:00, SPY 338.37 2020-10-26T14:30:00, VIX 32.78 2020-10-26T14:30:00, SPY 338.62 2020-10-26T14:35:00, VIX 32.9 2020-10-26T14:35:00, SPY 338.42 2020-10-26T14:40:00, VIX 32.88 2020-10-26T14:40:00, SPY 338.3 2020-10-26T14:45:00, VIX 33.21 2020-10-26T14:45:00, SPY 337.5 2020-10-26T14:50:00, VIX 33.01 2020-10-26T14:50:00, SPY 338.21 2020-10-26T14:55:00, VIX 32.56 2020-10-26T14:55:00, SPY 339.41 2020-10-26T15:00:00, VIX 32.28 2020-10-26T15:00:00, SPY 339.46 2020-10-26T15:05:00, VIX 32.38 2020-10-26T15:05:00, SPY 339.65 2020-10-26T15:10:00, VIX 32.46 2020-10-26T15:10:00, SPY 339.5
As you can see first vix changes, but than it become constant. On the other hand SPY changes all the time. I am confused about this data, becuase, when I use
ib_insync
package I get different data for two series:from ib_insync import * import datetime import pandas as pd util.startLoop() # uncomment this line when in a notebook ib = IB() ib.connect('127.0.0.1', 7496, clientId=90) contract_vix = Index(symbol='VIX', exchange='CBOE', localSymbol='VIX', currency='USD') contract_spy = Stock(symbol='SPY', exchange='SMART', currency='USD') vix = ib.reqHistoricalData( contract_vix, endDateTime=datetime.datetime(2020, 10, 27), durationStr='1 D', barSizeSetting='5 mins', whatToShow='TRADES', useRTH=True, formatDate=1 ) vix = util.df(vix) vix = vix.add_prefix('vix_') spy = ib.reqHistoricalData( contract_spy, endDateTime=datetime.datetime(2020, 10, 27), durationStr='1 D', barSizeSetting='5 mins', whatToShow='TRADES', useRTH=True, formatDate=1 ) spy = util.df(spy) spy_vix = pd.concat([spy, vix], axis=1).dropna() spy_vix = spy_vix[['date', 'close', 'vix_close']] spy_vix.head(50) spy_vix.tail(10) ib.disconnect()
As you can see, there is a difference between VIX and SPY match.
For exmaple in logs, VIX around 30 all the time, while in the 'ib_insync data', it is around 29
1 ) **Why backtrader shows different data than the one imported with **
2) If you try to run this backtest you can see it never ends. It ends only I add optionoldsync=True
, which change the joint data too.