There is an example of using backtrader for multiple stocks and one for using multiple timeframes. Is there an example available to use both together? I want to get a signal on weekly data and buy if another signal on end of day data is satisfied.
Best posts made by Suraj Thorat
-
How to deal with multiple stocks data with multiple timeframes?
Latest posts made by Suraj Thorat
-
How to deal with multiple stocks data with multiple timeframes?
There is an example of using backtrader for multiple stocks and one for using multiple timeframes. Is there an example available to use both together? I want to get a signal on weekly data and buy if another signal on end of day data is satisfied.
-
RE: Bug with live trading ibtest.py script?
I forgot to mention the seconds run is working fine when I change timeframe to minutes or any other timeframe the problem occurs.
-
Bug with live trading ibtest.py script?
Is there a bug with live trading when fetching the data feed when we use resample data? Especially with the high and low data.
I am using the exact ibtest script in the samples without any changes to it.
When I run the script for Seconds and compression 1 the desired data is received with no issues.
I used this to run the code which is pretty standard.
./ibtest.py --port 4002 --data0 'TCS-STK-NSE-INR' --resample --timeframe Seconds --compression 1Data0, 1797, 737468.1806481482, 2020-02-13T04:20:08.000000, 2194.6, 2194.6, 2194.6, 2194.6, 6.0, 0, 2194.45 Data0, 1798, 737468.1806597222, 2020-02-13T04:20:09.000000, 2194.6, 2194.6, 2194.6, 2194.6, 0.0, 0, 2194.3900000000003 Data0, 1799, 737468.1806712963, 2020-02-13T04:20:10.000000, 2194.55, 2194.55, 2194.55, 2194.55, 11.0, 0, 2194.33 Data0, 1800, 737468.1806828703, 2020-02-13T04:20:11.000000, 2194.2, 2194.2, 2194.2, 2194.2, 36.0, 0, 2194.3 ***** DATA NOTIF: LIVE Data0, 1801, 737468.1807060185, 2020-02-13T04:20:13.000000, 2193.55, 2193.55, 2193.55, 2193.55, 133894.0, 0, 2194.3 Data0, 1802, 737468.1807175926, 2020-02-13T04:20:14.000000, 2184.55, 2184.55, 2184.55, 2184.55, 165.0, 0, 2192.29 Data0, 1803, 737468.1807291667, 2020-02-13T04:20:15.000000, 2180.9, 2180.9, 2180.9, 2180.9, 13.0, 0, 2189.55 Data0, 1804, 737468.1807523149, 2020-02-13T04:20:17.000000, 2184.85, 2184.85, 2184.85, 2184.85, 145.0, 0, 2187.6099999999997 Data0, 1805, 737468.180775463, 2020-02-13T04:20:19.000000, 2240.0, 2240.0, 2186.1, 2186.1, 95.0, 0, 2185.9900000000002
Please notice the high and lows i.e. the 2nd and 3rd column after time when I use the code for Minutes with compression 1.
I use this to run the code for minutes.
./ibtest.py --port 4002 --data0 'TCS-STK-NSE-INR' --resample --timeframe Minutes --compression 1Data0, 0036, 737468.1805555555, 2020-02-13T04:20:00.000000, 2193.35, 2195.0, 2193.35, 2194.6, 850.0, 0, 2194.45 Data0, 0037, 737468.18125, 2020-02-13T04:21:00.000000, 2194.7, 2194.7, 2193.65, 2194.0, 623.0, 0, 2194.13 Data0, 0038, 737468.1819444444, 2020-02-13T04:22:00.000000, 2194.2, 2194.2, 2193.45, 2193.65, 1027.0, 0, 2193.8599999999997 Data0, 0039, 737468.182638889, 2020-02-13T04:23:00.000000, 2193.7, 2194.0, 2193.65, 2193.95, 138.0, 0, 2193.9 ***** DATA NOTIF: LIVE Data0, 0040, 737468.1833333333, 2020-02-13T04:24:00.000000, 2194.05, 2273.9, 2176.0, 2185.15, 23071.0, 0, 2192.27 Data0, 0041, 737468.1840277778, 2020-02-13T04:25:00.000000, 2184.8, 2228.8, 2176.0, 2185.6, 9375.0, 0, 2190.4700000000003 Data0, 0042, 737468.1847222223, 2020-02-13T04:26:00.000000, 2176.0, 2230.25, 2176.0, 2185.5, 8036.0, 0, 2188.77
The lows are constant i.e. the 3rd column and the highs i.e. the second column is not sensible. Just to cross check I opened Trader workstation and it has the right highs and lows. You can also compare the highs and lows with the historical data for further confirmation and this is 1 minute data so it is obvious.
Can someone please help fixing this? -
Is it possible to get a shorter timeframe backfill in Interactive brokers live paper trading?
I have backtested a strategy on 2000 stocks simultaneously. I am trying to go live paper trade with the strategy. Although as I am using 15 minutes data when I call ibstore data almost 10 days of backfill data is asked for which takes a lot of time to download. I just need 14 bars of previous data to be backfilled in order for my strategy to work.
Also if there is a better way to do what I am doing I would be grateful for any suggestion. (Strategy is basically applying technical indicators for buy and sell on 2000 stocks. The occurrence of indicators is very rare. )
Thank you in advance and a special thanks to developers of this awesome library. -
RE: Problem occurring when I change 'cerebro.adddata(data, name = i)' to 'cerebro.resampledata(data , timeframe = bt.TimeFrame.Minutes, compression=15,name = i)' Error: return self.array[self.idx + ago] IndexError: array index out of range
So here is an update for the same. I had added
def prenext(self): self.next()
so that my backtest runs the entire timeframe i.e. if I have 100 stocks the earliest possible date is taken into consideration since some stocks have IPO's later. When I removed the prenext the code worked with resample and replay data but again I am getting output for a lower timeframe i.e. if the entire timeframe is 100 days but 1 of the stocks is only 30 days I am getting the backtest of only that particular timeframe.
Is there a way to achieve both i.e. get the backtest for the entire timeframe as well as me not getting an error. I am sure I have made some silly mistake in understanding the docs but I am helpless regarding this. -
RE: Problem occurring when I change 'cerebro.adddata(data, name = i)' to 'cerebro.resampledata(data , timeframe = bt.TimeFrame.Minutes, compression=15,name = i)' Error: return self.array[self.idx + ago] IndexError: array index out of range
Thank you for the reply. I tried defining the original data feed time frame and compression and then using resampling on it.
Note: I don't really need to do this for backtesting but I am doing this as I need to resample the data for interactive brokers live paper trading.
But I still got the same error at the same place.for i in stocks_5: stock_ohlc = data_csv[data_csv['Symbol'] == i][['open','high','low','close','volume']] data = bt.feeds.PandasData(dataname=stock_ohlc, timeframe = bt.TimeFrame.Minutes, compression=15, sessionstart = datetime.time(9,15), sessionend=datetime.time(15,15)) cerebro.resampledata(data, timeframe = bt.TimeFrame.Minutes, compression=15, name = i)
-
Problem occurring when I change 'cerebro.adddata(data, name = i)' to 'cerebro.resampledata(data , timeframe = bt.TimeFrame.Minutes, compression=15,name = i)' Error: return self.array[self.idx + ago] IndexError: array index out of range
I can't thank the developers enough for this platform. But here I am again with another post trying to figure out what I am doing wrong. I know the mistake might be something really basic.
Important portions of the code are mentioned below.
init functiondef __init__(self): ''' Create an dictionary of indicators so that we can dynamically add the indicators to the strategy using a loop. This mean the strategy will work with any numner of data feeds. ''' self.inds = dict() self.order = {} for i, d in enumerate(self.datas): self.order[d._name] = None self.inds[d] = dict() self.inds[d]['atr14'] = bt.indicators.AverageTrueRange(d, period = self.params.atrperiod) self.inds[d]['macd'] = bt.indicators.MACD(d, period_me1=self.p.macd1, period_me2=self.p.macd2, period_signal=self.p.macdsig) self.inds[d]['mcross'] = bt.indicators.CrossUp(self.inds[d]['macd'].macd, self.inds[d]['macd'].signal)
Main logic in next():
def next(self): # self.logdata() for i, d in enumerate(self.datas): dt, dn = self.datetime.date(), d._name pos = self.getposition(d).size self.logdata() print(d._name) #ERROR OCCURSE BELOW THIS LINE if self.inds[d]['atr14'][0] * self.params.atrmultiplier < (d.high[0]-d.low[0]) and d.close[0]<d.close[-1]: # This is where the error occurs # self.buy(data=d, size=1) if self.order[d._name]: self.log('We already have a buy order for %s in place' % d._name) return self.log(' BUYING. ||| symbol: %s , atr14 : %.2f , high-low : %.2f, close[0]: %.2f, close[-1] %.2f ' % (d._name,self.inds[d]['atr14'][0],(d.high[0]-d.low[0]),d.close[0],d.close[-1])) self.order[d._name] = self.order_target_value(data=d, target = 30000)
Code for data insertion for multiple stocks written below. This code gives error.
for i in stocks_5: stock_ohlc = data_csv[data_csv['Symbol'] == i][['open','high','low','close','volume']] data = bt.feeds.PandasData(dataname=stock_ohlc, sessionstart = datetime.time(9,15), sessionend=datetime.time(15,15)) cerebro.resampledata(data , timeframe = bt.TimeFrame.Minutes, compression=15,name = i)
Keeping everything same but using this code works and I get a proper output.
for i in stocks_5: stock_ohlc = data_csv[data_csv['Symbol'] == i][['open','high','low','close','volume']] data = bt.feeds.PandasData(dataname=stock_ohlc, timeframe = bt.TimeFrame.Minutes, compression=15, sessionstart = datetime.time(9,15), sessionend=datetime.time(15,15)) cerebro.adddata(data, name = i)
The only reason I need to use resample data instead of addatla is because as far as I know there is a necessity to use resample data for 15 minute bars when paper trading I interactive brokers.
I can't post output here of the code that is working since it is really long.
Also 1 important thing to consider which I have deduced is the error occurs when there is no data for that stock. For example out of 5 stocks data of 4 stocks are starting on 21st January but the 5th is starting on 24th, this is where the error occurs.Here is the output when I have the error:
1,2018-12-21T09:15:00,O: 43.85,H: 43.85,L: 43.10,C: 43.30,V: 222.00 20MICRONS 1,2018-12-21T09:15:00,O: 43.85,H: 43.85,L: 43.10,C: 43.30,V: 222.00 21STCENMG 1,2018-12-21T09:15:00,O: 43.85,H: 43.85,L: 43.10,C: 43.30,V: 222.00 3IINFOTEC 1,2018-12-21T09:15:00,O: 43.85,H: 43.85,L: 43.10,C: 43.30,V: 222.00 3MINDIA 1,2018-12-21T09:15:00,O: 43.85,H: 43.85,L: 43.10,C: 43.30,V: 222.00 3PLAND --------------------------------------------------------------------------- IndexError Traceback (most recent call last) <ipython-input-43-a3e55db0bf8e> in <module> 291 # Run over everything 292 # cerebro.broker.set_coc(True) --> 293 cerebro.run() 294 295 #Get final portfolio Value /usr/local/Cellar/python35/3.5.7_1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/backtrader/cerebro.py in run(self, **kwargs) 1125 # let's skip process "spawning" 1126 for iterstrat in iterstrats: -> 1127 runstrat = self.runstrategies(iterstrat) 1128 self.runstrats.append(runstrat) 1129 if self._dooptimize: /usr/local/Cellar/python35/3.5.7_1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/backtrader/cerebro.py in runstrategies(self, iterstrat, predata) 1296 self._runnext_old(runstrats) 1297 else: -> 1298 self._runnext(runstrats) 1299 1300 for strat in runstrats: /usr/local/Cellar/python35/3.5.7_1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/backtrader/cerebro.py in _runnext(self, runstrats) 1628 self._check_timers(runstrats, dt0, cheat=False) 1629 for strat in runstrats: -> 1630 strat._next() 1631 if self._event_stop: # stop if requested 1632 return /usr/local/Cellar/python35/3.5.7_1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/backtrader/strategy.py in _next(self) 345 346 def _next(self): --> 347 super(Strategy, self)._next() 348 349 minperstatus = self._getminperstatus() /usr/local/Cellar/python35/3.5.7_1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/backtrader/lineiterator.py in _next(self) 273 self.nextstart() # only called for the 1st value 274 else: --> 275 self.prenext() 276 else: 277 # assume indicators and others operate on same length datas <ipython-input-43-a3e55db0bf8e> in prenext(self) 115 d.plotinfo.plotmaster = self.datas[0] 116 def prenext(self): --> 117 self.next() 118 119 def next(self): <ipython-input-43-a3e55db0bf8e> in next(self) 126 print(d._name) 127 if not pos and self.p.typeofentry == 1: # no market / no orders --> 128 if self.inds[d]['atr14'][0] * self.params.atrmultiplier < (d.high[0]-d.low[0]) and d.close[0]<d.close[-1]: 129 # self.buy(data=d, size=1) 130 if self.order[d._name]: /usr/local/Cellar/python35/3.5.7_1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/backtrader/lineseries.py in __getitem__(self, key) 465 466 def __getitem__(self, key): --> 467 return self.lines[0][key] 468 469 def __setitem__(self, key, value): /usr/local/Cellar/python35/3.5.7_1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/backtrader/linebuffer.py in __getitem__(self, ago) 161 162 def __getitem__(self, ago): --> 163 return self.array[self.idx + ago] 164 165 def get(self, ago=0, size=1): IndexError: array index out of range
Also I really don't mind implementing this exact code with add data but I have some issue when live paper trading it in interactive brokers thus I have to use resample data. I think there is a need I show the prenext() function to get further insight so here it is:
def prenext(self): self.next()
Again I might be wrong in many things I am considering. Please feel free to correct me. Any help would be appreciated. Thanks in advance.
-
RE: During market closing hours double buy or double sell orders are placed for multiple stocks back trading.
@ab_trader Thank you for the reply. I wasn't using the condition to check if self.order exists then return.
It worked for me. So anyone having the same issue in the future kindly make sure these 3 things are taken care of. I think I have solved the issue entirely but if not I'll update this in the future.- Use self.order condition in code to check if we already have a similar order in place.
if not pos: # no market / no orders if self.inds[d]['atr14'][-1] * 10 < (d.high[0]-d.low[0]) and (d.close[0]-d.open[0])>0: # self.buy(data=d, size=1) if self.order[d._name]: self.log('We already have a buy order for %s in place' % d._name) return self.log(' BUYING. ||| symbol: %s , atr14 : %.2f , close-open : %.2f , rsi3crossD: %.2f' % (d._name,self.inds[d]['atr14'][0],(d.close[0]-d.open[0]),self.inds[d]['rsi3crossD'][0])) self.order[d._name] = self.order_target_value(data=d, target = 30000) # elif self.inds[d]['cross'][0] == -1: # # self.sell(data=d, size=1) # self.order_target_percent(data=d, target = -0.16) elif pos: if self.inds[d]['rsi3'][0]<65: if self.order[d._name]: self.log('We already have a sell order for %s in place' % d._name) return self.order[d._name] = self.close(data=d) self.log(' SELLING. ||| symbol: %s ,rsi3crossD: %.2f' % (d._name,self.inds[d]['rsi3crossD'][0])) #
- Make sure that crossovers are used in the way above i.e. crossovercondition==1
- Make sure you add the pandas data with parameters timeframe, compression, session start and session end.
-
RE: During market closing hours double buy or double sell orders are placed for multiple stocks back trading.
Even this is not working in every circumstance? I really need to find the issue with this code of mine. Any help would be appreciated. Thanks in advance.
-
RE: During market closing hours double buy or double sell orders are placed for multiple stocks back trading.
This is the 3rd reply from me on my post. This is what worked for me. I am just posting it so that it might be helpful for someone.
elif pos: if self.inds[d]['rsi3crossD'][0]==1: self.close(data=d) self.log(' SELLING. ||| symbol: %s ,rsi3crossD: %.2f' % (d._name,self.inds[d]['rsi3crossD'][0]))
For crossovers you need to use ==1 .