Exception in line series length at paper trading
I'm trying to run backtrader with Interactive Broker where I fed data for 10 equities. But I get an error related to the length of some of the data lines at the beginning of the simulation. When I reduce the number of equities to three all run ok. I searched on this community and found a related problem but any
solution is yet proposed.
Here the full exception trace:
Traceback (most recent call last): File "/home/galvez/repos/research_factory/20180822_PortafolioBacktrader/XBranch_NN/backTesting/run_backtest.py", line 335, in <module> run_live(equity) File "/home/galvez/repos/research_factory/20180822_PortafolioBacktrader/XBranch_NN/backTesting/run_backtest.py", line 325, in run_live cerebro.run() File "/home/galvez/anaconda3/envs/backtrader/lib/python3.5/site-packages/backtrader/cerebro.py", line 1127, in run runstrat = self.runstrategies(iterstrat) File "/home/galvez/anaconda3/envs/backtrader/lib/python3.5/site-packages/backtrader/cerebro.py", line 1298, in runstrategies self._runnext(runstrats) File "/home/galvez/anaconda3/envs/backtrader/lib/python3.5/site-packages/backtrader/cerebro.py", line 1630, in _runnext strat._next() File "/home/galvez/anaconda3/envs/backtrader/lib/python3.5/site-packages/backtrader/strategy.py", line 325, in _next super(Strategy, self)._next() File "/home/galvez/anaconda3/envs/backtrader/lib/python3.5/site-packages/backtrader/lineiterator.py", line 255, in _next clock_len = self._clk_update() File "/home/galvez/anaconda3/envs/backtrader/lib/python3.5/site-packages/backtrader/strategy.py", line 305, in _clk_update newdlens = [len(d) for d in self.datas] File "/home/galvez/anaconda3/envs/backtrader/lib/python3.5/site-packages/backtrader/strategy.py", line 305, in <listcomp> newdlens = [len(d) for d in self.datas] File "/home/galvez/anaconda3/envs/backtrader/lib/python3.5/site-packages/backtrader/lineseries.py", line 464, in __len__ return len(self.lines) File "/home/galvez/anaconda3/envs/backtrader/lib/python3.5/site-packages/backtrader/lineseries.py", line 220, in __len__ return len(self.lines) ValueError: __len__() should return >= 0 Process finished with exit code 1
Thanks in advance and happy trading.
You are provoking pacing violations, due to the number of requests. That's why reducing the number of assets gets you up and running.
Thank you for your prompt response. Any clue how to overcome the issue? Or is there a limited number of assets we can work with Interactive Brokers and Backtrader?
Or is there a limited number of assets we can work with Interactive Brokers
The limit is there. Even if some of the limits have been softened over the years, IB praises to be a broker and not a data provider. That's why they also provide 250 ms snapshot-ticks of the market instead of real time ticks.
Using IB is of course fit for many purposes and trading situations, but working simultaneously with a large number of assets isn't one of them.
- Use data files (or any other source) as
Because the pacing violations are hit when requesting historical data (which is the backfilling source)
There is an additional limit which is the number of simultanteous assets you can request real-time information from (NBBO), but this also has to do with your account commissions level, if you have acquired a boost package and other things.
Let me add: if you didn't know these things before ... you should know all the rules that apply to your trading (data provision and brokerage) before you trade. It is the 1st rule when gambling (sorry trading): "Know the house rules".
@backtrader Thank you very much for your response and suggestions. I appreciate it.
Hi, @backtrader and Bactraders!
I tried the second option but it didn't go as expected. I read the documentation and review every post about the
backfill_fromand tried several things from there without success.
The issue is that the timestamp I get from the data when the
next()is called jump from
5:00of the other day instead of continue to
12:01and so on until
9:30of the next day. Here an extract of my prints:
2018-10-01T11:56:00 2018-10-01T11:57:00 2018-10-01T11:58:00 2018-10-01T11:59:00 2018-10-01T12:00:00 2018-10-02T05:31:00 2018-10-02T05:32:00 2018-10-02T05:33:00 2018-10-02T05:34:00 2018-10-02T05:35:00 2018-10-02T05:36:00 2018-10-02T05:37:00
I'm very sure that my local CSVs have data only between
Let me detail about my code. This is how I'm connecting to IB:
cerebro = bt.Cerebro() store = bt.stores.IBStore(host='127.0.0.1',port=4002, clientId=36, reconnect=-1)
My data is in minute bars and I'm reading it as follow:
data0 = btfeed.GenericCSVData( dataname=file_path, fromdate=datetime.datetime(2018, 10, 1, 9, 30, 0), todate=datetime.datetime(2018, 12, 12, 9, 30, 0), timeframe=bt.TimeFrame.Minutes, compression=1, dtformat='%Y%m%d', tmformat='%H:%M:%S', datetime=0, time=1, open=3, high=4, low=5, close=6, volume=8, openinterest=-1, )
Then, adding the data:
contract = 'TWTR-STK-SMART-USD' data = store.getdata(dataname=contract, rtbar=True, timeframe=bt.TimeFrame.Ticks, historical=False, #todate=datetime.datetime.now(), fromdate=datetime.datetime(2018, 10, 1, 9, 30, 0), tz=pytz.timezone('US/Eastern'), backfill_from=data0, #sessionstart=datetime.time(9, 30, 0), #sessionend=datetime.time(16, 0, 0), ) cerebro.resampledata(data, timeframe=bt.TimeFrame.Minutes, compression=1)
Finally, I run cerebro as:
cerebro.broker = store.getbroker() cerebro.addstrategy(TestStrategy) cerebro.run(exactbars=1, runonce=False)
I have tried several combinations of parameters both, in
btfeed.GenericCSVData()but none of them work. So, any help will be appreciated.
Thanks in advance.