General Discussion

• N

  Backtest vs live bars "off-by-1" discrepancy?
  • nooby_mcnoob

  18
  0
  Votes
  18
  Posts
  1675
  Views

  Z

  I bumped into similar situation. Looks like by default, backtrader is not touching DELAYED data and therefore inherits whatever "rightedge" convention from the original data feed, and in case of ibstore, it is "rightedge=False"; while for LIVE data part it will use the default convention of "rightedge=True", which causes the discrepancy. Many thanks for the tip from @dasch, now the bars match backteset/live trading.
• K

  Position analyzer with entry/exit details
  • kausality

  5
  1
  Votes
  5
  Posts
  403
  Views

  K

  Awesome work there! I don't need to add anything. I am currently doing some live bot like trading and therefore I though adding a continuous dump feature would be helpful. This little modification also allows one to dump the position details in a CSV file class trade_list(bt.Analyzer): params = ( ("continuous", False), ("fname", "position_{0}.csv".format(time.strftime("%Y%m%d-%H%M%S"))), ("dump", True) ) def __init__(self): self.trades = [] self.cumprofit = 0.0 def get_analysis(self): return self.trades def _write_to_file(self): if not self.p.dump: return df = pd.DataFrame(self.trades) df.to_csv(self.p.fname, index=False) def notify_trade(self, trade): if trade.isclosed: brokervalue = self.strategy.broker.getvalue() dir = "short" if trade.history[0].event.size > 0: dir = "long" pricein = trade.history[len(trade.history)-1].status.price priceout = trade.history[len(trade.history)-1].event.price datein = bt.num2date(trade.history[0].status.dt) dateout = bt.num2date(trade.history[len(trade.history)-1].status.dt) if trade.data._timeframe >= bt.TimeFrame.Days: datein = datein.date() dateout = dateout.date() pcntchange = 100 * priceout / pricein - 100 pnl = trade.history[len(trade.history)-1].status.pnlcomm pnlpcnt = 100 * pnl / brokervalue barlen = trade.history[len(trade.history)-1].status.barlen pbar = pnl / barlen self.cumprofit += pnl size = value = 0.0 for record in trade.history: if abs(size) < abs(record.status.size): size = record.status.size value = record.status.value highest_in_trade = max(trade.data.high.get(ago=0, size=barlen+1)) lowest_in_trade = min(trade.data.low.get(ago=0, size=barlen+1)) hp = 100 * (highest_in_trade - pricein) / pricein lp = 100 * (lowest_in_trade - pricein) / pricein if dir == "long": mfe = hp mae = lp else: mfe = -lp mae = -hp self.trades.append({"ref": trade.ref, "ticker": trade.data._name, "dir": dir, "datein": datein, "pricein": pricein, "dateout": dateout, "priceout": priceout, "chng%": round(pcntchange, 2), "pnl": pnl, "pnl%": round(pnlpcnt, 2), "size": size, "value": value, "cumpnl": self.cumprofit, "nbars": barlen, "pnl/bar": round(pbar, 2), "mfe%": round(mfe, 2), "mae%": round(mae, 2)}) if self.p.continuous: self._write_to_file() def stop(self): if not self.p.continuous: self._write_to_file()
• T

  Multiple Live Data Feeds
  • TOTORO

  8
  0
  Votes
  8
  Posts
  677
  Views

  K

  @totoro It depends on what you are doing inside the loop. If you are making some calculations like std devs, atr etc then I don't think that even with 100's of data feed your lag could reach 10s. If you are running some neural network based complex calculations then it can though. You can use something like the time command to get an idea about your execution time.
• K

  Closing trade on the same day
  • kausality

  6
  0
  Votes
  6
  Posts
  483
  Views

  K

  EDIT: Post created by mistake. As I cannot delete it, I have edited it out.
• 

  initializing resampled/replayed data
  • bb2

  3
  0
  Votes
  3
  Posts
  224
  Views

  

  @backtrader thank you for the suggestion. Though would that still work If I need to replay multiple timeframes? I was thinking of resampling my low tf data dataframe with pandas to higher timeframe and use it for backfilling source. (if I got you right above). If I have multiple replays, then I will do the pandas resampling multiple times, but I think concatenating and backfilling wouldn't be the right approach, because they have different compressions on bt PandasData level. What would you suggest for multiple replays?
• K

  speed market replay tool for stocks to acquire more experience?
  • katie matriana

  1
  0
  Votes
  1
  Posts
  124
  Views

  No one has replied

• V

  Running Multiple Strategies & Combining the Output
  • Vandan Chopra

  5
  0
  Votes
  5
  Posts
  495
  Views

  V

  @backtrader Thanks a ton for that. It does sort out quite a bit for me. Also gives me confidence in you saying that I wouldn't need spark for something like this and that python by itself should be able to handle the load. (Anyways, it's going to be a fairly small system to begin with). PS. Looks like i'm going to have to reach out to you again in the future. But SUPER appreciate your response (tipping my hat :)
• D

  Using multiple strategies runs only one?
  • duality_

  17
  0
  Votes
  17
  Posts
  1892
  Views

  B

  @vandan-chopra Sorry, but seeing your other posts, this is not what you are looking for, because this is about selecting a strategy based on its results and not about coordinating strategies.
• A

  Small question on the Documentation for Indicator Reference
  • Amien Johaadien

  2
  0
  Votes
  2
  Posts
  208
  Views

  B

  Indeed. Will be corrected.
• A

  Benchmarking & Actual Cash/Interest Rates over longer timeperiods
  • Amien Johaadien

  2
  0
  Votes
  2
  Posts
  184
  Views

  B

  @amien-johaadien said in Benchmarking & Actual Cash/Interest Rates over longer timeperiods: My thought was to build a synthetic cash index and load it as a datafeed and use that as a "Benchmark" using the observer method. Would that work and is there a better way of doing it? The Observer is only for visual representation. The functionality is actually in the Analyzer. But yes, that would be the right approach (imho) @amien-johaadien said in Benchmarking & Actual Cash/Interest Rates over longer timeperiods: Also when using optstrategy - can I still use benchmarks? Yes (unless you happen to trigger one of the pickling problems)
• 

  Bokeh Integration - Interactive Webbrowser Plotting
  • vbs

  32
  8
  Votes
  32
  Posts
  4608
  Views

  Z

  @vbs Hi, thanks for the response. Good news: I upgraded Bokeh to 0.13 and tried your minimised example with some yahoo csv daily data, and it did work in a notebook environment! Bad news: my own strategy code still didn't work : ( There's some new error information from the notebook though, please refer to the bottom of below message. Any guess of the issue? Is there something wrong with 2 resampled data? I am using minute data (data0) and two resampled datas 15-minutes, 60 minutes. Could this have caused some mess? /Users/kevin/anaconda2/envs/py36/lib/python3.6/site-packages/bokeh/models/sources.py:110: BokehUserWarning: ColumnDataSource's columns must be of the same length. Current lengths: ('4440411776close', 8475), ('4440411776colors_bars', 8475), ('4440411776colors_outline', 8475), ('4440411776colors_wicks', 8475), ('4440411776high', 8475), ('4440411776low', 8475), ('4440411776open', 8475), ('4440411776volume', 8475), ('4440411776volume_colors', 8475), ('4440443984', 8474), ('datetime', 8475), ('index', 8475) /Users/kevin/anaconda2/envs/py36/lib/python3.6/site-packages/bokeh/models/sources.py:110: BokehUserWarning: ColumnDataSource's columns must be of the same length. Current lengths: ('4440411776close', 8475), ('4440411776colors_bars', 8475), ('4440411776colors_outline', 8475), ('4440411776colors_wicks', 8475), ('4440411776high', 8475), ('4440411776low', 8475), ('4440411776open', 8475), ('4440411776volume', 8475), ('4440411776volume_colors', 8475), ('4440443984', 8474), ('4440444096', 8474), ('datetime', 8475), ('index', 8475) Exception Traceback (most recent call last) <ipython-input-5-fb7d00ce15b9> in <module>() 21 args.plot=True 22 ---> 23 es.runstrat(args) ~/Dropbox/quantiacs/InteractiveBroker/PyBackTrader/ES_bracketXover.py in runstrat(args) 251 #cerebro.plot(style='candle',iplot=False,volume=False) 252 b = Bokeh(style='bar', plot_mode='single', )# scheme=Tradimo()) --> 253 cerebro.plot(b) 254 255 def analyseDefaultArguments(args): ~/anaconda2/envs/py36/lib/python3.6/site-packages/backtrader/cerebro.py in plot(self, plotter, numfigs, iplot, start, end, width, height, dpi, tight, use, **kwargs) 989 rfig = plotter.plot(strat, figid=si * 100, 990 numfigs=numfigs, iplot=iplot, --> 991 start=start, end=end, use=use) 992 # pfillers=pfillers2) 993 ~/anaconda2/envs/py36/lib/python3.6/site-packages/backtrader_plotting/bokeh/bokeh.py in plot(self, obj, figid, numfigs, iplot, start, end, use, **kwargs) 402 403 if isinstance(obj, bt.Strategy): --> 404 self._blueprint_strategy(obj, start, end, **kwargs) 405 elif isinstance(obj, bt.OptReturn): 406 if not hasattr(obj, 'strategycls'): ~/anaconda2/envs/py36/lib/python3.6/site-packages/backtrader_plotting/bokeh/bokeh.py in _blueprint_strategy(self, strategy, start, end, **kwargs) 243 strat_figures.append(bf) 244 --> 245 bf.plot(master, strat_clk, None) 246 247 for s in slaves: ~/anaconda2/envs/py36/lib/python3.6/site-packages/backtrader_plotting/bokeh/figure.py in plot(self, obj, strat_clk, master) 164 height_set = self._scheme.plot_height_observer 165 else: --> 166 raise Exception(f"Unsupported plot object: {type(obj)}") 167 168 self.datas.append(obj) Exception: Unsupported plot object: <class 'backtrader.feed.DataClone'>
• K

  How to determine if the data is being replayed in strategy?
  • kausality

  2
  0
  Votes
  2
  Posts
  212
  Views

  B

  @kausality said in How to determine if the data is being replayed in strategy?: From what I gathered from the docs, the len(self.data) will remain the same if the data is being sent to next() at the original(1-minute) timeframe. It will change when it receives the 5-minute(replayed) candle. Your 5-minutes replayed bar is constantly delivered. Only the 1st tick will contain an original 1-minute bar. The other 4 ticks will mix the incoming values to replicate (replay) how the final 5-minutes bar is constructed. @kausality said in How to determine if the data is being replayed in strategy?: So in this case, how will I determine within next() that the current data value(say: self.data.close[0]) refers to its original 1-minute value or the resampled 5-minute ohlc data? You cannot, because you cannot also guarantee that for each replayed 5-minutes bar you have 5 underlying 1-minute bars. If your dataset is perfect and the time boundaries too, you could count the 5 ticks that each replayed bar generates. @kausality said in How to determine if the data is being replayed in strategy?: How do I make this distinction in the code? You cannot also when trading live, because you only know that a bar is closed because you enter the next one. The only feasible approach: Add a 2nd resampled feed from the original data. This bar will be delivered only once when complete, before the 1st tick of the next replayed data comes in.
• 

  Purpose of adding multiple Strategies
  • vbs

  11
  0
  Votes
  11
  Posts
  1509
  Views

  V

  @backtrader Got it. Thanks a ton for that. I didn't understand it all, but it gives me enough to start researching / learning/ building. :)
• 

  Is possible send a signal before a candle (1 minute) being closed feeding backtrader with tikcs?
  • dedeco

  2
  0
  Votes
  2
  Posts
  165
  Views

  B

  You rp @dedeco said in Is possible send a signal before a candle (1 minute) being closed feeding backtrader with tikcs?: noticed that the minimum GRANULARITY is to fetch on OHLC are minutes There is NOT such a limitation in backtrader. If a given data provieder attached to backtrader doesn't support a smaller granularity it's another story. @dedeco said in Is possible send a signal before a candle (1 minute) being closed feeding backtrader with tikcs?: But supposing the I need to get live data before the 1m candle being closed to send a signal. Is possible? You are looking for replaydata. See: Docs - Cerebro Docs - Data Replay This allows you see how a 1-minute replayed data is constructed (tick-by-tick) from a smaller timeframe. But your data feed @dedeco said in Is possible send a signal before a candle (1 minute) being closed feeding backtrader with tikcs?: I need the historical (15 M) data too. So i will need to feeds: historical and live. You need to combine it with resampled data which will only tick once for each 15 minutes timeframe. Docs - Multiple Timeframes Docs - Data Resampling
• 

  Testing signal backtest with add_signal function
  • sirhc

  2
  0
  Votes
  2
  Posts
  278
  Views

  

  My bad... Everything is working as expected. My pandas dataframe was not correctly indexed. Thank you
• S

  how to disable plotting resample data
  • stentor

  6
  0
  Votes
  6
  Posts
  385
  Views

  B

  @ab_trader said in how to disable plotting resample data: data.plotinfo.plot = False As indicated by @ab_trader you don't have to put necessarily in the constructor, but if you wish pass it to resampledata as indicated by the documentation Docs - Cerebro Or receive the resampled data instance as the result of calling resampledata and do as told by @ab_trader
• H

  Running copy/pasted samples from documentation gives different results than those presented in the documentation
  documentation quickstart samples • • hadrien-toma

  5
  0
  Votes
  5
  Posts
  264
  Views

  H

  @backtrader Perfect, thank you very much!
• M

  Backtesting with tick data and ask/bid prices
  • malapija

  4
  0
  Votes
  4
  Posts
  611
  Views

  B

  @malapija said in Backtesting with tick data and ask/bid prices: Concrete (simplified) example: Let's say we have two assets: A1, A2 with bid/ask prices A1.b, A1.a, A2.b, A2.a. An arbitrage opportunity between A1 and A2 appears and we need to go long on A1, short on A2. Therefore we need to use A1.a for entering the long position and A2.b for entering the short position (assuming Market orders). The difference between bid and ask is quite important in this case because with arbitrage we're going for tiny market inefficiencies. Your problem is that you are expecting an execution which is not part of backtrader. Backtrader works on the principle that a price which has been seen cannot used for matching. If you have seen A1.a and A2.b, the prices are gone. Your Market orders will be executed with the next incoming prices. In your case the cheat-on-close mode won't help, because for A2.b you would need to be matched with for example the current open price for A2 which is where you would code that information (A1.a would be encoded in the close price of A1) As stated at the beginning: backtrader was not conceived for bid/ask execution. @malapija said in Backtesting with tick data and ask/bid prices: My current understanding from what I tried with backtrader and from what you wrote is that this probably won't be possible, right? I also explored all other Python backtesting frameworks you mention in Github README and it seems that none of them will be easily applicable to my use case although I need to explore QSTrader and QTPyLib in more depth I don't think they are meant for bid/ask execution either, but the platforms (except for 2 or 3) were not explored in full detail.
• L

  General question related to memory usage and time range
  • Lucas Condé

  4
  0
  Votes
  4
  Posts
  299
  Views

  B

  @lucas-condé said in General question related to memory usage and time range: Would 1 month of data be enough for that? If your month was a melt-up month like the SP&500 has has in the last few years, your strategy results would probably be biased. I would go for different 1-monht scenarios (or 1-week) to make sure that the scalping works in each and every situation. It is also not the same to try scalping with a bank of $1M as to try scalping with a bank of $10K (the latter needs a lot more points to generate the absolute returns you need as a human being) @lucas-condé said in General question related to memory usage and time range: I'm also thinking of running it on 5m and 15m timeframe aswell, so maybe I would need to go for 2 months and 4 months? Same as above. It entirely depends also on how often your strategy trades.
• B

  Issue with date2num
  • BobDenar

  3
  0
  Votes
  3
  Posts
  255
  Views

  B

  On timers with tzdata parameters to trigger timers on market place's timezone. Our historicals datas are stored into a MongoDB through Arctic library, since ticks and bars are stored UTC, when building a feed I add tz parameter to point to the proper timezone. I guess that those uses of timezones pinpointed me to this issue.