lines declaration is a string, not a tuple (your are missing the trailing comma). Change it to
lines = ('numtrd',)
@hfrog713 It's hard to really provide any helpful information without knowing what "different results" means. Are you getting different EMA values? Are you getting different buy/sell signals from the crossover indicator? Are the signals the same but they result in different orders?
Additionally, without the full script and the data file, it's going to be hard to try it out locally. I ran your script against some AAPL prices and it just generates one sell signal during the period of test. Is that right or wrong? Is it consistent with what TradeView would show? It's hard for me to know.
@li-mike I don't think BT supports "boxed" positions (simultaneous long and short in the same security) but I would think you could fake it by adding the same feed for the security twice and just treating one of them as the long and the other as the short.
@kjiessar You are always receiving data in "bars". A bar is a summary of all the trades over a certain time interval. Each interval will have its own Open, High, Low and Close. You receive the bar in
next() just after the interval is complete so the Close is never in the future, it always just happened. The Open will be further in the past by the interval of time of that bar.
Typically if you are using daily data, then
next() is being called at the end of the trading day. If you place an order, the first opportunity for it to get filled is at the next days's Open price.
However, the most confusing issue still stands: the RSI indicator, which I initialize but do NOT use (intentionally, at least), is somehow necessary for the code to run (if you comment it out on your code, the code breaks).
Here is the issue
self.interval_low = min(self.data.low.get(size=3))
next you are trying to access the last 3 low values but in the first call to next, there is only one value in
self.data.low at this point. When you leave in the line for the RSI indicator, BT buffers the first 14 data points before calling
next so the RSI indicator will be available (whether you access it or not).
You could change the offending line to:
self.interval_low = min(self.data.low.get(size=min(3, len(self.data.low))))
or just create some indicator with a minimum period of 3 to work around this.
@dizzy0ny I think you'd need a custom feed derived from
PandasData like in my example with all the additional "lines". But I supposed you could create one that just dynamically added the lines based on the columns in your DataFrame. But doing it explicitly isn't that complicated, as my example shows.
As for the vector vs. event-based backtesting, I think it's probably true that a vector-based approach is more powerful in some ways and bound to be a lot faster, but I think there is some logic that is easier to express in an event-driven approach which might lead to fewer mistakes (although I'm speculating a bit). The event-driven approach is also "safer" in that you can't cheat and accidentally look at future values. Finally, Backtrader makes is pretty straightforward to switch from backtesting to live trading. That might be more challenging with a vector-based system.
BTW, here's a vector-based Python backtesting project I found that looks interesting: vectorbt
@new_trader Here is how limit orders work:
Hope that helps.
@punajunior It looks like the date format in your file is
"%m/%d/%Y" but you specified
"%Y-%m-%d" in your script. Try changing the
dtformat argument of
@catvin When you create expressions using lines and indicators, you need to use operations that Backtrader understands for those operations. It supports the basic arithmetic operators like
+, -, *, / but doesn't know about
math.log. But there is a helper indicator called
ApplyN that can apply a function across a line to create a new indicator, which fits your case nicely. The only caveat is that
ApplyN generally works with a list of values over a period, not a scalar, so below I've used a
lambda to just pass the one value in that period to the
self.lines.x = bt.ind.ApplyN(self.close, func=lambda c: math.log(c), period=1)
If you look at the implementation of
ApplyN here, (which derives from
BaseApplyN, which itself derives from
OperationN). You can see that it simply just calls the function in
next like you were originally doing!
def next(self): self.line = self.func(self.data.get(size=self.p.period))
ApplyN is really just syntactic sugar for your original implementation. But I think it's more readable.
@kuky I believe BT assumes the timestamp of a bar is for the end of that time interval. Thus at the simulator time of 22:30, you are seeing the 5m bar that ended at 22:30 (i.e. 22:25-22:30) and the 1h bar that ended at 22:00 (which is the last complete 1h bar at this point).
@alechter I started where you are at just a few weeks ago: comfortable Python developer and familiar with Pandas/numpy. I evaluated a number of backtesting platforms (zipline, QuantConnect/LEAN, QuantRocket, PyInvesting) and settled on Backtrader.
I think Backtrader is pretty comprehensive, has a good design with a lot of useful functionality built in but also lots of extension points to customize where you need to (custom feeds, indicators, analyzers, observers). The documentation is pretty good, although not as well organized as it could be (I've picked up useful nuggets from the Articles archive, for example, that weren't easy to find in the standard documentation). The code is also pretty easy to follow so I'll read through it when I want to better understand how something works.
And the community is really good. Many of my questions have been answered by simply searching this forum and reading through past responses. And I'm learning enough to even post the occasional answer to other people's questions now, too!
I'd encourage you to stick with it and fight through a real (but not overly complicated) example to force you to learn how all the pieces fit together and you'll probably find it makes a lot more sense at the end of that exercise.
If you decide then that it's not for you, a lot of what you've learned will apply to other backtesting environments. Many of them share the same concepts (strategies, indicators, analyzers, data feeds). Good luck!
@carameldragon I don't think this is an error, it's just a little confusing. The cost of an order doesn't change when you sell it. If you buy something for $100 and then sell it for $120, the cost is still $100 and the P&L is +$20. BT captures the P&L in the trade object (with and without commissions).
There is no proceeds tracked in an Order which is what I think you are after. You can get that with
order.executed.price * order.executed.size for gross proceeds, and then factor in commission to get net proceeds.
value field in Order is a bit ambiguous and I think leads to confusion. It would have been better named
@auth87 Yes, the
pre_next method in
Strategy will be called each time and you can simply call
next from it. But you then need to check which of the data items in
self.datas have data ready. Something like:
def pre_next(self): self.next() def next(self): available = [d for d in self.datas if len(d)] # do something with available list...
@dizzy0ny There is the
PandasData class for reading data feeds from Pandas but it really translates the DataFrame to a Backtrader line so you are still working with BT lines and indicators in your strategy. Note that BT indicators support some mathematical operations, like below to compute daily percent change (analogous to Pandas
self.rets = (self.datas.close / self.datas.close(-1) - 1)
But BT's math support is not nearly as rich as Pandas. However you could calculate all your indicators in Pandas and add them as additional columns in your DataFrame and load them into a custom feed. Something like this:
class CustomPandasFeed(bt.feeds.PandasData): lines = ('rtn1d', 'vol1m',) params = ( ('datetime', 'Date'), ('open', 'Open'), ('high', 'High'), ('low', 'Low'), ('close', 'Close'), ('volume', 'Volume'), ('openinterest', None), ('adj_close', 'Adj Close'), ('rtn1d', 'rtn1d'), ('vol1m', 'vol1m'), ) ... df = pd.read_csv(datapath, parse_dates=['Date']) df['rtn1d'] = df.Close.pct_change(1) df['vol1m'] = df.rtn1d.rolling(21).std() * (252 ** 0.5) df = df.dropna(axis=0) data = CustomPandasFeed(dataname=df) ... cerebro.adddata(data)
Then those additional fields are accessible in your strategy as another line (i.e.
@punajunior My guess is that this error is thrown when it's trying to parse the "Open" field which defaults to column 1, so after parsing the datetime correctly, it's then trying to parse that same field as a float. Try adding explicit column indicators for all fields in your call to GenericCSVData:
BTW, it looks like you created the CSV file using Pandas
.to_csv() method without the
index=False parameter. That's why it has that unnamed first column. If you can recreate the file without the index, this probably would have just worked.
@new_trader I'm not sure what your question is, but it seems that BT is doing what I'd expect:
But I don't understand whey you are adding 100 to the prices at the top:
p1 = 100+d.close *0.99. You will always be way off the market with that approach.
@atmps Sorry - what I meant in the last part is that you'll need to extend PandasData like so:
class PandasDataRev(bt.feeds.PandasData): lines=('rev',) params=( ('rev', None), )
Then you can use it like this:
for tkr, grp in df.groupby('ticker'): data = PandasDataRev( dataname=grp, datetime='date', close='close', rev='rev', open=None, high=None, low=None, volume=None, openinterest=None, ) cerebro.adddata(data, name=tkr)
You will need to figure out how to reshape your data from how you have it in Excel to a form where it's easy to iterate through it ticker by ticker. There are many ways you can do that. You could load it in Pandas and "unpivot" the data or just use it in the form you have but the indexing will be different.
And to your last question, you would access each dataset as
n is an integer for the n-th ticker. (You are loading one data feed for each ticker so you have n feeds now).
@new_trader You could just wait until the price reaches your threshold in
next() before you place an order. Right now you are placing an order regardless of the current price.
But what you are describing is a very odd strategy: you want to wait until the price is much higher before you buy it. Most strategies boil down to "buy low, sell high" in one form or another (but not necessarily in that order).
@atmps While I'm new to BT, here are my thoughts on your questions:
a/b) I would just load the whole file in a Pandas DataFrame and then add each subgroup. Say your file has
date, ticker,close,rev fields, the code might look like this (there might be mistakes in the example!):
df = pd.read_csv(filename, parse_dates=['date']) for tkr, grp in df.groupby('ticker'): data = bt.feeds.PandasData( dataname=grp, datetime='date', close='close', open=None, high=None, low=None, volume=None, openinterest=None, plot=False, ) cerebro.adddata(data, name=tkr)
c) no. BT handles data of different time resolutions fine.
d) I think this is covered by loading all the tickers as in (a) if I understand the question
As for handling an additional custom field,
rev, there is a good example in the docs where you extend GenericCSVData to add a
pe line: https://www.backtrader.com/docu/extending-a-datafeed/
In my example above, I would just extend PandasData rather than GenericCSVData to add the
rev line and then add it as
rev='rev' when creating it.