Posts made by davidavr

@the-world Yes, it uses C# but it's a completely different system than BT so you can't compare the two simply on implementation language. It may have something to do with how they process historical data? (I'm guessing here). I'd have to try writing two identical algorithms in each to really understand how they compare and it's not something I've tried or plan to do.

@the-world LEAN is not as fast as BT in my experience but I personally don't think that's the most important characteristic of a backtesting environment. You'll end up spending more time figuring out how to implement, analyze and debug your strategy, so finding the tool that allows you to work productively is - to me - the most important quality of a backtesting platform.

I think LEAN's ease of use with Docker, integrated VS Code debugging and good documentation with a very active community are really attractive features, though.

@kjiessar In that analyzer, self.rets does not track trades, it tracks entries which is a list of position size, price, data name and proceeds. Additionally, the position already aggregates multiple executions for multiple trades in a given data name (in notify_order), so there isn't an issue here using datetime as a key. It simply gives you a time series record of all position metrics at each bar in the analysis for all data names.

Here is the full function for context:

def next(self):
    # super(Transactions, self).next()  # let dtkey update
    entries = []
    for i, dname in self._idnames:
        pos = self._positions.get(dname, None)
        if pos is not None:
            size, price = pos.size, pos.price
            if size:
                entries.append([size, price, i, dname, -size * price])

    if entries:
        self.rets[self.strategy.datetime.datetime()] = entries

    self._positions.clear()

@sarah_james You can use order_target_percent(data, target=0.05) to buy 5% of your current portfolio value. Then when you want to sell, you can use order_target_percent(data, target=0.0)

@jacksonx I would think you could create an Analyzer to do this. It has access to the strategy, and therefore all the lines within it, so you could calculate on each day your margin requirement and then at the end of the analysis, sum up the total margin cost. You would need to incorporate that into your net P&L, though.

They other approach would be a custom broker (derived from the standard BackBroker) that only overrides the commission calculations, but that might be more work as it doesn't appear that brokers were designed to be as easily extensible as some of the other classes.

@auth87 Yes, the pre_next method in Strategy will be called each time and you can simply call next from it. But you then need to check which of the data items in self.datas have data ready. Something like:

def pre_next(self):
    self.next()

def next(self):
    available = [d for d in self.datas if len(d)]
    # do something with available list...

@sky_aka41 I think you are pretty close. I tried this all in a strategy (not a custom indicator) and it seems to be working.

Note I changed the prob_change and probability expressions slightly.

class St(bt.Strategy):
    params = (("momLength", 34), ("maLength", 55), ("pLength", 13))

    def __init__(self) :
        self.momentum = bt.ind.Momentum(self.datas[0].close, period=self.p.momLength)
        self.accelerate = bt.ind.Momentum(self.momentum, period=self.p.momLength)
        self.prob_change = bt.ind.Momentum(self.datas[0].close, period=1) > 0
        self.probability = bt.ind.SumN(self.prob_change) / self.p.pLength 
        
        self.adjusted_close = self.datas[0].close + (self.momentum + self.accelerate*0.5 ) * self.probability
        
        self.lines.MaMA = bt.ind.SMA(self.adjusted_close, period=self.p.maLength)

@brunof I think Option 2 sounds pretty reasonable. It may seem a bit kludgy to replace the Open field with a different value, but I looked at the execute method in BackBroker and it's pretty involved so subclassing and overriding would be complicated. Note you could replace the Open with whatever you like: VWAP, the first tick 10 seconds after the open, etc. - whatever you think is a reasonable fill price.

@brunof This sounds to me like an issue with the source data bars, not with Backtrader. If you are truly aggregating tick data into bars, then a given tick would only be included in one bar. So if the close price of one bar is the same as the open price of the next bar, that would only occur if there were two separate ticks at the same price (which is quite common, btw). It would be interesting to look at the underlying tick data for some of your ten-minute bars and see if they are consistent with what you'd expect.

I think Backtrader's assumption of filling at the open of the next bar is as good of an assumption as you can make. If indeed, there were two ticks at the same level that crossed the boundary of a bar, then you would expect to get filled at the same level as the previous bar's close.

If you want a much more realistic simulation of execution behavior on an exchange, I think you'd ultimately need to use tick data.

@jialeen You can use order_target_percent(data, target=1) to have BT use 100% of the portfolio value for the size of the order.

@chricco94 That error is coming from your use of EMA. You need to name the period parameter so use

esa = bt.ind.EMA(ap, period=n1)

instead of

esa = bt.ind.EMA(ap, n1)

BT makes heavy use of meta classes which can be confusing at times.

There are a couple of other issues in your code:

• There is no CrossUnder indicator in BT
• You aren't storing the longCondition and shortCondition indicators in self so you cannot access them in next()

@carameldragon I don't think this is an error, it's just a little confusing. The cost of an order doesn't change when you sell it. If you buy something for $100 and then sell it for $120, the cost is still $100 and the P&L is +$20. BT captures the P&L in the trade object (with and without commissions).

There is no proceeds tracked in an Order which is what I think you are after. You can get that with order.executed.price * order.executed.size for gross proceeds, and then factor in commission to get net proceeds.

The value field in Order is a bit ambiguous and I think leads to confusion. It would have been better named cost, IMO.

@dizzy0ny I think you'd need a custom feed derived from PandasData like in my example with all the additional "lines". But I supposed you could create one that just dynamically added the lines based on the columns in your DataFrame. But doing it explicitly isn't that complicated, as my example shows.

As for the vector vs. event-based backtesting, I think it's probably true that a vector-based approach is more powerful in some ways and bound to be a lot faster, but I think there is some logic that is easier to express in an event-driven approach which might lead to fewer mistakes (although I'm speculating a bit). The event-driven approach is also "safer" in that you can't cheat and accidentally look at future values. Finally, Backtrader makes is pretty straightforward to switch from backtesting to live trading. That might be more challenging with a vector-based system.

BTW, here's a vector-based Python backtesting project I found that looks interesting: vectorbt

@dizzy0ny There is the PandasData class for reading data feeds from Pandas but it really translates the DataFrame to a Backtrader line so you are still working with BT lines and indicators in your strategy. Note that BT indicators support some mathematical operations, like below to compute daily percent change (analogous to Pandas .pct_change() function).

self.rets = (self.datas[0].close / self.datas[0].close(-1) - 1)

But BT's math support is not nearly as rich as Pandas. However you could calculate all your indicators in Pandas and add them as additional columns in your DataFrame and load them into a custom feed. Something like this:

class CustomPandasFeed(bt.feeds.PandasData):
    lines = ('rtn1d', 'vol1m',)
    params = (
            ('datetime', 'Date'),
            ('open', 'Open'),
            ('high', 'High'),
            ('low', 'Low'),
            ('close', 'Close'),
            ('volume', 'Volume'),
            ('openinterest', None),
            ('adj_close', 'Adj Close'),
            ('rtn1d', 'rtn1d'),
            ('vol1m', 'vol1m'),
    )
...
        df = pd.read_csv(datapath, parse_dates=['Date'])
        df['rtn1d'] = df.Close.pct_change(1)
        df['vol1m'] = df.rtn1d.rolling(21).std() * (252 ** 0.5)
        df = df.dropna(axis=0)
        data = CustomPandasFeed(dataname=df)
...
        cerebro.adddata(data)

Then those additional fields are accessible in your strategy as another line (i.e. self.datas[0].rtn1d and sell.datas[0].vol1m)

@catvin When you create expressions using lines and indicators, you need to use operations that Backtrader understands for those operations. It supports the basic arithmetic operators like +, -, *, / but doesn't know about math.log. But there is a helper indicator called ApplyN that can apply a function across a line to create a new indicator, which fits your case nicely. The only caveat is that ApplyN generally works with a list of values over a period, not a scalar, so below I've used a lambda to just pass the one value in that period to the math.log function:

self.lines.x = bt.ind.ApplyN(self.close, func=lambda c: math.log(c[0]), period=1)

If you look at the implementation of ApplyN here, (which derives from BaseApplyN, which itself derives from OperationN). You can see that it simply just calls the function in next like you were originally doing!

    def next(self):
        self.line[0] = self.func(self.data.get(size=self.p.period))

So ApplyN is really just syntactic sugar for your original implementation. But I think it's more readable.

@tai Yes, but MaxN doesn't take an ago arg but you can just pass a delayed version of close like this:

max_close = bt.ind.MaxN(self.data.close(-2), period=self.p.num_bars)

@catvin I think you can just define the full value of v in the init method as this:

self.lines.v = (data.lines.x - data.lines.x(-5)) / 5

At least if I'm understanding what you are doing correctly.

@fanpeix I'm not quite sure what you mean by getting the quantile of an indicator. A quantile would apply to a series of data values, so you could get the last N values of the indicator and take the quantile of that, like below where I'm using NumPy's quantile function.

vals = self.sma.get(size=10)
quantiles = np.quantile(vals, [0.25, 0.5, 0.75], interpolation='linear')