Problem with BT Signal and Sizer

cnimativ

Hi all,

I wrote two different scripts using the SMA strategy on the website front page with some slight modifications and ran into two problems.

Data from the same source loaded in different ways resulted in different portfolio value.
SMA CrossOver signal Sizer issue that is documented somewhat here

Here's my questions:

How do I inspect trade history in BT?
How do I apply the same signal across several different securities?
How do I pull the result/portfolio balance time series out from BT?

The following are the codes and their respective outputs:

import backtrader as bt
import backtrader.filters as btfilters
from datetime import datetime
import pandas_datareader.data as web

class SmaCross(bt.SignalStrategy): 
    # using bt.SignalStrategy instead of bt.Strategy to utilize signals
    params = (('pfast', 50), ('pslow', 200),)
    def __init__(self):
        sma1, sma2 = bt.ind.SMA(period=self.p.pfast), bt.ind.SMA(period=self.p.pslow)
        self.signal_add(bt.SIGNAL_LONG, bt.ind.CrossOver(sma1, sma2))

def runstrat():
    start = datetime(1950,1,1)
    end = datetime(2017,1,2)
    spx = web.DataReader("^GSPC", 'yahoo', start, end)
    data = bt.feeds.PandasData(dataname=spx)
    
    cerebro = bt.Cerebro()
    cerebro.adddata(data)
    cerebro.addstrategy(SmaCross)
    cerebro.broker.setcash(10000.0)
    cerebro.addsizer(bt.sizers.PercentSizer, percents=99)
    
    results = cerebro.run()
        
    cerebro.plot()
    strat = results[0]
        
if __name__ == '__main__':
    runstrat()

from datetime import datetime
import backtrader as bt
    
class SmaCross(bt.SignalStrategy):
    params = (('pfast', 50), ('pslow', 200),)
    def __init__(self):
        sma1, sma2 = bt.ind.SMA(period=self.p.pfast), bt.ind.SMA(period=self.p.pslow)
        self.signal_add(bt.SIGNAL_LONG, bt.ind.CrossOver(sma1, sma2))
    
cerebro = bt.Cerebro()
    
data = bt.feeds.YahooFinanceData(dataname='^GSPC', fromdate=datetime(1950, 1, 1),
                                     todate=datetime(2017, 1, 2))
cerebro.adddata(data)

cerebro.addstrategy(SmaCross)
cerebro.broker.setcash(10000.0)
cerebro.addsizer(bt.sizers.PercentSizer, percents=99)
cerebro.run()
cerebro.plot()

backtrader

@cnimativ said in Problem with BT Signal and Sizer:

Data from the same source loaded in different ways resulted in different portfolio value.

The final portfolio value in both cases is 767892.24 (from the chart). Some elaboration may be needed here.

SMA CrossOver signal Sizer issue that is documented somewhat here

Some extra elaboration here would also help.

How do I inspect trade history in BT?

Override notify_trade and store the notifications.
Or access the _trades attribute of the strategy, where the trades are kept in a list.

How do I apply the same signal across several different securities?

The signal in the example calculates SMAs over data0 (because nothing else is specified)
Specify to the SMA to which dataX pertains

If you are looking for something else it might be the _data parameter in SignalStrategy. See the reference: Docs - Strategy Reference

How do I pull the result/portfolio balance time series out from BT?

You need to add an analyzer. Choose yours: Docs - Analyzers Reference
You may also override notify_cashvalue(self, cash, value) and keep the values yourself.

backtrader is conceived as a plug-in tool to which each user plugs whatever he prefers, rather than having many always built-in analyzers running.

cnimativ

The final portfolio value in both cases is 767892.24 (from the chart). Some elaboration may be needed here.

One is 767892.34, the other one is 76892.24. Very minor differences, but given the same data set and same signal, it should be exactly the same.

Some extra elaboration here would also help.

The same issue that trades are not being executed when sizer uses previous day close to calculate # of shares in an order, sees a gap up next day open, determines there's not enough cash balance, and abandons the whole trade.

For now I am just setting sizer to 99% to make sure trade fills.

Override notify_trade and store the notifications.

Or access the _trades attribute of the strategy, where the trades are kept in a list.

How do I access _trades after backtesting? I've looked into cerebro.strats[0][0][0]._trades and it says AttributeError: type object 'SmaCross' has no attribute '_trades'.

The signal in the example calculates SMAs over data0 (because nothing else is specified)

Specify to the SMA to which dataX pertains

How do you specify/attach an SMA calculation over data A to a security data B, where A doesn't have to be the same as B?
When I look at SMA it reads,

    def __init__(self):
        # Before super to ensure mixins (right-hand side in subclassing)
        # can see the assignment operation and operate on the line
        self.lines[0] = Average(self.data, period=self.p.period)

        super(MovingAverageSimple, self).__init__()

so it appears to be that I can use the following to add sma signal for self.data0 this way?

sma1, sma2 = bt.ind.SMA(self.data0, period=self.p.pfast), bt.ind.SMA(self.data0, period=self.p.pslow)

However, when I look at strategy.py, it doesn't appear that I can specify which signal to attach to which data when I do signal_add(...)?

    def signal_add(self, sigtype, signal):
        self._signals[sigtype].append(signal)

Basically, what I am trying to do is to place trades for security B based on signal derived from security A. The other case is to close A, buy B based on -crossover signal in A and to buy A, close B on +crossover signal in A.

Thanks for the help.

cnimativ

Hacked together a solution. Here's my code. It's probably not the best way to use Backtrader. Instead of using self.signal_add(bt.SIGNAL_LONG, bt.ind.CrossOver(sma1, sma2)), I pulled out sma1 and sma2 and use them directly in next. Also, I've peaked ahead for next day open price to order_target_percent as I would've done in a manual rebalacning process and used try except to catch errors.

Please advise if there's a more 'backtrader' way of implementing the same thing.

import backtrader as bt
from datetime import datetime
import pandas_datareader.data as web
import matplotlib.pylab as pylab
pylab.rcParams['figure.figsize'] = 30, 20  # that's default image size for this interactive session
pylab.rcParams["font.size"] = "100"

class SmaCross2(bt.SignalStrategy): 
    params = (('pfast', 50), ('pslow', 200),)
    def __init__(self):
        self.sma1 = bt.ind.SMA(self.data0, period=self.p.pfast)
        self.sma2 = bt.ind.SMA(self.data0, period=self.p.pslow)
        
    def next(self):
        if self.sma1[0] >= self.sma2[0]: # SPY bull cross
            try:
                self.order = self.order_target_percent(data=self.data0,
                                                        target=1,
                                                        exectype=bt.Order.Limit,
                                                        price=self.data0.open[1])
                self.order = self.order_target_percent(data=self.data1,
                                                        target=0,
                                                        exectype=bt.Order.Limit,
                                                        price=self.data1.open[1])
            except IndexError:
                self.order = self.order_target_percent(data=self.data0,
                                                       target=1)
                self.order = self.order_target_percent(data=self.data1,
                                                       target=0)
        else:
            try:
                self.order = self.order_target_percent(data=self.data0,
                                                        target=0,
                                                        exectype=bt.Order.Limit,
                                                        price=self.data0.open[1])
                self.order = self.order_target_percent(data=self.data1,
                                                        target=1,
                                                        exectype=bt.Order.Limit,
                                                        price=self.data1.open[1])
            except IndexError:
                self.order = self.order_target_percent(data=self.data0,
                                                       target=0)
                self.order = self.order_target_percent(data=self.data1,
                                                       target=1)

def run_strat():
    start = datetime(2002,7,30)
    end = datetime(2017,1,2)
    spy = bt.feeds.PandasData(dataname=web.DataReader("SPY", 'yahoo', start, end), name='SPY')
    tlt = bt.feeds.PandasData(dataname=web.DataReader("TLT", 'yahoo', start, end), name='TLT')
    
    cerebro = bt.Cerebro()
    cerebro.adddata(spy)
    cerebro.adddata(tlt)
    cerebro.addstrategy(SmaCross2)
    cerebro.broker.setcash(10000.0)
    
    results = cerebro.run()
        
    cerebro.plot()
    strat = results[0]

if __name__ == '__main__':
    run_strat()

backtrader

One is 767892.34, the other one is 76892.24. Very minor differences, but given the same data set and same signal, it should be exactly the same.

Screen resolution difference was not appreciated. At the end of the day you don't necessarily have the same data. There is rounding, because Yahoo provides unadjusted close prices which have to be adjusted. How much pandas rounds is unknown. But you can control that with backtrader. Given the .10 difference, the rounding seems not be a determining factor.

See Docs - Data Feeds Reference for the parameters for YahooFinanceCSVData (or online)

The same issue that trades are not being executed when sizer uses previous day close to calculate # of shares in an order, sees a gap up next day open, determines there's not enough cash balance, and abandons the whole trade.

That's NOT an issue. Future prices are not known and the number of shares can only be calculated with a known price. You can use cheat-on-close to get matched on the same bar with the closing price.

How do I access _trades after backtesting? I've looked into cerebro.strats[0][0][0]._trades and it says AttributeError: type object 'SmaCross' has no attribute '_trades'.

See Docs - Cerebro and the section Execute the backtesting for the return value of cerebro.run. In a non-optimization case you get the strategy with: strategy = result_from_run[0]. The attribute _trades should be there.

Basically, what I am trying to do is to place trades for security B based on signal derived from security A. The other case is to close A, buy B based on -crossover signal in A and to buy A, close B on +crossover signal in A.

You have to pass the _data parameter to the addstrategy(MyStrategy, _data=XXX) to indicate what's the target of your data when operating with signals. If nothing is passed the 1st data in the system (aka self.data or self.data0) is the target.

See Docs - Strategy Reference and look for SignalStrategy and specifically for _data under Params. You can pass a data feed instance, an int, a string.

backtrader

Hacked together a solution. Here's my code. It's probably not the best way to use Backtrader. Instead of using self.signal_add(bt.SIGNAL_LONG, bt.ind.CrossOver(sma1, sma2)), I pulled out sma1 and sma2 and use them directly in next

If you don't use the signals, there is no need to subclass from class SmaCross2(bt.SignalStrategy), you can subclass directly from bt.Strategy.

if self.sma1[0] >= self.sma2[0]: # SPY bull cross

This is not a cross. One sma is greater than the other and this will happen for a long time. If you want a cross you can calculate it manually or use CrossOver and await it's indication. It will only deliver an indication when a cross happens.

cnimativ

@backtrader Good catch! I will find a better name for it. I think for the frequency and nature of the strategy, it works perfectly fine, especially when dividend/reinvestments are taken into account in these slow moving strategies.