Multi-Data Long & Short

Mango Loco

Hi there,

For a few months now I have been piecing together different parts of a framework with backtrader to handle multiple datas and be able to trade both long and short. However, I keep running into an error. Firstly, I will summarise what the code should achieve, post the code, and then post the error. Any help in helping me get this strategy off the ground would be much appreciated!

Target:
For each data (1D timeframe) in the trading universe:
-> Is it a long environment or a short environment?

If long environment and not in trade already:
Does it meet conditions to take a long?
-> If yes, order with or without tp
-> If no, do nothing

If long environment and in a trade:
Does it meet condition to close trade?
-> If yes, close trade
-> If no, leave trade open / do nothing

Exactly vice versa for the short side logic.

Here is the code I currently have to try to achieve the above:
note: BackTrader is the folder where I keep my own creations, such as indicators.

import backtrader as bt
import math
from BackTrader import indicators

class NHL(bt.Strategy):
    params = (
        ('use_tp', True),
        ('tp_percent', 12),
        ('ma_fast_len', 10),
        ('ma_slow_len', 100),
        ('ind1_period', 80)
    )

    def log(self, txt, dt=None):
        ''' Logging function for this strategy'''
        dt = dt or self.datas[0].datetime.date(0)
        print('%s, %s' % (dt.isoformat(), txt))

    def __init__(self):
        self.name = 'Multi_Long_Short'

        self.inds = dict()
        self.trades = dict()
        self.orders = dict()

        self.cash = self.broker.get_value()
        self.risk = 0.01
        self.onePlot = False

        for i, d in enumerate(self.datas):
            plot_ind1 = True
            plot_vstop1 = True
            plot_vstop2 = False

            self.inds[d] = dict()
            self.inds[d]['ind1'] = indicators.Ind1(d, plot=plot_ind1, n=self.p.ind1_period)
            self.inds[d]['vstop1'] = indicators.Vol_Stop(d, mult=1, plot=plot_vstop1)
            self.inds[d]['vstop2'] = indicators.Vol_Stop(d, mult=2, plot=plot_vstop2)

            use_sma = False
            if use_sma:
                self.inds[d]['fast_ma'] = bt.indicators.SMA(d, period=self.p.ma_fast_len)
                self.inds[d]['slow_ma'] = bt.indicators.SMA(d, period=self.p.ma_slow_len)
            else:
                self.inds[d]['fast_ma'] = bt.indicators.ExponentialMovingAverage(d, period=self.p.ma_fast_len)
                self.inds[d]['slow_ma'] = bt.indicators.ExponentialMovingAverage(d, period=self.p.ma_slow_len)

            if i > 0:
                if self.onePlot:
                    d.plotinfo.plotmaster = self.datas[0]

    def notify_order(self, order):
        order_name = order.info.name
        d = order.data
        data_name = d._name
        order_price = order.executed.price

        if order.status in [order.Submitted, order.Accepted]:
            return

        if order.status in [order.Completed]:
            self.log('%s, %s, %s, Price: %.5f' % (data_name, self.name, order_name, order_price))

        elif order.status in [order.Canceled]:
            # if not order.info.name in ['Long SL', 'Short SL', 'Long Entry', 'Short Entry', 'Long TP', 'Short TP']:
                self.log('Order Canceled')
        elif order.status in [order.Margin, order.Rejected]:
            self.log('Order Margin/Rejected')

        if not order.alive():
            orders = self.orders[d]
            idx = orders.index(order)
            orders[idx] = None
            if all(x is None for x in orders):
                orders[:] = []

    def notify_trade(self, trade):
        if not trade.isclosed:
            return

        self.log('OPERATION PROFIT, GROSS %.2f, NET %.2f, PORTFOLIO %.2f' %
                 (trade.pnl, trade.pnlcomm, self.broker.get_value()))

    def next(self):
        for i, d in enumerate(self.datas):
            name = d._name
            close = d.close[0]
            close1 = d.close[-1]
            high = d.high[0]
            low = d.low[0]
            self.log('{}: {}'.format(name, close))

            ind1_1 = self.inds[d]['ind1'].lines.ind1_1[0]
            ind1_2 = self.inds[d]['ind1'].lines.ind1_2[0]
            ma_fast = self.inds[d]['fast_ma'][0]
            ma_slow = self.inds[d]['slow_ma'][0]
            vstop1 = self.inds[d]['vstop1'][0]
            vstop2 = self.inds[d]['vstop2'][0]

            long_environment = ma_fast > ma_slow
            short_environment = ma_fast < ma_slow
            go_long = close >= ind1_1
            go_short = close <= ind1_2
            close_long = close < ma_fast
            close_short = close > ma_fast

            position = self.getposition(d).size
            r_percent = self.p.tp_percent / 100

            if not position and not self.orders.get(d, None):
                if long_environment and go_long:
                    sl = vstop2 if (close > vstop2) else (low - (close*0.01))
                    qty = math.floor(self.cash * self.risk / abs(close - sl))
                    qty = qty if (qty > 0) else 1 # Just in case
                    tp = close + (close*r_percent)

                    if self.p.use_tp:
                        # Open Long with TP
                        o1 = self.buy(data=d, price=close, size=qty, exectype=bt.Order.Limit, transmit=False)
                        o2 = self.sell(data=d, price=sl, size=o1.size, exectype=bt.Order.Stop, parent=o1, transmit=False)
                        o3 = self.sell(data=d, price=tp, size=o1.size, exectype=bt.Order.Limit, parent=o1, transmit=True)
                        o1.addinfo(name='Long Entry')
                        o2.addinfo(name='Long SL')
                        o3.addinfo(name='Long TP')
                        self.orders[d] = [o1, o2, o3]
                    else:
                        # Open Long without TP
                        o1 = self.buy(data=d, price=close, size=qty, exectype=bt.Order.Limit, transmit=False)
                        o2 = self.sell(data=d, price=sl, size=o1.size, exectype=bt.Order.Stop, parent=o1, transmit=True)
                        o1.addinfo(name='Long Entry')
                        o2.addinfo(name='Long SL')
                        self.orders[d] = [o1, o2]


                elif short_environment and go_short:
                    sl = vstop2 if (close < vstop2) else high + (close*0.01)
                    qty = math.floor(self.cash * self.risk / abs(close - sl))
                    qty = qty if (qty > 0) else 1 # Just in case
                    tp = close - (close*r_percent)

                    if self.p.use_tp:
                        # Open Short with TP
                        o1 = self.sell(data=d, price=close, size=qty, exectype=bt.Order.Limit, transmit=False)
                        o2 = self.buy(data=d, price=sl, size=o1.size, exectype=bt.Order.Stop, parent=o1, transmit=False)
                        o3 = self.buy(data=d, price=tp, size=o1.size, exectype=bt.Order.Limit, parent=o1, transmit=True)
                        o1.addinfo(name='Short Entry')
                        o2.addinfo(name='Short SL')
                        o3.addinfo(name='Short TP')
                        self.orders[d] = [o1, o2, o3]
                    else:
                        # Open Short without TP
                        o1 = self.sell(data=d, price=close, size=qty, exectype=bt.Order.Limit, transmit=False)
                        o2 = self.buy(data=d, price=sl, size=o1.size, exectype=bt.Order.Stop, parent=o1, transmit=True)
                        o1.addinfo(name='Short Entry')
                        o2.addinfo(name='Short SL')
                        self.orders[d] = [o1, o2]

            elif position:
                if (position > 0) and close_long:
                    # Close Long
                    long_close = self.close(data=d)
                    if long_close:
                        long_close.addinfo(name='Close Long')
                        self.orders[d].append(long_close)
                        self.cancel(self.orders[d][1])

                elif (position < 0) and close_short:
                    # Close Short
                    short_close = self.close(data=d)
                    if short_close:
                        short_close.addinfo(name='Close Short')
                        self.orders[d].append(short_close)
                        self.cancel(self.orders[d][1])

The above has been pieced together mainly through docs, examples, forum posts, etc.

Here is an example of the error that I am hitting at the moment:

Traceback (most recent call last):
  File "/Users/.../Desktop/Python Projects/AlgoTrading/.../backtester.py", line 170, in <module>
    run_strat()
  File "/Users/.../Desktop/Python Projects/AlgoTrading/.../backtester.py", line 88, in run_strat
    run = cerebro.run(stdstats=False, tradehistory=True)
  File "/anaconda3/lib/python3.7/site-packages/backtrader/cerebro.py", line 1127, in run
    runstrat = self.runstrategies(iterstrat)
  File "/anaconda3/lib/python3.7/site-packages/backtrader/cerebro.py", line 1293, in runstrategies
    self._runonce(runstrats)
  File "/anaconda3/lib/python3.7/site-packages/backtrader/cerebro.py", line 1695, in _runonce
    strat._oncepost(dt0)
  File "/anaconda3/lib/python3.7/site-packages/backtrader/strategy.py", line 309, in _oncepost
    self.next()
  File "/Users/.../Desktop/Python Projects/AlgoTrading/.../LT_2.py", line 120, in next
    self.cancel(self.orders[d][1])
IndexError: list index out of range

I am struggling to see where it is going wrong in achieving what I have in mind, the ordering and "orderbook" are implemented almost identically to:
https://www.backtrader.com/blog/posts/2017-04-09-multi-example/multi-example/

I believe what I have is very close to being fully functional for my purposes - can anyone help me get past this error and/or achieve what I would like in-full?

Please let me know if I need to provide more code.

Thanks in advance!

backtrader

@Mango-Loco said in Multi-Data Long & Short:

        if not order.alive():
            orders = self.orders[d]
            idx = orders.index(order)
            orders[idx] = None
            if all(x is None for x in orders):
                orders[:] = []

You empty the list here.

And then try to access it here ...

@Mango-Loco said in Multi-Data Long & Short:

    self.cancel(self.orders[d][1])
IndexError: list index out of range

Mango Loco

@backtrader
Ok, I see that now. That is the same way you have shown it to be done in the link I pointed out?

Can you give me an example of how to cleanly handle ordering and orders for a multi-data, long and short strategy? Or at least point me in the direction of getting me closer to that.

Thanks in advance.

run-out

This is a working example of a long/short strategy with multiple datas.
****This was my first backtrader project so you'll excuse any crudities.

Essentially, I was calculating the daily long and shorts from a 5,000 stock universe, resulting in 50 long and 50 shorts a day. This part was using an external module and importing the long and short recommendations per day as a dictionary. Then, I would adjust the portfolio with buys and sells each day according to the dictionary.

I hope there's some useful reference. Again, this was my first project, so use with a grain of salt.

https://bit.ly/2kcMfSZ

Mango Loco

@run-out
Hey, thank you for that. I plan on transferring one of my factor models over soon so that will be very useful as a start/reference point!

To be honest, I think it's more trouble with opening/closing the bracket orders, and keeping track of those orders. I just want something that I can trust there is no "funny business" going on where it is entering where it shouldn't, orders/trades are being accounted for etc. The "realest" possible backtest. :)

Mango Loco

@backtrader
My understanding is that self.close will close the entry order. From seeing the code at the link I attached, I assumed that attached bracket orders (SL and/or TP) had to still be cancelled manually (one of them, at least).

Commenting out both the parts of code you highlighted, results in running, and accurate looking, backtest. Could you clarify whether self.close will cancel an attached child order or whether we need to do it manually? Have I set up my ordering correctly?

backtrader

@Mango-Loco said in Multi-Data Long & Short:

My understanding is that self.close will close the entry order.

Either your naming convention is wrong or there is some fundamental misunderstanding. An Order cannot be closed. One can cancel an order.

From the documentation: Counters a long/short position closing it. It works with the open position in the market.

It has absolutely nothing to do with bracket orders. Obviously, if you close an open position, which was opened with a bracket order set and the two bracketing orders are still active in the market, you will have to take care of them, or else a new open position will sooner or later be there.