Strategy auto-generation

[ab_trader]

Hi all! I am seeking you help on coding advice on auto-generating strategies within bt. Lets say I have 2 ideas for enter position (enter 1 & 2) and two ideas to exit position (exit 1 & 2). Using the regular way to create the strategy I need to create 4 scrips like this:

Script 1 - strategy 1- enter 1 - exit 1
Script 2 - strategy 2 - enter 1 - exit 2
Script 3 - strategy 3 - enter 2 - exit 1
Script 4 - strategy 4 - enter 2 - exit 2

How I can mix them in single script in order to have backtested all 4 strategies in a sequence without intermediate manual updating of the code?

I estimate typical strategy N code as follows:

# strategy N
class strategyN(bt.Strategy):
params = (('pN1', x), (pN2, y), (pN3, z), ... (pNM, zz),)

def __init__(self):
    self.indN1 = indicator N1
    self.indN2 = indicator N2

    self.buysigN = expression for buy signal conditions
    self.sellsigN = expression for sell signal conditions

def next(self):
    if not self.position and self.buysigN:
        self.order = self.buy()
    
    if self.position and self.sellsigN:
       self.order = self.sell()

def stop(self):
    write strategy stats to the file

[backtrader]

A possible approach is to pack the entry/exit signals into Indicator subclasses and then simply use the standard optimization approach. Something along these lines:

class Entry1(bt.Indicator):
    pass  # define something sensible here
...

class Exit2(bt.Indicator):
    pass  # define something sensible here

class MyStrategy(bt.Strategy):

    params = (('entry', None), ('exit', None),)

    def __init__(self):
        self.enter = self.p.entry()
        self.exit = self.p.exit()

    def next(self):
        pass  # do the buy/sell logic here

cerebro = bt.Cerebro()
cerebro.optstrategy(MyStrategy, entry=(Entry1, Entry2), exit=(Exit1, Exit2))

cerebro.run()

[ab_trader]

Actually I was thinking about individual optimization of the newly generated systems rather then backtests. :)
Thank you for the idea with indicators subclassing, good start to think further.

[ab_trader]

Does this code make any sense, if I want to optimize all strategies generated?

# generate list of strategies
cerebro = bt.Cerebro()
strats = cerebro.optstrategy(MyStrategy, entry=(Entry1, Entry2), exit=(Exit1, Exit2))
cerebro.run()

# optimize each strategy
for strategyN in strats:
    cerebroN = bt.Cerebro()
    cerebroN.optstrategy(strategyN, InternalStrategyNParameterRanges)
    cerebroN.run()

[backtrader]

No, it doesn't.

If each of those self-generated strategies is going to be optimized, you need several loops creating a cerebro and running it. In each of those loops you need to call optstrategy.

For example

import itertools

signals = itertools.product((Entry1, Entry2), (Exit1, Exit2))

for entry, exit in signals:
    cerebro = bt.Cerebro()
    cerebro.optstrategy(MyStrategy, entry=entry, exit=exit, period=list(range(10, 20)))
    cerebro.run()

period is in this case what it will be iterated over and will allow optimization of entry and exit. You'll probably need a lot more.

[ab_trader]

Thank you!
I'll try to implement it.

[backtrader]

Another insight can be found in the blog: Strategy Selection

[ab_trader]

@backtrader

When develop indicator in the __init__ we use series of data. Is there any they to use shifted series in this case?

As example, I want to compare today close and yesterday highest values. CrossOver compares today close and today highest values, so I have no signals at all.

[backtrader]

CrossOver(self.data.close, self.data.high(-1))

[ab_trader]

Following proposed approach:

Entry signals are set as indicator:

class MA2Intersection(bt.Indicator):
    lines = ('longsig', 'shortsig', 'ma1', 'ma2')
    params = (('ma1_period', 2), ('ma2_period', 20))
    
    def __init__(self):

        self.ma1 = bt.indicators.MovingAverageSimple(self.data.close, period=self.p.ma1_period)
        self.ma2 = bt.indicators.MovingAverageSimple(self.data.close, period=self.p.ma2_period)

        self.longsig = bt.indicators.CrossUp(self.ma1, self.ma2)
        self.shortsig = bt.indicators.CrossDown(self.ma1, self.ma2)

Then initialize strategy and signals:

class MasterStrategy(bt.Strategy):

    params = (('entry', MA2Intersection), ('exit', PriceMACrossOver),
              ('plot_entry', True), ('plot_exit', True),)

    def __init__(self):

        self.order = None

        self.entry = self.p.entry(plot=self.p.plot_entry)
        self.longsig = self.entry.longsig
        self.shortsig = self.entry.shortsig
        self.ma1 = self.entry.ma1
        self.ma2 = self.entry.ma2

And then do buy/short logic in the strategy next. The problem is that only last values of the indicator series are transferred to the strategy - so all elements of the strategy series contain single value. Could you please advice something?

[backtrader]

The last statement is unclear.

Where there is only 1 value?

self.entry holds a regular indicator and the values can be queried anytime. For example:

self.entry.longsig[0]  # actual value
self.entry.longsig[-1]  # previous value

[ab_trader]

In the strategies next

    def next(self):

        print ('MA1 %0.2f, MA2 %0.2f, LONG %d, SHORT %d' % (self.ma1[0], self.ma2[0],
                              self.longsig[0], self.shortsig[0]))

It outputs the same value for each bar and this value equal to the last value of the indicator. But in the indicator next

    def next(self):
        
        print ('MA1 %0.2f, MA2 %0.2f, LONG %d, SHORT %d' % (self.ma1[0], self.ma2[0],
                            self.longsig[0], self.shortsig[0]))

output is correct. Each bar contains appropriately calculated data.

[backtrader]

Because at least the initial code is wrong (upon extra reading)

        self.longsig = bt.indicators.CrossUp(self.ma1, self.ma2)
        self.shortsig = bt.indicators.CrossDown(self.ma1, self.ma2)

instead of

self.lines.longsig = bt ...

[ab_trader]

@backtrader

Yes, thats my bad. Thank you, it works great now!

[ab_trader]

Returning to the system generator. I was following your idea and things were going nice. Thank you again! The only problem came up recently - how to pass to optimizer the parameters for particular entries/exits/etc (formally indicator parameters)? If you can advice something that will be great.

The current structure of the script:

# entries
class Entry_XXX(bt.Indicator):
    lines = ('longsig', 'shortsig')
    params = (('XXX_param1', 5), ('XXX_param2', 10),)

    def __init__(self):
        self.lines.longsig = # function of XXX_param1 & XXX_param2
        self.lines.shortsig = # other function of XXX_param1 & XXX_param2

# exits
class Exit_YYY(bt.Indicator):
    lines = ('sellsig', 'coversig')
    params = (('YYY_param1', 5), ('YYY_param2', 10), ('YYY_param3', 2),)

    def __init__(self):
        self.lines.sellsig = # function of YYY_param1, YYY_param2 & YYY_param3
        self.lines.coversig = # other function of YYY_param1, YYY_param2 & YYY_param3

# strategy
class MasterStrategy(bt.Strategy):

    params = (('entry', None), ('exit', None), ('support', None),
              ('plot_entry', True), ('plot_exit', True), ('plot_support', True),)

    def __init__(self):
        self.longsig = None
        self.shortsig = None
        self.sellsig = None
        self.coversig = None

        if self.p.entry:
            self.entry = self.p.entry(plot=self.p.plot_entry)
            self.longsig = self.entry.lines.longsig
            self.shortsig = self.entry.lines.shortsig

        if self.p.exit:
            self.exit = self.p.exit(plot=self.p.plot_exit)
            self.sellsig = self.exit.lines.sellsig
            self.coversig = self.exit.lines.coversig

    def next(self):
        # process buy, short, sell and cover signals

#main code
if __name__ == '__main__':
    cerebro = bt.Cerebro()
    strats = cerebro.optstrategy(MasterStrategy,
        entry = (Entry_1, Entry_2, Entry_XXX),
        exit = (Exit_1, Exit_2, Exit_YYY)
        )
    data = bt.feeds.YahooFinanceCSVData(Parameters set as usual)
    cerebro.adddata(data)
    cerebro.run()

[backtrader]

That cannot be directly solved, because you are trying to optimize a strategy, which is optimizing substrategies. And the system cannot known in advance how many of these indirection levels you actually want to do.

Hence the comment in one of the posts above:

If each of those self-generated strategies is going to be optimized, you need several loops creating a cerebro and running it. In each of those loops you need to call optstrategy.

The only solution as also pointed out above is to implement the Strategy Selection approach. You pass parameters to the strategy, which let the strategy stub select which entries/exits and with which actual parameters are going to be put in motion.

Another insight can be found in the blog: Strategy Selection

[ab_trader]

Thank you for your answer!

I was thinking about code generation script, which will generate me separate script for each new strategy replacing pieces of code related to entries and exits. Finally each strategy script will be typical bt system with entries and exits calculated by indicators with cerebro.optstrategy. But it is not clear for me how to pass indicators parameters (=entry/exit parameters) to optimizer of this simple script. Is it possible?

[backtrader]

One option would be to pass iterables in iterables. The optimization code will then mix the different iterables of iterables to give you what you want.

cerebro.optstrategy(entries=(Et1, Et2, Et3), exits=(Ex1, Ex2, Ex3), pentries=(Iterable1, Iterable2, Iterable3), pexits=(Iter4, Iter5, Iter6))

And you will a constant mix of Entries, Exits and the iterables associated to them

[ab_trader]

@backtrader Thank you!

So, these pentries and pexits should be parameters of the strategy. Then in the strategy __init__ when I declare my entry/exit indicators I need to pass these parameters to entry/exit indicators. Is it a way to do it?

[backtrader]

That would indeed be the procedure for such an approach.