Optimization: Can you reproduce this bug?

bigdavediode

Okay, challenge time. Can you reproduce this problem? Here's code for optimization. When the line cerebro = bt.Cerebro(optreturn=False) is used without maxcpus=1, the following correct results are obtained showing what the results are during optimization testing:

/usr/bin/python2.7 /docker_stocks/DELETETEMP.py
1.9.58.122
('UNDERLYING:', ['YHOO'])
Results: Ordered by period:
Period: 14, PnL: 2062.0
Period: 15, PnL: 2079.0
Period: 16, PnL: 1718.0
Period: 17, PnL: 2141.0
Period: 18, PnL: 2002.0
Period: 19, PnL: 1451.0
Period: 20, PnL: 1883.0
Results: Ordered by Profit:
Period: 17, PnL: 2141.0
Period: 15, PnL: 2079.0
Period: 14, PnL: 2062.0
Period: 18, PnL: 2002.0
Period: 20, PnL: 1883.0
Period: 16, PnL: 1718.0
Period: 19, PnL: 1451.0

BUT when maxcpus=1 is used : cerebro = bt.Cerebro(optreturn=False, maxcpus=1)

/usr/bin/python2.7 /docker_stocks/DELETETEMP.py
1.9.58.122
('UNDERLYING:', ['YHOO'])
Results: Ordered by period:
Period: 14, PnL: 1883.0
Period: 15, PnL: 1883.0
Period: 16, PnL: 1883.0
Period: 17, PnL: 1883.0
Period: 18, PnL: 1883.0
Period: 19, PnL: 1883.0
Period: 20, PnL: 1883.0
Results: Ordered by Profit:
Period: 14, PnL: 1883.0
Period: 15, PnL: 1883.0
Period: 16, PnL: 1883.0
Period: 17, PnL: 1883.0
Period: 18, PnL: 1883.0
Period: 19, PnL: 1883.0
Period: 20, PnL: 1883.0

Process finished with exit code 0

Here's the simple test code:

import backtrader as bt
import datetime


class firstStrategy(bt.Strategy):
    params = (
        ('period', 21),
    )

    def __init__(self):
        self.startcash = self.broker.getvalue()
        self.rsi = bt.indicators.RSI_SMA(self.data.close, period=self.params.period)

    def next(self):
        if not self.position:
            if self.rsi < 30:
                self.buy(size=100)
        else:
            if self.rsi > 70:
                self.sell(size=100)

print(bt.__version__)
sunderlying = ["YHOO"]


print("UNDERLYING:", sunderlying)

#Variable for our starting cash
startcash = 10000

#Create an instance of cerebro
cerebro = bt.Cerebro(optreturn=False, maxcpus=1)

#Add our strategy
cerebro.optstrategy(firstStrategy, period=range(14,21))

#Get Apple data from Yahoo Finance.
data = bt.feeds.YahooFinanceData(
    dataname='AAPL',
    fromdate = datetime.datetime(2016,1,1),
    todate = datetime.datetime(2017,1,1),
    buffered= True
    )
cerebro.adddata(data)

# Set our desired cash start
cerebro.broker.setcash(startcash)

# Run over everything
opt_runs = cerebro.run()

# Generate results list
final_results_list = []
for run in opt_runs:
    for strategy in run:
        value = round(strategy.broker.get_value(),2)
        PnL = round(value - startcash,2)
        period = strategy.params.period
        final_results_list.append([period,PnL])

#Sort Results List
by_period = sorted(final_results_list, key=lambda x: x[0])
by_PnL = sorted(final_results_list, key=lambda x: x[1], reverse=True)

#Print results
print('Results: Ordered by period:')
for result in by_period:
    print('Period: {}, PnL: {}'.format(result[0], result[1]))
print('Results: Ordered by Profit:')
for result in by_PnL:
    print('Period: {}, PnL: {}'.format(result[0], result[1]))

backtrader

@bigdavediode said in Optimization: Can you reproduce this bug?:

final_results_list = []
for run in opt_runs:
    for strategy in run:
        value = round(strategy.broker.get_value(),2)

Your problem is probably there. Get the broker value inside the stop method of the strategy and not from outside the strategy.

bigdavediode

@backtrader said in Optimization: Can you reproduce this bug?:

Fascinating difference and gracias. I altered the code and it now works, as below (the original code is from https://backtest-rookies.com/2017/06/26/optimize-strategies-backtrader/ )



import backtrader as bt
import datetime


class firstStrategy(bt.Strategy):
    params = (
        ('period', 21),
    )

    def __init__(self):
        self.startcash = self.broker.getvalue()
        self.rsi = bt.indicators.RSI_SMA(self.data.close, period=self.params.period)

    def next(self):
        if not self.position:
            if self.rsi < 30:
                self.buy(size=100)
        else:
            if self.rsi > 70:
                self.sell(size=100)

    def stop(self):
        self.value = round(self.broker.get_value(), 2)

print(bt.__version__)
sunderlying = ["YHOO"]


print("UNDERLYING:", sunderlying)

#Variable for our starting cash
startcash = 10000

#Create an instance of cerebro
cerebro = bt.Cerebro(optreturn=False, maxcpus=1)

#Add our strategy
cerebro.optstrategy(firstStrategy, period=range(14,21))

#Get Apple data from Yahoo Finance.
data = bt.feeds.YahooFinanceData(
    dataname='AAPL',
    fromdate = datetime.datetime(2016,1,1),
    todate = datetime.datetime(2017,1,1),
    buffered= True
    )
cerebro.adddata(data)

# Set our desired cash start
cerebro.broker.setcash(startcash)

# Run over everything
opt_runs = cerebro.run()

# Generate results list
final_results_list = []
for run in opt_runs:
    for strategy in run:
        value = strategy.value
        PnL = round(value - startcash,2)
        period = strategy.params.period
        final_results_list.append([period,PnL])

#Sort Results List
by_period = sorted(final_results_list, key=lambda x: x[0])
by_PnL = sorted(final_results_list, key=lambda x: x[1], reverse=True)

#Print results
print('Results: Ordered by period:')
for result in by_period:
    print('Period: {}, PnL: {}'.format(result[0], result[1]))
print('Results: Ordered by Profit:')
for result in by_PnL:
    print('Period: {}, PnL: {}'.format(result[0], result[1]))

The output is now correct:

/usr/bin/python3.6 /docker_stocks/DELETETEMP.py
1.9.64.122
UNDERLYING: ['YHOO']
Results: Ordered by period:
Period: 14, PnL: 2062.0
Period: 15, PnL: 2079.0
Period: 16, PnL: 1718.0
Period: 17, PnL: 2141.0
Period: 18, PnL: 2002.0
Period: 19, PnL: 1451.0
Period: 20, PnL: 1883.0
Results: Ordered by Profit:
Period: 17, PnL: 2141.0
Period: 15, PnL: 2079.0
Period: 14, PnL: 2062.0
Period: 18, PnL: 2002.0
Period: 20, PnL: 1883.0
Period: 16, PnL: 1718.0
Period: 19, PnL: 1451.0

Process finished with exit code 0

backtrader

With multiple cores and processes, a new cerebro per process is created. When you restrict things to a single core, the same cerebro and broker are reused.

It was not foreseen that someone would be accessing the broker from outside the methods.