Mutidata + Multicore Optimization AttributeError

ThatBlokeDave

Hello All,
I have run into some peculiar results when trying to optimise a strategy that uses multiple data feeds. The results I am getting also seem to be effected by maxcpus. The key word here is "seems". I am not really sure how they are interrelated (if at all).

I have put together a little demonstration code to see if others can reproduce it. However, first let me describe what I am seeing.

If I run a simple script with a single data feed, I can optimise without issue.
If I add a second data feed, my strategy exits with an AttributeError
If I reduce the maxcpus to 1, essentially disabling multicore support, then the optimisation runs but all of the final results are the same.

The test script is below. To drive it, we just need to change the following 4 lines

# ------------------------ PLAY WITH THESE PARAMS -------------------------/
    optimize  = True
    multicore = 1
    testStrategy = multTestStrategy
    second_data = True
# -------------------------------------------------------------------------/

And the script:

import backtrader as bt
from datetime import datetime


class singleTestStrategy(bt.Strategy):
    params = (
        ('period',21),
        )

    def __init__(self):
        self.startcash = self.broker.getvalue()
        self.rsi = bt.indicators.RSI_SMA(self.data.close, period=self.params.period)

    def next(self):
        if not self.position:
            if self.rsi < 30:
                self.buy(size=100)
        else:
            if self.rsi > 70:
                self.sell(size=100)



class multTestStrategy(bt.Strategy):
    params = (
        ('period',21),
        )

    def __init__(self):
        self.startcash = self.broker.getvalue()
        self.data_dict = dict()

        for i, d in enumerate(self.datas):
            self.data_dict[d] = dict()
            self.data_dict[d]['rsi'] = bt.indicators.RSI_SMA(d, period=self.params.period)

    def next(self):

        for i, d in enumerate(self.datas):
            pos = self.broker.getposition(d).size
            if not pos:
                if self.data_dict[d]['rsi'] < 30:
                    self.buy(d,size=10)
            else:
                if self.data_dict[d]['rsi'] > 70:
                    self.sell(d,size=10)


if __name__ == '__main__':
    #Variable for our starting cash
    startcash = 10000

    # ------------------------ PLAY WITH THESE PARAMS -------------------------/
    optimize  = True
    multicore = 0
    testStrategy = singleTestStrategy
    second_data = False
    # -------------------------------------------------------------------------/

    #Create an instance of cerebro
    cerebro = bt.Cerebro(optreturn=False, maxcpus=multicore)

    #Add our strategy
    if optimize:
        cerebro.optstrategy(testStrategy, period=[14,21,28])
    else:
        cerebro.addstrategy(testStrategy, period=21)


    data = bt.feeds.Quandl(
        dataname='AAPL',
        fromdate = datetime(2016,1,1),
        todate = datetime(2017,1,1),
        buffered= True
        )

    data2 = bt.feeds.Quandl(
        dataname='AMZN',
        fromdate = datetime(2016,1,1),
        todate = datetime(2017,1,1),
        buffered= True
        )

    #Add the data to Cerebro
    cerebro.adddata(data)

    #Add the data to Cerebro
    if second_data:
        cerebro.adddata(data2)

    # Set our desired cash start
    cerebro.broker.setcash(startcash)

    # Run over everything
    if optimize:
        opt_runs = cerebro.run()

        # Generate results list
        final_results_list = []
        for run in opt_runs:
            for strategy in run:
                value = round(strategy.broker.get_value(),2)
                PnL = round(value - startcash,2)
                period = strategy.params.period
                final_results_list.append([period,PnL])

        #Sort Results List
        by_period = sorted(final_results_list, key=lambda x: x[0])
        by_PnL = sorted(final_results_list, key=lambda x: x[1], reverse=True)

        #Print results
        print('Results: Ordered by period:')
        for result in by_period:
            print('Period: {}, PnL: {}'.format(result[0], result[1]))
        print('Results: Ordered by Profit:')
        for result in by_PnL:
            print('Period: {}, PnL: {}'.format(result[0], result[1]))
    else:
        # Run over everything
        cerebro.run()

        #Get final portfolio Value
        portvalue = cerebro.broker.getvalue()
        pnl = portvalue - startcash

        #Print out the final result
        print('Final Portfolio Value: ${}'.format(portvalue))
        print('P/L: ${}'.format(pnl))

        #Finally plot the end results
        cerebro.plot(style='candlestick')

Test Case 1 - Single Data with Optimization

This is the baseline - Working example

Settings:

# ------------------------ PLAY WITH THESE PARAMS -------------------------/
    optimize  = True
    multicore = 0
    testStrategy = singleTestStrategy
    second_data = False
# -------------------------------------------------------------------------/

Output:

Results: Ordered by period:
Period: 14, PnL: 2153.0
Period: 21, PnL: 1710.0
Period: 28, PnL: 1327.0
Results: Ordered by Profit:
Period: 14, PnL: 2153.0
Period: 21, PnL: 1710.0
Period: 28, PnL: 1327.0

Test Case 2 - Multi-Data / Multicore with Optimization

Settings:

# ------------------------ PLAY WITH THESE PARAMS -------------------------/
    optimize  = True
    multicore = 0
    testStrategy = multTestStrategy
    second_data = True
# -------------------------------------------------------------------------/

Output:

Exception in thread Thread-3:
Traceback (most recent call last):
  File "C:\Users\Dave\AppData\Local\Programs\Python\Python36-32\lib\threading.py", line 916, in _bootstrap_inner
    self.run()
  File "C:\Users\Dave\AppData\Local\Programs\Python\Python36-32\lib\threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Users\Dave\AppData\Local\Programs\Python\Python36-32\lib\multiprocessing\pool.py", line 463, in _handle_results
    task = get()
  File "C:\Users\Dave\AppData\Local\Programs\Python\Python36-32\lib\multiprocessing\connection.py", line 251, in recv
    return _ForkingPickler.loads(buf.getbuffer())
AttributeError: Can't get attribute 'Lines_LineSeries_LineIterator_DataAccessor_ObserverBase_Observer_DataTrades_aa890439e0cc4922a6091c60d2c6e113' on <module 'backtrader.lineseries' from 'C:\\Users\\Dave\\AppData\\Local\\Programs\\Python\\Python36-32\\lib\\site-packages\\backtrader\\lineseries.py'>

Test Case 3 - Multi-Data / Multicore WITHOUT Optimization

Again, just a positive working test case showing that things are working on the multTestStrategy

Settings:

# ------------------------ PLAY WITH THESE PARAMS -------------------------/
    optimize  = False
    multicore = 0
    testStrategy = multTestStrategy
    second_data = True
# -------------------------------------------------------------------------/

Output:

Final Portfolio Value: $10046.32851701423
P/L: $46.328517014229874

Test Case 4 - Multi-Data / Single Core with Optimization

The final test case is also quite perculiar. Limiting maxcpus to 1. I get the correct period from the strategyreturned by cerebrobut the get_value() broker calls all seem to return the same figure.

Settings:

# ------------------------ PLAY WITH THESE PARAMS -------------------------/
    optimize  = True
    multicore = 1
    testStrategy = multTestStrategy
    second_data = True
# -------------------------------------------------------------------------/

Output:

Results: Ordered by period:
Period: 14, PnL: 333.98
Period: 21, PnL: 333.98
Period: 28, PnL: 333.98
Results: Ordered by Profit:
Period: 14, PnL: 333.98
Period: 21, PnL: 333.98
Period: 28, PnL: 333.98

As we can see the period is changing but the PnL remains the same.

So overall, I am not sure what is causing it but I am also not sure how where to begin with debugging!

backtrader

@thatblokedave said in Mutidata + Multicore Optimization AttributeError:

If I reduce the maxcpus to 1, essentially disabling multicore support, then the optimisation runs but all of the final results are the same.

The problem is that you have to store the PnL value in the stop method of the Strategy. Because unless you do that you talk to the last instance of the PnL which cerebro keeps record of. (With maxcpus > 1 it would be the same)

See here: Community - Optimization: Can you reproduce this bug?

@thatblokedave said in Mutidata + Multicore Optimization AttributeError:

AttributeError: Can't get attribute 'Lines_LineSeries_LineIterator_DataAccessor_ObserverBase_Observer_DataTrades_aa890439e0cc4922a6091c60d2c6e113' on <module 'backtrader.lineseries' from 'C:\\Users\\Dave\\AppData\\Local\\Programs\\Python\\Python36-32\\lib\\site-packages\\backtrader\\lineseries.py'>

Run with cerebro.run(stdstats=False) and the observers won't run and won't generate the error.

ThatBlokeDave

@backtrader Awesome! Thanks, I will take a look