Throwing an exception when run in multiple processes

soulmachine

I write a strategy in this Jupyter notebook: Clenow trend following strategy - Github

I try to run a grid search to find out the optimal parameters combination. If I run sequentially as the following, no problem at all:

stats = []
    for params in tqdm(params_list):
        result = run_strategy(params)
        stats.append(result)

If I run in parallel as the following:

stats = db.from_sequence(params_list).map(run_strategy).compute()

It always throw out an exception:

~/anaconda3/lib/python3.7/site-packages/backtrader/lineiterator.py in dopostinit()
    133 
    134         # my minperiod is as large as the minperiod of my lines
--> 135         _obj._minperiod = max([x._minperiod for x in _obj.lines])
    136 
    137         # Recalc the period

ValueError: max() arg is an empty sequence

Full stack trace is in the Jupyter notebook.

Any ideas? Thanks!

soulmachine

I guess the issue is in the Donchian channel indicator:

class DonchianChannelsIndicator(bt.Indicator):
    '''Donchian channel.'''

    alias = ('DCH', 'DonchianChannel',)

    lines = ('dcm', 'dch', 'dcl',)  # dc middle, dc high, dc low

    params = (
        ('period', 20), # lookback period
    )

    plotinfo = dict(subplot=False)  # plot along with data
    plotlines = dict(
        dcm=dict(ls='--'),  # dashed line
        dch=dict(_samecolor=True),  # use same color as prev line (dcm)
        dcl=dict(_samecolor=True),  # use same color as prev line (dch)
    )

    def __init__(self):
        super().__init__()
        self.addminperiod(self.params.period + 1)
        self.lines.dch = bt.indicators.Highest(self.data.high(-1), period=self.params.period)
        self.lines.dcl = bt.indicators.Lowest(self.data.low(-1), period=self.params.period)
        self.lines.dcm = (self.lines.dch + self.lines.dcl) / 2.0  # avg of the above

In sequential mode the indicator has 3 lines, while in parallel mode the indicator has 0 lines, why? This is so weird.

Probably Backtrader is not compatible with Dask?

soulmachine

It seems Backtrader is not compatible with Dask.

Dask version not working:

with ProgressBar(): # Backtrader is not compatible with Dask
    stats = db.from_sequence(params_list).map(run_strategy).compute()

Plain multiprocessing version works:

with multiprocessing.Pool(os.cpu_count()) as p:
    stats = p.map(run_strategy, params_list)