For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

Optimization error when multiple data feed



  • While carrying out optimization with multiple data feed I am getting threading(pickling) error.
    I have added multiple data feed in this way:

    backtester = bt.Cerebro(maxcpus = 6)
    data0 = bt.feeds.GenericCSVData(dataname=inFilePath0,
                                                datetime=0,
                                                open=1,
                                                high=2,
                                                low=3,
                                                close=4,
                                                volume=5,
                                                openinterest=-1,
                                                timeframe=bt.TimeFrame.Ticks,
                                                dtformat="%m/%d/%Y %H:%M",
                                                tmformat='%H:%M',
                                                )
    
    backtester.resampledata(
                    dataname=data0,
                    timeframe=timeFrameInfo.tframes[timeframe],
                    compression=compression)
    
    data1 = bt.feeds.GenericCSVData(dataname=inFilePath0,
                                                datetime=0,
                                                open=1,
                                                high=2,
                                                low=3,
                                                close=4,
                                                volume=5,
                                                openinterest=-1,
                                                timeframe=bt.TimeFrame.Ticks,
                                                dtformat="%m/%d/%Y %H:%M", 
                                                )
    
    backtester.resampledata(
                    dataname=data1,
                    timeframe=timeFrameInfo.tframes["daily"],
                    compression=1)
    

    Using optimization with these data I am getting this error:

    Exception in thread Thread-3:
    Traceback (most recent call last):
      File "C:\ProgramData\Anaconda2\envs\Python36\lib\threading.py", line 916, in _bootstrap_inner
        self.run()
      File "C:\ProgramData\Anaconda2\envs\Python36\lib\threading.py", line 864, in run
        self._target(*self._args, **self._kwargs)
      File "C:\ProgramData\Anaconda2\envs\Python36\lib\multiprocessing\pool.py", line 463, in _handle_results
        task = get()
      File "C:\ProgramData\Anaconda2\envs\Python36\lib\multiprocessing\connection.py", line 251, in recv
        return _ForkingPickler.loads(buf.getbuffer())
    AttributeError: Can't get attribute 'Lines_LineSeries_LineIterator_DataAccessor_ObserverBase_Observer_DataTrades_a45e1dc626d14c80b0da9e30da5b6b0c' on <module 'backtrader.lineseries' from 'C:\\ProgramData\\Anaconda2\\envs\\Python36\\lib\\site-packages\\backtrader\\lineseries.py'>
    

    If I remove the second data feed(data1), code runs perfectly fine.



  • You may try to use

    timeframe = bt.TimeFrame.Daily
    

    in backtester.resampledata()
    I'm new here, I don't know if this could work, but, emmmm, have a try, maybe it's good.



  • @yufeng-xin Thanks.
    But I got the actual problem(I am assuming it, may be I am wrong also). Actually I am using the second datafeed only for indicator calculation and thus it has not any observer(BuySell,CashValue etc.).So I am thinking that after the code completetion it is trying to fetch these values and thus giving this error.
    So when I put this for optimization:

    bt.Cerebro(maxcpus = 8, stdstats = False))
    

    it worked.


  • administrators

    @ry-93 said in Optimization error when multiple data feed:

    stdstats = False
    

    This disables the addition of observers, and the Datatrades observer is the one which cannot be pickled.