optreturn ofc... I will never get this right...

However,I don't use the backtrader's original optimize function,because there is something wrong when I do it,I can't sovle it,so every time,my backtest must consume 13%(data._preload()),is there some ways to pcikle the preloaded data?

There is a bug with optresult=True in the current backtrader code when using particular analyzers which will result in unnecessary large result objects. As every process needs to pickle its result back to the one main process this main process will become a bottleneck with large result objects. The processes will pile up waiting for their chance to send their results which effectively brings them to a halt.
Not sure if you encountered this situation though.

I think I fixed it. Seems to work with the observers.

Thank you, yes I know, but then all observers are gone :)

I am experiencing crashes when trying to run an optimization. I reduced it to quite a simple test case to reproduce the error:

import datetime

import backtrader as bt


class TestStrategy(bt.Strategy):
    params = (
        ('period', 15),
        ('printdata', True),
        ('printops', True),
    )


if __name__ == '__main__':
    cerebro = bt.Cerebro(maxcpus=0, optreturn=False)

    cerebro.optstrategy(TestStrategy, period=[5, 6, 7])

    data = bt.feeds.YahooFinanceCSVData(
        dataname=r"..\datas\nvda-1999-2014.txt",
        fromdate=datetime.datetime(2000, 1, 1),
        todate=datetime.datetime(2002, 12, 31),
        reverse=False,
        swapcloses=True,
    )
    cerebro.adddata(data)

    data = bt.feeds.YahooFinanceCSVData(
        dataname=r"..\datas\orcl-1995-2014.txt",
        fromdate=datetime.datetime(2000, 1, 1),
        todate=datetime.datetime(2002, 12, 31),
        reverse=False,
        swapcloses=True,
    )
    cerebro.adddata(data)

    cerebro.run()

This is the result:

(qrawler) C:\Users\Robin\Projects\backtrader\tests>python test_multidata_optimize.py
Exception in thread Thread-3:
Traceback (most recent call last):
  File "C:\Users\Robin\Miniconda3\envs\qrawler\lib\threading.py", line 916, in _bootstrap_inner
    self.run()
  File "C:\Users\Robin\Miniconda3\envs\qrawler\lib\threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Users\Robin\Miniconda3\envs\qrawler\lib\multiprocessing\pool.py", line 463, in _handle_results
    task = get()
  File "C:\Users\Robin\Miniconda3\envs\qrawler\lib\multiprocessing\connection.py", line 251, in recv
    return _ForkingPickler.loads(buf.getbuffer())
AttributeError: Can't get attribute 'Lines_LineSeries_LineIterator_DataAccessor_ObserverBase_Observer_DataTrades_32f0d39362e84be2898055ff9eaf034a' on <module 'backtrader.lineseries' from 'c:\\users\\robin\\projects\\backtrader\\backtrader\\lineseries.py'>

My Python:

(qrawler) C:\Users\Robin\Projects\backtrader\tests>python -V
Python 3.6.9 :: Anaconda, Inc.

Any idea whats wrong on my end or actually backtrader's fault?

Sorry, I didn't see this thread earlier. Does it still happend in 1.0.0? If yes then could you post an example of the code that produced this in the plotting thread or as a Github issue? Thanks.

I recently found time to work on this a bit again and finally reached something I call now version 1.0.0. I did alot of polishing and cleaning up the code base. All open issues on GitHub are fixed with this as far as I know.

Some additions include:

• being able to configure plotting completely independent from strategy code (which bothered me)
• configurable hover tooltip
• much easier optimization result browser
• etc.

It was not possible to edit the first post of this thread but the project is still available on GitHub :)
https://github.com/verybadsoldier/backtrader_plotting

I also tried my best to document the core components:
https://github.com/verybadsoldier/backtrader_plotting/wiki

I am very happy to hear your feedback and critics. Feel free to use also the issue tracker on GitHub:
https://github.com/verybadsoldier/backtrader_plotting/issues

I know the generated pages to not look very great as I am bad at designing. I would be happy if someone of you who is maybe good with css could give it a try spice it a visually. I would be happy to include an updated css scheme!

@lampalork
Which data feed are you using?

I would also be interested to learn about your stance regarding the future of Backtrader. I am not demanding anything but just knowing what your plans are (or are not) would help everyone to make good decisions in my opinion.

@backtrader said in Backtrader 2.0?:

Let's put it the other way round

What are you missing?

While Backtrader is awesome already (you did a hell of a job actually, thank you) here are some ideas:

• an external interface that makes integrating backtrader in other system or further processing easy. It could be MQTT, ZeroMQ, HTTP/Rest or whatever. It could fire events when trades happen, a bar moves or whatever...
• Speedup backtrader (poor Python) for example by implementing parts of it in Cython. Would probably mean profiling to find bottlenecks to know where to start
• As already mentioned the plotting could be improved. Maybe even a (web-based) interactive UI?
• Support user-defined custom data to take into account (like fundamental data or even weather data if the user wants to)
• General bugfixing. In the last year I noticed like 3 or 4 behaviors I don't understand. I guess that at least some of them are bugs actually

I have configured a trading calendar that contains daily sessions and I have the problem that the timestamp of the end of session is not being updated after the end of session has been reached.

I have tracked it down to the code where I think a condition is not met which makes it dismiss the end of session condition.

The scenario is like this:

• live paper trading using tick data getting resampled to H1 bars (10:00 to 18:00, 9 bars)
• The trading hours are 9:00 to 17:35
• As said, data is tick-based, but backfilling fills H1 bars (!)

The code I have pinpointed is in resamplefilter.py in function _eoscheck:

        equal = dt == self._nextdteos
        grter = dt > self._nextdteos

        if exact:
            is_eos = equal
        else:
            # if the compared data goes over the endofsession
            # make sure the resampled bar is open and has something before that
            # end of session. It could be a weekend and nothing was delivered
            # until Monday
            if grter:
                is_eos = self.bar.isopen() and self.bar.datetime <= self._nextdteos
            else:
                is_eos = equal

In particular this condition:

                is_eos = self.bar.isopen() and self.bar.datetime <= self._nextdteos

In the moment a backfilling H1 bar with timestamp 18:00 is coming in (and eos being 17:35) then equal is False but grter is True so the if-clause in question is getting reached. The clause will render is_eos to False which in my opinion is the problem which will lead to not considering the session to have ended and therefore not fetching the next following eos timestamp.

The condition returns False since the bar is not open because the 18:00-bar is the first data in the range from 17:00 - to 18:00 (because H1 bars always deliver a single data in each hour). This situation that a single bar opens a bar and goes beyond the end of the session is the core of the problem I guess cause of the bar.isopen condition.

I am tempted to just remove the condition for me and set is_eos to True when grter is True but I feel I don't really understand what the condition is meant to do. I know there even is a multiline comment above it but I still fail to see under which condition the code. Would you give me a hint please regarding that if-condition? Especially what will go wrong when I remove it?

Also in general:
If I do live tick data resampling to H1 bars. Is it ok for the backtrader to still do backfill with H1 bars? I mean the resampler is expecting tick data but gets H1 bars fed in the beginning. But I think the Oanda data feed does the same so I thought it would be ok.

Many thanks for your help!