A real (as real as possible) diagnostic could be attempted if at least a couple of data samples and buy/sell operations were provided. And furthermore if you use Market or Limit orders for example.

A code sample would be even more helpful.

In any other case one can only make an educated guess.

The educated guess assumes you use Market orders and that MT4 lets you buy the close of the bar you are evaluating. Be it the case, that is unrealistic, because a bar which you see is already closed, and giving you the close of the bar is giving you a price which is no longer available (it may become available later in a yet-to-be-delivered bar)

It can be a good approximation. Such a behavior can be enabled with cheat-on-close in the backtrader broker. All assuming the educated guess is not educated and not a guess.

Probably is just some kind of "lost in translation", but the pyfoliotest sample provides you with a working example that buys, sells and has the pyfolio analyzer plugged in and later fetched to look at the results.

It seems a rather complete example. The rest of the analyzers are all present in the documentation and can be added and later fetched like pyfolio.

And the other docs, samples that were pointed do the same with some other analyzers (trade to give the analyzer a meaningful role in the sample)

@randyt has added a list of those other analyzers, which are also present in other samples, but it's just a matter of taste which analyzers to use. Settling for pyfolio may be perfectly fine for you and not for other people. Yes it's rather complete and yet some people may not like that it doesn't work with sub-day data (at least it didn't when implemented) or because it doesn't implement what people are looking for.

The example from above uses replaydata. If this were not used, something that could be considered to speed up things would be:

• Replace the load cycle in a data feed with a plain one which simply loops over the data.

This precludes the usage of filters (including resampling and replaying) With that in mind the usage of pd.Series would make sense when one is for example using a PandasData feed.

In that case instead of copying the data from the Dataframe, each column from the DataFrame would take the place of a given line and a pointer could be used to indicate the current location to access the data.

This is a snippet that shows how it could be done ...

class InfluxDB(bt.DataBase):
    def start(self):
        super(InfluxDB, self).start()

        self.ndb = GetConnectionToInflux()

        # The query could already consider parameters like fromdate and todate
        # to have the database skip them and not the internal code
        q = ('SELECT mean("close_p") AS "close", mean("open_p") '
             'AS "open", mean("volume") AS "volume" FROM "autogen".'
             '"{dataname}" WHERE time > now() - 5d GROUP BY time(1d)'
             'fill(linear)')

        dbars = self.ndb.query(q.format(dataname=self.p.dataname))

        # The sample returns something called ResultSet which seems to contain
        # a dict with a single key and a list of dictionaries with the values
        # Let's assume for the sake of the example that it can be addressed
        # like a dict
        self.bars = dbars[(self.p.dataname, None)]
        self.biter = iter(bars)

    def _load(self):
        bar = next(self.biter)
        if bar['close'] is None:
            return False

        self.l.datetime[0] = datetime.strptime(bar['time'],
                                               '%Y-%m-%dT%H:%M:%SZ')

        self.l.open[0] = bar['open']
        self.l.high[0] = bar['high']
        self.l.low[0] = bar['low']
        self.l.close[0] = bar['close']
        self.l.volume[0] = bar['volume']

        return True

Real operational details pertaining to the connection to the InfluxDB and the result object ResultSet need to be worked out.

Adding a use case to support uses cases which don't simply go over the entire range of parameters could be considered.

The question is:

• how does one decide if a given parameter combination is the path to follow?

Some options:

• Greater total value
• Larger Sharpe Ratio
• Many others

Some of the analyzers return dict which contain for example datetime timestamps and returns (for a given timeframe) and the core code cannot consider analyzers that people may plug in with unknown values.

Some kind of callback would have to be implemented in each strategy to return a score.

The Raspberry Pi was never in doubt, because there are several complete Linux distributions. The other target (a freetzed Fritz!Box 7490) has no pandas, numpy or anything else.

Version number is for sure a good start. In case you have been following the latest discussions in the community, the resampling code has been fine tuned across the last 3-4 releases to improve bar delivery and synchronization. The end of session is being used as much as possible to decide where the limit is for subdays and days resampling.

The tests done and the experiences in the other threads seem to confirm the validity of the changes, but some corners could still be there.

In order to understand what happens, it would be good to know:

• What is not working
• How the use case looks like (a simple streamlined script, no logic in next)

That's how things have positively been worked out, along several sessions, in the other thread working on this.

@Ed-Bartosh said in How to speed up backtest:

You can imagine how many days users would wait if they want to run a strategy that operates for example all S&P 500 securities on the Raspberry Pi mentioned above

The Raspberry Pi would be doing live trading with a 2 line logic:

1. Buy when it's cheap
2. Sell when it's expensive. Go back to 1

@Ed-Bartosh said in How to speed up backtest:

btw, "stdstats=False" approach didn't help. It saved 3 sec out of more than 2 minutes of backtest run.

It wasn't meant to give you a 50% savings.

Can you point out which parts of backtester have to be changes to support pandas? How complex this task would be from your point of view?

It's not just supporting pandas, as in making pd.Series the underlying buffer. There is plenty of code which is pure python doing the overriding of operators and managing the indirection of the logic to make 0 the pivoting point pointing to the current data and -1 the last data that was delivered. And also having single lines objects and multiple lines objects being compatible.

In the case of exactbars=1, the entire buffering system is switched to collections.deque and there is no real speed improvement.

As an experiment, it may be worth (or fun?) to produce a version which can switch the underlying buffer for lines to pd.Series.

This was outlined here: Community - Importing orders #219

Your options:

• The data feed contains the extra data. You then read it directly from the data feed

• With separate data you load it into an indicator and check if the timestamp of the data feed matches the next timestamp of the action to be executed. And the indicator may signal with 1 (Buy) and -1 (sell), plus potentially extra lines for size, price and some other things.

• As you point out ... you can pass it as a parameter to the strategy and do the check with the trading logic.

The indicator solution seems the cleanest, because it does isolate data and logic.