Machine learning + backtrader

junajo10

Hi,

Do you have on your mind to add any machine learning library in backtrader or any ml sample?

Rgds,

Jj

backtrader

Not at the moment.

junajo10

Ok, thanks. In the future if you consider it please let me know. I would like to suggest some ideas

RandyT

Please suggest some ideas. This is an area I plan to go in near future after getting my head fully around current backtrader capability.

revelator101

What's stopping you from doing any machine learning? Why cant you just import the libraries, run the models & execute based on the parameters?

There's plenty of examples and ideas on Quantopian in python:

https://www.quantopian.com/posts/machine-learning-on-quantopian-part-3-building-an-algorithm

RandyT

Great thread. Thanks for sharing.

junajo10

Hi,

In the first phase the kind of things that i want to learn is the indicators that i should use depend on the asset class

https://www.linkedin.com/pulse/selecting-your-indicators-machine-learning-tad-slaff

Finally q-learning Could be the target

ab_trader

At this point machine learning opportunities are endless from both ML feature building and from ML algo selecting points of view. imho implementing and further support of special ML functions/procedures into bt will be just wasting of time.

junajo10

hi,

I was thinking more in defining a style guide that how to combine bt with the best ML libraries. For example in the regression linear example, the use is not direct .

remroc

@junajo10 I maybe we could give it a try if you have an article with an example...

junajo10

Here, i have found a case that could be a good starting point.

https://www.quantopian.com/posts/582de686358fe9440b0007f2#587e9dae9ea10879ef9ecb16

Kalman filter : Pairs trading

remroc

@junajo10 Thanks Junajo10 ! I need some documentation to understand this filter and it is related to ML ?

Do you have any tips ?

Thanks,

Remroc

junajo10

https://www.quantopian.com/posts/quantopian-lecture-series-kalman-filters

https://www.quantstart.com/articles/kalman-filter-based-pairs-trading-strategy-in-qstrader

http://numericalmethod.com/papers/course1/lecture5.pdf

Maxim Korobov

From 101 (csv, pandas) to some ML usage and only for trading + code, not just ML + code or trading theory.
https://www.udacity.com/course/machine-learning-for-trading--ud501

remroc

thanks guys. really good stuff

remroc

Hi guys,

Here is an implementation of Kalman filter in a quantopian algo :
https://www.quantopian.com/posts/ernie-chans-ewa-slash-ewc-pair-trade-with-kalman-filter

Let's tackle this...

Remroc

junajo10

https://github.com/quantopian/research_public/blob/master/lectures/kalman_filter/kalman_filter_pair_trade.py

For me the problem is to migrate this code to backtrader .

backtrader

@junajo10 said in Machine learning + backtrader:

https://www.quantstart.com/articles/kalman-filter-based-pairs-trading-strategy-in-qstrader

Here is a version of the KalmanFilter described in that article (seems 1-to-1 identical to the one that implements Ernie Chan's strategy)

import numpy as np
import pandas as pd


class KalmanFilter(bt.Indicator):
    _mindatas = 2  # needs at least 2 data feeds

    lines = ('et', 'sqrt_qt',)

    params = dict(
        delta=1e-4,
        vt=1e-3,
    )

    def __init__(self):
        self.wt = self.p.delta / (1 - self.p.delta) * np.eye(2)
        self.theta = np.zeros(2)
        self.P = np.zeros((2, 2))
        self.R = None

        self.d1_prev = self.data1(-1)  # data1 yesterday's price

    def next(self):
        F = np.asarray([self.data0[0], 1.0]).reshape((1, 2))
        y = self.d1_prev[0]

        if self.R is not None:  # self.R starts as None, self.C set below
            self.R = self.C + self.wt
        else:
            self.R = np.zeros((2, 2))

        yhat = F.dot(self.theta)
        et = y - yhat

        # Q_t is the variance of the prediction of observations and hence
        # \sqrt{Q_t} is the standard deviation of the predictions
        Qt = F.dot(self.R).dot(F.T) + self.p.vt
        sqrt_Qt = np.sqrt(Qt)

        # The posterior value of the states \theta_t is distributed as a
        # multivariate Gaussian with mean m_t and variance-covariance C_t
        At = self.R.dot(F.T) / Qt
        self.theta = self.theta + At.flatten() * et
        self.C = self.R - At * F.dot(self.R)

        # Fill the lines
        self.lines.et[0] = et
        self.l.sqrt_qt[0] = sqrt_Qt

The filter per-se is not directly used in the decision making process. Knowing (from the article) which relations et and sqrt_qt need to have to trigger signals, the behavior can be packed in another indicator

class KalmanSignals(bt.Indicator):
    _mindatas = 2  # needs at least 2 data feeds

    lines = ('long', 'short',)

    def __init__(self):
        kf = KalmanFilter(self.data0, self.data1)
        et, sqrt_qt = kf.lines.et, kf.lines.sqrt_qt

        self.lines.long = et < -1.0 * sqrt_qt
        # longexit is et > -1.0 * sqrt_qt ... the opposite of long
        self.lines.short = et > sqrt_qt
        # shortexit is et < sqrt_qt ... the opposite of short

And the signals can be easily applied in the strategy next method for an easy to follow logic

class St(bt.Strategy):
    params = (
    )

    def __init__(self):
        self.ksig = KalmanSignals(self.data0, self.data1)

    def next(self):
        size = self.position.size
        if not size:
            if self.ksig.long:
                self.buy()
            elif self.ksig.short:
                self.sell()

        elif size > 0:
            if not self.ksig.long:
                self.close()
        elif not self.ksig.short:  # implicit size < 0
            self.close()

The difficult thing here is to select assets for which it makes sense to apply the filter.

junajo10

Thanks @backtrader .

@backtrader said "The difficult thing here is to select assets for which it makes sense to apply the filter".

For solving thsi problem use these tecniques:
Augmented Dickey Fuller
Hurst exponent
Half Life

Maxim Korobov

@backtrader, add this filter to main package.