Hurst Exponent behavior in a Synthetic Trend Serie
Hi, Backtrader is fabulous! Mementum, congratulations on your work.
I tried to replicate the analysis done by Duk2 in this article El Exponente de Hurst y la memoria de los precios en bolsa, referring to the exponent of Hurst, but using Backtrader.
In her article she calculates a single value of the Hurst exponent for all the time serie, but I tried to use the Backtrader rolling indicator with three different periods (which generate different lags).
For this I created a syntetic trend serie. But the result surprised me because in the three periodsHurst exponent is less than 0.5, when it should be close to 1.
Here the code:
from __future__ import (absolute_import, division, print_function, unicode_literals) import datetime import backtrader as bt import pandas as pd from numpy import * class HurstST(bt.Strategy): params = ( ) def __init__(self): # Three Hurst Exponents with diferent periods hurst_1 = bt.ind.HurstExponent(period=40) hurst_2 = bt.ind.HurstExponent(period=200) hurst_3 = bt.ind.HurstExponent(period=1200) def next(self): pass if __name__ == '__main__': cerebro = bt.Cerebro() # Create a synthetic trend time serie in a pandas dataframe tr = pd.DataFrame(pd.date_range(start='2005-1-1', end='2017-01-31', freq='D'), columns=['date']) tr['open'] = tr['high'] = tr['low'] = tr['close'] = pd.DataFrame(log(cumsum(random.randn(100000) + 1) + 1000)) tr.set_index(['date'], inplace=True) # Pass the syntectic dataframe to the backtrader datafeed and add it to the cerebro data = bt.feeds.PandasDirectData(dataname=tr, volume=-1, openinterest=-1 ) cerebro.adddata(data) cerebro.addstrategy(HurstST) cerebro.run(stdstats=False) cerebro.plot()
Here the chart :
Is this a mistake of mine or some problem with the indicator?
The same was stated when it was first developed. See Community - linear regression and std #211
It was, so to say, peer-reviewed. If you feel it's wrong (and that was the 1st opinion) and you can point to an implementation formula/description, it can be for sure changed.
I have checked that the code for the Hurst Exponent is correct and consistent with the method used. However the results are not correct due to the parameters used for the calculation.
Because the nature of the method the result is sensitive to the size of the sample.
Thus I have done simulations using synthetic series (Browian Geometrical Movement, Mean Reversal and Trend), and checking that the values of the Hurst exponent are not consistent with samples of less than about 2000 observations, being stable from about 3000 And 5000 observations according to the sample.
Similarly, the value used to define the lags must be less than about 10 and the upper value of the range should be above 500.
Backtrader uses a range for lags between 2 and half of sample size (period), and I have verified that it is not a good system because even with large samples and large periods is unstable.
The simulations can be reproduced and tested in this Jupyter Notebook:
I also propose to modify the calculation of the Exponent of Hurst in Backtrader with the following code, and to advise in the documentation that is not reliable with period values inferior to 2,000 bars.
from __future__ import (absolute_import, division, print_function, unicode_literals) from . import PeriodN __all__ = ['HurstExponent', 'Hurst'] class HurstExponent(PeriodN): frompackages = ( ('numpy', ('asarray', 'log10', 'polyfit', 'sqrt', 'std', 'subtract')), ) alias = ('Hurst',) lines = ('hurst',) params = (('period', 2000), ('lag_start', 10), ('lag_end', 500),) def __init__(self): super(HurstExponent, self).__init__() # Prepare the lags array self.lags = asarray(range(self.p.lag_start, self.p.lag_end)) self.log10lags = log10(self.lags) def next(self): # Fetch the data ts = asarray(self.data.get(size=self.p.period)) # Calculate the array of the variances of the lagged differences tau = [sqrt(std(subtract(ts[lag:], ts[:-lag]))) for lag in self.lags] # Use a linear fit to estimate the Hurst Exponent poly = polyfit(self.log10lags, log10(tau), 1) # Return the Hurst exponent from the polyfit output self.lines.hurst = poly * 2.0
In any case I find that this index is useful for an initial study of the series to determine what type of strategy is more propitious, but difficult to use in the strategy itself.
Those are very reasonable proposals. Will be added.
Looking into this and to avoid breaking the code anyone may have using this, the best alternative seems to:
lag_endparameters you added but set them to
If any of both values is specified when adding the indicator to the mix, it will be used.
Else: the original values will be used, namely:
self.p.period // 2
And of course:
- Add the recommendations to carefully look into the values and propose longer periods like
2000and the proposed
500for the lag.
Already in the
These is a great solution. Thanks for taking into account my proposals and include them in this great framework.