Navigation

    Backtrader Community

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

    Hidden Markov Model - Create Indicator & use in strategy

    Indicators/Strategies/Analyzers
    3
    9
    1283
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Amw_775
      Amw_775 last edited by Amw_775

      Hi I'm trying to create a hidden markov model. I'd create the model as an indicator.
      The model requires to be fitted to the data first, using the fit function. Which requires 'period' number of data points to be present.
      I have created the indicator with a basic strategy template but I keep getting the error below.
      I'd really appreciate if any one can help me with the code, whether its placed correctly. There's plenty of documentation for simple indicators but there's limited guidance in the documentation for any advanced models.

      from future import (absolute_import, division, print_function, unicode_literals)

      import datetime # For datetime objects
      import os.path # To manage paths
      import sys # To find out the script name (in argv[0])
      import backtrader as bt
      import numpy as np
      from hmmlearn import hmm

      Create Indicator

      class HMM(bt.Indicator):
      lines = ('q', 'p')
      params = (('n', 500), ('states', 2) ,('period',100)) # n iterations and number of states

      def __init__(self):
           self.addminperiod(self.params.period)
           #self._dlast = self.data(-1)  # get previous day value
      
      def next(self):
          r = self.data.get(ago = 0, size = self.params.period)
          ret = bt.ind.PctChange(self.data, period=1)
          mdl2 = hmm.GaussianHMM(n_components=2, n_mix=1, covariance_type='diag', covars_prior=0.01,
                                     algorithm='viterbi',n_iter=100)
      
          # Fits the model till the t-1 observation
          mdl2.fit(self.data0,lengths = None)
          # print("Model Score : " , score  )
          #score = mdl2.score(ret, lengths=None)
      
          self.lines.q = q = (mdl2.predict(ret, lengths=None))
      
          a = (mdl2.transmat_)
          a0j = a[0, :]  # State 0 to 0, or 0 to 1
          a1j = a[1, :]  # State 1 to 0, or 1 to 1
      
              # print("Trans Probaility:  " , a)
              # print("If Current State =0, then TransProb = ", a0j)
              # print("If Current State =1, then TransProb = ",a1j)
          q0 = np.where(a0j == a0j.max())
          q1 = np.where(a1j == a1j.max())
              # If Current predicted state is 0, then q0[0] Returns the next State Value with highest Trans Probability
          if q[-1] == 0:
              self.lines.p = q0[0]
              print("Q=0, thus Next Predicted State  ", self.lines.p)
                  # If CUrrent predicted state is 1, then q1[1] Returns the next State Value with highest Trans Probability
          if q[-1] == 1:
              self.lines.p = q1[0]
              print("Q=0, thus Next Predicted State  ", self.lines.p)
      

      Create a Stratey

      class TestStrategy(bt.Strategy):
      params = {'n': 500, 'states': 2 , 'period': 500}

      def log(self, txt, dt=None):
          ''' Logging function for this strategy'''
          dt = dt or self.datas[0].datetime.date(0)
          print('%s, %s' % (dt.isoformat(), txt))
      
      def __init__(self):
          # Keep a reference to the "close" line in the data[0] dataseries
          self.dataclose = self.datas[0].close
      
          # Add an indicator
          self.indicator = HMM(500, self.data0, 100)
      
      
      
      def next(self):
          # Simply log the closing price of the series from the reference
          self.log('Close, %.2f' % self.dataclose[0])
      
          if self.dataclose[0] < self.dataclose[-1]:
              # current close less than previous close
      
              if self.dataclose[-1] < self.dataclose[-2]:
                  # previous close less than the previous close
      
                  # BUY, BUY, BUY!!! (with all possible default parameters)
                  self.log('BUY CREATE, %.2f' % self.dataclose[0])
                  self.buy()
      

      if name == 'main':
      # Create a cerebro entity
      cerebro = bt.Cerebro()

      # Add a strategy
      cerebro.addstrategy(TestStrategy)
      
      # Datas are in a subfolder of the samples. Need to find where the script is
      # because it could have been called from anywhere
      modpath = os.path.dirname(os.path.abspath(sys.argv[0]))
      datapath = os.path.join(modpath, 'D:/data/GLD.txt')
      
      # Create a Data Feed
      data = bt.feeds.GenericCSVData(dataname=datapath ,
          fromdate=datetime.datetime(2005, 1, 1), todate=datetime.datetime(2015, 12, 31),
          nullvalue=0.0, headers=True,dtformat=('%m/%d/%Y'),tmformat=('%H.%M'),seperator=",",datetime=0,time=-1,
          open=2,high=3,low=4,close=5,volume=6,openinterest=7 # reverse=True,
      )
      
      # Add the Data Feed to Cerebro
      cerebro.adddata(data)
      
      # Set our desired cash start
      cerebro.broker.setcash(100000.0)
      
      # Print out the starting conditions
      print('Starting Portfolio Value: %.2f' % cerebro.broker.getvalue())
      #print('Close: %.2f' % self.dataclose())
      # Run over everything
      cerebro.run()
      
      # Print out the final result
      print('Final Portfolio Value: %.2f' % cerebro.broker.getvalue())
      

      Results in the following Error:

      D:\WinPython-3670\backtrader\Scripts\python.exe D:/WinPython-3670/backtrader/HMM_indicator.py
      Starting Portfolio Value: 100000.00
      Traceback (most recent call last):
      File "D:/WinPython-3670/backtrader/HMM_indicator.py", line 113, in <module>
      cerebro.run()
      File "D:\WinPython-3670\backtrader\backtrader\cerebro.py", line 1127, in run
      runstrat = self.runstrategies(iterstrat)
      File "D:\WinPython-3670\backtrader\backtrader\cerebro.py", line 1217, in runstrategies
      strat = stratcls(*sargs, **skwargs)
      File "D:\WinPython-3670\backtrader\backtrader\metabase.py", line 88, in call
      _obj, args, kwargs = cls.doinit(_obj, *args, **kwargs)
      File "D:\WinPython-3670\backtrader\backtrader\metabase.py", line 78, in doinit
      _obj.init(*args, **kwargs)
      File "D:/WinPython-3670/backtrader/HMM_indicator.py", line 66, in init
      self.indicator = HMM(500, self.data0, 100)
      File "D:\WinPython-3670\backtrader\backtrader\indicator.py", line 53, in call
      return super(MetaIndicator, cls).call(*args, **kwargs)
      File "D:\WinPython-3670\backtrader\backtrader\metabase.py", line 88, in call
      _obj, args, kwargs = cls.doinit(_obj, *args, **kwargs)
      File "D:\WinPython-3670\backtrader\backtrader\metabase.py", line 78, in doinit
      _obj.init(*args, **kwargs)
      TypeError: init() takes 1 positional argument but 2 were given

      Process finished with exit code 1

      B 1 Reply Last reply Reply Quote 0
      • A
        ab_trader last edited by ab_trader

        I might be wrong, but you call your indicator with three arguments:

        self.indicator = HMM(500, self.data0, 100)
        

        but based on your implementation it doesn't need any arguments:

        class HMM(bt.Indicator):
        
        def __init__(self):
             self.addminperiod(self.params.period)
        

        You need to describe these arguments in the __init()__.

        1 Reply Last reply Reply Quote 0
        • B
          backtrader administrators last edited by

          The poster is trying to assign a value to the declared params, which are barely readable due to the lack of formattting.

          self.myindicator = MyIndicator(n=xxxx)
          

          Because each parameter has a name.

          Amw_775 1 Reply Last reply Reply Quote 2
          • A
            ab_trader last edited by ab_trader

            @backtrader I see, missed during first look.

            It was a chat here some time ago. I am not able to find now, did you remove it?

            1 Reply Last reply Reply Quote 0
            • Amw_775
              Amw_775 @backtrader last edited by

              @backtrader Hi What should I do ? Any help would be greatly appreciated ?
              Would you like me to clean up the code and report ?

              I 've searched the community forum and web...I cannot understand which params need need to be initiated/declared under which section?

              every article has the user doing it differently.

              1 Reply Last reply Reply Quote 0
              • B
                backtrader administrators @Amw_775 last edited by backtrader

                @arjun-whabi said in Hidden Markov Model - Create Indicator & use in strategy:

                There's plenty of documentation for simple indicators but there's limited guidance in the documentation for any advanced models.

                Sorry, but the trees are not letting you see the forest. This has nothing to do with advanced, simple or rocket science models. It's about the arguments you are passing and how you are passing them.

                It's clear in the error message

                @arjun-whabi said in Hidden Markov Model - Create Indicator & use in strategy:

                TypeError: __init__() takes 1 positional argument but 2 were given
                

                @arjun-whabi said in Hidden Markov Model - Create Indicator & use in strategy:

                I cannot understand which params need need to be initiated/declared under which section?

                You apparently already understand it.

                @arjun-whabi said in Hidden Markov Model - Create Indicator & use in strategy:

                params = (('n', 500), ('states', 2) ,('period',100))  # n iterations and number of states
                

                But you are expecting magic

                @arjun-whabi said in Hidden Markov Model - Create Indicator & use in strategy:

                    # Add an indicator
                    self.indicator = HMM(500, self.data0, 100)
                

                How should the indicator know what the usage of 500 and 100 is and to what has to be assigned to?

                My suggestion. Read this:

                • Docs - Platform Concepts - Parameters

                And probably also this section within the same page

                • Docs - Platform Concepts - Data Feeds - Passing them around
                1 Reply Last reply Reply Quote 1
                • Amw_775
                  Amw_775 last edited by

                  @backtrader thanks for the help and the links.
                  I made the mistake that I was not naming the params in the strategy section. Once I named self.p.n , self.p.states and so on it worked.

                  I'm stuck again in the below bt.indicator section:
                  I'm required to convert the price to an array before fitting it to the model.

                  class HMM(bt.Indicator):
                  lines = ('q', 'p')
                  params = (('n', 500), ('states', 2) ,('period',500)) # n iterations and number of states

                  def __init__(self):
                       self.addminperiod(self.params.period)
                  
                  def next(self):
                      r = np.array(self.data.get( size=self.params.period))
                      print(' type r : ' , type(r))
                      print(' len r : ' , len(r))
                      print(' r shape: ', r.shape)
                      ret = r.reshape(-1,1)
                  
                      print(ret)
                  
                      mdl2 = hmm.GaussianHMM(n_components=self.p.states, covariance_type='diag', covars_prior=0.01,
                                             algorithm='viterbi', n_iter=self.p.n)
                  
                      # Fits the model 
                      mdl2.fit(ret,lengths = None)
                  

                  it throws the following error:

                  Traceback (most recent call last):
                  File "D:/WinPython-3670/backtrader/HMM_indicator.py", line 117, in <module>
                  cerebro.run()
                  File "D:\WinPython-3670\backtrader\backtrader\cerebro.py", line 1127, in run
                  runstrat = self.runstrategies(iterstrat)
                  File "D:\WinPython-3670\backtrader\backtrader\cerebro.py", line 1293, in runstrategies
                  self._runonce(runstrats)
                  File "D:\WinPython-3670\backtrader\backtrader\cerebro.py", line 1652, in _runonce
                  strat._once()
                  File "D:\WinPython-3670\backtrader\backtrader\lineiterator.py", line 297, in _once
                  indicator._once()
                  File "D:\WinPython-3670\backtrader\backtrader\lineiterator.py", line 317, in _once
                  self.oncestart(self._minperiod - 1, self._minperiod)
                  File "D:\WinPython-3670\backtrader\backtrader\indicator.py", line 124, in oncestart_via_nextstart
                  self.nextstart()
                  File "D:\WinPython-3670\backtrader\backtrader\lineiterator.py", line 347, in nextstart
                  self.next()
                  File "D:/WinPython-3670/backtrader/HMM_indicator.py", line 40, in next
                  self.lines.q = q = (mdl2.predict(ret, lengths=None))
                  File "D:\WinPython-3670\backtrader\backtrader\lineseries.py", line 79, in set
                  value = value(0)
                  TypeError: 'numpy.ndarray' object is not callable

                  Any suggestions ?

                  1 Reply Last reply Reply Quote 0
                  • Amw_775
                    Amw_775 last edited by

                    @backtrader Sorry just moving over from TradeStation platform to python so I'm not the best with the language. Appreciate your help a lot

                    1 Reply Last reply Reply Quote 0
                    • B
                      backtrader administrators last edited by

                      You should take 2 (or 3) steps back and develop a simple indicator.

                      From the documentation - Docs - Indicator Development (you should read the entire doc)

                          def next(self):
                              self.lines.dummyline[0] = max(0.0, self.params.value)
                      

                      Now look for that in your code.

                      1 Reply Last reply Reply Quote 1
                      • 1 / 1
                      • First post
                        Last post
                      Copyright © 2016, 2017, 2018 NodeBB Forums | Contributors
                      $(document).ready(function () { app.coldLoad(); }); }