Navigation

    Backtrader Community

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

    Help: Correlation between two symbols from IB

    General Code/Help
    2
    6
    990
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • A
      aGiant last edited by aGiant

      Hi guys,

      the codes from backtrader web for correlation between two symbols from IB didn't work, is this a bug?

      class PearsonR(bt.ind.PeriodN):
          _mindatas = 2  # hint to the platform
          lines = ('correlation',)
          params = (('period', 20),)
          def next(self):
              #print(dir(self))
              print(self.dnames)
              #print(self.datas[0])
              c = scipy.stats.pearsonr(np.array(self.data0.get(size=self.p.period)),
                                       np.array(self.data1.get(size=self.p.period)))
              self.lines.correlation[0] = c[0]
      
      cerebro = bt.Cerebro(stdstats=False)
      store = bt.stores.IBStore(port=4002)#7499
      data0 = store.getdata(dataname='EUR.USD-CASH-IDEALPRO', timeframe=bt.TimeFrame.Minutes)
      cerebro.adddata(data0, name='d0')
      cerebro.resampledata(data0, timeframe=bt.TimeFrame.Minutes, compression=1)
      
      data1 = store.getdata(dataname='EUR.GBP-CASH-IDEALPRO', timeframe=bt.TimeFrame.Minutes)
          #cerebro.adddata(data1, name='d1')
      cerebro.resampledata(data1, timeframe=bt.TimeFrame.Minutes, compression=1)
      print(dir(cerebro))
      print(cerebro.datasbyname)
      print(cerebro.datas)
      cerebro.addstrategy(St)
      

      There was no self.data1 that should be transferred to this indicator. And there was no get() for self.data0 too if we printed dir(self.data0).

      Many thanks! @backtrader

      A 1 Reply Last reply Reply Quote 0
      • A
        aGiant @aGiant last edited by

        Error:

         File "C:\ProgramData\Anaconda3\lib\site-packages\backtrader\cerebro.py", line 1127, in run
            runstrat = self.runstrategies(iterstrat)
          File "C:\ProgramData\Anaconda3\lib\site-packages\backtrader\cerebro.py", line 1264, in runstrategies
            strat._start()
          File "C:\ProgramData\Anaconda3\lib\site-packages\backtrader\strategy.py", line 385, in _start
            self._stage2()
          File "C:\ProgramData\Anaconda3\lib\site-packages\backtrader\lineiterator.py", line 181, in _stage2
            lineiterator._stage2()
          File "C:\ProgramData\Anaconda3\lib\site-packages\backtrader\lineiterator.py", line 177, in _stage2
            data._stage2()
          File "C:\ProgramData\Anaconda3\lib\site-packages\backtrader\lineiterator.py", line 181, in _stage2
            lineiterator._stage2()
          File "C:\ProgramData\Anaconda3\lib\site-packages\backtrader\lineiterator.py", line 177, in _stage2
            data._stage2()
          File "C:\ProgramData\Anaconda3\lib\site-packages\backtrader\lineiterator.py", line 181, in _stage2
            lineiterator._stage2()
          File "C:\ProgramData\Anaconda3\lib\site-packages\backtrader\lineiterator.py", line 177, in _stage2
            data._stage2()
          File "C:\ProgramData\Anaconda3\lib\site-packages\backtrader\lineiterator.py", line 181, in _stage2
            lineiterator._stage2()
        1 Reply Last reply Reply Quote 0
        • A
          aGiant last edited by

          There is no get() function for self.data1:

          b__', '__subclasshook__', '__truediv__', '__weakref__', '_add2stack', '_barstack', '_barstash', '_calendar', '_check', '_clone', '_compensate', '_compression', '_dataname', '_dlen', '_env', '_feed', '_ffilters', '_filters', '_fromstack', '_getline', '_getnexteos', '_getstatusname', '_gettz', '_gettzinput', '_id', '_last', '_laststatus', '_load', '_makeoperation', '_makeoperationown', '_minperiod', '_name', '_operation', '_operation_stage1', '_operation_stage2', '_operationown', '_operationown_stage1', '_operationown_stage2', '_opstage', '_owner', '_plotlabel', '_preloading', '_qcheck', '_roperation', '_save2stack', '_stage1', '_stage2', '_start', '_start_finish', '_started', '_store', '_tick_fill', '_tick_nullify', '_timeframe', '_timeoffset', '_tmoffset', '_tz', '_tzinput', '_updatebar', 'addfilter', 'addfilter_simple', 'addminperiod', 'advance', 'advance_peek', 'alias', 'aliased', 'array', 'backwards', 'clone', 'compensate', 'copyas', 'csv', 'data', 'date2num', 'do_qcheck', 'extend', 'forward', 'fromdate', 'frompackages', 'get_notifications', 'getenvironment', 'getfeed', 'getwriterheaders', 'getwriterinfo', 'getwritervalues', 'haslivedata', 'home', 'incminperiod', 'islive', 'l', 'line', 'line0', 'line1', 'line2', 'line3', 'line4', 'line5', 'line6', 'line_0', 'line_1', 'line_2', 'line_3', 'line_4', 'line_5', 'line_6', 'linealias', 'lines', 'load', 'minbuffer', 'next', 'nextstart', 'notifs', 'num2date', 'once', 'oncestart', 'p', 'packages', 'params', 'plotinfo', 'plotlabel', 'plotlines', 'preload', 'prenext', 'preonce', 'put_notification', 'qbuffer', 'replay', 'replaying', 'resample', 'resampling', 'reset', 'rewind', 'sessionend', 'sessionstart', 'setenvironment', 'setminperiod', 'start', 'stop', 'todate', 'updateminperiod']
          1 Reply Last reply Reply Quote 0
          • B
            backtrader administrators last edited by

            Sorry, but it is very difficult to follow the logic of what you are trying to show. There is no complete code sample, which makes it impossible to track any error.

            For example:

            @agiant said in Help: Correlation between two symbols from IB:

            And there was no get() for self.data0 too if we printed dir(self.data0).

            You have the wrong expectation that dir is going to tell you what's going to be executed. If you call get it will be there.

            A 1 Reply Last reply Reply Quote 0
            • A
              aGiant @backtrader last edited by

              @backtrader

              from __future__ import (absolute_import, division, print_function,
                                      unicode_literals)
              import backtrader as bt
              import redis
              import itertools
              import scipy.stats
              import numpy as np
              
              set_currency = ['EUR.USD-CASH-IDEALPRO', 'EUR.GBP-CASH-IDEALPRO', 'USD.GBP-CASH-IDEALPRO']
              set_period_ind = ['MOM', 'RSI', 'STDDEV', 'LINEARREG_ANGLE', 'LINEARREG_SLOPE', 'SMA', 'CMO', 'ROCP']
              set_period_ind_param = [3, 5, 8, 13, 15, 18, 23, 31, 44, 59, 80, 114, 169, 258, 402, 635]
              set_state_ind_1 = ['HT_TRENDMODE'] #'HT_DCPERIOD', 'HT_DCPHASE',
              set_state_ind_2 = ['HT_PHASOR'] #, 'HT_SINE'
              set_oscillator_stoch = ['STOCH']
              set_oscillator_stochrsi = ['STOCHRSI']
              set_oscillator_macd = ['MACD']
              set_oscillator_param = itertools.combinations(set_period_ind_param, 3)
              
              r = redis.Redis(host='localhost', port=6379, db=0)
              #r.set('foo','bar')
              #r.set('dict_test',{"Name":"Pradeep", "Company":"SCTL", "Address":"Mumbai", "Location":"RCP"})
              
              class PearsonR(bt.Indicator):
                  #_mindatas = 2  # hint to the platform
              
                  lines = ('correlation',)
                  params = (('period', 20),)
              
                  #def __init__(self):#, *args, **kwargs):
                      #if 'datas' not in kwargs:
                      #    raise bt.errors.BacktraderError()
              
                      #self.addminperiod(self.p.period)
              
                      #self.datasByName = kwargs['datas']
              
                      #self.first = self.datasByName['first']
                      #self.second = self.datasByName['second']
                      #self.lines.correlation = scipy.stats.pearsonr
                      #print(dir(self.datas))
                      #print('*****************')
                  def next(self):#, *args, **kwargs):
                      #print(dir(self))
                      print(self.dnames)
                      #print(self.datas[0])
                      c = scipy.stats.pearsonr(np.array(self.data0.get(size=self.p.period)),
                                               np.array(self.data1.get(size=self.p.period)))
                      print(self.p.period,": ", c)
                      #for i in np.array(self.data0.get(size=self.p.period)):
                      #    print(i)
                      print("-----------")
                      #for i in np.array(self.data1.get(size=self.p.period)):
                      #    print(i)
                      exit()
                      self.lines.correlation[0] = c[0]
              
              class St(bt.Strategy):
                  def __init__(self):
                      print(self.dnames)
                      self.indicators = {}
                      self.indicators_names = []
                      #mxL = max(set_period_ind_param)
                      #L = len(self.data.open) - mxL - 1  # - max(set_pioder_ind_param)
                      for s in set_period_ind:
                          ret = getattr(bt.talib, s)
                          for i, t in enumerate(set_period_ind_param):
                              # print(s+'_'+str(t))
                              self.indicators[s + '_' + str(t)] = ret(self.data0, timeperiod=t)
                              # print(len(ret(data,timeperiod=t)))
                      '''
                      #oscilators
                      set_oscillator_param = itertools.combinations(set_period_ind_param, 2)
                      for s in set_oscillator_stoch:
                          func = getattr(bt.talib, s)
                          for i, item in enumerate(set_oscillator_param):
                              # high, low, close and slow, fast1, type = 0, fast2, type = 0
                              # d[s + '_fast_' + str(i)]
                              #self.indicators[s + '_slowk_' + str(i)], self.indicators[s + '_slowd_' + str(i)]  = func(self.data.high, self.data.low, self.data.close, fastk_period=item[1], slowk_period=item[0], slowd_period=item[0])
                              self.indicators[s + '_slowk_' + str(i)] = func(self.data.high, self.data.low, self.data.close, fastk_period=item[1], slowk_period=item[0], slowd_period=item[0])
                              self.indicators[s + '_slowk_' + str(i)] = func(self.data.high, self.data.low, self.data.close, fastk_period=item[1], slowk_period=item[0], slowd_period=item[0])
                              #MACD([input_arrays], [fastperiod=12], [slowperiod=26], [signalperiod=9])
              
                      for s in set_oscillator_stoch:
                          func = getattr(bt.talib, s)
                          for i, item in enumerate(set_oscillator_param):
                              # high, low, close and slow, fast1, type = 0, fast2, type = 0
                              # d[s + '_fast_' + str(i)]
                              #self.indicators[s + '_slowk_' + str(i)], self.indicators[s + '_slowd_' + str(i)]  = func(self.data.high, self.data.low, self.data.close, fastk_period=item[1], slowk_period=item[0], slowd_period=item[0])
                              self.indicators[s + '_slowk_' + str(i)] = func(self.data.high, self.data.low, self.data.close, fastk_period=item[1], slowk_period=item[0], slowd_period=item[0])
                              #MACD([input_arrays], [fastperiod=12], [slowperiod=26], [signalperiod=9])
              
                      set_oscillator_param = itertools.combinations(set_period_ind_param, 3)
                      for s in set_oscillator_macd:
                          func = getattr(bt.talib, s)
                          for i, item in enumerate(set_oscillator_param):
                              # fastk, fastd = STOCHRSI(close, timeperiod=14, fastk_period=5, fastd_period=3, fastd_matype=0)
                              # item = (14, 21, 34)
                              self.indicators_names.append(s + '_' + str(i))
                              self.indicators[s + '_' + str(i)] = func(self.data.close, fastperiod=item[1], slowperiod=item[2], signalperiod=item[0])
                              #print(self.indicators[s + '_' + str(i)][0])
                      '''
                      #print(dir(self.data0))
                      #print(dir(self.data1))
                      #exit()
                      #PearsonR(self.data0, self.data1)
                      for i, t in enumerate(set_period_ind_param):
                              # print(s+'_'+str(t))
                              self.indicators['PearsonR' + '_' + str(t)] = bt.talib.CORREL(self.data0.close, self.data1.close, timeperiod=t)
                      print(bt.talib.CORREL.__doc__)
                      #print(dir(self))
                      #print('{:.6f}'.format(self.data0.open[0]))
                      #print('{:.6f}'.format(self.data1.open[0]))
                      #print('{:.6f}'.format(self.data2.open[0]))
                      #exit()
                      #for i in self.indicators:
                      #    print(self.indicators[i][0])
                      #exit()
                      #bt.indicators.Stochastic(self.data)
                      #self.sma = getattr(bt.talib, "SMA")(self.data, period=20)#btind.SMA(self.data, period=20)
                      #getattr(self, "sma") = getattr(bt.talib, "SMA")(self.data, period=20)#btind.SMA(self.data, period=20)
                      #self.stoc = bt.talib.STOCH(self.data.high, self.data.low, self.data.close, fastk_period=14, slowk_period=3, slowd_period=3)
              
              
                  def logdata(self):
                      txt = []
                      prices = {}
                      txt.append('{}'.format(len(self)))
                      key = '{}'.format(self.data.datetime.datetime(0).isoformat())
                      txt.append(key)
                      #print(self.data.Open)#(dir(self.data))
                      open = self.data0.open[0]#(dir(self.data)
                      txt.append(open)
                      high = '{:.6f}'.format(self.data0.high[0])
                      txt.append(high)
                      low = '{:.6f}'.format(self.data0.low[0])
                      txt.append(low)
                      close = '{:.6f}'.format(self.data0.close[0])
                      txt.append(close)
                      volume = '{:.6f}'.format(self.data0.volume[0])
                      txt.append(volume)
                      #print(self.getindicators())
                      #for i in self.indicators:
                      #    print(i)
              
                      #self.sma = bt.talib.SMA(self.data.open-self.data.close, period = 20)
                      #print(bt.talib.STOCH.__doc__)
                      #print(bt.talib.STOCHRSI.__doc__)
                      #print(bt.talib.MACD.__doc__)
                      #print(dir(sma2))
                      #print(self.dnames)
                      print('{:.6f}'.format(self.data0.open[0]))
                      print('{:.6f}'.format(self.data1.open[0]))
                      print('{:.6f}'.format(self.data2.open[0]))
                      #print('{:.6f}'.format(self.data.open[0]))
                      #print('{:.6f}'.format(self.data0_1[0]))
                      #print('{:.6f}'.format(self.data1_1[0]))
                      exit()
                      #print(len(self.data.open))
                      #print('{:.6f}'.format(self.sma.data.high[0]))
                      #print(dir(sma2))0
                      #print('{:.6f}'.format(self.sma[0]))
                      #print(dir(bt.talib))
                      #for i in self.indicators:
                      #    print(self.indicators[i][0])
                      #for i in self.indicators_names:
                      #    print(len(self.indicators[i][0]))
                      #exit()
                      print(txt)
              
                      #print(','.join(txt))
                      #write data to redis should be here
                      prices = {'Key':key, 'Open':open, 'High':high, 'Low':low, 'Close':close, 'Volume':volume}
                      indicat = dict()
                      for i in self.indicators:
                          indicat[i] = self.indicators[i][0]
                      prices.update(indicat)
                      r.hmset(key, prices)
              
                  def next(self):
                      self.logdata()
              
              
              def run(args=None):
                  #TWTR
                  cerebro = bt.Cerebro(stdstats=False)
                  store = bt.stores.IBStore(port=4002)#7499
                  #data0 = store.getdata(dataname='EUR.USD-CASH-IDEALPRO', timeframe=bt.TimeFrame.Minutes)
                  #cerebro.adddata(data0, name='data0')
                  #cerebro.resampledata(data0, timeframe=bt.TimeFrame.Minutes, compression=1)
              
                  for c in set_currency:
                      data1 = store.getdata(dataname=c, timeframe=bt.TimeFrame.Minutes)
                  #cerebro.adddata(data1, name='data1')
                      cerebro.resampledata(data1, timeframe=bt.TimeFrame.Minutes, compression=1)
                  #print(dir(cerebro))
                  #print(cerebro.datasbyname)
                  #print(cerebro.datas)
                  cerebro.addstrategy(St)
              
                  cerebro.run()
              
              
              if __name__ == '__main__':
                  run()
              

              The ideal is to calculate the indicators and save them in databse. Triangel correlations are considerred. But it didn't work at all.

              B 1 Reply Last reply Reply Quote 0
              • B
                backtrader administrators @aGiant last edited by

                @agiant said in Help: Correlation between two symbols from IB:

                But it didn't work at all.

                Again: nobody can actually understand what didn't work or what worked. There is no error log, no log at all. The only thing you present is a bunch of code lines in which many are simply commented out, whether for a good reason or not, who knows. And many of those commented lines show things which are astonishing. For example:

                @agiant said in Help: Correlation between two symbols from IB:

                    #def __init__(self):#, *args, **kwargs):
                        #if 'datas' not in kwargs:
                        #    raise bt.errors.BacktraderError()
                
                        #self.addminperiod(self.p.period)
                
                        #self.datasByName = kwargs['datas']
                
                

                Why are (or were) you expecting datas in kwargs?

                Furthermore,

                @agiant said in Help: Correlation between two symbols from IB:

                    def next(self):#, *args, **kwargs):
                        #print(dir(self))
                        print(self.dnames)
                        #print(self.datas[0])
                        c = scipy.stats.pearsonr(np.array(self.data0.get(size=self.p.period)),
                                                 np.array(self.data1.get(size=self.p.period)))
                        print(self.p.period,": ", c)
                        #for i in np.array(self.data0.get(size=self.p.period)):
                        #    print(i)
                        print("-----------")
                        #for i in np.array(self.data1.get(size=self.p.period)):
                        #    print(i)
                        exit()
                

                How is anything going to work if you call exit()?

                1 Reply Last reply Reply Quote 1
                • 1 / 1
                • First post
                  Last post
                Copyright © 2016, 2017, 2018, 2019, 2020, 2021 NodeBB Forums | Contributors