For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

Logarithmic plotting indicator fail



  • Hi,

    I'm trying to make some sort of logarithmic plotting indicator. By means of cargo cult programming I arrived at the stuff below which doesn't work. Any ideas on how to fix this? As an aside, can we Try, Except the ValueError that will occur with math.log(0)?

    Added to basicops.py:

    class Logarithm(OperationN):
        lines = ('logarithm',)
        func = math.log
    

    logplot.py:

    #!/usr/bin/env python
    # -*- coding: utf-8; py-indent-offset:4 -*-
    
    import backtrader as bt
    from . import Logarithm
    
    class LogPlot(bt.Indicator):
        '''
        Logarithmic plot
        '''
        lines = ('logplot',)
    
        def __init__(self):
            self.lines[0] = Logarithm(self.data + 1)
    
            super(LogPlot, self).__init__()
    

    My wonky idea:

    #!/usr/bin/env python3
    
    import datetime
    import os.path
    import sys
    import backtrader as bt
    from backtrader import Indicator
    
    
    class PeakingStrategy(bt.Strategy):
        params = (('level_period',  500),
                  ('short_period',  100),
                  ('bias',           10),
                  ('printlog',     True))
    
        def log(self, txt, dt=None, doprint=False):
            ''' Logging function for this strategy'''
            if self.params.printlog or doprint:
                dt = dt or self.datas[0].datetime.datetime(0)
                print('%s, %s' % (dt.isoformat(), txt))
    
        def __init__(self):
    
            self.resistance = bt.ind.Highest(self.data.high, period=self.p.level_period, subplot=False)
            self.support    = bt.ind.Lowest(self.data.low,   period=self.p.level_period, subplot=False)
            self.SRL_diff = (self.resistance - self.support)
    
            self.high_trig = bt.ind.Highest(self.data.high, period=self.p.short_period, subplot=False)
            self.low_trig  = bt.ind.Lowest(self.data.low,   period=self.p.short_period, subplot=False)
    
            self.high_diff = self.high_trig - self.data.high
            self.low_diff  = self.low_trig  - self.data.low
    
            self.hma_diff = bt.ind.ExpHullMovingAverage(self.high_diff, period=400) \
                     / bt.ind.ExpHullMovingAverage(self.low_diff, period=400)
            self.lma_diff = bt.ind.ExpHullMovingAverage(self.low_diff, period=400) \
                    / -bt.ind.ExpHullMovingAverage(self.high_diff, period=400)
            log_high = bt.ind.LogPlot(self.hma_diff, subplot=True, plotname='log_high')
            log_low  = bt.ind.LogPlot(self.lma_diff, subplot=True, plotname='log_low')
    
    
    if __name__ == '__main__':
        cerebro = bt.Cerebro()
    
        modpath = os.path.dirname(os.path.abspath(sys.argv[0]))
        datapath = os.path.join(modpath, '../10_min_eurusd_2013.txt')
    
        data = bt.feeds.GenericCSVData(
            dataname=datapath,
            dtformat='%Y%m%d %H%M%S',
            separator=',',
            openinterest=-1,
            timeframe=bt.TimeFrame.Minutes,
            compression=10)
    
        cerebro.adddata(data)
        cerebro.addstrategy(PeakingStrategy)
        cerebro.broker.setcash(1000.0)
        cerebro.addsizer(bt.sizers.PercentSizer, percents=90)
        cerebro.broker.setcommission(commission=0.0002)
    
        cerebro.run()
        cerebro.plot()
    

    The trace back:

    Traceback (most recent call last):
      File "./logtest.py", line 60, in <module>
        cerebro.run()
      File "/mnt/data/code/forex/backtrader/backtrader/cerebro.py", line 1127, in run
        runstrat = self.runstrategies(iterstrat)
      File "/mnt/data/code/forex/backtrader/backtrader/cerebro.py", line 1293, in runstrategies
        self._runonce(runstrats)
      File "/mnt/data/code/forex/backtrader/backtrader/cerebro.py", line 1652, in _runonce
        strat._once()
      File "/mnt/data/code/forex/backtrader/backtrader/lineiterator.py", line 292, in _once
        indicator._once()
      File "/mnt/data/code/forex/backtrader/backtrader/lineiterator.py", line 292, in _once
        indicator._once()
      File "/mnt/data/code/forex/backtrader/backtrader/lineiterator.py", line 312, in _once
        self.oncestart(self._minperiod - 1, self._minperiod)
      File "/mnt/data/code/forex/backtrader/backtrader/lineiterator.py", line 322, in oncestart
        self.once(start, end)
      File "/mnt/data/code/forex/backtrader/backtrader/indicators/basicops.py", line 70, in once
        dst[i] = func(src[i - period + 1: i + 1])
    TypeError: a float is required
    

  • administrators

    @b-steer said in Logarithmic plotting indicator fail:

    TypeError: a float is required
    

    You are passing something which isn't a float. Check your data.

    @b-steer said in Logarithmic plotting indicator fail:

    As an aside, can we Try, Except the ValueError that will occur with math.log(0)?

    You can only do that by checking the value for each step and not in a declarative manner. try ... except is not something which can be overridden in Python.



  • @backtrader said in Logarithmic plotting indicator fail:

    You are passing something which isn't a float. Check your data.

    When defining next() in my strategy like below it appears that the indicator first produces a load of nan's although these are said to be floats. I would have expected to see nan's in the prenext() phase but not in the next() phase.

        def next(self):
            print(self.high_diff[0], type(self.high_diff[0]), isinstance(self.high_diff[0], float))
    

    Also when trying to take the logarithm of an indicator with period=1 (that doesn't produce nan's in next()) I still get the same TypeError. I can't see anything wrong with the input file either, snippet below.

    20130101 170000, 1.32041, 1.32066, 1.32009, 1.32063, 13.203729999999998
    20130101 171000, 1.32064, 1.32072, 1.31997, 1.32062, 13.20387
    20130101 172000, 1.3206, 1.32066, 1.32027, 1.32048, 13.203940000000001
    20130101 173000, 1.32049, 1.32191, 1.32042, 1.32178, 13.210249999999998
    20130101 174000, 1.32177, 1.32179, 1.32112, 1.32116, 13.21461
    20130101 175000, 1.32117, 1.32133, 1.32044, 1.32055, 13.210460000000001
    20130101 180000, 1.32056, 1.32079, 1.31886, 1.3193, 13.196820000000002
    20130101 181000, 1.31929, 1.32057, 1.31883, 1.32012, 13.195849999999998
    20130101 182000, 1.32013, 1.32026, 1.31879, 1.31954, 13.195020000000001
    20130101 183000, 1.31952, 1.31998, 1.3193, 1.31957, 13.195599999999999
    20130101 184000, 1.31956, 1.31984, 1.31954, 1.31973, 13.19666
    
    

  • administrators

    @b-steer said in Logarithmic plotting indicator fail:

       def __init__(self):
            self.lines[0] = Logarithm(self.data + 1)
    

    You are replacing a line in the lines array, which does nothing. You have to assign to the line

    self.lines.logarithm = ...
    

    The former cannot be caught to make lines be bound, the latter can. Hence the source of the NaN values. Although technically a NaN is a float, so it is unclear whether that's your actual problem.



  • @backtrader said in Logarithmic plotting indicator fail:

    You are replacing a line in the lines array, which does nothing. You have to assign to the line

    self.lines.logarithm = ...
    

    The former cannot be caught to make lines be bound, the latter can. Hence the source of the NaN values. Although technically a NaN is a float, so it is unclear whether that's your actual problem.

    It looks like math.log doesn't like the array it is getting, or actually I think it's getting a list with one float. With your suggested fix (self.lines.logarithm = ...) and a hack like below log plotting now works. This will probably break all other operations that accept a list as input so I would need a new class OperationX or similar for operations that accept a single value only. Thanks for your support!

    class OperationN(PeriodN):
        '''
        Calculates "func" for a given period
    
        Serves as a base for classes that work with a period and can express the
        logic in a callable object
    
        Note:
          Base classes must provide a "func" attribute which is a callable
    
        Formula:
          - line = func(data, period)
        '''
        def next(self):
            self.line[0] = self.func(self.data.get(size=self.p.period))
    
        def once(self, start, end):
            dst = self.line.array
            src = self.data.array
            period = self.p.period
            func = self.func
            for i in range(start, end):
                dst[i] = func(src[i - period + 1: i + 1][0])
    

  • administrators

    Indeed, I hadn't checked math.log and it takes a single value. The rationale behind an iterable is that the operations are done on a period of N values. Even if N=1, you are still working with a period.

    The solution in your case is simple

    Instead of this

    @b-steer said in Logarithmic plotting indicator fail:

    class Logarithm(OperationN):
        lines = ('logarithm',)
        func = math.log
    

    do this

    class Logarithm(OperationN):
        lines = ('logarithm',)
        func = lambda x: math.log(x[0])
    

    Which means you have now a function which takes an array, but uses only the 1st value of it.



  • @backtrader said in Logarithmic plotting indicator fail:

    do this

    class Logarithm(OperationN):
        lines = ('logarithm',)
        func = lambda x: math.log(x[0])
    

    Which means you have now a function which takes an array, but uses only the 1st value of it.

    That looks like a better idea than a separate OperationX class.
    I'm guessing that lambda somehow doesn't honour the slice notation overloading put in place by the engine, resulting in:

        dst[i] = func(src[i - period + 1: i + 1])
    TypeError: <lambda>() takes 1 positional argument but 2 were given
    

    A small tweak fixes this:

     class Logarithm(OperationN):
         lines = ('logarithm',)
         func = lambda _, x: math.log(x[0])
    

    LogPlot is now functional if you take care to avoid math domain errors :)


  • administrators

    If you go down the hierarchy, you have ApplyN, which defines a func parameter and is intended for your use case. See

    class Logarithm(bt.ind.ApplyN):
        params = (('func', lambda x: math.log(x[0])),)
    

    In your case because func is defined at class level, the lambda is promoted to be a method and is therefore taking a self parameter, which doesn't happen, when func is defined as a parameter.

    You should be able to directly do:

    class LogPlot(bt.Indicator):
        '''
        Logarithmic plot
        '''
        lines = ('logplot',)
    
        def __init__(self):
            self.lines.logplot = bt.ind.ApplyN(self.data + 1, func=lambda x: math.log(x[0]))
    

    which entirely forgoes having to define an indicator for something which simply is the application of a function over a period.