For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

keep (custom) observer value in a list.



  • I would like to keep the value of the observer in a list (rather than write it out to a csv and read the csv latter). Can this be done?

    Here is my custom observer:

    class PositionObserver(backtrader.observer.Observer):
        lines = ('position', 'price', )
    
        plotinfo = dict(plot=True, subplot=True, plotlinelabels=True)
    
        plotlines = dict(
            executed_position = dict(marker='*', markersize=8.0, color='lime', fillstyle='full'),
            executed_price    = dict(marker='*', markersize=8.0, color='lime', fillstyle='full')
        )
    
        def next(self):
    
            for order in self._owner._orderspending:
    
                if order.data is not self.data:
                    continue
    
                self.lines.executed_position[0] = order.executed.size
                self.lines.executed_price[0]    = order.executed.price
    

  • administrators

    You probably need to elaborate a bit more

    You have this

    @kav said in keep (custom) observer value in a list.:

    self.lines.executed_position[0] = order.executed.size
    

    you can do this

    self.executed_pos.append(order.executed.size)
    

    So you probably mean something else.



  • @backtrader : sorry, my question was poorly formulated.

    What I meant is that after I run the code:

            cerebro   = backtrader.Cerebro()
            cerebro.addobserver(PositionObserver)
            # Add a strategy
            cerebro.addstrategy(StrategyBaseTrading)
    
            data = PandasDataStrat1(dataname     = dataframe, 
                                    datetime     = -1, 
                                    open         = -1, 
                                    high         = -1, 
                                    low          = -1, 
                                    close        = -1,
                                    volume       = -1,
                                    openinterest = None)
            cerebro.adddata(data)
            cerebro.broker\
                   .setcash(100000.0)
            cerebro.broker\
                   .setcommission(commission = 2.0, 
                                  margin     = 10000.0, 
                                  mult       = 10.0)
    

    is there a way to extract the content of lines.executed_position created by the Observer out of the cerebero object?

    basically, right now, I do:

            cerebro.addwriter(backtrader.WriterFile, 
                              csv = True, 
                              out = 'results.csv')
            cerebro.run()
    
            def find_skipped_rows(file_name, string):
                sentinel = []
                counter  = 0
                with open(file_name) as fh:
                    for line in fh:
                        counter += 1
                        if line.startswith(string):
                            sentinel.append(counter)
                return [i for i in range(counter) if i < sentinel[0] or i >= sentinel[1] - 1]
    
            skip_rows = find_skipped_rows(outfile, '===')
            executed_position  = pandas.read_csv(outfile, skiprows = skip_rows).executed_position
    
    

    so basically I'm looking for a way to get the content of executed_position from the Cerebro object directly (without the write to csv, read from csv step)

    thanks


  • administrators

    You can access the observers as attribute observers in the strategies returned by cerebro. And from there you have full access to the lines.

    See: Docs - Cerebro and the section "Returning the Results"



  • @backtrader Thanks for the pointer.. But I must be doing something wrong because the output I get don't seem to match the one from the csv file.

    result[0].observers[2].lines.executed_size
    
    In [44]: len(list(result[0].observers[2].lines.executed_size))
    Out[44]: 3364
    

    (this is the correct length: that of the data).

    but I must be doing something wrong because these are all nan values:

    In [39]: pandas.DataFrame(list(result[0].observers[2].lines.executed_size)).dropna()
    Out[39]:
    Empty DataFrame
    Columns: [0]
    Index: []
    

    whereas when I export the result to a csv:

            cerebro.addwriter(backtrader.WriterFile, 
                              csv = True,
                              out = outfile)
    

    I can clearly see the executed_size column is not all empty.

    full code

    class maCross(backtrader.Strategy):
        '''
        For an official backtrader blog on this topic please take a look at:
    
        https://www.backtrader.com/blog/posts/2017-04-09-multi-example/multi-example.html
    
        oneplot = Force all datas to plot on the same master.
        '''
        params = (
        ('sma1', 80),
        ('sma2', 200),
        ('oneplot', True),
        ('list_plus', [])
        )
    
        def __init__(self):
            '''
            Create an dictionary of indicators so that we can dynamically add the
            indicators to the strategy using a loop. This mean the strategy will
            work with any number of data feeds. 
            '''
            self.inds = dict()#           p0 = pandas.Series(self.data.close.get(size=5))
    #           print(p0.head(5).transpose())
            for i, d in enumerate(self.datas):
                self.inds[d] = dict()
                self.inds[d]['sma1'] = backtrader.indicators.SMA(d.close, 
                                                                 period=self.params.sma1)
                self.inds[d]['sma2'] = backtrader.indicators.SMA(d.close, 
                                                                 period=self.params.sma2)
                if d._name in self.params.list_plus:
                    self.inds[d]['cross'] = backtrader.indicators.CrossOver(self.inds[d]['sma1'],
                                                                            self.inds[d]['sma2'])
                else:
                    self.inds[d]['cross'] = backtrader.indicators.CrossOver(self.inds[d]['sma2'],
                                                                            self.inds[d]['sma1']) 
              
                if i > 0: #Check we are not on the first loop of data feed:
                    if self.p.oneplot == True:
                        d.plotinfo.plotmaster = self.datas[0]
    
        def next(self):
            for i, d in enumerate(self.datas):
                dt, dn = self.datetime.date(), d._name
                pos = self.getposition(d).size
    
                if not pos:  # no market / no orders
                    if self.inds[d]['cross'][0] == 1:
                        self.buy(data=d, size=1)
                    elif self.inds[d]['cross'][0] == -1:
                        self.sell(data=d, size=1)
                else:
                    if self.inds[d]['cross'][0] == 1:
                        self.close(data=d)
                        self.buy(data=d, size=1)
                    elif self.inds[d]['cross'][0] == -1:
                        self.close(data=d)
                        self.sell(data=d, size=1)
    
    class SimpleObserver(backtrader.observer.Observer):
        lines = ('executed_size', )
    
        def next(self):
    
            for order in self._owner._orderspending:
    
                if order.data is not self.data:
                    continue
    
                if order.status in [backtrader.Order.Completed]:
                    self.lines.executed_size[0]  = order.executed.size
    
    

    then

            dataframe = self.get_data(file_name)
            margin    = 10000
            # Create a cerebro entity
            cerebro   = backtrader.Cerebro()
            cerebro.addobserver(SimpleObserver)
            # Add a strategy
            cerebro.addstrategy(maCross)
            data      = PandasDataStrat1(dataname     = dataframe, 
                                         datetime     = -1, 
                                         open         = -1, 
                                         high         = -1, 
                                         low          = -1, 
                                         close        = -1,
                                         volume       = -1,
                                         openinterest = None)
            cerebro.adddata(data)
            start_cash = 100000.0
            cerebro.broker\
                   .setcash(start_cash)
            cerebro.broker\
                   .setcommission(commission = 2.0, 
                                  margin     = margin, 
                                  mult       = 10.0)
            result = cerebro.run()
    

  • administrators

    @kav said in keep (custom) observer value in a list.:

    In [44]: len(list(result[0].observers[2].lines.executed_size))
    

    If you have read the docs, you know that the lines don't follow the indexing convention in Python to make them fit the trading pattern. [0] is the actual time and [-1] is the last value delivered before the current point (i.e.: one instant in time before)

    See: Docs - Platform Concepts and Slicing. The get method allows you to get the actual data. There are some other methods available for a line like getzero, plot or you can even access the underlying array attribute which holds the values.



  • @backtrader: Perfect. Thanks for the pointer, it indeed addresses my problem. Just to be sure, could you confirm that the backtrader-preferred way to add an observer as a new column to the original dataframe (that was fed to backtrader) is:

    dataframe['observer'] = result[0].observers[2].executed_size.get(size = dataframe.shape[0])
    

    thank you