3. keep (custom) observer value in a list.
For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

keep (custom) observer value in a list.

  • K

    I would like to keep the value of the observer in a list (rather than write it out to a csv and read the csv latter). Can this be done?

    Here is my custom observer:

    class PositionObserver(backtrader.observer.Observer):
    lines = ('position', 'price', )

    plotinfo = dict(plot=True, subplot=True, plotlinelabels=True)

    plotlines = dict(
        executed_position = dict(marker='*', markersize=8.0, color='lime', fillstyle='full'),
        executed_price    = dict(marker='*', markersize=8.0, color='lime', fillstyle='full')
    )

    def next(self):

        for order in self._owner._orderspending:

            if order.data is not self.data:
                continue

            self.lines.executed_position[0] = order.executed.size
            self.lines.executed_price[0]    = order.executed.price
    0
  • B
    administrators

    You probably need to elaborate a bit more

    You have this

    @kav said in keep (custom) observer value in a list.:

    self.lines.executed_position[0] = order.executed.size

    you can do this

    self.executed_pos.append(order.executed.size)

    So you probably mean something else.

    0
  • K

    @backtrader : sorry, my question was poorly formulated.

    What I meant is that after I run the code:

            cerebro   = backtrader.Cerebro()
        cerebro.addobserver(PositionObserver)
        # Add a strategy
        cerebro.addstrategy(StrategyBaseTrading)

        data = PandasDataStrat1(dataname     = dataframe, 
                                datetime     = -1, 
                                open         = -1, 
                                high         = -1, 
                                low          = -1, 
                                close        = -1,
                                volume       = -1,
                                openinterest = None)
        cerebro.adddata(data)
        cerebro.broker\
               .setcash(100000.0)
        cerebro.broker\
               .setcommission(commission = 2.0, 
                              margin     = 10000.0, 
                              mult       = 10.0)

    is there a way to extract the content of lines.executed_position created by the Observer out of the cerebero object?

    basically, right now, I do:

            cerebro.addwriter(backtrader.WriterFile, 
                          csv = True, 
                          out = 'results.csv')
        cerebro.run()

        def find_skipped_rows(file_name, string):
            sentinel = []
            counter  = 0
            with open(file_name) as fh:
                for line in fh:
                    counter += 1
                    if line.startswith(string):
                        sentinel.append(counter)
            return [i for i in range(counter) if i < sentinel[0] or i >= sentinel[1] - 1]

        skip_rows = find_skipped_rows(outfile, '===')
        executed_position  = pandas.read_csv(outfile, skiprows = skip_rows).executed_position

    so basically I'm looking for a way to get the content of executed_position from the Cerebro object directly (without the write to csv, read from csv step)

    thanks

    0
  • B
    administrators

    You can access the observers as attribute observers in the strategies returned by cerebro. And from there you have full access to the lines.

    See: Docs - Cerebro and the section "Returning the Results"

    0
    K
     1 Reply
  • K

    @backtrader Thanks for the pointer.. But I must be doing something wrong because the output I get don't seem to match the one from the csv file.

    result[0].observers[2].lines.executed_size

    In [44]: len(list(result[0].observers[2].lines.executed_size))
Out[44]: 3364

    (this is the correct length: that of the data).

    but I must be doing something wrong because these are all nan values:

    In [39]: pandas.DataFrame(list(result[0].observers[2].lines.executed_size)).dropna()
Out[39]:
Empty DataFrame
Columns: [0]
Index: []

    whereas when I export the result to a csv:

            cerebro.addwriter(backtrader.WriterFile, 
                          csv = True,
                          out = outfile)

    I can clearly see the executed_size column is not all empty.

    full code

    class maCross(backtrader.Strategy):
    '''
    For an official backtrader blog on this topic please take a look at:

    https://www.backtrader.com/blog/posts/2017-04-09-multi-example/multi-example.html

    oneplot = Force all datas to plot on the same master.
    '''
    params = (
    ('sma1', 80),
    ('sma2', 200),
    ('oneplot', True),
    ('list_plus', [])
    )

    def __init__(self):
        '''
        Create an dictionary of indicators so that we can dynamically add the
        indicators to the strategy using a loop. This mean the strategy will
        work with any number of data feeds. 
        '''
        self.inds = dict()#           p0 = pandas.Series(self.data.close.get(size=5))
#           print(p0.head(5).transpose())
        for i, d in enumerate(self.datas):
            self.inds[d] = dict()
            self.inds[d]['sma1'] = backtrader.indicators.SMA(d.close, 
                                                             period=self.params.sma1)
            self.inds[d]['sma2'] = backtrader.indicators.SMA(d.close, 
                                                             period=self.params.sma2)
            if d._name in self.params.list_plus:
                self.inds[d]['cross'] = backtrader.indicators.CrossOver(self.inds[d]['sma1'],
                                                                        self.inds[d]['sma2'])
            else:
                self.inds[d]['cross'] = backtrader.indicators.CrossOver(self.inds[d]['sma2'],
                                                                        self.inds[d]['sma1']) 
          
            if i > 0: #Check we are not on the first loop of data feed:
                if self.p.oneplot == True:
                    d.plotinfo.plotmaster = self.datas[0]

    def next(self):
        for i, d in enumerate(self.datas):
            dt, dn = self.datetime.date(), d._name
            pos = self.getposition(d).size

            if not pos:  # no market / no orders
                if self.inds[d]['cross'][0] == 1:
                    self.buy(data=d, size=1)
                elif self.inds[d]['cross'][0] == -1:
                    self.sell(data=d, size=1)
            else:
                if self.inds[d]['cross'][0] == 1:
                    self.close(data=d)
                    self.buy(data=d, size=1)
                elif self.inds[d]['cross'][0] == -1:
                    self.close(data=d)
                    self.sell(data=d, size=1)

class SimpleObserver(backtrader.observer.Observer):
    lines = ('executed_size', )

    def next(self):

        for order in self._owner._orderspending:

            if order.data is not self.data:
                continue

            if order.status in [backtrader.Order.Completed]:
                self.lines.executed_size[0]  = order.executed.size

    then

            dataframe = self.get_data(file_name)
        margin    = 10000
        # Create a cerebro entity
        cerebro   = backtrader.Cerebro()
        cerebro.addobserver(SimpleObserver)
        # Add a strategy
        cerebro.addstrategy(maCross)
        data      = PandasDataStrat1(dataname     = dataframe, 
                                     datetime     = -1, 
                                     open         = -1, 
                                     high         = -1, 
                                     low          = -1, 
                                     close        = -1,
                                     volume       = -1,
                                     openinterest = None)
        cerebro.adddata(data)
        start_cash = 100000.0
        cerebro.broker\
               .setcash(start_cash)
        cerebro.broker\
               .setcommission(commission = 2.0, 
                              margin     = margin, 
                              mult       = 10.0)
        result = cerebro.run()
    0
    B
     1 Reply
  • B
    administrators

    @kav said in keep (custom) observer value in a list.:

    In [44]: len(list(result[0].observers[2].lines.executed_size))

    If you have read the docs, you know that the lines don't follow the indexing convention in Python to make them fit the trading pattern. [0] is the actual time and [-1] is the last value delivered before the current point (i.e.: one instant in time before)

    See: Docs - Platform Concepts and Slicing. The get method allows you to get the actual data. There are some other methods available for a line like getzero, plot or you can even access the underlying array attribute which holds the values.

    0
    K
     1 Reply
  • K

    @backtrader: Perfect. Thanks for the pointer, it indeed addresses my problem. Just to be sure, could you confirm that the backtrader-preferred way to add an observer as a new column to the original dataframe (that was fed to backtrader) is:

    dataframe['observer'] = result[0].observers[2].executed_size.get(size = dataframe.shape[0])

    thank you

    0