Can observer output text/characters in lines?



  • Hello,

    Thanks again for creating this wonderful tool. I have a question regarding observer.

    Below code taken from https://www.backtrader.com/docu/observers-and-statistics/observers-and-statistics.html.

    For example, instead of settingself.lines.created[0] to order.created.price, is it possible to set self.lines.created[0] with some text? Reason being I want to create some descriptions in the output, i.e Bought xx stock at xx price for xx amount and output this in the writer alone with other information.

    When i tried to set it to text, i got below errors:

    Traceback (most recent call last):
      File "strategy\pair-trading-dev.py", line 299, in <module>
        cerebro.run()
      File "C:\Users\u8010137\Envs\quant\lib\site-packages\backtrader\cerebro.py", line 873, in run
        runstrat = self.runstrategies(iterstrat)
      File "C:\Users\u8010137\Envs\quant\lib\site-packages\backtrader\cerebro.py", line 1005, in runstrategies
        self._runonce(runstrats)
      File "C:\Users\u8010137\Envs\quant\lib\site-packages\backtrader\cerebro.py", line 1380, in _runonce
        strat._oncepost(dt0)
      File "C:\Users\u8010137\Envs\quant\lib\site-packages\backtrader\strategy.py", line 274, in _oncepost
        self._next_observers(minperstatus, once=True)
      File "C:\Users\u8010137\Envs\quant\lib\site-packages\backtrader\strategy.py", line 326, in _next_observers
        observer.prenext()
      File "C:\Users\u8010137\Envs\quant\lib\site-packages\backtrader\observer.py", line 59, in prenext
        self.next()
      File "strategy\pair-trading-dev.py", line 74, in next
        self.lines.created[0] = 'd'
      File "C:\Users\u8010137\Envs\quant\lib\site-packages\backtrader\linebuffer.py", line 222, in __setitem__
        self.array[self.idx + ago] = value
    TypeError: a float is required
    
    from __future__ import (absolute_import, division, print_function,
                            unicode_literals)
    
    import math
    
    import backtrader as bt
    
    
    class OrderObserver(bt.observer.Observer):
        lines = ('created', 'expired',)
    
        plotinfo = dict(plot=True, subplot=True, plotlinelabels=True)
    
        plotlines = dict(
            created=dict(marker='*', markersize=8.0, color='lime', fillstyle='full'),
            expired=dict(marker='s', markersize=8.0, color='red', fillstyle='full')
        )
    
        def next(self):
            for order in self._owner._orderspending:
                if order.data is not self.data:
                    continue
    
                if not order.isbuy():
                    continue
    
                # Only interested in "buy" orders, because the sell orders
                # in the strategy are Market orders and will be immediately
                # executed
    
                if order.status in [bt.Order.Accepted, bt.Order.Submitted]:
                    self.lines.created[0] = order.created.price
    
                elif order.status in [bt.Order.Expired]:
                    self.lines.expired[0] = order.created.price
    

  • administrators

    No. The lines in the objects contain a float (even the ones carrying timestamps)

    The BuySell observer can (by changing the parameters) plot at the exact price at which an operation happens.

    To plot text labels a model would be needed to let the plotting code replace the markers with text output produced by the indicator.



  • @backtrader Thanks for the prompt reply!

    I am not trying to plot text label in chart, actually I want to add an additional column to the csv file produced by adding cerebro.addwriter(bt.WriterFile, csv=True) to my strategy.

    I noticed all the pricing data / observers / indicators are included in the csv file automatically, which is quite helpful to understand what's going on behind the hood and is useful to verify the backtesting results.

    But I find it a difficult to output orders/trade details. I understand i can write out orders/traders using notify_order and notify_trade functions, however the output text from these function are not aligned with other columns.

    To give an example, Below is the output(partial results) from the writer, from line 130 to line 131 are the text I'd like to add to a new column so that they don't take a new line.

    Current results:

    130	U74271810	130	2016-7-8 0:00	84.979996	85.899994	84.770005	85.770005	6827348	6827348	U92826C83	130	2016-7-8 0:00
    BACKTRADER - ORDER: OrderId: 1 Stock: stock1 Action: SELL CreatePrice: 85.77 CreateSize:-58 ExecutePrice: 85.61 Cost: -4965.38 Comm: 0.00                          												
    BACKTRADER - ORDER: OrderId: 2 Stock: stock2 Action: BUY CreatePrice: 76.42 CreateSize:65 ExecutePrice: 76.81 Cost: 4992.65 Comm: 0.00                          												
    131	U74271810	131	2016-7-11 0:00	85.610001	85.949997	84.910004	85.75	6156824	6156824	U92826C83	131	2016-7-11 0:00
    132	U74271810	132	2016-7-12 0:00	85.490006	85.940003	85.190003	85.75	6649848	6649848	U92826C83	132	2016-7-12 0:00
    133	U74271810	133	2016-7-13 0:00	85.809998	86.149994	85.410004	85.89	7572149	7572149	U92826C83	133	2016-7-13 0:00
    
    

    intended results

    130	U74271810	130	2016-7-8 0:00	84.979996	85.899994	84.770005	85.770005	6827348	6827348	U92826C83	130	2016-7-8 0:00	BACKTRADER - ORDER: OrderId: 1 Stock: stock1 Action: SELL CreatePrice: 85.77 CreateSize:-58 ExecutePrice: 85.61 Cost: -4965.38 Comm: 0.00                          	BACKTRADER - ORDER: OrderId: 2 Stock: stock2 Action: BUY CreatePrice: 76.42 CreateSize:65 ExecutePrice: 76.81 Cost: 4992.65 Comm: 0.00                          												
    131	U74271810	131	2016-7-11 0:00	85.610001	85.949997	84.910004	85.75	6156824	6156824	U92826C83	131	2016-7-11 0:00	76.809998	77.192002
    
    

    Below is the observer I used to output the "Backtrader..." line.

    class OrderObserver(bt.observer.Observer):
        lines = ('created', 'expired',)
    
        plotinfo = dict(plot=False, subplot=False, plotlinelabels=False)
    
        plotlines = dict(
            created=dict(marker='*', markersize=8.0, color='lime', fillstyle='full'),
            expired=dict(marker='s', markersize=8.0, color='red', fillstyle='full')
        )
    
        def next(self):
            results = []
            for order in self._owner._orderspending:
                # if order.data is not self.data:
                #     continue
                if order.status in [bt.Order.Completed]:
                    content = '''BACKTRADER - ORDER: OrderId: %s Stock: %s Action: %s CreatePrice: %.2f CreateSize:%s ExecutePrice: %.2f Cost: %.2f Comm: %.2f
                              ''' % (  order.ref,
                                       order.info['addinfo']['stock'],
                                       order.info['addinfo']['buysell'],
                                       order.created.price,
                                       order.created.size,
                                       order.executed.price,
                                       order.executed.value,
                                       order.executed.comm
                          )
                    content = content.replace('\n', '')
                    results.append(content)
                    print(content)
    
    

  • administrators

    Understanding your target result, that's not possible with the goal which the csv output has, which is automated printing of any of the objects which are present in the hierarchy. With a further goal to allow automated processing of the output.

    With all that in mind you could very much achieve the desired effect. The automated facility uses the actual information present in the object: class name, name of the lines, values of the lines.

    You want for example to print out Create Size. As such you would:

    • Define a line for the observer with a name like: created_size

      This name would be printed in the header row in column N of the output.

    • The rest of the rows in the owould look as follows:

      • Rows with no order: *they would be empty(i.e.: 2 consecutive **separators**, being the default separator a,`)

      • Rows which detect an operation like above would print out: -58

    You can use the sign of the created size to separate buy from sell, or have an additional line named: buy, which would have a value of 1 for buy operations, a value of 0 for sell operations and be empty when no operation is recorded (or any other convention you see fit)

    At the end of the day you have a complete table you can filter in tools like Excel or load to a database and easily query.



  • @backtrader

    Many thanks for the advice, I solved the problem using the method you provided!

    
    # https://www.backtrader.com/docu/observers-and-statistics/observers-and-statistics.html
    class OrderObserver(bt.observer.Observer):
    
        lines = ('ref0', 'createdPrice0', 'createdSize0', 'executedPrice0', 'executedValue0', 'Comm0',
                 'ref1', 'createdPrice1', 'createdSize1', 'executedPrice1', 'executedValue1', 'Comm1',)
    
        plotinfo = dict(plot=False, subplot=False, plotlinelabels=False)
    
        def next(self):
    
            results = []
            for order in self._owner._orderspending:
                # if order.data is not self.data:
                #     continue
                if order.status in [bt.Order.Completed]:
                    results.append([
                        order.ref,
                        order.created.price,
                        order.created.size,
                        order.executed.price,
                        order.executed.value,
                        order.executed.comm
                        ])
    
            if len(results) > 0:  # orders are available
                self.lines.ref0[0] = results[0][0]
                self.lines.createdPrice0[0] = results[0][1]
                self.lines.createdSize0[0] = results[0][2]
                self.lines.executedPrice0[0] = results[0][3]
                self.lines.executedValue0[0] = results[0][4]
                self.lines.Comm0[0] = results[0][5]
                self.lines.ref1[0] = results[1][0]
                self.lines.createdPrice1[0] = results[1][1]
                self.lines.createdSize1[0] = results[1][2]
                self.lines.executedPrice1[0] = results[1][3]
                self.lines.executedValue1[0] = results[1][4]
                self.lines.Comm1[0] = results[1][5]
    
    


  • @derek2017 can you show piece of output?



  • @ab_trader

    I use this for a pair trading strategy, so i output the order content for both stocks.

    OrderObserver	len	ref0	createdPrice0	createdSize0	executedPrice0	executedValue0	Comm0	ref1	createdPrice1	createdSize1	executedPrice1	executedValue1	Comm1
    OrderObserver	128												
    OrderObserver	129												
    OrderObserver	130												
    OrderObserver	131	1	85.770005	-58	85.610001	-4965.380058	0	2	76.419999	65	76.809998	4992.64987	0
    OrderObserver	132												
    OrderObserver	133												
    OrderObserver	134												
    OrderObserver	17												
    
    


  • @derek2017 Thank you! Can be a good alternative for usual logging.


Log in to reply
 

Looks like your connection to Backtrader Community was lost, please wait while we try to reconnect.