Can observer output text/characters in lines?

derek2017

Hello,

Thanks again for creating this wonderful tool. I have a question regarding observer.

Below code taken from https://www.backtrader.com/docu/observers-and-statistics/observers-and-statistics.html.

For example, instead of settingself.lines.created[0] to order.created.price, is it possible to set self.lines.created[0] with some text? Reason being I want to create some descriptions in the output, i.e Bought xx stock at xx price for xx amount and output this in the writer alone with other information.

When i tried to set it to text, i got below errors:

Traceback (most recent call last):
  File "strategy\pair-trading-dev.py", line 299, in <module>
    cerebro.run()
  File "C:\Users\u8010137\Envs\quant\lib\site-packages\backtrader\cerebro.py", line 873, in run
    runstrat = self.runstrategies(iterstrat)
  File "C:\Users\u8010137\Envs\quant\lib\site-packages\backtrader\cerebro.py", line 1005, in runstrategies
    self._runonce(runstrats)
  File "C:\Users\u8010137\Envs\quant\lib\site-packages\backtrader\cerebro.py", line 1380, in _runonce
    strat._oncepost(dt0)
  File "C:\Users\u8010137\Envs\quant\lib\site-packages\backtrader\strategy.py", line 274, in _oncepost
    self._next_observers(minperstatus, once=True)
  File "C:\Users\u8010137\Envs\quant\lib\site-packages\backtrader\strategy.py", line 326, in _next_observers
    observer.prenext()
  File "C:\Users\u8010137\Envs\quant\lib\site-packages\backtrader\observer.py", line 59, in prenext
    self.next()
  File "strategy\pair-trading-dev.py", line 74, in next
    self.lines.created[0] = 'd'
  File "C:\Users\u8010137\Envs\quant\lib\site-packages\backtrader\linebuffer.py", line 222, in __setitem__
    self.array[self.idx + ago] = value
TypeError: a float is required

from __future__ import (absolute_import, division, print_function,
                        unicode_literals)

import math

import backtrader as bt


class OrderObserver(bt.observer.Observer):
    lines = ('created', 'expired',)

    plotinfo = dict(plot=True, subplot=True, plotlinelabels=True)

    plotlines = dict(
        created=dict(marker='*', markersize=8.0, color='lime', fillstyle='full'),
        expired=dict(marker='s', markersize=8.0, color='red', fillstyle='full')
    )

    def next(self):
        for order in self._owner._orderspending:
            if order.data is not self.data:
                continue

            if not order.isbuy():
                continue

            # Only interested in "buy" orders, because the sell orders
            # in the strategy are Market orders and will be immediately
            # executed

            if order.status in [bt.Order.Accepted, bt.Order.Submitted]:
                self.lines.created[0] = order.created.price

            elif order.status in [bt.Order.Expired]:
                self.lines.expired[0] = order.created.price

backtrader

No. The lines in the objects contain a float (even the ones carrying timestamps)

The BuySell observer can (by changing the parameters) plot at the exact price at which an operation happens.

To plot text labels a model would be needed to let the plotting code replace the markers with text output produced by the indicator.

derek2017

@backtrader Thanks for the prompt reply!

I am not trying to plot text label in chart, actually I want to add an additional column to the csv file produced by adding cerebro.addwriter(bt.WriterFile, csv=True) to my strategy.

I noticed all the pricing data / observers / indicators are included in the csv file automatically, which is quite helpful to understand what's going on behind the hood and is useful to verify the backtesting results.

But I find it a difficult to output orders/trade details. I understand i can write out orders/traders using notify_order and notify_trade functions, however the output text from these function are not aligned with other columns.

To give an example, Below is the output(partial results) from the writer, from line 130 to line 131 are the text I'd like to add to a new column so that they don't take a new line.

Current results:

130	U74271810	130	2016-7-8 0:00	84.979996	85.899994	84.770005	85.770005	6827348	6827348	U92826C83	130	2016-7-8 0:00
BACKTRADER - ORDER: OrderId: 1 Stock: stock1 Action: SELL CreatePrice: 85.77 CreateSize:-58 ExecutePrice: 85.61 Cost: -4965.38 Comm: 0.00                          												
BACKTRADER - ORDER: OrderId: 2 Stock: stock2 Action: BUY CreatePrice: 76.42 CreateSize:65 ExecutePrice: 76.81 Cost: 4992.65 Comm: 0.00                          												
131	U74271810	131	2016-7-11 0:00	85.610001	85.949997	84.910004	85.75	6156824	6156824	U92826C83	131	2016-7-11 0:00
132	U74271810	132	2016-7-12 0:00	85.490006	85.940003	85.190003	85.75	6649848	6649848	U92826C83	132	2016-7-12 0:00
133	U74271810	133	2016-7-13 0:00	85.809998	86.149994	85.410004	85.89	7572149	7572149	U92826C83	133	2016-7-13 0:00

intended results

130	U74271810	130	2016-7-8 0:00	84.979996	85.899994	84.770005	85.770005	6827348	6827348	U92826C83	130	2016-7-8 0:00	BACKTRADER - ORDER: OrderId: 1 Stock: stock1 Action: SELL CreatePrice: 85.77 CreateSize:-58 ExecutePrice: 85.61 Cost: -4965.38 Comm: 0.00                          	BACKTRADER - ORDER: OrderId: 2 Stock: stock2 Action: BUY CreatePrice: 76.42 CreateSize:65 ExecutePrice: 76.81 Cost: 4992.65 Comm: 0.00                          												
131	U74271810	131	2016-7-11 0:00	85.610001	85.949997	84.910004	85.75	6156824	6156824	U92826C83	131	2016-7-11 0:00	76.809998	77.192002

Below is the observer I used to output the "Backtrader..." line.

class OrderObserver(bt.observer.Observer):
    lines = ('created', 'expired',)

    plotinfo = dict(plot=False, subplot=False, plotlinelabels=False)

    plotlines = dict(
        created=dict(marker='*', markersize=8.0, color='lime', fillstyle='full'),
        expired=dict(marker='s', markersize=8.0, color='red', fillstyle='full')
    )

    def next(self):
        results = []
        for order in self._owner._orderspending:
            # if order.data is not self.data:
            #     continue
            if order.status in [bt.Order.Completed]:
                content = '''BACKTRADER - ORDER: OrderId: %s Stock: %s Action: %s CreatePrice: %.2f CreateSize:%s ExecutePrice: %.2f Cost: %.2f Comm: %.2f
                          ''' % (  order.ref,
                                   order.info['addinfo']['stock'],
                                   order.info['addinfo']['buysell'],
                                   order.created.price,
                                   order.created.size,
                                   order.executed.price,
                                   order.executed.value,
                                   order.executed.comm
                      )
                content = content.replace('\n', '')
                results.append(content)
                print(content)

backtrader

Understanding your target result, that's not possible with the goal which the csv output has, which is automated printing of any of the objects which are present in the hierarchy. With a further goal to allow automated processing of the output.

With all that in mind you could very much achieve the desired effect. The automated facility uses the actual information present in the object: class name, name of the lines, values of the lines.

You want for example to print out Create Size. As such you would:

Define a line for the observer with a name like: created_size

This name would be printed in the header row in column N of the output.
The rest of the rows in the owould look as follows:
- Rows with no order: *they would be empty(i.e.: 2 consecutive **separators**, being the default separator a,`)
- Rows which detect an operation like above would print out: -58

You can use the sign of the created size to separate buy from sell, or have an additional line named: buy, which would have a value of 1 for buy operations, a value of 0 for sell operations and be empty when no operation is recorded (or any other convention you see fit)

At the end of the day you have a complete table you can filter in tools like Excel or load to a database and easily query.

derek2017

@backtrader

Many thanks for the advice, I solved the problem using the method you provided!


# https://www.backtrader.com/docu/observers-and-statistics/observers-and-statistics.html
class OrderObserver(bt.observer.Observer):

    lines = ('ref0', 'createdPrice0', 'createdSize0', 'executedPrice0', 'executedValue0', 'Comm0',
             'ref1', 'createdPrice1', 'createdSize1', 'executedPrice1', 'executedValue1', 'Comm1',)

    plotinfo = dict(plot=False, subplot=False, plotlinelabels=False)

    def next(self):

        results = []
        for order in self._owner._orderspending:
            # if order.data is not self.data:
            #     continue
            if order.status in [bt.Order.Completed]:
                results.append([
                    order.ref,
                    order.created.price,
                    order.created.size,
                    order.executed.price,
                    order.executed.value,
                    order.executed.comm
                    ])

        if len(results) > 0:  # orders are available
            self.lines.ref0[0] = results[0][0]
            self.lines.createdPrice0[0] = results[0][1]
            self.lines.createdSize0[0] = results[0][2]
            self.lines.executedPrice0[0] = results[0][3]
            self.lines.executedValue0[0] = results[0][4]
            self.lines.Comm0[0] = results[0][5]
            self.lines.ref1[0] = results[1][0]
            self.lines.createdPrice1[0] = results[1][1]
            self.lines.createdSize1[0] = results[1][2]
            self.lines.executedPrice1[0] = results[1][3]
            self.lines.executedValue1[0] = results[1][4]
            self.lines.Comm1[0] = results[1][5]

ab_trader

@derek2017 can you show piece of output?

derek2017

@ab_trader

I use this for a pair trading strategy, so i output the order content for both stocks.

OrderObserver	len	ref0	createdPrice0	createdSize0	executedPrice0	executedValue0	Comm0	ref1	createdPrice1	createdSize1	executedPrice1	executedValue1	Comm1
OrderObserver	128												
OrderObserver	129												
OrderObserver	130												
OrderObserver	131	1	85.770005	-58	85.610001	-4965.380058	0	2	76.419999	65	76.809998	4992.64987	0
OrderObserver	132												
OrderObserver	133												
OrderObserver	134												
OrderObserver	17

ab_trader

@derek2017 Thank you! Can be a good alternative for usual logging.