Confused with pnlcomm and pnl

AliAskari

I have just started working with backtrader and tried a simple strategy but I'm getting very weird results.

I'm using the addwriter method to save the trade results to a csv file:

cerebro.addwriter(bt.WriterFile, csv=True, out="trade_history.csv")

and I'm also setting the commission to zero.

This is part of the csv results

0_1525004691472_eff42c45-5dc5-4373-bdd9-3061c6470795-image.png

at the 85th iteration (don't know if it's the right term, correct me please) I buy ~500$ worth of BTC at a price of ~13400$, which means I have ~0.037 BTC which is then sold at ~13760. My calculations are telling me that after selling my cash balance should be more than what I had before entering the trade because the trade is clearly profitable but it is not the case and I'm in a 3$ loss! Could someone please explain what is going on?

I have attached my script

import backtrader as bt
import backtrader.feeds as btfeed
import numpy as np


class custom_csv(btfeed.GenericCSVData):
    params = (
        ('datetime', 0),
        ('time', -1),
        ('open', 1),
        ('high', 2),
        ('low', 3),
        ('close', 4),
        ('volume', 5),
        ('openinterest', -1),
        ('timeframe', bt.TimeFrame.Minutes),
        ('compression', 1),
    )

from datetime import datetime

class ma_strat(bt.Strategy):
    params = (
        ('fast_ma', 12),
        ('slow_ma', 60),
    )

    def __init__(self):
        self.dataclose = self.datas[0].close
        self.volume = self.datas[0].volume
        self.latest_order = None
        self.first_run = True
        self.order_size = 0
        self.trade_number = 0

        self.fast_ma = bt.indicators.SimpleMovingAverage(self.datas[0], period=self.params.fast_ma)
        self.slow_ma = bt.indicators.SimpleMovingAverage(self.datas[0], period=self.params.slow_ma)

        self.fast_ma.csv = True
        self.slow_ma.csv = True

    def notify_order(self, order):

        if order.isbuy():
            order_type = 'Buy'
        else:
            order_type = 'Sell'

        if order.status == order.Completed:
            self.latest_order = None
            return
        elif order.status == order.Canceled:
            print(order_type + ' order canceled')
            self.latest_order = None
            return
        elif order.status == order.Margin:
            current_cash_balance = self.broker.get_cash()
            print(order_type + ' order cancled due to margin Error')
            print('Current balance: ' + str(current_cash_balance))
            # Write down: no pending order
            self.latest_order = None
            return
        elif order.status == order.Rejected:
            print(order_type + ' order rejected')
            # Write down: no pending order
            self.latest_order = None
            return



    def notify_trade(self, trade):

        if trade.isopen:
            self.trade_number += 1
            print('Opening trade #%02d with size %.4f commision %.4f value %.2f price %.2f' %
                     (self.trade_number, trade.size, trade.commission, trade.value, trade.price))
        elif trade.isclosed:
            print('Closing trade #%02d gross %.2f, net %.2f' %
                     (self.trade_number, trade.pnl, trade.pnlcomm))

    def next(self):

        # Check if an order is pending ... if yes, we cannot send a 2nd one
        if self.latest_order is not None:
            return

        if not self.position:
            if self.fast_ma[0] > self.slow_ma[0]:
                    current_cash_balance = self.broker.get_cash()
                    self.order_size = 0.5 * current_cash_balance / float(self.dataclose[0])
                    # place the buy order
                    self.latest_order = self.buy(size=self.order_size)

        else:
            if self.fast_ma[0] < self.slow_ma[0]:
                    self.latest_order = self.sell(size=self.order_size)


if __name__ == '__main__':

    # Create a cerebro entity
    cerebro = bt.Cerebro()
    cerebro.addwriter(bt.WriterFile, csv=True, out="trade_history.csv")

    csv_data_file =  'bitfinex_btc_price.csv'
    bitfinex_minut_data = custom_csv(dataname=csv_data_file)

    # Add the Data Feed to Cerebro
    unknown = cerebro.adddata(bitfinex_minut_data)

    cerebro.addstrategy(ma_strat)

    cerebro.broker.setcash(1000.0)

    print('Starting Portfolio Value: %.2f USD$' % cerebro.broker.getvalue())

    cerebro.run()

    print('Final Portfolio Value: %.2f USD$' % cerebro.broker.getvalue())

backtrader

Anyone would probably need some extra info.

You have the data and that's not seen in your post. Seeing the data points (and around) at the buy/sell points could help.

@aliaskari said in Confused with pnlcomm and pnl:

def notify_trade(self, trade):

    if trade.isopen:
        self.trade_number += 1
        print('Opening trade #%02d with size %.4f commision %.4f value %.2f price %.2f' %
                 (self.trade_number, trade.size, trade.commission, trade.value, trade.price))
    elif trade.isclosed:
        print('Closing trade #%02d gross %.2f, net %.2f' %
                 (self.trade_number, trade.pnl, trade.pnlcomm))

You also have this logging code, but the output of it (which for sure could help) is also not shown (since you have single orders, it would seem even better to log out during notify_order when the order is Completed)

The output of the writer omits the most important parts:

datetime timestamp
prices in the data stream

The only thing the screenshot shows is that the value decreases from the very first moment after your buy operation. Even if you believe that a screenshot may be more helpful than the raw data: it isn't. Put at least the header of the writer output and the lines at (and around) your sell/buy points and the

AliAskari

Thanks for your time @backtrader. I was not sure how to add csv file as there are a lot of columns. I have attached both the entire csv file and also a snapshot from the data around this particular trade.

0_1525189002436_71adc4ea-e066-47a7-87da-550e93f2f849-image.png

EDIT: edited to add the link to csv data file: https://www.dropbox.com/s/cpdbzf839kpx5mb/bitfinex_btc_price.csv?dl=0

You also have this logging code, but the output of it (which for sure could help) is also not shown

I modified the script to print more info and also used notify_order as suggested. Here is the output:

Starting Portfolio Value: 1000.00 USD$
Open trade #01 with size 0.0367 value 500.5144 price 13621.0000
Close trade #01 with size -0.0367 value 500.5144 price 13569.0000
Open trade #02 with size 0.0366 value 499.0812 price 13631.0000
Close trade #02 with size -0.0366 value 499.0812 price 13548.0000

I just realized that the price mentioned in the log does not match the buy/sell price in the csv file. The info pasted in the screenshot is from trade#2.

i.e.

            notify_order print log        csv 

open        13621.0000              13400.9250
close       13569.0000              13760.3550

The only thing the screenshot shows is that the value decreases from the very first moment after your buy operation.

I am not sure why you say the price decreases? By looking at the csv and the buy/sell columns it appears to me that the buy price is lower than the sell price.

I am under the impression that those columns reflect the price at which the trade was completed. But apparently it is not the case.

What am I missing here?

Thanks!

backtrader

It is clear now why you are losing.

Screenshots even if sometimes visually helpful don't really help. The header of the output csv with the offending lines (and a couple of the surrounding ones) allow to see what Buy and Sell are. But let me speculate:

They happen to be the output values of the BuySellObserver, the one plotting green/red triangles to mark your operations on the chart.

Those triangles are plotted above and below your entry point to give you a visual indication of when the operation has happened (if the values were at the price, they would be mostly not visible most of the times, given they would be in between several price bars)

And being above and below, the points you see are the observer points and not the actual operations (which are available to you via the notified orders)

AliAskari

I see. Makes sense. Thanks!

So if I want to capture the real buy/sell prices, along with other metrics, I'm better off writing my own logger and dump the log into a csv file.

backtrader

You can do it in different ways:

Pass stdstats=False to cerebro.run() (or to cerebro = bt.Cerebro()) and add your own observers. In this case you add a BuySellObserver with barplot=False, which will then use the exact buy and sell prices.

See:
- Observers and Statistics
- Observer Reference

If you simply want to capture the prices and avoid them being plotted, you can also pass plot=False (or subclass the observer and set that as a default in plotinfo)

You can do it intercepting the notifications in notify_order and add them to any structure you like

There would be other ways, but I would call them dirty (the fact is that you can do anything in Python, but it is usually better to play by the rules of the library/framework/pattern you are using)

AliAskari

Thanks for your time, it's now working as expected.