For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

Questions about outputs of addwriter method



  • Hello,

    I have some questions about the outputs of the addwriter method.

    I looked at the csv the addwriter method outputs, and I noticed some numbers I didn't understand. The "buy" prices don't seem to match with any OHLC figures.

    Another user reported seeing these discrepancies between buy prices and the OHLC prices for those days in his backtest here: https://community.backtrader.com/topic/920/wrong-execution-price-during-the-backtesting

    It was suggested that the unreliable data source (yahoo) was to blame. I replicated the strategy, using a more reliable data source (Alpha Vantage), for the same ticker, over the same time range, and the error (?) replicated.

    Reproducible example:

    import datetime
    import backtrader as bt
    import backtrader.feeds as feeds
    import pandas as pd
    
    class SmaCross(bt.SignalStrategy):
        def log(self, txt, dt=None):
            ''' Logging function fot this strategy'''
            dt = dt or self.datas[0].datetime.date(0)
            print('%s, %s' % (dt.isoformat(), txt))
    
        def __init__(self):
            self.dataclose = self.datas[0].close
    
        def notify_order(self, order):
            print("Created: ", order.created.price)
            print("Executed: ", order.executed.price)
    
        def next(self):
            # Simply log the closing price of the series from the reference
            self.log('Close, %.2f' % self.dataclose[0])
    
            if self.dataclose[0] < self.dataclose[-1]:
                # current close less than previous close
    
                if self.dataclose[-1] < self.dataclose[-2]:
                    # previous close less than the previous close
                    # BUY, BUY, BUY!!! (with all possible default parameters)
                    #self.log('BUY CREATE, %.2f' % self.dataclose[0])
                    self.buy()
    
    
    cerebro = bt.Cerebro(writer=True)
    cerebro.addstrategy(SmaCross)
    
    url = 'https://raw.githubusercontent.com/VSMomentum/data/master/spy.csv'
    stuff = pd.read_csv(url,index_col=0,parse_dates=[0])
    
    data0 = feeds.PandasData(dataname=stuff,
        timeframe=1,
        openinterest=None,
        close=4,
        open=0,
        high=1,
        low=2,
        fromdate=datetime.datetime(2017, 1, 1),
        todate=datetime.datetime(2017, 12, 31))
    
    cerebro.adddata(data0)
    cerebro.addwriter(bt.WriterFile, csv=True, out="trade_history.csv")
    cerebro.run()
    

    The logging in the terminal shows that the created and executed buy values are correct, but the file outputted by the addwriter method ("trade_history.csv") contains buy values that don't match the logged values. For example:

    • Log shows a market order created on 2017-01-03 at $220.95, which is the correct close price for that day. The order then fills the next day at $225.62, which is the correct open price for that day (2017-01-04). I understand so far.

      • But, if you look at the csv, you'll see a buy price for that day of $222.22585. I'm curious where this figure comes from. It isn't within the low-high range for either day.

      • The portfolio values in the "value" column don't seem to be informed by this buy price. Rather the executed price in the log seems to be the relevant value.

    My other question is about the value column in the csv. I don't understand the arithmetic. For ex:

    • Log shows that the first order is executed on 2018-01-04 for $225.62. We then see that the portfolio value is $9996.6483. How is this calculated? I'm expecting that the value would still be $10,000, as it's the first purchase. Cash at that bar is $9774.38 (which is 10K - $225.62; Starting Capital - First Filled Order), so why is my portfolio value not equal to my cash plus the value of my one share, which would be 10K? Thought maybe it was due to commissions, but the documentation says commissions are set to 0 by default.

    Thanks. -Joe


  • administrators

    @cowboy said in Questions about outputs of addwriter method:

    Log shows that the first order is executed on 2018-01-04 for $225.62. We then see that the portfolio value is $9996.6483. How is this calculated? I'm expecting that the value would still be $10,000, as it's the first purchase. Cash at that bar is $9774.38 (which is 10K - $225.62; Starting Capital - First Filled Order), so why is my portfolio value not equal to my cash plus the value of my one share, which would be 10K? Thought maybe it was due to commissions, but the documentation says commissions are set to 0 by default.

    You buy at 225.62 and commission is 0, so the remaining cash is 10.000 - 225.62 = 9774.38. Unless you buy or sell something else or you are working with futures-like instruments, your cash will remain fixed at that point. It would be a real primer if it didn't.

    Why is the total portfolio value 9996.6483? Unless your stock almost magically managed to close at the same price at the end of the day ... the value of your stock will either be greater or lower than the buy price. In your case it is obvious that the price at the end of the day was lower than the acquisition price (which is the open) of 225.62, hence the lower value.

    @cowboy said in Questions about outputs of addwriter method:

    a market order created on 2017-01-03 at $220.95,

    Market orders have no price, it is only the close price what's quoted.

    @cowboy said in Questions about outputs of addwriter method:

    Log shows ...

    And the rest of the world can't see the log ... but even with no log ... magic can be done

    @cowboy said in Questions about outputs of addwriter method:

    But, if you look at the csv, you'll see a buy price for that day of $222.22585. I'm curious where this figure comes from. It isn't within the low-high range for either day.

    That's not the buy price. Hint: look at the chart and you will see something which is exactly located at $222.22585 on that day.

    @cowboy said in Questions about outputs of addwriter method:

    Reproducible example:

    Sorry, but there is nothing here which is being reproduced.