Wrong execution price during the backtesting

andreyapopov

Hi all,

I'm pretty new to all of this so maybe I'm missing something very obvious, however - I copy-pasted the piece of code from Quickstart section that deals with the super basic strategy of buying after 3 days down. I executed it on SPY. And noticed serious discrepancies between execution price and the actual price on that day.

Example:

The strategy generates buy signal on 2017-12-12 at a close price of 265.42 (which is the correct close price according to several sources I checked).
The strategy then executes the order the next, on 2017-12-13, presumably on open, at a price of 267.06. Thing is though... Open price on that day was 265.69. Even the high on that day was 266.20. Again, I checked in several places, including the feed csv file.

Virtually every trade I checked has the wrong execution price - it is always higher, usually by 0.5-3 points. What could be the reason for this?

The piece of relevant code is:

        if self.dataclose[0] < self.dataclose[-1]:
                # current close less than previous close

                if self.dataclose[-1] < self.dataclose[-2]:
                    # previous close less than the previous close

                    # BUY, BUY, BUY!!! (with default parameters)
                    self.log('BUY CREATE, %.2f' % self.dataclose[0])

                    # Keep track of the created order to avoid a 2nd order
                    self.order = self.buy()

andreyapopov

FWIW I tried cheating on close and it shows correct execution price of the close, so that makes me think something's up with determining the open price on the next bar?

backtrader

@andreyapopov said in Wrong execution price during the backtesting:

Virtually every trade I checked has the wrong execution price - it is always higher, usually by 0.5-3 points. What could be the reason for this?
...
The piece of relevant code is:
...

That's the only non-relevant part for your problem. For anyone to be able to provide an answer, you would have to provide a working script, logs of when orders are generated and execution and some sample of the data.

How you come to generate a buy/sell signal is not relevant for what you claim your problem to be.

andreyapopov

Fair enough. I did copy text from Quickstart with only one modification of taking the data directly from the yahoo finance and no the csv file. Here's the strategy class

class TestStrategy(bt.Strategy):

def log(self, txt, dt=None):
    ''' Logging function fot this strategy'''
    dt = dt or self.datas[0].datetime.date(0)
    print('%s, %s' % (dt.isoformat(), txt))

def __init__(self):
    # Keep a reference to the "close" line in the data[0] dataseries
    self.dataclose = self.datas[0].close

def next(self):
    # Simply log the closing price of the series from the reference
    self.log('Close, %.2f' % self.dataclose[0])

    if self.dataclose[0] < self.dataclose[-1]:
        # current close less than previous close

        if self.dataclose[-1] < self.dataclose[-2]:
            # previous close less than the previous close

            # BUY, BUY, BUY!!! (with all possible default parameters)
            self.log('BUY CREATE, %.2f' % self.dataclose[0])
            self.buy()

And the cerebro call:

# Add a strategy
cerebro.addstrategy(TestStrategy)


# Create a Data Feed
data = bt.feeds.YahooFinanceData(
    dataname='SPY',
    # Do not pass values before this date
    fromdate=datetime.datetime(2017, 1, 1),
    # Do not pass values before this date
    todate=datetime.datetime(2017, 12, 31),
    # Do not pass values after this date
    reverse=False)

# Add the Data Feed to Cerebro
cerebro.adddata(data)

# Set our desired cash start
cerebro.broker.setcash(100000.0)

# Print out the starting conditions
print('Starting Portfolio Value: %.2f' % cerebro.broker.getvalue())

# Run over everything
cerebro.run()

# Print out the final result
print('Final Portfolio Value: %.2f' % cerebro.broker.getvalue())

Few executions from the logs:

2017-12-04, BUY CREATE, 262.80
2017-12-05, BUY EXECUTED, 263.19
2017-12-12, SELL CREATE, 265.42
2017-12-13, SELL EXECUTED, 267.06
2017-12-14, BUY CREATE, 264.31
2017-12-15, BUY EXECUTED, 265.45

The created price matches the close price of the day when it was created, no issues there. The execution price is always higher than the high of the day when the execution is done though.

ab_trader

@andreyapopov since you use Yahoo finance, the problem can be in their data. Try to use different data source to get the same error.

andreyapopov

@ab_trader I think you're right. I checked the source and it appears the adjusted close at Yahoo is lower than the actual close most of the time (?). Open prices are consistent with the actual execution, so the problem is not in the code. Onwards to trying to figure out how to get data from a different source :)

In fact, to elaborate, here are the results I'm getting:

2017-12-04, BUY CREATE, 262.80
2017-12-05, BUY EXECUTED, 263.19
2017-12-12, SELL CREATE, 265.42
2017-12-13, SELL EXECUTED, 267.06
2017-12-14, BUY CREATE, 264.31
2017-12-15, BUY EXECUTED, 265.45

And here's the extract from CSV file for the dates for which I posted the log above:

2017-12-04,266.309998,266.799988,264.079987,264.140015,262.796753,94040600
2017-12-05,263.190002,265.149994,263.040009,263.190002,261.851563,77994500
.......
2017-12-12,267.209991,267.320007,266.350006,266.779999,265.423309,85195800
2017-12-13,267.059998,267.559998,266.649994,266.750000,265.393463,102905400
......
2017-12-14,267.089996,267.220001,265.600006,265.660004,264.308990,100666700
2017-12-15,265.450012,267.040009,265.390015,266.510010,266.510010,144610300

ab_trader

Probably Quandl or Google should have price data for SPY. Yahoo after their shut down last year generates more confusion then clarity.

andreyapopov

@ab_trader Well for right now when I changed the feed to Quandl I'm getting the following error:

Traceback (most recent call last):
File "test.py", line 115, in <module>
cerebro.run()
File "/Library/Python/2.7/site-packages/backtrader/cerebro.py", line 1127, in run
runstrat = self.runstrategies(iterstrat)
File "/Library/Python/2.7/site-packages/backtrader/cerebro.py", line 1209, in runstrategies
data.preload()
File "/Library/Python/2.7/site-packages/backtrader/feed.py", line 692, in preload
self.f.close()
AttributeError: 'NoneType' object has no attribute 'close'

Need to figure out how to use their feeds; I only used Yahoo feeds before...

I think another option would be to use YahooFinanceFeed with parameter adjclose=False, especially for things like SPY. It's too bad we can't choose to NOT adjust for dividends, but do adjust for splits.

backtrader

@andreyapopov said in Wrong execution price during the backtesting:

I think another option would be to use YahooFinanceFeed with parameter adjclose=False, especially for things like SPY. It's too bad we can't choose to NOT adjust for dividends, but do adjust for splits.

Yahoo is 100% broken. With quirks and random changes in what's the adjusted close across a data set. Unless you can identify a segment which is safe, don't use it.

@andreyapopov said in Wrong execution price during the backtesting:

Traceback (most recent call last):
File "test.py", line 115, in <module>
cerebro.run()
File "/Library/Python/2.7/site-packages/backtrader/cerebro.py", line 1127, in run
runstrat = self.runstrategies(iterstrat)
File "/Library/Python/2.7/site-packages/backtrader/cerebro.py", line 1209, in runstrategies
data.preload()
File "/Library/Python/2.7/site-packages/backtrader/feed.py", line 692, in preload
self.f.close()
AttributeError: 'NoneType' object has no attribute 'close'

It will be really difficult for @ab_trader (or anyone else) to identify what your problem is if you don't supply sample code. The error alone doesn't help.