next() called multiple times with resampled data feeds

nimrare

Hello backtrader community

First of all, thanks for this awesome piece of software. My favorite backtesting framework by far. ;)
Recently, I started working with mixed time frames and stumbled upon the following problem.
I have a daily data feed and a weekly data feed. At the end of the week, when the weekly bar forms, next seems to be called twice. The output for the code below looks like the following:

2018-02-02 27.3 27.3
2018-02-05 36.51 27.3
2018-02-06 35.5 27.3
2018-02-07 36.94 27.3
2018-02-08 45.79 27.3
2018-02-09 41.48 27.3
2018-02-09 41.48 41.48
2018-02-12 39.71 41.48
2018-02-13 39.75 41.48
2018-02-14 35.16 41.48
2018-02-15 34.41 41.48
2018-02-16 34.95 41.48
2018-02-16 34.95 34.95
2018-02-20 36.4 34.95
2018-02-21 36.81 34.95
2018-02-22 36.3 34.95
2018-02-23 33.49 34.95
2018-02-23 33.49 33.49
2018-02-26 32.1 33.49
2018-02-27 35.13 33.49
2018-02-28 36.7 35.13

import matplotlib.pyplot as plt

import backtrader as bt
import backtrader.indicators as btind
import backtrader.analyzers as btanalyzers

import pandas as pd

class MultiTimeFrameStrategy(bt.Strategy):

	def __init__(self):
		pass

        def next(self):
		print(self.datetime.date(), self.datas[0].close[0], self.datas[1].close[0])

	def stop(self):
		pass

def run_strategy():

	fromdate = "2018-02-01"
	todate = "2018-03-01"

	fromdate = datetime.datetime.strptime(fromdate, "%Y-%m-%d")
	todate = datetime.datetime.strptime(todate, "%Y-%m-%d")

	cerebro = bt.Cerebro()

	# add a strategy
	cerebro.addstrategy(
		MultiTimeFrameStrategy
	)

	try:
		# load data
		dataframe = pd.read_csv("/path/to/data.csv")
		dataframe = dataframe[["timestamp", "open", "high", "low", "close", "volume"]].copy()
		dataframe = dataframe.set_index("timestamp")
		dataframe.index = pd.to_datetime(dataframe.index)

		# convert the dataframe to a data feed, the timeframe of the original data feed must be set
		data = bt.feeds.PandasData(dataname=dataframe, timeframe=bt.TimeFrame.Minutes, compression=1, fromdate=fromdate, todate=todate, plot=True)

		# add the original data feed to cerebro
		#cerebro.adddata(data)

		# add upsampled data feeds
		cerebro.resampledata(data, timeframe=bt.TimeFrame.Days, compression=1)
		cerebro.resampledata(data, timeframe=bt.TimeFrame.Weeks, compression=1)

	except Exception as e:
		print(e)

	cerebro.broker.setcash(100000)

	results = cerebro.run()

if __name__ == "__main__":
	run_strategy()

I don't really understand this behavior. Maybe you could help me out. When I plot the results, the bars seem to be correctly aligend. Thanks a lot for your help.

nimrare

UPDATE: I found out that this does not seem to be happening if I use the original data feed and only one resampled data feed like so:

# convert the dataframe to a data feed, the timeframe of the original data feed must be set
data = bt.feeds.PandasData(dataname=dataframe, timeframe=bt.TimeFrame.Minutes, compression=1, fromdate=fromdate, todate=todate, plot=True)

# add the original data feed to cerebro
cerebro.adddata(data)

# add upsampled data feeds
cerebro.resampledata(data, timeframe=bt.TimeFrame.Days, compression=1)

If I run this, I don't get the duplicated timestamps as shown above. However, another thing that I noticed is that the close of a resampled bar will only be available during the next bar of the fast data feed (in the case above the minute data feed). Is this expected behavior?

backtrader

next is NOT called multiple times. It delivers you what it can.

You are not using a trading calendar and the platform does not know when the trading week ends ... until it sees the start of the next trading week (which is usually a Monday, but must not be). And it then gives you the last resampling.

You could easily argue that when it is Friday, the trading week is over, but that doesn't always apply, because a Friday can be a trading holiday and some assets may trade 7 days a week.

Use a trading calendar and the synchronization will be perfect. You can use pre-existing ones or create your own.

See Docs - Trading Calendars - https://www.backtrader.com/docu/tradingcalendar/tradingcalendar/

nimrare

@backtrader

Hello backtrader

Thanks for your reply. What you say makes sense and I will use the trading calendar feature. But just to make sure I understand the behavior correctly, let's look at the duplicate timestamp of 2018-02-09 shown in my first post above.

2018-02-09 is a Friday. So the the backtrader resampling mechanism creates a daily bar at the end of the day 2018-02-09. At this point it cannot know whether trading might continue on the weekend so it basically looks at the days 2018-02-10 and 2018-02-11. It cannot find that data, so it produces a weekly bar with the timestamp 2018-02-09 in a separate call to next().

Am I assuming correctly that backtrader assumes by default that a week is ending on Sunday if nothing else is specified in terms of trading calendars?

Also in the above code you see that I am using two resampled data feeds but NOT the original data feed. If I am using the original data feed together with one resampled data feed this alignment problem does not seem to happen. I understand that you may need more information to give an answer to this but maybe it's obvious and I am missing something.