Posts made by nimrare

Update:

Okay, I have solved it. I found this article detailing a bit about the matter: https://www.backtrader.com/blog/posts/2016-04-19-sync-different-markets/sync-different-markets/

However, I did not really like the possible solutions so I solved it like that:

...

        datamaster = "VIXY"
        universe   = ["SPY"]

        # add a strategy
	cerebro.addstrategy(MultiFeedStrategy)

	# synchronize data by dropping all indices that are not present in the datamaster
	data_feed_dfs = []

	# load datamaster
	datamaster_df = pd.read_csv("../../data/{}.csv".format(datamaster), index_col="datetime", parse_dates=True)
	datamaster_df = datamaster_df[["open", "high", "low", "close", "volume"]].copy()

	data_feed_dfs.append(datamaster_df)

	for symbol in universe:

		# load data
		dataframe = pd.read_csv("../../data/{}.csv".format(symbol), index_col="datetime", parse_dates=True)
		dataframe = dataframe[["open", "high", "low", "close", "volume"]].copy()

		dataframe = dataframe.loc[dataframe.index.isin(datamaster_df.index)].copy()

		data_feed_dfs.append(dataframe)

	for data_feed_df in data_feed_dfs:

		try:

			# convert the dataframe to data feed
			data = bt.feeds.PandasData(dataname=data_feed_df, fromdate=fromdate, todate=todate, plot=True)

			# add the data feed to cerebro
			cerebro.adddata(data, name=symbol)

		except Exception as e:
			print(e)

	# set our desired cash start
	cerebro.broker.setcash(100000)

...

Basically I am only selecting timestamps that are also present in the datamaster (i.e. first data feed added to the strategy)

Hope this helps others that run into similar problems. :)

However, I would still appreciate if someone could clarify my questions in the first post.

Best,
nimrare

I found something out. The first data feed (on the left side of each column) has a length of 2263 bars and the second data feed has a length of 2264 bars. So it seems there is one bar missing in the first data feed. Now makes sense that there is lag present. In the doc it says the following about indicators:

Background: Indicators are smart dumb objects.

    They are smart because they can make complex calculations.

    They are dumb because they operate with no knowledge of what sources are providing the data for the calculations

Is my assumption therefore correct that in the strategy itself, data feeds are matched by timestamp but in indicators they are not?

Dear backtrader community,

I am having a problem with an indicator taking multiple data feeds as input and I would really appreciate your help.
When I access the data feeds in the next method of the strategy, they are aligned by datetime. In the indicator, however, they are not. Here is the code:

class MultiFeedIndicator(bt.Indicator):

	lines = ('feed_0', 'feed_1')

	def __init__(self):
		pass

	def next(self):
		self.lines.feed_0[0] = self.datas[0].close[0]
		self.lines.feed_1[0] = self.datas[1].close[0]

class MultiFeedStrategy(bt.Strategy):

	def __init__(self):
		
		self.multi_feed_indicator = MultiFeedIndicator(self.datas[0], self.datas[1])

	def next(self):

		print(self.datetime.date(), " - ", 
                      self.datas[0].close[0], 
                      self.multi_feed_indicator.feed_0[0], " - ", 
                      self.datas[1].close[0], 
                      self.multi_feed_indicator.feed_1[0])

def run_strategy():

	fromdate = "2011-01-01"
	todate   = "2020-01-01"

	universe = ["VIXY", "SPY"]

	# setup
	fromdate = datetime.datetime.strptime(fromdate, "%Y-%m-%d")
	todate   = datetime.datetime.strptime(todate, "%Y-%m-%d")

	cerebro = bt.Cerebro()

	# add a strategy
	cerebro.addstrategy(MultiFeedStrategy)

	for symbol in universe:

		try:
			# load data
			dataframe = pd.read_csv("../../data/{}.csv".format(symbol))
			dataframe = dataframe[["datetime", "open", "high", "low", "close", "volume"]].copy()
			dataframe = dataframe.set_index("datetime")
			dataframe.index = pd.to_datetime(dataframe.index)

			# convert the dataframe to data feed
			data = bt.feeds.PandasData(dataname=dataframe, fromdate=fromdate, todate=todate, plot=True)

			# add the data feed to cerebro
			cerebro.adddata(data, name=symbol)

		except Exception as e:
			print(e)

	# set our desired cash start
	cerebro.broker.setcash(100000)

	# run over everything
	results = cerebro.run()

	cerebro.plot(style="candlestick")

if __name__ == "__main__":
	run_strategy()

You see that in the next method of the strategy I simply print the current close and the corresponding output of the indicator next to each other. The output looks as follows:

...
2019-12-19  -  12.22 12.22  -  320.9 319.59
2019-12-20  -  12.35 12.35  -  320.73 320.9
2019-12-23  -  12.41 12.41  -  321.22 320.73
2019-12-24  -  12.28 12.28  -  321.23 321.22
2019-12-26  -  12.25 12.25  -  322.94 321.23
2019-12-27  -  12.52 12.52  -  322.86 322.94
2019-12-30  -  12.97 12.97  -  321.08 322.86
2019-12-31  -  12.43 12.43  -  321.86 321.08

The left side of the columns (i.e. the close prices accessed in the strategy directly) are correctly aligned with the timestamps. However, the right side of the right column is shifted by one. So it seems that in the indicator, timestamps of multiple data feeds are not correctly aligned.

Could help me out here? From the docs I gather that timestamps of multiple data feeds should be automatically aligned. (I still don't quite know what the default behavior is for missing data in the primary or secondary data feed, maybe you can clarify that for me). So I though I am also guaranteed to access data of the same timestamp when I access the current element [0] of a data feed in an indicator. However, this does not seem to be the case. I hope you can help me shed some light on this problem.

Thanks a lot in advance,
nimrare

PS: Is there a way to access the current datetime in an indicator? Like I can use self.datetime.date() in the strategy itself. Would help a lot for debugging problems such as above being able to print timestamps for each data feed in the indicator.

@backtrader

Ah, that makes sense. Thanks a lot for your response.

Hey together

I am having a problem with a strategy where I am rebalancing two assets. See the code below:

class RebalancingStrategy(bt.Strategy):

	def __init__(self, verbose=1):

		# verbosity
		self.verbose = verbose

	def next(self):

		asset_0_target   = 0.75
		asset_1_target   = 0.25
		rebalance_spread = 0.05

		# calculate current position weights
		asset_0_weight = round((self.getposition(self.datas[0]).size * self.datas[0].close[0]) / self.broker.getvalue(), 4)
		asset_1_weight = round((self.getposition(self.datas[1]).size * self.datas[1].close[0]) / self.broker.getvalue(), 4)

		asset_0_spread = abs(asset_0_target - asset_0_weight)
		asset_1_spread = abs(asset_1_target - asset_1_weight)

		total_spread   = asset_0_spread + asset_1_spread

		if total_spread > rebalance_spread:

			# determine sell order and issue it first
			if asset_0_weight > asset_0_target:
				
				sell_asset_num    = 0
				sell_asset_target = asset_0_target
				
				buy_asset_num    = 1
				buy_asset_target = asset_1_target

			else:
			
				sell_asset_num = 1
				sell_asset_target = asset_1_target

				buy_asset_num  = 0
				buy_asset_target = asset_0_target

			self.order_target_percent(data=self.datas[sell_asset_num], target=sell_asset_target, exectype=bt.Order.Limit, price=self.datas[sell_asset_num].open[1])
			self.order_target_percent(data=self.datas[buy_asset_num],  target=buy_asset_target,  exectype=bt.Order.Limit, price=self.datas[buy_asset_num].open[1])

For optical reasons I excluded the notify functions. From what I see in the docs, I assume that the order that gets issued first, will be executed first. As you can see, I am issuing the "sell" order first to get free cash for the buy order. Below is the output from the notify functions.

2018-10-02 - BUY[1]      VIXY    1148    @ 21.76
2018-10-02 - BUY[2]      SPY     257     @ 291.56
2018-10-11 - SELL[3]     VIXY    -222    @ 27.6
2018-10-11 - BUY[4]      SPY     22      @ 277.08
2018-10-25 - SELL[5]     VIXY    -121    @ 31.58
2018-10-25 - BUY[6]      SPY     14      @ 267.38
2018-11-07 - REJ[8]      VIXY    99      @ 0.0
2018-11-07 - SELL[7]     SPY     -9      @ 277.56
2018-11-08 - REJ[10]     VIXY    165     @ 0.0
2018-11-08 - SELL[9]     SPY     -6      @ 280.11
2018-12-10 - SELL[11]    VIXY    -19     @ 33.23
2018-12-10 - BUY[12]     SPY     19      @ 263.37
2018-12-21 - SELL[13]    VIXY    -92     @ 36.91
2018-12-21 - BUY[14]     SPY     13      @ 246.74
2018-12-26 - SELL[15]    VIXY    -73     @ 40.38
2018-12-26 - BUY[16]     SPY     13      @ 235.97

What I don't understand is why sometimes the BUY order is still issued first and therefore obviously rejected. Could you point me to what I am missing. Would be really grateful. :)

@backtrader

Hello backtrader

Thanks for your reply. What you say makes sense and I will use the trading calendar feature. But just to make sure I understand the behavior correctly, let's look at the duplicate timestamp of 2018-02-09 shown in my first post above.

2018-02-09 is a Friday. So the the backtrader resampling mechanism creates a daily bar at the end of the day 2018-02-09. At this point it cannot know whether trading might continue on the weekend so it basically looks at the days 2018-02-10 and 2018-02-11. It cannot find that data, so it produces a weekly bar with the timestamp 2018-02-09 in a separate call to next().

Am I assuming correctly that backtrader assumes by default that a week is ending on Sunday if nothing else is specified in terms of trading calendars?

Also in the above code you see that I am using two resampled data feeds but NOT the original data feed. If I am using the original data feed together with one resampled data feed this alignment problem does not seem to happen. I understand that you may need more information to give an answer to this but maybe it's obvious and I am missing something.

UPDATE: I found out that this does not seem to be happening if I use the original data feed and only one resampled data feed like so:

# convert the dataframe to a data feed, the timeframe of the original data feed must be set
data = bt.feeds.PandasData(dataname=dataframe, timeframe=bt.TimeFrame.Minutes, compression=1, fromdate=fromdate, todate=todate, plot=True)

# add the original data feed to cerebro
cerebro.adddata(data)

# add upsampled data feeds
cerebro.resampledata(data, timeframe=bt.TimeFrame.Days, compression=1)

If I run this, I don't get the duplicated timestamps as shown above. However, another thing that I noticed is that the close of a resampled bar will only be available during the next bar of the fast data feed (in the case above the minute data feed). Is this expected behavior?

Hello backtrader community

First of all, thanks for this awesome piece of software. My favorite backtesting framework by far. ;)
Recently, I started working with mixed time frames and stumbled upon the following problem.
I have a daily data feed and a weekly data feed. At the end of the week, when the weekly bar forms, next seems to be called twice. The output for the code below looks like the following:

2018-02-02 27.3 27.3
2018-02-05 36.51 27.3
2018-02-06 35.5 27.3
2018-02-07 36.94 27.3
2018-02-08 45.79 27.3
2018-02-09 41.48 27.3
2018-02-09 41.48 41.48
2018-02-12 39.71 41.48
2018-02-13 39.75 41.48
2018-02-14 35.16 41.48
2018-02-15 34.41 41.48
2018-02-16 34.95 41.48
2018-02-16 34.95 34.95
2018-02-20 36.4 34.95
2018-02-21 36.81 34.95
2018-02-22 36.3 34.95
2018-02-23 33.49 34.95
2018-02-23 33.49 33.49
2018-02-26 32.1 33.49
2018-02-27 35.13 33.49
2018-02-28 36.7 35.13

import matplotlib.pyplot as plt

import backtrader as bt
import backtrader.indicators as btind
import backtrader.analyzers as btanalyzers

import pandas as pd

class MultiTimeFrameStrategy(bt.Strategy):

	def __init__(self):
		pass

        def next(self):
		print(self.datetime.date(), self.datas[0].close[0], self.datas[1].close[0])

	def stop(self):
		pass

def run_strategy():

	fromdate = "2018-02-01"
	todate = "2018-03-01"

	fromdate = datetime.datetime.strptime(fromdate, "%Y-%m-%d")
	todate = datetime.datetime.strptime(todate, "%Y-%m-%d")

	cerebro = bt.Cerebro()

	# add a strategy
	cerebro.addstrategy(
		MultiTimeFrameStrategy
	)

	try:
		# load data
		dataframe = pd.read_csv("/path/to/data.csv")
		dataframe = dataframe[["timestamp", "open", "high", "low", "close", "volume"]].copy()
		dataframe = dataframe.set_index("timestamp")
		dataframe.index = pd.to_datetime(dataframe.index)

		# convert the dataframe to a data feed, the timeframe of the original data feed must be set
		data = bt.feeds.PandasData(dataname=dataframe, timeframe=bt.TimeFrame.Minutes, compression=1, fromdate=fromdate, todate=todate, plot=True)

		# add the original data feed to cerebro
		#cerebro.adddata(data)

		# add upsampled data feeds
		cerebro.resampledata(data, timeframe=bt.TimeFrame.Days, compression=1)
		cerebro.resampledata(data, timeframe=bt.TimeFrame.Weeks, compression=1)

	except Exception as e:
		print(e)

	cerebro.broker.setcash(100000)

	results = cerebro.run()

if __name__ == "__main__":
	run_strategy()

I don't really understand this behavior. Maybe you could help me out. When I plot the results, the bars seem to be correctly aligend. Thanks a lot for your help.

Hi,

I am a new backtrader user and I really love it so far. Thanks for this awesome piece of software. One thing I noticed though, was that the order reference always keeps increasing, even after creating a new cerebro instance.

Concretely, I have a function that calculates a portfolio performance, given some input assets. This function is called in a loop. Every function call should create a new cerebro instance, yet the order reference always keeps increasing. Is this expected behavior and why?

The program looks like the following:

def calculate_performance(assets, start, end):

    cerebro = bt.Cerebro()

    ...

    results = cerebro.run()
    return cerebro.broker.getvalue()

if __name__ == "__main__":

    for assets in asset_list:

        ending_value = calculate_performance(assets, start, end)

Thanks a lot in advance!