For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

Data Feeds Not Aligned In Custom Indicator



  • Dear backtrader community,

    I am having a problem with an indicator taking multiple data feeds as input and I would really appreciate your help.
    When I access the data feeds in the next method of the strategy, they are aligned by datetime. In the indicator, however, they are not. Here is the code:

    class MultiFeedIndicator(bt.Indicator):
    
    	lines = ('feed_0', 'feed_1')
    
    	def __init__(self):
    		pass
    
    	def next(self):
    		self.lines.feed_0[0] = self.datas[0].close[0]
    		self.lines.feed_1[0] = self.datas[1].close[0]
    
    class MultiFeedStrategy(bt.Strategy):
    
    	def __init__(self):
    		
    		self.multi_feed_indicator = MultiFeedIndicator(self.datas[0], self.datas[1])
    
    	def next(self):
    
    		print(self.datetime.date(), " - ", 
                          self.datas[0].close[0], 
                          self.multi_feed_indicator.feed_0[0], " - ", 
                          self.datas[1].close[0], 
                          self.multi_feed_indicator.feed_1[0])
    
    def run_strategy():
    
    	fromdate = "2011-01-01"
    	todate   = "2020-01-01"
    
    	universe = ["VIXY", "SPY"]
    
    	# setup
    	fromdate = datetime.datetime.strptime(fromdate, "%Y-%m-%d")
    	todate   = datetime.datetime.strptime(todate, "%Y-%m-%d")
    
    	cerebro = bt.Cerebro()
    
    	# add a strategy
    	cerebro.addstrategy(MultiFeedStrategy)
    
    	for symbol in universe:
    
    		try:
    			# load data
    			dataframe = pd.read_csv("../../data/{}.csv".format(symbol))
    			dataframe = dataframe[["datetime", "open", "high", "low", "close", "volume"]].copy()
    			dataframe = dataframe.set_index("datetime")
    			dataframe.index = pd.to_datetime(dataframe.index)
    
    			# convert the dataframe to data feed
    			data = bt.feeds.PandasData(dataname=dataframe, fromdate=fromdate, todate=todate, plot=True)
    
    			# add the data feed to cerebro
    			cerebro.adddata(data, name=symbol)
    
    		except Exception as e:
    			print(e)
    
    	# set our desired cash start
    	cerebro.broker.setcash(100000)
    
    	# run over everything
    	results = cerebro.run()
    
    	cerebro.plot(style="candlestick")
    
    if __name__ == "__main__":
    	run_strategy()
    

    You see that in the next method of the strategy I simply print the current close and the corresponding output of the indicator next to each other. The output looks as follows:

    ...
    2019-12-19  -  12.22 12.22  -  320.9 319.59
    2019-12-20  -  12.35 12.35  -  320.73 320.9
    2019-12-23  -  12.41 12.41  -  321.22 320.73
    2019-12-24  -  12.28 12.28  -  321.23 321.22
    2019-12-26  -  12.25 12.25  -  322.94 321.23
    2019-12-27  -  12.52 12.52  -  322.86 322.94
    2019-12-30  -  12.97 12.97  -  321.08 322.86
    2019-12-31  -  12.43 12.43  -  321.86 321.08
    

    The left side of the columns (i.e. the close prices accessed in the strategy directly) are correctly aligned with the timestamps. However, the right side of the right column is shifted by one. So it seems that in the indicator, timestamps of multiple data feeds are not correctly aligned.

    Could help me out here? From the docs I gather that timestamps of multiple data feeds should be automatically aligned. (I still don't quite know what the default behavior is for missing data in the primary or secondary data feed, maybe you can clarify that for me). So I though I am also guaranteed to access data of the same timestamp when I access the current element [0] of a data feed in an indicator. However, this does not seem to be the case. I hope you can help me shed some light on this problem.

    Thanks a lot in advance,
    nimrare

    PS: Is there a way to access the current datetime in an indicator? Like I can use self.datetime.date() in the strategy itself. Would help a lot for debugging problems such as above being able to print timestamps for each data feed in the indicator.



  • I found something out. The first data feed (on the left side of each column) has a length of 2263 bars and the second data feed has a length of 2264 bars. So it seems there is one bar missing in the first data feed. Now makes sense that there is lag present. In the doc it says the following about indicators:

    Background: Indicators are smart dumb objects.
    
        They are smart because they can make complex calculations.
    
        They are dumb because they operate with no knowledge of what sources are providing the data for the calculations
    
    

    Is my assumption therefore correct that in the strategy itself, data feeds are matched by timestamp but in indicators they are not?



  • Update:

    Okay, I have solved it. I found this article detailing a bit about the matter: https://www.backtrader.com/blog/posts/2016-04-19-sync-different-markets/sync-different-markets/

    However, I did not really like the possible solutions so I solved it like that:

    ...
    
            datamaster = "VIXY"
            universe   = ["SPY"]
    
            # add a strategy
    	cerebro.addstrategy(MultiFeedStrategy)
    
    	# synchronize data by dropping all indices that are not present in the datamaster
    	data_feed_dfs = []
    
    	# load datamaster
    	datamaster_df = pd.read_csv("../../data/{}.csv".format(datamaster), index_col="datetime", parse_dates=True)
    	datamaster_df = datamaster_df[["open", "high", "low", "close", "volume"]].copy()
    
    	data_feed_dfs.append(datamaster_df)
    
    	for symbol in universe:
    
    		# load data
    		dataframe = pd.read_csv("../../data/{}.csv".format(symbol), index_col="datetime", parse_dates=True)
    		dataframe = dataframe[["open", "high", "low", "close", "volume"]].copy()
    
    		dataframe = dataframe.loc[dataframe.index.isin(datamaster_df.index)].copy()
    
    		data_feed_dfs.append(dataframe)
    
    	for data_feed_df in data_feed_dfs:
    
    		try:
    
    			# convert the dataframe to data feed
    			data = bt.feeds.PandasData(dataname=data_feed_df, fromdate=fromdate, todate=todate, plot=True)
    
    			# add the data feed to cerebro
    			cerebro.adddata(data, name=symbol)
    
    		except Exception as e:
    			print(e)
    
    	# set our desired cash start
    	cerebro.broker.setcash(100000)
    
    ...
    

    Basically I am only selecting timestamps that are also present in the datamaster (i.e. first data feed added to the strategy)

    Hope this helps others that run into similar problems. :)

    However, I would still appreciate if someone could clarify my questions in the first post.

    Best,
    nimrare


Log in to reply
 

});