Calling data by date

mac0

Is it possible to call data by date in the next() loop?

Namely, I would like to perform some computations in the next() operation to check if last close value is higher/lower than its value at some specific date.

Thx!

backtrader

Data feeds offer you the datetime line which conveys timestamps. And lines can be converted to python datetime instances (of course it only makes sense for the datetime line)

You just have to decide how to perform the accounting.

mac0

Maybe my question was not so clear. I don't have problem getting the date from a line by its index. E.g.:

print datetime.datetime.fromordinal(int(self.getdatabyname(ticker).datetime[-10]))

My problem is how do I get the price associated with a date? e.g. how do I recover the close for 2017-01-31?

In pure pandas dataframe I would do something like this to get the line:

df['2017-01-31']

backtrader

It was crystal clear. by keeping the accounting as to when (in terms of data length) that specific timestamp happened (i.e.: len(data))

mac0

@backtrader said in Calling data by date:

It was crystal clear. by keeping the accounting as to when (in terms of data length) that specific timestamp happened (i.e.: len(data))

Thx!

backtrader

@mac0 said in Calling data by date:

print datetime.datetime.fromordinal(int(self.getdatabyname(ticker).datetime[-10]))

This is far easier done as this, which also takes care of timezone management:

print(self.dnames[ticker].datetime.datetime(-10))

In any case and although it may make sense to refer to something like 2017-01-31 for specific price research around a set of dates, the background in backtrader is:

You throw something in, and you don't really care about the date

This isn't 100% true, because an algorithm could be looking into:

Is it the last day of the month?
Is it 15 minutes before the closing time on the 24th of December? (which happens earlier as other closing times)
Is it already 2017-01-31 because Trump is going to tweet something and the algorithm has to stop trading on that day?

But those things, as stated, are checked when one is at instant 0 (the pivot index which in backtrader always refers to the current instant) during live trading or walk-forward-like actions.

In the backtrader ecosystem, datetime is not an index, the index is the number of instants that have elapsed (number of bars if you wish), in which you can always refer to the current instant as 0. This 0 keeps moving forward, because from the point of view of the system, it doesn't know in advance when the action is going to terminate (even if you are backtesting with a fixed amount of data, the system makes no assumption about it)
A pandas.DataFrame is mostly a static structure and that's why an append is a very expensive operation, but keeping an index and finding things is easy (things like dask build on it by managing multiple Dataframe as chunks of a larger structure to offer dynamic Dataframes

That's why the indexing in backtrader refers to the length of the data (later you have to look backwards from instant 0)

mac0

Thanks for this explanation! Much clearer in my mind.

I am indeed in the need of knowing the exact date to get the return between two end of month business days.

I am trying to replicate a simple tactical asset allocation paper with monthly rebalancing and I need to compute the 1 year return each month. At first, I did it by using the PercentChange function with a period of 252 days but my results are not in line with the source I am following for comparison. This is probably due to the fact that I am not using exactly the same window to compute returns. Hence, I would like to compute the exact percent change over a year.

I was looking into resample but I don't want to resample my daily data to monthly as I am cheating-on-open and by resampling I would lose the next open (correct?).

backtrader

Why not using the TimeReturn which calculates the returns with the timeframe of your choice in real-time? It can even calculate using a data feed for benchmarkig purposes.

mac0

I actually didn't know about it! Still some learning to do!

I have been trying TimeReturn with no luck. I have added the analyzer:

cerebro.addanalyzer(btanalyzers.TimeReturn, timeframe=TimeFrame.Years, data=data, _name='myret')

But how do I catch the returns computed in the next() of my strategy?

backtrader

myret = self.analyzers.myret will get you to the analyzer. The strategy has some attributes holding things like that

See Docs - Strategy (under Member attributes)

And then ... the TimeFrameAnalyzerBase is not documented ... and probably should ... being the base class for analyzers that calculate things on a regular basis based on a given timeframe. Quickly:

Canonically you would call analysis = myanalyzer.get_analysis()

The answer will be a collections.OrderedDict with the keys being datetime.datetime instances
Hack-wise you
- You know that the current results (the OrderedDict) are held in an attribute of the analyzer named myret.rets
- You know that the current key for the insertion in the dictionary is held in an attributed named: myret.dtkey
And you play with it

mac0

Getting much clearer thanks!

I could make it work but came across something strange.

I am on the 31st of January 2011 after the close according to this line in my next():

print self.dnames[ticker1].datetime.datetime()

I get my analyzer details like so:

tmp = self.analyzers.ret_spy.get_analysis()
for i in tmp.items()[-12:]:
    print i

and get the following results (ticker is SPY):

(datetime.datetime(2010, 2, 28, 0, 0), 0.031194710866933484)
(datetime.datetime(2010, 3, 31, 0, 0), 0.05652880621275069)
(datetime.datetime(2010, 4, 30, 0, 0), 0.015470085470085548)
(datetime.datetime(2010, 5, 31, 0, 0), -0.07945459136436328)
(datetime.datetime(2010, 6, 30, 0, 0), -0.05623114199506263)
(datetime.datetime(2010, 7, 31, 0, 0), 0.06830071691532646)
(datetime.datetime(2010, 8, 31, 0, 0), -0.04498050240319207)
(datetime.datetime(2010, 9, 30, 0, 0), 0.08375273003513439)
(datetime.datetime(2010, 10, 31, 0, 0), 0.03820205029352497)
(datetime.datetime(2010, 11, 30, 0, 0), 0.0)
(datetime.datetime(2010, 12, 31, 0, 0), 0.06127099333277064)
(datetime.datetime(2011, 1, 31, 0, 0), 0.01566600397614315)

I manually cross-checked the number and they seem fine for all dates but the last , i.e. 31st of January 2011. To get to the result given by the analyzer I need to compute the result from the close of 31/12/2010 to the close of 28/01/2011. Why the last day isn't taken into account? I cheat-on-open so everything should be after the close.

Side question: is there a way to compute TimeReturn on each ticker without having to call addanalyzer(TimeReturn, data=cerebro.datas[x]) for each one of them?

Thanks again for your help!

backtrader

@mac0 said in Calling data by date:

Side question: is there a way to compute TimeReturn on each ticker without having to call addanalyzer(TimeReturn, data=cerebro.datas[x]) for each one of them?

No.

@mac0 said in Calling data by date:

I manually cross-checked the number and they seem fine for all dates but the last , i.e. 31st of January 2011. To get to the result given by the analyzer I need to compute the result from the close of 31/12/2010 to the close of 28/01/2011. Why the last day isn't taken into account? I cheat-on-open so everything should be after the close.

It seems your data would end on 2011-01-28. The analyzer doesn't know your data ends there ... and the end of the montha and/or next month has not been seen. It can only calculate it when stopping (stop method) and you can therefore only see that in the corresponding strategy stop method.