Trading Calendar possible implementation problems?
-
I am trying to implement my own class derived from
TradingCalenderBase
and I am trying for some time to understand the existing implementationTradingCalendar
. I have problems implementing the functionschedule
(and I do not fully understand the implementation inTradingCalendar
either).So either its my bad or maybe there even is a bug in the current implementation. The first is more probable :)
I will just explain what I see and what I do not understand, please feel free to correct me where I am wrong. I do not say my findings are facts.The current implementation in
TradingCalendar
is this:def schedule(self, day, tz=None): ''' Returns the opening and closing times for the given ``day``. If the method is called, the assumption is that ``day`` is an actual trading day The return value is a tuple with 2 components: opentime, closetime ''' while True: dt = day.date() try: i = self._earlydays.index(dt) o, c = self.p.earlydays[i][1:] except ValueError: # not found o, c = self.p.open, self.p.close closing = datetime.combine(dt, c) if tz is not None: closing = tz.localize(closing).astimezone(UTC) closing = closing.replace(tzinfo=None) if day > closing: # current time over eos day += ONEDAY continue opening = datetime.combine(dt, o) if tz is not None: opening = tz.localize(opening).astimezone(UTC) opening = opening.replace(tzinfo=None) return opening, closing
This is what I am struggling with the most:
while True: ... if day > closing: # current time over eos day += ONEDAY continue
I am not totally sure what this code means to do. But when analyzing where
schedule
is used (infeed._getnexteos
) I guess the whole point ofschedule
is actually not to return opening and closing time for the date specified bydate
but to return the session that ends next. This would be in contradiction to the function header sayingReturns the opening and closing times for the given day
. Is this correct so far?When thinking about that, some possible problems came to mind:
-
Regarding that while-loop: let's say we have regular trading hours
Mon
toFri
from09:00
to18:00
. If the function gets called withday
being a random trading day date with its time being18:05
then I suspect it will run into an endless loop causeday
will always be greater thanclosing
.ONEDAY
will be added again and again but the actual time will never chance so it will always be greater than the current date'sclosing
time. Is this a bug or did I misinterpret? Maybe there are assumptions this function makes to avoid this scenario to ever happen?
Suggestion: Do the compareday > closing
always with the originaldatetime
value passed to the function (and not with the variableday
which got increment byONEDAY
) -
I think there is a problem for trading hours that start before UTC midnight
00:00
and end after. Let's sayopening
is22:00 UTC
and closing06:00 UTC
.
Let's take New Zealand exchange which opens at22:00 UTC
and closes at05:00 UTC
(+12h UTC offset). So when callingschedule
withday
being e.g.2005-05-23 04:00 UTC
we are actually in NZSX's trading hours (2005-05-22 22:00 UTC
to2005-05-23 05:00 UTC
) from date2005-05-22
(22th !) which should be the expected return value. But the function would start search at the provided date2005-05-23
and therefore miss the 22th and return then opening and closing times for the 23th which are2005-05-23 22:00 UTC
to2005-05-24 22:00 UTC
. This would be wrong I suspsect.
Suggestion: Do not start searching exactly from the provided date inday
but from the day before (SubtractONEDAY
before starting the search). -
If this is true
day > closing
thenONEDAY
will be added and then searched again for the next day. But there is no check if the next day is a trading day at all. I think this is missing and can lead to returning trading hours for a non-trading day.
Suggestion: Instead of addingONEDAY
, call_nextday
to jump the actual next trading day.
-
-
Could you maybe comment on my assumption that the interface of the function
schedule
is to return the opening and closing time of the session that ends next (instead of the opening/closing for the provided day)?
It's the important point for me. I can hopefully deal with the rest alone. -
@vbs said in Trading Calendar possible implementation problems?:
while True:
...
if day > closing: # current time over eos
day += ONEDAY
continueWhat this code does: if the current
day
given as param is not a trading day, go for the next. Aday
has no time (which can be equaled to00:00:00
, butclosing
contains a time component which should be greater than00:00:00
)If you pass a Monday and it is an active trading day, the loop won't go to the next day. If you pass a Sunday and it isn't a trading day (which is most likely the case), the loop will go to the next day, i.e.: Monday
@vbs said in Trading Calendar possible implementation problems?:
Could you maybe comment on my assumption that the interface of the function schedule is to return the opening and closing time of the session that ends next (instead of the opening/closing for the provided day)?
As explained above. It will return the
closing
of the given day if its a trading day and move to the next if it isn't.@vbs said in Trading Calendar possible implementation problems?:
- I think there is a problem for trading hours that start before UTC midnight ...
2005-05-23
is not greater than2005-05-23 05:00 UTC
(which when comparing carries no timezone payload and is simply2005-05-23 05:00
) soday > closing
will not happen@vbs said in Trading Calendar possible implementation problems?:
- If this is true day > closing then ONEDAY will be added and then searched again for the next day. But there is no check if the next day is a trading day at all. I think this is missing and can lead to returning trading hours for a non-trading day.
If
day
has been found in the database, it won't go over. This is for the case in which the day is not found and the default times from the parameters are taken. In that case there can be no check to see if the day is a trading day or not, because it wasn't found and one works blindly. -
Thanks for your answer! Sorry I could not get back to this earlier. And sorry but I still have problemts to see how it is intended to work.
My main issues are that you say thatday
does not carry time information and that it is not needed to check whetherday
is a trading day.
I wrote some little test cases that are runnable and are intended to demonstrate my issues. Possibly they do not meet some assumptions the functionschedule
made, I am not sure. I thought it would be easier to discuss directly about source code.@backtrader said in Trading Calendar possible implementation problems?:
@vbs said in Trading Calendar possible implementation problems?:
while True:
...
if day > closing: # current time over eos
day += ONEDAY
continueWhat this code does: if the current
day
given as param is not a trading day, go for the next. Aday
has no time (which can be equaled to00:00:00
, butclosing
contains a time component which should be greater than00:00:00
)If you pass a Monday and it is an active trading day, the loop won't go to the next day. If you pass a Sunday and it isn't a trading day (which is most likely the case), the loop will go to the next day, i.e.: Monday
So, as you say
day
has no time I conclude that it is supposed to be an instance ofdatetime.date
? But as far as I can see at runtimeDataFeed
callsschedule
passing a regulardatetime.datetime
instance so it will have a time component:dt = self.lines.datetime[0] dtime = num2date(dt) ... _, nexteos = self._calendar.schedule(dtime, self._tz)
I think this is an important point in understanding the code. My following thoughts are based on the assumption that the function really excepts an instance of
datetime.datetime
which carries a time component. If this is really not true then the following text can be ignored I think. :)if the time component of the provided parameter
day
is after theclosing
time then it will result in an endless loop as demonstrated here:import datetime import backtrader as bt tradingcal = bt.TradingCalendar(open=datetime.time(hour=12), close=datetime.time(hour=22)) # results in an endless loop cause 22:00:02 is always outside of trading hours sched = tradingcal.schedule(datetime.datetime(2018, 1, 1, 22, 0, 2))
@vbs said in Trading Calendar possible implementation problems?:
Could you maybe comment on my assumption that the interface of the function schedule is to return the opening and closing time of the session that ends next (instead of the opening/closing for the provided day)?
As explained above. It will return the
closing
of the given day if its a trading day and move to the next if it isn't.Ok, but as I read the code I always goes forward exactly one day but I cannot see the code checking if that day is a trading day at all. I cannot understand why it would not be needed to check if that following day is a trading day.
I made a small test script that demonstrates my concern about that nextday. The script is runnable so it can be analyzed what is happening.
import datetime import backtrader as bt tradingcal = bt.TradingCalendar(open=datetime.time(hour=12), close=datetime.time(hour=22), earlydays=[(datetime.date(2018, 11, 23), datetime.time(hour=12), datetime.time(hour=20))]) sched = tradingcal.schedule(datetime.datetime(2018, 11, 23, 20, 10, 0)) print(sched) # should skip saturday and sunday and return monday (26-11-2018) assert(sched == (datetime.datetime(2018, 11, 26, 12), datetime.datetime(2018, 11, 26, 22)))
It starts searching on a friday but after close time. So it skips to the next day which is saturday. It does not check if saturday is a trading day and returns it which is wrong in my opinion. I expected the function to return the upcoming monday (which is the next trading day).
@vbs said in Trading Calendar possible implementation problems?:
- I think there is a problem for trading hours that start before UTC midnight ...
2005-05-23
is not greater than2005-05-23 05:00 UTC
(which when comparing carries no timezone payload and is simply2005-05-23 05:00
) soday > closing
will not happenHm, ok, but it will use the trading hours for the date
2005-05-23
which is wrong in my opinion. Because of the timezone of the exchange the trading hours of2005-05-22
should be used which could be different than the ones for2005-05-23
.Also a small test script for demonstration:
import datetime import backtrader as bt import pytz tradingcal = bt.TradingCalendar(open=datetime.time(hour=10), close=datetime.time(hour=16, minute=45), earlydays=[(datetime.date(2018, 11, 20), datetime.time(hour=9), datetime.time(hour=18))], ) sched = tradingcal.schedule(datetime.datetime(2018, 11, 21, 3, 0, 0), pytz.timezone('Pacific/Auckland')) # the requested timestamp actually was date '2018-11-20' (not 21th) according to exhchange's timezone 'Pacific/Auckland' so it should return trading hours # for that date. those trading hours differ since they are defined by the earlydays parameter (instead of regular trading hours) assert(sched == (datetime.datetime(2018, 11, 20, 20), datetime.datetime(2018, 11, 21, 5)))
-
So I guess 'no answer' means I am totally on the wrong track here, right?
-
It only means that time is scarce and sometimes messages, especially those with complex content, are quickly overrun by simpler things and other-life things
-
Ok, no problem, I understand. Knowing your reason is already valuable to me, so thanks for that reply.
-
Hey @vbs I observed the same error when creating my own calendar; I also feel this code might be erroneous. My data holds price information after the trading session. Once it gets outside of the session, it enters an infinite loop as the time is always after the end of the session.