Bug building bars in live mode regarding last tick?
-
Hello gentlemen,
I am new to this place and I am not quite experienced with automated trading. Currently I am experimenting with Backtrader's live mode and I have a problem (which I think it is a bug):
When feeding live ticks into Backtrader then bars are built in real-time as ticks fly in. I might be wrong but I think there is a bug about how BT handles the first/last tick of a bar.
As far as I know it is the standard way, that when (for example) using M1 bars then the bar at, let's say,
"14:00"
includes tick data from13:00.000
to13:59.999.
So a tick at exactly14:00.000
would be the first tick for the following"15:00"
-bar. Is this correct so far?I think BT actually is treading a tick at
14:00.000
as the last tick of"14:00"
-bar (instead as the first tick of the"15:00"
-bar).An example output generated by my test code to illustrate the problem:
{'Time': datetime.datetime(2017, 12, 22, 20, 55, 57, tzinfo=<UTC>), 'Last': 142.0, {'Time': datetime.datetime(2017, 12, 22, 20, 55, 58, tzinfo=<UTC>), 'Last': 143.0, {'Time': datetime.datetime(2017, 12, 22, 20, 55, 59, tzinfo=<UTC>), 'Last': 144.0, {'Time': datetime.datetime(2017, 12, 22, 20, 56, tzinfo=<UTC>), 'Last': 145.0, 2017-12-22, Bar finished: Time: 2017-12-22 20:56:00 O:101.0 H:145.0 L:101.0 C:145.0 - Len: 1
The first 4 lines in the log are ticks that are coming in with values starting at
142
incrementing with every tick. Then at20:56:00
the bar finishes. The close price is145
but the tick which included the value145
had the timestamp20:56.000
. So I had expected BT to not included that tick into the20:56
-bar but to be the opening tick for the following20:57
-bar. So the20:56
-bar's close price would be144
.So, do I misunderstand something or is it really a bug? If yes, is there any chance that it could be fixed?
Thanks in advance! -
@vbs said in Bug building bars in live mode regarding last tick?:
As far as I know it is the standard way,
After reading your post ... I think that's the key in your question. You are describing that standard way as your expectation, and the platform is giving you something slightly different.
As such ... it doesn't seem like a bug, but rather a mismatched expectation. You may also try to use the parameters available for resampling:
rightedge
(Of course I cannot personally guarantee that it will deliver to your expectations, but it seems the closest thing to what you seem to be asking)
-
Ok I see, maybe it is not as standardized as I thought. :) Great conincident btw, I was just about to click the "Submit" button when your answer popped up ;)
I think
rightedge
does again mean something different though.Behaviors for H1 bars (example with bar timestamp
14:00:00
):- Current (
rightedge
enabled):13:00:01
to14:00:00
rightedge
disabled:14:00:00
to14:59:59
(or maybe14:00:01
to15:00:00
?)- my assumption was:
13:00:00
to13:59:59
When writing the following paragraph I assumed the current behavior was wrong. I have just learned it is not wrong but just different from by expectations:
Anyway, I digged my way several hours through the code of BT trying to understand the resampler mechanics. I think I have build a a bit of comprehension and found the issue (which might not be an issue as I have just learned).The case is about detecting if a tick (or a bar) is on the ending edge of the current bar and should actually close the bar. Then the question is if the tick (or bar) in question should be included in the current bar which is about to be closed or not.
Current behavior is correct when upsampling for example M1 bars to M5 as the last M1 bar should be fully included in the M5 bar.
But this behavior is wrong when upsampling ticks since for example a tick at 14:00:00 should not be included into the bar ending at 14:00:00. Instead a new bar should be opened and the tick should be included there.I have commited a fix into my repo here:
https://github.com/verybadsoldier/backtrader/commit/76d52d72c550606eb06b84138dca8f08d3b85130I think the variable
consumed
controls if the current data should be included before or after switching bars.
So, maybe I should make my "fix" configurable so both ways are possible? Maybe someone being more sound with the BT code could give a comment if I am on the right path with my patch or if there is a more cleaner or better way to do it?
Thank you guys & merry xmas!
- Current (
-
@vbs another parameter
boundoff
can be used to shift the boundaries of the bar. It seems to me that it can work in your case. -
Thanks for the hint. I fiddled with that parameter before but in my understanding it is not exactly the same (please correct me if I am wrong).
As far as I can see thatboundoff
can be used to shift in target units and only is useful when using compression values greater than 1 (shifting M1 boundaries by minutes makes no difference). But I need to shift the source units, e.g. 14:00:00.000 to 13:59:59.999.I still tried it and when setting
boundoff
to 1 when upsampling ticks to M1 bars then I rarely get bars at all. I think something wents quite wrong in that case. -
@vbs
no problem, but based on description of theboundoff
parameter it uses number of source units to shift the new bar. You need to shift your 1 min bar 1 tick forward.bt
approach - 14:00:01 - 15:00:00- you need (if I understand correctly) - 14:00:00 - 14:59:59 - 1 tick forward shift
Description shows same case but on 1 min to 15 min transformation. Maybe it doesn't work for
ticks
as source units. -
@ab_trader
Hm, I have to admit that I cannot find that in the documentation ofboundoff
. The example is a bit different since both source and target units are minutes (unlike in my caseticks
->minutes
).
Also judging from the source code, I think the code in question is solely working on resampler target units. -
https://www.backtrader.com/docu/data-resampling/data-resampling.html
boundoff
(default: 0)Push the boundary for resampling/replaying by an amount of units.
If for example the resampling is from 1 minute to 15 minutes, the default behavior is to take the 1-minute bars from 00:01:00 until 00:15:00 to produce a 15-minutes replayed/resampled bar.
If boundoff is set to 1, then the boundary is pushed 1 unit forward. In this case the original unit is a 1-minute bar.
Consequently the resampling/replaying will now:Use the bars from 00:00:00 to 00:14:00 for the generation of the 15-minutes bar
-
@Paska-Houso
Thank you but I didn't mean that I cannot the documentation in general :). When I said "I cannot find that in the documentation" I meant this statement from @ab_trader :based on description of the boundoff parameter it uses number of source units to shift the new bar
Sorry for confusion...
-
Misread your message. After re-reading ... I understand why @ab_trader believes it does so. From the doc:
If boundoff is set to 1, then the boundary is pushed 1 unit forward. In this case the original unit is a 1-minute bar.
The "source unit" (in your description) seems to be the "original unit" (in the documentation). That doesn't mean that it is actually doing what you are expecting.
-
I've played a bit with
boundoff
, daily and resampled weekly data trying to get shifted weekly bar, but wasn't able to make it. Maybe it should be sametimeframe
specified for both original and target bars in order to force it to work.