@sylvain Problem solved after making the change of shift the timestamp of hist 1 min bar to 1 min later. Thank you !

@rami-ramich I see your question now. After reading through the source code, I feel in your case some customized code needed to accomplish this.

@sylvain Problem solved after making the change of shift the timestamp of hist 1 min bar to 1 min later. Thank you !

@rami-ramich
you can try adding the following in your next()

    print('The current account value is %s' % self.broker.getvalue(self.data))

I

@rami-ramich try cerebro.broker.getvalue() ?

@sylvain I think one possible solution can be: change the timestamp of the historical data to 1 minute later. this would make sense since, IB 1 min hist bar starts at 9:30 ends at 15:59, and after the shift, it will starts at 9:31 ends at 16:00, this will line up with the resampled 1 min live bar. However still unclear about the issue mentioned above.
Also, it looks like the first 1min LIVE bar of the day is:

next run started @ 2019-02-15T09:30:05.705076
Data0, 24601, 2019-02-15T09:30:00.000000, 304.4, 304.56, 304.15, 304.39, 681.0, nan

should it be at 9:31:XX ?
Since the 1min bar would have enough info only after 1min of the mark opened, really not sure how to interpret this. Any help would be appreciated @backtrader

Hello all, I got ran into an issue with live data resampling from 1 min to 1 day. As you can see from some of my results below, the resampling works very well during backfilling, as well as resampling from 1min to 10min or 30min. However, it ran into this issue, the 1day timeframe data feeds did not get updated to the previous day's value.

My code for getdata() and resampling:

data = ibstore.getdata(dataname='TSLA-STK-SMART-USD', timeframe = bt.TimeFrame.Minutes, compression =1, rtbar=False,
                       qcheck=1.0, backfill_start=True, backfill=True, backfill_from = dataHist, useRTH=True, latethrough =False,
                       sessionstart=datetime.time(9,30,00), sessionend=datetime.time(16,00,00))


data.addfilter(bt.filters.SessionFilter(data))

cerebro.resampledata(data, timeframe=bt.TimeFrame.Minutes, compression=1) ## Adding 1min bar （data0)
cerebro.resampledata(data, timeframe=bt.TimeFrame.Minutes, compression=10) ## Adding 10min bar (data1)
cerebro.resampledata(data, timeframe=bt.TimeFrame.Minutes, compression=30) ## Adding 30min bar (data2)
cerebro.resampledata(data, timeframe=bt.TimeFrame.Days, compression=1) ## Adding 1day bar (data3)

datafeeds during backfilling:

.......
next run started @ 2019-02-15T09:01:17.282031
Data0, 24209, 2019-02-13T15:58:00.000000, 308.53, 308.55, 308.14, 308.34, 366.0,
Data1, 2421, 2019-02-13T15:50:00.000000, 307.97, 307.97, 307.37, 307.92, 1027.0,
Data2, 0807, 2019-02-13T15:30:00.000000, 309.31, 309.47, 307.79, 307.91, 1514.0
Data3, 0062, 2019-02-12T16:00:00.000000, 316.2, 318.19, 309.62, 311.81, 41598.0,

next run started @ 2019-02-15T09:01:17.332751
Data0, 24210, 2019-02-13T15:59:00.000000, 308.36, 308.43, 308.0, 308.11, 273.0, nan
Data1, 2421, 2019-02-13T15:50:00.000000, 307.97, 307.97, 307.37, 307.92, 1027.0, nan
Data2, 0807, 2019-02-13T15:30:00.000000, 309.31, 309.47, 307.79, 307.91, 1514.0
Data3, 0062, 2019-02-12T16:00:00.000000, 316.2, 318.19, 309.62, 311.81, 41598.0,

next run started @ 2019-02-15T09:01:17.386498
Data0, 24210, 2019-02-13T15:59:00.000000, 308.36, 308.43, 308.0, 308.11, 273.0, nan
Data1, 2421, 2019-02-13T15:50:00.000000, 307.97, 307.97, 307.37, 307.92, 1027.0, nan
Data2, 0807, 2019-02-13T15:30:00.000000, 309.31, 309.47, 307.79, 307.91, 1514.0
Data3, 0063, 2019-02-13T16:00:00.000000, 312.35, 312.75, 305.57, 308.11, 38587.

next run started @ 2019-02-15T09:01:17.435305
Data0, 24211, 2019-02-14T09:30:00.000000, 303.38, 304.62, 302.95, 304.1, 1294.0, nan
Data1, 2422, 2019-02-14T09:30:00.000000, 307.99, 308.75, 302.95, 304.1, 3439.0, -1.0
Data2, 0808, 2019-02-14T09:30:00.000000, 307.81, 308.75, 302.95, 304.1, 4910.0
Data3, 0063, 2019-02-13T16:00:00.000000, 312.35, 312.75, 305.57, 308.11, 38587.0,

next run started @ 2019-02-15T09:01:17.481209
Data0, 24212, 2019-02-14T09:31:00.000000, 304.11, 304.63, 303.21, 303.38, 387.0, nan
Data1, 2422, 2019-02-14T09:30:00.000000, 307.99, 308.75, 302.95, 304.1, 3439.0, -1.0
Data2, 0808, 2019-02-14T09:30:00.000000, 307.81, 308.75, 302.95, 304.1, 4910.0
Data3, 0063, 2019-02-13T16:00:00.000000, 312.35, 312.75, 305.57, 308.11, 38587.0,
.......

Resampling while turn into live:

.......
next run started @ 2019-02-15T09:01:39.194076
Data0, 24600, 2019-02-14T15:59:00.000000, 304.0, 304.05, 303.35, 303.83, 635.0, nan
Data1, 2460, 2019-02-14T15:50:00.000000, 304.6, 304.86, 304.34, 304.42, 907.0, nan
Data2, 0820, 2019-02-14T15:30:00.000000, 304.37, 305.87, 304.2, 304.88, 1797.0
Data3, 0063, 2019-02-13T16:00:00.000000, 312.35, 312.75, 305.57, 308.11, 38587.0,

***** DATA NOTIF: DELAYED
***** DATA NOTIF: LIVE
next run started @ 2019-02-15T09:30:05.705076
Data0, 24601, 2019-02-15T09:30:00.000000, 304.4, 304.56, 304.15, 304.39, 681.0, nan
Data1, 2461, 2019-02-15T09:30:00.000000, 304.44, 304.56, 303.35, 304.39, 2784.0, nan
Data2, 0821, 2019-02-15T09:30:00.000000, 304.95, 305.09, 303.35, 304.39, 4159.0
Data3, 0063, 2019-02-13T16:00:00.000000, 312.35, 312.75, 305.57, 308.11, 38587.0,

next run started @ 2019-02-15T09:31:01.533144
Data0, 24602, 2019-02-15T09:31:00.000000, 304.31, 304.53, 304.09, 304.51, 204.0, nan
Data1, 2461, 2019-02-15T09:30:00.000000, 304.44, 304.56, 303.35, 304.39, 2784.0, nan
Data2, 0821, 2019-02-15T09:30:00.000000, 304.95, 305.09, 303.35, 304.39, 4159.0
Data3, 0063, 2019-02-13T16:00:00.000000, 312.35, 312.75, 305.57, 308.11, 38587.0,
.......

One can see the daily bar(Data3) is not getting update to the correct timestamp. For the information, the dataHist I am backfilling from is the RTH 1min bar stored locally, it will get update once a day after market close, so it alwasy up to yesterday's 15:59:00.

I really appreciate if you could provide any guidance.

bt.filters.SessionFilter is the solution.

data = ibstore.getdata(dataname='AAPL-STK-SMART-USD', timeframe = bt.TimeFrame.Minutes, compression =1,rtbar=True,
                       qcheck=1.0,backfill_start=True,backfill=True,backfill_from =dataHist,latethrough =False,
                       tz='US/Eastern', tzinput ='US/Eastern', useRTH=True, sessionstart=datetime.time(9,30,00),
                        sessionend=datetime.time(16,00,00))
data.addfilter(bt.filters.SessionFilter)

Hello sir, thank you in advance!! I have been struggling with live trading with IB, in my other topic you have provided a great solution on backfilling only regular trading hours' data feeds by set useRTH=Ture.
However, some new question pops out I feel it's hard to solve. Ideally, my program will run 24/7 on VPS, with live-1min-bar being feed to next(). Even with useRTH=True, next() still getting live data that outside RTH.
By the nature of bt indicators, say SMA, it is calculated on time window from data[0] to data[-n], which rely on a well-formatted data.
My target is to have the datafeeds only contains RTH data(3:59 day1 connected to 9:30 day2 for US stocks), in such case the indicators do not mass up by non-trading-hours' data.

Is there any way I could formate such data from live data feeds? Even with the custom code needed, which approaches you to recommend me to go for, would work better with your package? (1. make the indicator "smarter", so they know non-trading hours' data should be ignored. 2, formate the live data feeds, if it's outside RTH, do not pass to next(). 3. If a non-trading hour bar has been pass to next, use some function to delete that bar in datafeeds, so the data[-1] still inside RTH. 4. Pause the program when the time is outside RTH(might need some good control for such pause) )

@backtrader I really appreciate the answer, I think "useRTH" would work, sorry I missed that variable in the documentation. One last thing about this backfilling I wish to point out or get confirmed is that I feel the default time zone on backfill_from has been set to UTC-1. My settings: TWS is @ us-Pacific, ContractDetails @ us-Eastern, getdata(tz=us-Pacific), the local historical data stored for backfill_from is timestamped@us-Pacific, however, the histData does not line up with the LIVE feeds until I manually it by adding +7 hours on the timestamps.
Thanks again for all the help, I have been using your package for 2 months, really a great platform, saved a lot of time for me while building out my strategy.

@backtrader Thank you for much for clarifying that. I have one more question, for the build in backfill_start, particularly on 1 min bar.
When I start running the program at Jan 29 10:00 am US/Eastern the backfill period went from Jan29 might night to Jan29 9:59 am. However, any data before regular market hours (9:30) am is relatively meaningless(for most strategies). So ideally the backfilling period I am looking for at 10:00 am Jan 29, is 10:00 am ~ 3:59 pm Jan 28 and 9:30 am to 9:59 am Jan 29.
In such a case, I would consider using my own historical data (say every 1 min bar in trading hours till 3:59 pm Jan 28), but when backfilling the gap, is there any way to limit backfilling only on regular market hours? (in this case only backfill 9:30-9:59 Jan 29)