Why the datetime is set to the end of day for minute period bars in the csv data feed?

mothee

I want get the datetime series of the data, and I am not sure whether is right of my using/ understing the csv data feed, I fed the minutes period bar data to the GenericCSVData, when I check the datatime for the loaded data by 'data.lines.datetime.array', I found they were coverted to the end of the day. I debug the code found the logic for the datetime feeding in GenericCSVData._loadline() is as below, the line remarked by ' #*****WHY REPLACE THE dtnum BY dteosnum?', how do I get the right datetime of fed data? thanks!

if self.p.timeframe >= TimeFrame.Days:
            # check if the expected end of session is larger than parsed
            if self._tzinput:
                dtin = self._tzinput.localize(dt)  # pytz compatible-ized
            else:
                dtin = dt

            dtnum = date2num(dtin)  # utc'ize

            dteos = datetime.combine(dt.date(), self.p.sessionend)
            dteosnum = self.date2num(dteos)  # utc'ize

            if dteosnum > dtnum: #*****WHY REPLACE THE dtnum BY dteosnum?
                self.lines.datetime[0] = dteosnum
            else:
                # Avoid reconversion if already converted dtin == dt
                self.l.datetime[0] = date2num(dt) if self._tzinput else dtnum
        else:
            self.lines.datetime[0] = date2num(dt)

backtrader

you say:

@mothee said in Why the datetime is set to the end of day for minute period bars in the csv data feed?:

I fed the minutes period bar data

And the code you are checking has an if guard at the beginning that says:

@mothee said in Why the datetime is set to the end of day for minute period bars in the csv data feed?:

if self.p.timeframe >= TimeFrame.Days:

So, you are either not feeding minutes or you are checking the wrong code section.

In any case and for timeframe 1-Day and above, the bar is placed at the end of the session eos == end of session, which if not specified is the end of the day. If this weren't done for such timeframes, any minute of the session would happen after the 1-Day timeframe which cannot be.

mothee

Thanks you help @backtrader !

I am sure my data are minutes bars, a segment as below:
"Date","Time","Open","High","Low","Close","TotalVolume"
2010-04-16,09:16:00,3450.0,3488.0,3450.0,3468.0,489
2010-04-16,09:17:00,3468.0,3473.8,3467.0,3467.0,302
2010-04-16,09:18:00,3467.0,3471.0,3466.0,3467.0,203
2010-04-16,09:19:00,3467.0,3468.2,3448.0,3448.0,280
2010-04-16,09:20:00,3448.0,3459.0,3448.0,3454.0,250
2010-04-16,09:21:00,3454.0,3456.8,3454.0,3456.8,109
2010-04-16,09:22:00,3457.0,3458.8,3456.4,3458.8,162
2010-04-16,09:23:00,3458.0,3460.0,3455.2,3455.2,202
2010-04-16,09:24:00,3455.0,3455.0,3450.2,3454.0,203
2010-04-16,09:25:00,3454.0,3457.6,3453.6,3456.6,137
2010-04-16,09:26:00,3456.0,3457.0,3447.0,3448.4,282
2010-04-16,09:27:00,3449.4,3451.8,3449.4,3450.0,174
2010-04-16,09:28:00,3450.0,3450.0,3442.0,3442.0,267
2010-04-16,09:29:00,3440.0,3447.0,3438.0,3444.0,401
2010-04-16,09:30:00,3444.0,3447.0,3440.0,3447.0,178
2010-04-16,09:31:00,3447.0,3447.6,3443.0,3445.8,301
2010-04-16,09:32:00,3445.8,3447.6,3443.0,3447.0,207
2010-04-16,09:33:00,3447.8,3458.0,3445.8,3458.0,361
2010-04-16,09:34:00,3458.2,3458.2,3450.0,3452.0,234
2010-04-16,09:35:00,3451.8,3454.0,3451.6,3451.8,204
2010-04-16,09:36:00,3451.6,3451.8,3449.0,3451.8,262

was my using of the csv feed wrong? my code of feed is as below:

datafile = os.path.join(modpath, '../data/IF0000_1min.csv')
    # Create a Data Feed
    data = bt.feeds.GenericCSVData(
        dataname=datafile,
        fromdate=datetime.datetime(2012, 1, 1),
        todate=datetime.datetime(2012, 1, 31),
        dtformat=('%Y-%m-%d'),
        tmformat=('%H:%M:%S'),
        datetime=0,
        time=1,
        high=3,
        low=4,
        open=2,
        close=5,
        volume=6,
        openinterest=-1)

mothee

@backtrader , It works well now, it need set 'timeframe = bt.TimeFrame.Minutes' explicitly.

backtrader

@mothee said in Why the datetime is set to the end of day for minute period bars in the csv data feed?:

it need set 'timeframe = bt.TimeFrame.Minutes' explicitly.

If you don't tell the platform what your data feed is, it cannot know it.