Time in Bar's Datetime is not correct in version 1.9.29.108, comparing to 1.9.14.102

wanyuan

    current_time=self.data.datetime.datetime()

In the new version, it will always yield something like 2016-01-01T23:59:59.999989, even if I tested it on 1-min data. The code works fine on 1.9.14.102

The data were loaded from generic csv with format like this (I removed space, : and - in the datetime string, but it is not the cause of this issue)

    dataname='2016.csv',
    headers=False,
    separator=";",
    fromdate=datetime.datetime(2016, 9, 1),
    todate=datetime.datetime(2016, 12, 30),
    nullvalue=0.0,
    dtformat=('%Y%m%d%H%M%S'),
    datetime=0,
    high=2,
    low=3,
    open=1,
    close=4,
    volume=5,
    openinterest=-1

wanyuan

Not sure about the correct way to use the bar's datetime in the new version. Is it still self.data.datetime.datetime() ?

junajo10

I have the same problem

backtrader

To make resampling safer if the timeframe of the data is equal or greater than Days, the designated end of session is appended.

You are giving the platform neither the timeframe nor a specific sessionend indication. The data may be minute based, but you are not stating it.

Add

timeframe=bt.TimeFrame.Minutes,

And the timestamp will not get the end of session time.

dimitar-petrov

@backtrader,

I extended a bit your example for resampling datafiles and am passing timeframe and compression for both input and output data:

dfkwargs['dataname'] = args.dataname
dfkwargs['timeframe'] = custfeeds.TIMEFRAMES[args.intf]
dfkwargs['compression'] = args.incmp
dfcls = custfeeds.DATAFORMATS[args.format]

# Resample the data
data = dfcls(**dfkwargs)
data = cerebro.resampledata(data,
                 timeframe=custfeeds.TIMEFRAMES[args.outtf],
                 compression=args.outcmp)

 ./bt_resample.py --dataname  <( egrep '03/02/1983|03/03/1983|03/04/1983' /home/dpetrov/develop/data/stester/indices/SP.asc)  --format customOHLC --intf minutes --incmp 5 --outtf minutes --outcmp 60
> /home/dpetrov/develop/python/py-envs/py2env/lib/python2.7/site-packages/backtrader/feed.py(114)dopostinit()
    113         import ipdb; ipdb.set_trace()
--> 114         return _obj, args, kwargs
    115 

ipdb>
ipdb> from pprint import pprint
ipdb> pprint(vars(_obj))
{'_barstack': deque([]),
 '_barstash': deque([]),
 '_compression': 5,
 '_dataname': '/proc/self/fd/11',
 '_feed': None,
 '_ffilters': [],
 '_filters': [],
 '_name': u'',
 '_owner': None,
 '_timeframe': 4,
...

Results

ipdb> c
===============================================================================
11,len,datetime,open,high,low,close,volume,openinterest,Strategy,len,datetime
11,1,1983-03-02 10:00:00,151.09,151.6,151.09,151.38,,151.38,Strategy,1,723971.416667
11,2,1983-03-02 11:00:00,151.39,152.28,151.39,152.28,,152.28,Strategy,2,723971.458333
11,3,1983-03-02 12:00:00,152.29,152.63,152.29,152.51,,152.51,Strategy,3,723971.5
11,4,1983-03-02 13:00:00,152.49,152.49,152.15,152.16,,152.16,Strategy,4,723971.541667
11,5,1983-03-02 14:00:00,152.15,152.22,151.87,152.21,,152.21,Strategy,5,723971.583333
11,6,1983-03-02 23:59:59.999989,152.18,152.23,152.02,152.03,,152.03,Strategy,6,723972.0
11,7,1983-03-03 10:00:00,152.31,153.25,152.31,153.22,,153.22,Strategy,7,723972.416667
11,8,1983-03-03 11:00:00,153.24,153.36,153.24,153.25,,153.25,Strategy,8,723972.458333
11,9,1983-03-03 12:00:00,153.23,153.56,153.23,153.55,,153.55,Strategy,9,723972.5
11,10,1983-03-03 13:00:00,153.57,154.08,153.53,154.08,,154.08,Strategy,10,723972.541667
11,11,1983-03-03 14:00:00,154.12,154.16,153.38,153.57,,153.57,Strategy,11,723972.583333
11,12,1983-03-03 15:00:00,153.58,153.63,153.33,153.35,,153.35,Strategy,12,723972.625
11,13,1983-03-03 23:59:59.999989,153.38,153.48,153.38,153.48,,153.48,Strategy,13,723973.0
11,14,1983-03-04 10:00:00,153.47,153.47,152.53,152.65,,152.65,Strategy,14,723973.416667
11,15,1983-03-04 11:00:00,152.68,152.95,152.68,152.95,,152.95,Strategy,15,723973.458333
11,16,1983-03-04 12:00:00,152.96,153.3,152.96,153.23,,153.23,Strategy,16,723973.5
11,17,1983-03-04 13:00:00,153.24,153.24,152.72,152.79,,152.79,Strategy,17,723973.541667
11,18,1983-03-04 14:00:00,152.78,153.02,152.76,153.02,,153.02,Strategy,18,723973.583333
11,19,1983-03-04 15:00:00,153.05,153.62,153.04,153.57,,153.57,Strategy,19,723973.625
11,20,1983-03-04 16:00:00,153.61,153.67,153.61,153.67,,153.67,Strategy,20,723973.666667
===============================================================================

I am using the following feed:

class CustomOHLC(bt.feeds.GenericCSVData):
    params = (
        ('dtformat', '%m/%d/%Y'),
        ('datetime', 1),
        ('time', 2),
        ('open', 3),
        ('high', 4),
        ('low', 5),
        ('close', 6),
        ('volume', -1),
    )

If I pass sessionend it is setting it correctly but it is not ideal since it is skipping an hour if in particular day the trading ends earlier. (for example 15:00 is missing on 1983-03-02 bellow)

11,len,datetime,open,high,low,close,volume,openinterest,Strategy,len,datetime
11,1,1983-03-02 10:00:00,151.09,151.6,151.09,151.38,,151.38,Strategy,1,723971.416667
11,2,1983-03-02 11:00:00,151.39,152.28,151.39,152.28,,152.28,Strategy,2,723971.458333
11,3,1983-03-02 12:00:00,152.29,152.63,152.29,152.51,,152.51,Strategy,3,723971.5
11,4,1983-03-02 13:00:00,152.49,152.49,152.15,152.16,,152.16,Strategy,4,723971.541667
11,5,1983-03-02 14:00:00,152.15,152.22,151.87,152.21,,152.21,Strategy,5,723971.583333
11,6,1983-03-02 16:00:00,152.18,152.23,152.02,152.03,,152.03,Strategy,6,723971.666667
11,7,1983-03-03 10:00:00,152.31,153.25,152.31,153.22,,153.22,Strategy,7,723972.416667
11,8,1983-03-03 11:00:00,153.24,153.36,153.24,153.25,,153.25,Strategy,8,723972.458333
11,9,1983-03-03 12:00:00,153.23,153.56,153.23,153.55,,153.55,Strategy,9,723972.5
11,10,1983-03-03 13:00:00,153.57,154.08,153.53,154.08,,154.08,Strategy,10,723972.541667
11,11,1983-03-03 14:00:00,154.12,154.16,153.38,153.57,,153.57,Strategy,11,723972.583333
11,12,1983-03-03 15:00:00,153.58,153.63,153.33,153.35,,153.35,Strategy,12,723972.625
11,13,1983-03-03 16:00:00,153.38,153.48,153.38,153.48,,153.48,Strategy,13,723972.666667
11,14,1983-03-04 10:00:00,153.47,153.47,152.53,152.65,,152.65,Strategy,14,723973.416667
11,15,1983-03-04 11:00:00,152.68,152.95,152.68,152.95,,152.95,Strategy,15,723973.458333
11,16,1983-03-04 12:00:00,152.96,153.3,152.96,153.23,,153.23,Strategy,16,723973.5
11,17,1983-03-04 13:00:00,153.24,153.24,152.72,152.79,,152.79,Strategy,17,723973.541667
11,18,1983-03-04 14:00:00,152.78,153.02,152.76,153.02,,153.02,Strategy,18,723973.583333
11,19,1983-03-04 15:00:00,153.05,153.62,153.04,153.57,,153.57,Strategy,19,723973.625
11,20,1983-03-04 16:00:00,153.61,153.67,153.61,153.67,,153.67,Strategy,20,723973.666667

Could you please point me why passing timeframe in my case does not have the effect of clossing the day session with the last hour with active trading.
Thanks in advance.

@OP: Sorry for highjacking the thread.

backtrader

This somehow got under the radar. In any case the recommendation would be to use now a trading calendar which should cover all those cases, including those of days which end up early.