Datetime format internally?

tw00000

Hi,

I'm trying to retrieve a chunk of 150 datetime objects in next() like the following:

self.datas[0].datetime.get(size=150)

However, this behaves strangely and instead returns an array of floats, for example:

def next(self):
        if len(self.datas[0].close) > 150:
                print(self.datas[0].datetime.get(size=150)[0]),
                print(type(self.datas[0].datetime.get(size=150)[0]))

Upon running cerebro, returns:

736684.375
<class 'float'>

This doesn't look like any format I've seen before for date time objects, and doesn't easily convert via ms or seconds or ns.

...

I'm using the Pandas datafeed feature without customization, like so: data = bt.feeds.PandasData(dataname=df), and my index is as expected (prior to loading into backtrader):

print(df.index[0])
print(type(df.index[0]))
Timestamp('2017-12-21 08:55:00')
pandas.tslib.Timestamp

I noticed under the hood that the pandas functionality in backtrader uses to_pydatetime() to convert the pandas datetime objects into native python datetime objects, but this shouldn't be a problem.

Any thoughts what could be happening here? Perhaps something in the .get() ?

EDIT:

The result seems to be the same if I use GenericCSV as well:

data = btfeeds.GenericCSVData(
    dataname='test_data.csv',
    datetime=0,
    high=2,
    low=3,
    open=1,
    close=4,
    volume=5,
    openinterest=-1
)

tw00000

For anyone else who stumbles on this, I found this to be a successful workaround (gleaned from another help request post about a different topic):

Using: [bt.utils.date.num2date(date) for date in self.datas[0].datetime.get(size=150)]

In place of: self.datas[0].datetime.get(size=150)

algoguy235

There is an alias version of that which is a little simpler - bt.num2date and bt.date2num for converting back and forth.

Backtrader uses timestamps so that datetimes can be represented as floats. The timestamp version it uses is the same as Matplotlib. From matplotlib.dates documentation:

Matplotlib represents dates using floating point numbers specifying the number of days since 0001-01-01 UTC, plus 1. For example, 0001-01-01, 06:00 is 1.25, not 0.25. Values < 1, i.e. dates before 0001-01-01 UTC are not supported

backtrader

@tw00000 said in Datetime format internally?:

Using: [bt.utils.date.num2date(date) for date in self.datas[0].datetime.get(size=150)]
In place of: self.datas[0].datetime.get(size=150)

This is partially correct. If you are working with timezones, that will only give you UTC time. The correct form would be:

current_datetime = self.data.num2date()

For those 150 objects

those_150_datetimes = [self.data.num2date(x) for x in self.data.datetime.get(size=150)]

tw00000

@backtrader said in Datetime format internally?:

those_150_datetimes = [self.data.numdate(x) for x in self.data.datetime.get(size=150)]

That code returns: 'Lines_LineSeries_DataSeries_OHLC_OHLCDateTime_Abst' object has no attribute 'numdate' EXCEPTION IN (<ipython-input-6-3ad54d8e5ed1>, LINE 22 "[self.datas[0].numdate(x) for x in self.data.datetime.get(size=window)]"): 'Lines_LineSeries_DataSeries_OHLC_OHLCDateTime_Abst' object has no attribute 'numdate'

But, I think it's just a typo, this seems to work: [self.datas[0].num2date(x) for x in self.data.datetime.get(size=window)]