For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

Pandas Dataframe issue (int object has no attribute to_pydatetime)



  • Re: Pandas Dataframe issue with datetime index

    Re: Bug using Pandas hdf?

    Anyone has got a solution to this?
    I am still not able to figure out why this error is occurring. (AttributeError: 'int' object has no attribute 'to_pydatetime')

    Below is my code which is giving the error. I have tried the above two solutions. They haven't worked either and have tried multiple ways to solve this.

    try:
            connection = psycopg2.connect(user="<ABC>",
                                          host="<ABC>",
                                          port="<ABC>",
                                          database="<ABC>")
    
            cursor = connection.cursor()
            # Print PostgreSQL Connection properties
            print(connection.get_dsn_parameters(), "\n")
    
            # Print PostgreSQL version
            cursor.execute("SELECT version();")
            record = cursor.fetchone()
            print("You are connected to - ", record, "\n")
    
    except (Exception, psycopg2.Error) as error:
        print("Error while connecting to PostgreSQL", error)
    ticker_data = psql.read_sql("""select tick_date as datetime,open_price as open,high_price as high,low_price as low,close_price as close,volume from stock_ticker_daily_new where stock_code = 'PNC' and tick_date > '2019-09-01' and tick_date < '2019-11-25';"""  , connection)
    ticker_data['datetime'] = pd.to_datetime(ticker_data['datetime'])
    data = bt.feeds.PandasData(dataname=ticker_data)
    cerebro.adddata(data, name = i)
        
    cerebro.addanalyzer(Screener_SMA, period=14)
    cerebro.run(runonce=False, stdstats=False, writer=True)
    # for i in ticker_symbols['stock_code']:
    #     ticker_data[i] = psql.read_sql("""select tick_date as datetime,open_price as open,high_price as high,low_price as low,close_price as close,volume from stock_ticker_daily_new where stock_code = '%s' and tick_date > '2019-11-01' and tick_date < '2019-11-25';""" % i, connection)
    print(ticker_data)# print(ticker_data)
    # closing database connection.
    if (connection):
        cursor.close()
        connection.close()
    

    This is the error.

    You are connected to -  ('PostgreSQL 11.6 on x86_64-apple-darwin16.7.0, compiled by Apple LLVM version 8.1.0 (clang-802.0.42), 64-bit',) 
    
         datetime   open   high    low  close  volume
    0  2019-09-03  15.50  15.50  14.30  14.45     681
    1  2019-09-04  14.20  15.45  14.10  14.90    5120
    2  2019-09-05  15.00  15.00  14.40  14.40     179
    3  2019-09-06  15.00  15.00  14.35  14.95   12216
    4  2019-09-09  15.00  15.00  14.55  14.65    3073
    5  2019-09-11  15.00  15.00  14.40  14.50     510
    6  2019-09-12  15.00  15.00  14.25  14.35    3827
    7  2019-09-13  14.85  14.95  14.60  14.90    5684
    8  2019-09-16  14.80  14.80  14.20  14.55   11345
    9  2019-09-17  14.80  14.80  14.80  14.80     180
    10 2019-09-18  14.75  14.90  14.35  14.85    9394
    11 2019-09-19  15.00  15.00  14.30  14.45    2896
    12 2019-09-20  14.30  17.30  14.30  15.05   14271
    13 2019-09-23  15.90  15.90  14.75  14.75   14584
    14 2019-09-24  15.40  15.40  14.35  15.05   12124
    15 2019-09-25  16.80  16.80  13.50  14.10   32398
    16 2019-09-26  14.55  15.50  14.30  14.40    9016
    17 2019-09-27  15.25  15.25  14.20  14.40    2886
    18 2019-09-30  15.20  15.20  14.00  14.05    2671
    19 2019-10-01  14.75  14.75  13.25  13.35    2039
    20 2019-10-03  14.85  14.85  13.10  13.65    3409
    21 2019-10-04  14.45  14.45  13.25  13.35    2180
    22 2019-10-07  13.10  14.20  13.10  13.35     570
    23 2019-10-09  14.85  14.85  13.35  13.55     406
    24 2019-10-10  14.40  14.45  13.10  13.40     300
    25 2019-10-11  14.20  14.20  13.10  13.35     530
    26 2019-10-14  13.00  14.75  13.00  13.50   30290
    27 2019-10-15  14.75  14.75  12.50  12.80    1043
    28 2019-10-16  13.90  13.90  12.60  12.75    1266
    29 2019-10-17  14.30  14.30  12.10  12.65    2504
    30 2019-10-18  13.95  13.95  12.80  13.00     253
    31 2019-10-22  14.70  14.70  12.55  13.25    2360
    32 2019-10-23  14.80  14.80  12.40  13.70   35380
    33 2019-10-24  14.90  14.90  13.35  13.80    1362
    34 2019-10-25  14.80  14.80  13.35  13.30     198
    35 2019-10-29  14.50  14.50  13.25  13.50    1911
    36 2019-10-30  14.00  14.00  12.75  14.00    1137
    37 2019-10-31  15.45  15.45  13.10  14.60    2528
    38 2019-11-01  14.95  14.95  13.35  14.00    1435
    39 2019-11-04  14.00  14.75  14.00  14.70    2088
    40 2019-11-05  15.00  15.00  14.05  14.70     268
    41 2019-11-06  14.50  14.50  13.65  13.75     309
    42 2019-11-07  14.90  14.90  14.15  14.70     222
    43 2019-11-08  14.50  14.55  13.60  13.95    5583
    44 2019-11-11  14.80  14.80  14.00  14.70     857
    45 2019-11-13  14.70  14.80  14.05  14.35     267
    46 2019-11-14  14.95  14.95  14.00  14.25    1025
    47 2019-11-15  13.60  14.90  13.20  14.65   13032
    48 2019-11-18  14.80  14.80  13.50  14.50     428
    49 2019-11-19  14.05  14.70  13.75  14.15    7511
    50 2019-11-20  14.00  14.70  13.75  14.65    1511
    51 2019-11-21  14.70  14.70  13.75  14.05    2347
    52 2019-11-22  13.95  15.10  13.80  14.70    7812
    ---------------------------------------------------------------------------
    AttributeError                            Traceback (most recent call last)
    <ipython-input-226-20a6becfd09f> in <module>
         23 
         24 cerebro.addanalyzer(Screener_SMA, period=14)
    ---> 25 cerebro.run(runonce=False, stdstats=False, writer=True)
         26 # for i in ticker_symbols['stock_code']:
         27 #     ticker_data[i] = psql.read_sql("""select tick_date as datetime,open_price as open,high_price as high,low_price as low,close_price as close,volume from stock_ticker_daily_new where stock_code = '%s' and tick_date > '2019-11-01' and tick_date < '2019-11-25';""" % i, connection)
    
    ~/py_37_env/lib/python3.7/site-packages/backtrader/cerebro.py in run(self, **kwargs)
       1125             # let's skip process "spawning"
       1126             for iterstrat in iterstrats:
    -> 1127                 runstrat = self.runstrategies(iterstrat)
       1128                 self.runstrats.append(runstrat)
       1129                 if self._dooptimize:
    
    ~/py_37_env/lib/python3.7/site-packages/backtrader/cerebro.py in runstrategies(self, iterstrat, predata)
       1210                 data._start()
       1211                 if self._dopreload:
    -> 1212                     data.preload()
       1213 
       1214         for stratcls, sargs, skwargs in iterstrat:
    
    ~/py_37_env/lib/python3.7/site-packages/backtrader/feed.py in preload(self)
        436 
        437     def preload(self):
    --> 438         while self.load():
        439             pass
        440 
    
    ~/py_37_env/lib/python3.7/site-packages/backtrader/feed.py in load(self)
        477 
        478             if not self._fromstack(stash=True):
    --> 479                 _loadret = self._load()
        480                 if not _loadret:  # no bar use force to make sure in exactbars
        481                     # the pointer is undone this covers especially (but not
    
    ~/py_37_env/lib/python3.7/site-packages/backtrader/feeds/pandafeed.py in _load(self)
        266 
        267         # convert to float via datetime and store it
    --> 268         dt = tstamp.to_pydatetime()
        269         dtnum = date2num(dt)
        270         self.lines.datetime[0] = dtnum
    
    AttributeError: 'int' object has no attribute 'to_pydatetime'
    

    There is a chance I am making some mistake. Can someone please help me with it? If any other information is needed or more of the code is needed I can uploaded it but I thought this would be sufficient since I think the issue is in the format of the data I am getting out from sql.
    Few things to know about the data types I have tried

    • Tried making the date column into date time and then into index. Didn't work.

    • Just for the information only the volume column is an int. Others are float and I am converting datetime column into pandas date time and then index.

    • EVEN IF I add this line to set index same error is given.

    '''
    ticker_data = ticker_data.set_index('datetime')
    '''

    Any help would be appreciated and if anything else is needed I would be happy to add on to the question. Thanks in advance.


  • administrators

    @Suraj-Thorat said in Pandas Dataframe issue (int object has no attribute to_pydatetime):

    Anyone has got a solution to this?

    Yes. Read the documentation.

    The error is self-explaining.

    int object has no attribute to_pydatetime
    

    @Suraj-Thorat said in Pandas Dataframe issue (int object has no attribute to_pydatetime):

         datetime   open   high    low  close  volume
    0  2019-09-03  15.50  15.50  14.30  14.45     681
    1  2019-09-04  14.20  15.45  14.10  14.90    5120
    

    And you have an index which is made up of int values.

    @Suraj-Thorat said in Pandas Dataframe issue (int object has no attribute to_pydatetime):

    • EVEN IF I add this line to set index same error is given.

    Basic engineering principles:

    • You say you have done something
    • But you show code with something else
    • That means that for all intent and purposes you haven't done that.

    The PandasData has a dedicated documentation page, which shows the parameters available for the data feed, and the following is stated

    Link: Docs - PandasData - https://www.backtrader.com/docu/pandas-datafeed/pandas-datafeed/

    ...
        params = (
            # Possible values for datetime (must always be present)
            #  None : datetime is the "index" in the Pandas Dataframe
            #  -1 : autodetect position or case-wise equal name
            #  >= 0 : numeric index to the colum in the pandas dataframe
            #  string : column name (as index) in the pandas dataframe
            ('datetime', None),
    ...
    

Log in to reply
 

});