Algotopian is starting. Let it be the future of backtrader. Check it out! https://www.algotopian.com
For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

Problem with tick time data from pandas Dataframe index



  • Hello,
    I have issues importing dataframes using pandas it seems.
    My best guess is that the datetime64[ns] index is not handled correctly in my code.

    Example of my dataframe:
    df.xs('ABCN').head()
    Out[10]: 
                         close  high    low   open  sharevolume type
    startdatetime                                                   
    2018-01-19 20:59:00  37.48  37.5  37.44  37.47        32445  MKT
    2018-01-19 20:58:00  37.48  37.5  37.47  37.48        13926  MKT
    2018-01-19 20:57:00  37.47  37.5  37.47  37.49         6716  MKT
    2018-01-19 20:56:00  37.49  37.5  37.45  37.48        15021  MKT
    2018-01-19 20:55:00  37.46  37.5  37.45  37.49         8297  MKT
    
    df.xs('ABCN').head().index
    Out[7]: 
    DatetimeIndex(['2018-01-19 20:59:00', '2018-01-19 20:58:00',
                   '2018-01-19 20:57:00', '2018-01-19 20:56:00',
                   '2018-01-19 20:55:00'],
                  dtype='datetime64[ns]', name='startdatetime', freq=None)
    
    
    How I import my data:
    df = pd.read_hdf('./data/marketdata.hdf', key='marketdata').xs('ABCN').head(1000)
        param = {
            'datetime': None,
            'open': 'open',
            'high': 'high',
            'low': 'low',
            'close': 'close',
            'volume': 'sharevolume',
            'openinterest': None
        }
    
        data = feed.PandasData(dataname=df, timeframe=bt.TimeFrame.Seconds, **param)
    
    

    On inspection I see that my datetime object lacks data.

    self.datas[0].datetime.date(0)
    Out[12]: datetime.date(2018, 1, 19)
    

    Any help is highly appreciated!


  • administrators

    @phophopho said in Problem with tick time data from pandas Dataframe index:

    On inspection I see that my datetime object lacks data.

    self.datas[0].datetime.date(0)
    Out[12]: datetime.date(2018, 1, 19)
    

    There must something missing here. You have read in a set of data points which have the following date: 2018-01-19. Now you query the date and you get 2018-01-19.

    Which data is lacking (missing)?



  • Dear backtrader,

    first of all thank you very much for this project!
    So my initial problem is that my orders do not get executied. I thought this might be duo to incorrect index parsing but it is something else.
    I've attached my demo code.
    Here only the first self.buy() is actually executed by the broker.

    import backtrader as bt
    import backtrader.feeds as feed
    import pandas as pd
    
    import numpy as np
    import sys
    
    class foo(bt.Strategy):
        def log(self, msg, dt=None):
            dt = dt or self.datas[0].datetime.datetime(0)
            print('{}, {}'.format(dt, msg))
    
        def __init__(self):
            self.order = None
            self.order = self.buy(size=10, exectype=bt.Order.Close)
            self.dataclose = self.datas[0].close
            print(self.dataclose)
    
        def notify_order(self, order):
            if order.status != order.Completed:
                return
    
            self.order = None
            self.log('ORDER SUCCESSFUL')
    
        def next(self):
            self.buy(size=5, exectype=bt.Order.Close)
    
    if __name__ == '__main__':
    
    
        df = pd.read_hdf('./data/marketdata.hdf', key='marketdata').xs('ABCN').head(1000)
        param = {
            'datetime': None,
            'open': 'open',
            'high': 'high',
            'low': 'low',
            'close': 'close',
            'volume': 'sharevolume',
            'openinterest': None
        }
    
        data = feed.PandasData(dataname=df, timeframe=bt.TimeFrame.Seconds, **param)
    
        cerebro = bt.Cerebro(stdstats=False)
        cerebro.addstrategy(foo)
    
        cerebro.adddata(data)
    
        cerebro.broker.set_cash(10000.0)
        cerebro.run()
        cerebro.plot(style='bar')
        print('portfolio fundvalue: {}'.format(cerebro.broker.get_fundvalue()))
        print('portfolio cash: {}'.format(cerebro.broker.get_cash()))
        sys.exit(1)
    

  • administrators

    Without logs and data it would seem you only have money for a single buy operation.



  • Okay here is a small example that should show that the buy orders are not executed somehow.
    Once again your help is highly appreciated.

    import backtrader as bt
    import backtrader.feeds as feed
    import pandas as pd
    
    import numpy as np
    import sys
    import os
    import io
    
    
    class foo(bt.Strategy):
        def log(self, msg, dt=None):
            dt = dt or self.datas[0].datetime.datetime(0)
            print('{}, {}'.format(dt, msg))
    
        def __init__(self):
            self.order = None
            self.order = self.buy(size=10, exectype=bt.Order.Close)
            self.dataclose = self.datas[0].close
            print(self.dataclose)
    
        def notify_order(self, order):
            if order.status != order.Completed:
                return
    
            self.order = None
            self.log('ORDER SUCCESSFUL')
    
        def next(self):
            self.buy(size=1, exectype=bt.Order.Close)
    
    
    if __name__ == '__main__':
        cnt = """startdatetime,close,high,low,open,sharevolume,type
        2018-01-19 20:59:00,2.52,2.54,2.52,2.53,6188,MKT
        2018-01-19 20:59:00,2.52,2.54,2.52,2.53,6188,MKT
        2018-01-19 20:58:00,2.53,2.53,2.52,2.53,9300,MKT
        2018-01-19 20:57:00,2.53,2.53,2.53,2.53,19600,MKT
        2018-01-19 20:56:00,2.53,2.53,2.52,2.53,9850,MKT
        2018-01-19 20:55:00,2.53,2.53,2.53,2.53,600,MKT"""
        fn = io.StringIO(cnt)
    
        df = pd.read_csv(fn, parse_dates=['startdatetime'], infer_datetime_format=True)
        # df = pd.read_hdf('./data/marketdata.hdf', key='marketdata').xs('ABCN').head(1000)
        param = {
            #'datetime': None,
            'datetime': 'startdatetime',
            'open': 'open',
            'high': 'high',
            'low': 'low',
            'close': 'close',
            'volume': 'sharevolume',
            'openinterest': None
        }
    
        data = feed.PandasData(dataname=df, timeframe=bt.TimeFrame.Seconds, **param)
    
        cerebro = bt.Cerebro(stdstats=False)
        cerebro.addstrategy(foo)
    
        cerebro.adddata(data)
    
        cerebro.broker.set_cash(10000.0)
        cerebro.run()
        cerebro.plot(style='bar')
        print('portfolio fundvalue: {}'.format(cerebro.broker.get_fundvalue()))
        print('portfolio cash: {}'.format(cerebro.broker.get_cash()))
        sys.exit(1)
    
    

  • administrators

    @phophopho said in Problem with tick time data from pandas Dataframe index:

        cnt = """startdatetime,close,high,low,open,sharevolume,type
        2018-01-19 20:59:00,2.52,2.54,2.52,2.53,6188,MKT
        2018-01-19 20:59:00,2.52,2.54,2.52,2.53,6188,MKT
        2018-01-19 20:58:00,2.53,2.53,2.52,2.53,9300,MKT
        2018-01-19 20:57:00,2.53,2.53,2.53,2.53,19600,MKT
        2018-01-19 20:56:00,2.53,2.53,2.52,2.53,9850,MKT
        2018-01-19 20:55:00,2.53,2.53,2.53,2.53,600,MKT"""
    

    Even if called backtrader, the platform cannot yet trade backwards.



  • This is slightly embarrassing. Thank you.


Log in to reply
 

Looks like your connection to Backtrader Community was lost, please wait while we try to reconnect.