Help! Strange dates in resampled data!

Ed Bartosh

Hi,

Please, help!

I'm getting strange list of dates for the resampled data in Daily time frame.
Here is testing code:

#!/usr/bin/env python

import datetime

import backtrader as bt
import pandas as pd

class TestStrategy(bt.Strategy):

    def notify_data(self, data, status, *args, **kwargs):
        print '*' * 5, 'DATA NOTIF:', data._getstatusname(status)

    def notify_store(self, msg, *args, **kwargs):
        print '*' * 5, 'STORE NOTIF:', msg

    def next(self):
        print '*' * 5, 'NEXT:', self.data1._name, bt.TimeFrame.getname(self.data1._timeframe), len(self.data1), \
              [bt.utils.date.num2date(date).strftime("%Y-%m-%d") for date in self.data1.datetime.get(size=len(self.data1))]

def runstrategy():

    # Create a cerebro
    cerebro = bt.Cerebro()

    store = bt.stores.IBStore(port=7497)
    cerebro.setbroker(store.getbroker())

    # Add data feeds
    ticker = 'AAPL-STK-SMART-USD'
    
    data_min = store.getdata(dataname=ticker, name="%s_min" % ticker, tz='US/Eastern')
    cerebro.resampledata(dataname=data_min, timeframe=bt.TimeFrame.Minutes, compression=1)
    
    hist_start_date = pd.bdate_range(end=datetime.datetime.now(), periods=30)[0].to_pydatetime()                                                      
    data_day = store.getdata(dataname=ticker, name="%s_day" % ticker, backfill=True,                                                                                      
                             backfill_start=True, fromdate=hist_start_date, tz='US/Eastern')
    cerebro.resampledata(dataname=data_day, timeframe=bt.TimeFrame.Days, compression=1)
 
    # Add the strategy
    cerebro.addstrategy(TestStrategy)

    # Live data ... avoid long data accumulation by switching to "exactbars"
    cerebro.run()

if __name__ == '__main__':
    runstrategy()

and the output:

$ ./ibtest-2tfs.py
Server Version: 76
TWS Time at connection:20170928 19:21:06 UTC
***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm connection is OK:usfarm>
***** STORE NOTIF: <error id=-1, errorCode=2106, errorMsg=HMDS data farm connection is OK:ilhmds>
***** STORE NOTIF: <error id=-1, errorCode=2106, errorMsg=HMDS data farm connection is OK:euhmds>
***** STORE NOTIF: <error id=-1, errorCode=2106, errorMsg=HMDS data farm connection is OK:fundfarm>
***** STORE NOTIF: <error id=-1, errorCode=2106, errorMsg=HMDS data farm connection is OK:ushmds>
***** DATA NOTIF: DELAYED
***** DATA NOTIF: DELAYED
***** DATA NOTIF: LIVE
***** NEXT: AAPL-STK-SMART-USD_day Day 14 ['2017-08-20', '2017-08-24', '2017-08-26', '2017-08-30', '2017-09-01', '2017-09-03', '2017-09-08', '2017-09-10', '2017-09-14', '2017-09-16', '2017-09-20', '2017-09-22', '2017-09-24', '2017-09-28']
***** NEXT: AAPL-STK-SMART-USD_day Day 14 ['2017-08-20', '2017-08-24', '2017-08-26', '2017-08-30', '2017-09-01', '2017-09-03', '2017-09-08', '2017-09-10', '2017-09-14', '2017-09-16', '2017-09-20', '2017-09-22', '2017-09-24', '2017-09-28']
***** NEXT: AAPL-STK-SMART-USD_day Day 14 ['2017-08-20', '2017-08-24', '2017-08-26', '2017-08-30', '2017-09-01', '2017-09-03', '2017-09-08', '2017-09-10', '2017-09-14', '2017-09-16', '2017-09-20', '2017-09-22', '2017-09-24', '2017-09-28']
...

The length of data is only 14 days instead of expected 30, a lot of dates are missing and some dates are not business days at all.

Regards,
Ed

backtrader

@Ed-Bartosh said in Help! Strange dates in resampled data!:

data_day = store.getdata(dataname=ticker, name="%s_day" % ticker, backfill=True,                                                                                      
                         backfill_start=True, fromdate=hist_start_date, tz='US/Eastern')
cerebro.resampledata(dataname=data_day, timeframe=bt.TimeFrame.Days, compression=1)

Don't resample what's already daily.

Ed Bartosh

@backtrader it worked! thank you very much!

Just out of curiosity, how first call of resampledata works? It asks to resample daily data into minute timeframe, right?

backtrader

@Ed-Bartosh said in Help! Strange dates in resampled data!:

Just out of curiosity, how first call of resampledata works? It asks to resample daily data into minute timeframe, right?

That would be a miracle. Working with Interactive Brokers data feeds is troublesome (even if it happens to be the most popular broker it also hast the most challenging - i.e.: broken - API)

The best path for real time is to ask for Ticks and resample that to the target timeframe. The historical download will be done in what the resample filter wants.

Have a look at the ibtest sample in the sources.

Richard O'Regan

Hi Backtrader, I've had all sorts of issues with IB with my manual trading. I was going to use them for running a strategy, that makes 1 or 2 trades a day on an ETF using Backtrader. Before I invest time into this, would you recommend I use another broker instead? Thanks