Oanda OHLC reporting same value

[ThatBlokeDave]

I am having an issue resampling the data from Oanda. I can connect and stream prices no problem but my "Open, High, Low and Close prices seem to always be the same value or with a micro pip of each other even when my resample timeframe is set to minutes and compression set to 1.

Interestingly when the backfill appears to resample correctly.

Please see below: (Backfill data is before the DATA NOTIF: Live Line)

'''
O: 1.23646 H: 1.23676 L: 1.23641 C: 1.23673
Date: 2017-03-17 15:24:00
O: 1.23685 H: 1.2369 L: 1.2367 C: 1.23683
Date: 2017-03-17 15:26:00
O: 1.23672 H: 1.23673 L: 1.23642 C: 1.23671
Date: 2017-03-17 15:28:00
O: 1.23685 H: 1.2371 L: 1.2368 C: 1.2371
Date: 2017-03-17 15:30:00
O: 1.23689 H: 1.23693 L: 1.23653 C: 1.23664
Date: 2017-03-17 15:32:00
O: 1.23628 H: 1.23632 L: 1.23601 C: 1.23614
Date: 2017-03-17 15:34:00
O: 1.23642 H: 1.23642 L: 1.23633 C: 1.23633
***** DATA NOTIF: LIVE
Date: 2017-03-17 15:35:00
O: 1.23633 H: 1.23633 L: 1.23633 C: 1.23633
Date: 2017-03-17 15:36:00
O: 1.23642 H: 1.23642 L: 1.23642 C: 1.23642
'''

Here is how I am calling the data

    cerebro = bt.Cerebro()
    oandastore = bt.stores.OandaStore(token=apikey, account=acc, practice=True)
    cerebro.broker = oandastore.getbroker()
    data0 = oandastore.getdata(dataname="GBP_USD")
    cerebro.resampledata(data0, timeframe=bt.TimeFrame.Minutes, compression=1)
    cerebro.adddata(data0)

    cerebro.addstrategy(One_Indicator_One_Filter, **strat_params)
    cerebro.run()

I have also tried without resampling. But when I do that, I still tend to get ticks coming in. (or at least data more frequent then per minute)

data0 = oandastore.getdata(dataname="GBP_USD", timeframe=bt.TimeFrame.Minutes,
                            compression=1, backfill_start=False)

I noticed in this help post. There is an ouput example from the oandatest.py sample posted by @backtrader also shows the output following live having the same values for open, high, low and close.

BTW here is my next() method:

    def next(self):

        self.data_date = self.datas[0].datetime.date(0)
        self.data_time = self.datas[0].datetime.time(0)
        # Simply log the closing price of the series from the reference
        print('Date: {} {}'.format(self.data_date, self.data_time))
        print('O: {} H: {} L: {} C: {}'.format(
                self.data.open[0],
                self.data.high[0],
                self.data.low[0],
                self.data.close[0],
                ))

[backtrader]

@ThatBlokeDave said in Oanda OHLC reporting same value:

cerebro.resampledata(data0, timeframe=bt.TimeFrame.Minutes, compression=1)
cerebro.adddata(data0)

Is it right to assume that you think you have to use addata after resampledata?

Because you don't. resampledata creates and adds the resampled version to the sytem. Using adddata adds an extra unfiltered version in the system which delivers ticks.

[ThatBlokeDave]

@backtrader

Actually I was not sure. I now see it was wrong to assume :) I think I missed that in the docs.

So I guess the ticks are messing up my re-sampling and each minute I am receiving one tick as output instead of the full minute?

Ok, I will try removing it and test again when the markets re-open.

I appreciate the help!

[backtrader]

The other reasons being that trading is really thin or that the practice environment offers such values.

[ThatBlokeDave]

@backtrader

Ok - I have tried testing again this morning without the adding the data a second time incorrectly. I am still seeing issues reporting the same figures. I don't think the issue is due to thin trading or the practice environment as I am testing against GBP_USD and I have cross checked against Oanda's practice server.

Here is my output....

dave@dave-ThinkPad-T410:~/Dropbox/Documents/Development/Python/Python3/backtrader$ python3 Oanda-1-Indicator-1-Filter.py 
-- Contract Details:
{'displayName': 'GBP/USD', 'instrument': 'GBP_USD', 'pip': '0.0001', 'maxTradeUnits': 10000000}
Started
Account Cash = 1000.0028
Account Value = 1000.0028
***** DATA NOTIF: LIVE
Date: 2017-03-20 01:34:00
O: 1.23831 H: 1.23831 L: 1.2383 C: 1.2383
Date: 2017-03-20 01:35:00
O: 1.23823 H: 1.23823 L: 1.23823 C: 1.23823
Date: 2017-03-20 01:36:00
O: 1.23834 H: 1.23834 L: 1.23834 C: 1.23834
Date: 2017-03-20 01:37:00
O: 1.23835 H: 1.23835 L: 1.23835 C: 1.23835
Date: 2017-03-20 01:38:00
O: 1.23837 H: 1.23837 L: 1.23837 C: 1.23837
Date: 2017-03-20 01:39:00
O: 1.23851 H: 1.23852 L: 1.23851 C: 1.23852

Here is a chart from Oanda Practice server at the same time. (Note I am in Asia GMT+8, hence the difference in reported times between the log and Oanda practice)

Here is my full code... (minus my API key and and ACC number)

import argparse
import datetime
import logging
import backtrader as bt
from extensions.classes import myStratExTest
import extensions.misc as misc
import datetime

apikey = 'REMOVED'
acc = 'REMOVED'


# Create a Stratey
class One_Indicator_One_Filter(myStratExTest):

    def __init__(self):
        self.dataclose = self.datas[0].close


        #Get class name for logging:
        self.class_name = self.__class__.__name__

        self.ind1 = ind1(**ind1_params)
        #initialize the filter
        self.filt1 = filt1(self, **filt1_params)

        #Set the BE indicator
        self.be_indicator = False

        #Set the stop strategy to just opened
        self.set_stop_strategy('JOPN')

        #setup PNL log
        self.pnl_report = []



    #Provides any noticficaitons about the data.
    def notify_data(self, data, status, *args, **kwargs):
        print('*' * 5, 'DATA NOTIF:', data._getstatusname(status), *args)
        #if status == data.LIVE:
        #
        #    self.counttostop = self.p.stopafter
        #    self.datastatus = 1

    def notify_store(self, msg, *args, **kwargs):
        print('*' * 5, 'STORE NOTIF:', msg)


    def next(self):

        self.data_date = self.datas[0].datetime.date(0)
        self.data_time = self.datas[0].datetime.time(0)
        # Simply log the closing price of the series from the reference
        print('Date: {} {}'.format(self.data_date, self.data_time))
        print('O: {} H: {} L: {} C: {}'.format(self.data.open[0],
                                                self.data.high[0],
                                                self.data.low[0],
                                                self.data.close[0],
                                                ))

        if self.ind1 == 1:
            #filter_signal = filt1(self)
            filter_signal = self.filt1.filter(self)
            if filter_signal == True:
                #get the item then set it.
                x = getattr(self.ind1, ind1_line1)
                x.__setitem__(0, 0.25)
                #self.ind1.candle.__setitem__(0, 0.25)
            else: #buy
                if not self.position:
                    entryStrat(self,'LONG', **entry_params)

        elif self.ind1 == -1:
            #filter_signal = filt1(self)
            filter_signal = self.filt1.filter(self)
            if filter_signal == True:
                x = getattr(self.ind1, ind1_line1)
                x.__setitem__(0, -0.25)
            else:
                if not self.position:
                    entryStrat(self,'SHORT', **entry_params)

        if self.position:
            if self.p.break_even_asap == True and self.be_indicator == False:
                self.break_even()
            else:
                exitStrat(self,**exit_params)

    def start(self):
        if self.data0.contractdetails is not None:
            print('-- Contract Details:')
            print(self.data0.contractdetails)
        print('Started')
        acc_cash = cerebro.broker.getcash()
        acc_val = cerebro.broker.getvalue()
        print('Account Cash = {}'.format(acc_cash))
        print('Account Value = {}'.format(acc_val))



if __name__ == '__main__':

    #GENERAL CONFIGS
    config, sections = misc.get_config('Oanda-1-Indicator-1-Filter.config')

    #Get General Settings
    fromdate, todate, logfile, loglevel = misc.get_general_settings(config[sections[0]])

    #Get Script Specific Settings
    timeframe = misc.get_timeframe(config['Script Settings']['timeframe'])
    compression = int(config['Script Settings']['compression'])

    #               #
    #  INDICATOR 1  #
    #               #

    #Get Indicator 1 Params
    ind1_params = misc.get_parameter_dict(config.items('Indicator 1 Settings'))

    #Indicator 1 import
    exec(misc.import_indicator(config['Indicators']['Indicator 1 Type'].lower(),
                config['Indicators']['Indicator 1'],'1'))

    #Get Indicator 1 Params
    filt1_params = misc.get_parameter_dict(config.items('Filter 1 Settings'))

    #Get Inidcator 1 Line(s)
    ind1_line1 = config['Indicator 1 Lines']['line']

    #               #
    #   Filter 1    #
    #               #

    #Filter 1 import
    exec(misc.import_filter(config['Filters']['Filter 1'],'1'))


    #               #
    #  Entry Strat  #
    #               #

    #Entry Strategy import
    exec(misc.import_entry(config['Entry Strategy']['Entry']))

    #Get Entry Strat Params
    entry_params = misc.get_parameter_dict(config.items('Entry Strategy Settings'))


    #               #
    #  Exit Strat   #
    #               #

    #Entry Strategy import
    exec(misc.import_exit(config['Exit Strategy']['Exit']))

    #Get Entry Strat Params
    exit_params = misc.get_parameter_dict(config.items('Exit Strategy Settings'))


    #Get Strategy Params
    strat_params = misc.get_parameter_dict(config.items('Strategy Settings'))

    #Setup Logging
    logging.basicConfig(filename=logfile,
                        level=loglevel,
                        format='%(levelname)s:%(message)s')

 

    ###########################################################################
                                #OANDA SPECIFIC#
    ###########################################################################
    cerebro = bt.Cerebro()
    oandastore = bt.stores.OandaStore(token=apikey, account=acc, practice=True)
    cerebro.broker = oandastore.getbroker()

    data0 = oandastore.getdata(dataname="GBP_USD", timeframe=bt.TimeFrame.Ticks, compression=1, backfill_start=False)
    ###########################################################################
    ###########################################################################

    cerebro.resampledata(data0, timeframe=bt.TimeFrame.Minutes, compression=1)
    #cerebro.adddata(data0)

    cerebro.addstrategy(One_Indicator_One_Filter, **strat_params)
    cerebro.run()

Have I made any incorrect assumptions again?

[backtrader]

Your sample seems complex, contains exec statements. See the standard sample included in the samples, delivering data

with initial backfilling

$ ./oandatest.py --account X --token Y --data0 GBP_USD --resample --timeframe Minutes --compression 1
--------------------------------------------------
Strategy Created
--------------------------------------------------
-- Contract Details:
{u'pip': u'0.0001', u'instrument': u'GBP_USD', u'maxTradeUnits': 10000000, u'displayName': u'GBP/USD'}
Datetime, Open, High, Low, Close, Volume, OpenInterest, SMA
***** DATA NOTIF: DELAYED
Data0, 0001, 736408.051389, 2017-03-20T01:14:00.000000, 1.238845, 1.239035, 1.238845, 1.239035,     36, 0, nan
Data0, 0002, 736408.052083, 2017-03-20T01:15:00.000000, 1.239030, 1.239030, 1.238720, 1.238810,     90, 0, nan
Data0, 0003, 736408.052778, 2017-03-20T01:16:00.000000, 1.238825, 1.238840, 1.238825, 1.238840,      4, 0, nan
Data0, 0004, 736408.053472, 2017-03-20T01:17:00.000000, 1.238840, 1.238845, 1.238840, 1.238845,      4, 0, nan
Data0, 0005, 736408.054167, 2017-03-20T01:18:00.000000, 1.238770, 1.238785, 1.238720, 1.238720,     10, 0, 1.238850
...
Data0, 0499, 736408.436806, 2017-03-20T10:29:00.000000, 1.240720, 1.240850, 1.240700, 1.240850,     28, 0, 1.240746
Data0, 0500, 736408.437500, 2017-03-20T10:30:00.000000, 1.240780, 1.240780, 1.240505, 1.240505,     11, 0, 1.240717
***** DATA NOTIF: LIVE
Data0, 0501, 736408.438194, 2017-03-20T10:31:00.000000, 1.240410, 1.240450, 1.240410, 1.240450,      0, 0, 1.240659
Data0, 0502, 736408.438889, 2017-03-20T10:32:00.000000, 1.240450, 1.240550, 1.240440, 1.240490,      0, 0, 1.240604

without initial backfilling

$ ./oandatest.py --account X --token Y --data0 GBP_USD --resample --timeframe Minutes --compression 1 --no-backfill_start
--------------------------------------------------
Strategy Created
--------------------------------------------------
-- Contract Details:
{u'pip': u'0.0001', u'instrument': u'GBP_USD', u'maxTradeUnits': 10000000, u'displayName': u'GBP/USD'}
Datetime, Open, High, Low, Close, Volume, OpenInterest, SMA
***** DATA NOTIF: LIVE
Data0, 0001, 736408.434722, 2017-03-20T10:26:00.000000, 1.240440, 1.240750, 1.240430, 1.240560,      0, 0, nan
Data0, 0002, 736408.435417, 2017-03-20T10:27:00.000000, 1.240500, 1.240660, 1.240500, 1.240660,      0, 0, nan
Data0, 0003, 736408.436111, 2017-03-20T10:28:00.000000, 1.240650, 1.240690, 1.240640, 1.240680,      0, 0, nan

[ThatBlokeDave]

@backtrader
Thanks for the continued help so far. It is great that you spend so much time to help the community in addition to the good work done to develop the framework.

I tried comparing using the script and initially got the same results. I.e The same values were being output. Eventually, later in the day I managed to get it working but if I am honest, I am not sure how. One thing I did was sync my clock with an NTP server as my time was a little off. However I can't imagine how that would have helped though.

BTW - The exec statements in my script are there just to import custom indicators under the same name name (i.e ind1) so I can easily switch the indicator used in my config file without going into the code. There might be a better way of doing that but that is where experience comes into play. :)