How to: change currency in datafeed

Jacob

I just got started with Backtrader and I want to start by experimenting a bit with arbitrage concepts in the crypto world.

what i'm trying to achieve is to use backtrader to convert data (OHLC) from currency 'X' to 'Y' using historical price data.

using the following data (.csv):

daily OHLC in 'EUR'
daily OHLC in 'USD'
weekly 'EUR' to 'USD' (close only)

I can easily convert all the data files outside of backtrader but I want to try and do it from within using 'simple' backtrader arithmetic.

any ideas on how to make it happen?

blonc

@Jacob add another set of data that is daily data for forex for the conversion needed. then do arithmetic with the two datasets self.datas[0].close vs self.datas[1].close to give you the needed outputs.

Jacob

@blonc thanks for your answer.

I played around with your suggested solution and it works well from inside a strategy class.
From my understanding this way on every bar the conversion is taking place on the fly. but it seem a bit limited, as I cannot override the original data (I can only make the calculation)

what i wanted was to change the original data with the new currency before so i can pass the 'new data' to the cerebro instance.

this way I can easily apply indicators, plot...

blonc

@Jacob To better understand how a data feed works maybe read this first CSV Data Feed Development , then move onto this Extending a Datafeed , and everything you want to do should be possible.

Jacob

maybe using 'filters' can be a solution.
as like resampling I can change the data before it enter cerebro.
this would need to work with 2 datafeeds, to create the data with the new currency.
is this a good approach or I'm just bending it completely? (passing the second datafeed in *args or **kwargs).

or maybe loading both datafeed (.csv) first with Pandas, playing around with the data and add it to cerebro directly?

blonc

@jacob gonna be a lot easier to just write a simple csv data feed handler and a lot cleaner. most the code is there to do it, just change as needed.. import the two files, do the conversion and pass it along. this will be totally reusable too for any two files you need.

blonc

if you are looking for something quick and dirty this bit of code can do literally everything you want. just import the CSV files, do the math in a np.array and then pass it to _load as a iter . you can add params at the top to pass from the strategy side and there you go.

class MyCryptoCSVData(bt.CSVDataBase):
    params = (
   ('filepath1', None),
   ('filepath2', None),
   ('anything you want', None),
  )
def start(self):
    # read in the CSV values, do your math and put the list into a iter()
    pass

def stop(self):
    # close your read files      

def _loadline(self, linetokens):
    i = itertools.count(0)

    dttxt = linetokens[next(i)]
    # Format is YYYY-MM-DD
    y = int(dttxt[0:4])
    m = int(dttxt[5:7])
    d = int(dttxt[8:10])

    dt = datetime.datetime(y, m, d)
    dtnum = date2num(dt)

    self.lines.datetime[0] = dtnum
    self.lines.open[0] = float(linetokens[next(i)])
    self.lines.high[0] = float(linetokens[next(i)])
    self.lines.low[0] = float(linetokens[next(i)])
    self.lines.close[0] = float(linetokens[next(i)])
    self.lines.volume[0] = float(linetokens[next(i)])
    self.lines.openinterest[0] = float(linetokens[next(i)])

    return True

backtrader

@jacob said in How to: change currency in datafeed:

as I cannot override the original data (I can only make the calculation)

You can add a filter to the target data feed, and the filter may have a reference to the data which holds the conversion rate.

The values in the target data feed for the filter will be overridden with anything you decide to output.

See: Docs - Filters

Jacob

thanks @blonc and @backtrader for your responses.
I'm exploring both solutions to learn what works for me best.

I really like the idea of using a filter to manipulate the data.
but I not sure the solution I have in mind is doable (I want to use as much as backtrader flexibility)

so I have 2 sets of data feed:

OHLC of some crypto in X currency
OHLC of X to Y currency

I import the 2 csv files and apply the conversion filter to the first one (adding also resample to both just to make it interesting :))

now if I understand correctly after cerebro.run() the system will go bar by bar and apply all the filters one bar at a time. so I will have the corresponding bar for the crypto data but not to conversion rate data. meaning if I pass the conversion data as a param for the main data filter I would have to do some date matching on each bar to get the right rate? or is there a cleaver way to make the 2 data feed bars in sync?

Jacob

Wowwww my mind is completely blown away.
I did a little bit of debugging just to find out that the 2 data feeds are actually in sync!!!

and on top of it the last value is accessible via [0].
meaning it automatically points to the last conversion rate, just out of the box

I'm really loving this library :)

I did find in the process something that seem a bit buggy with the dates (working with timestamps).
but i need to double check and if so I will open a new post for it.

will also post a short code snippet once i'm done testing it.
maybe someone can find it handy

again thanks @backtrader and @blonc for the help :)

Jacob

after testing and debugging there is some limitation that should be noted:

for both data feed to be properly synced both need to invoke the filter mechanism, otherwise the conversion filter doesn't get any data form the currency rate data. to solve this I resampled the currency rate data feed (which is a filter).
first raw of main data get called prior to currency data so there is no rate reference. making the data not consistence to the conversion
trying to delete the raw to solve problem (2) just make it propagate to the next raw causing problem (2) recursively. so a workaround i used is to change all values to 0

in the main code:
data.addfilter(CurrencyConverterFilter, rate = data_rate)

filter:
class CurrencyConverterFilter():

def __init__(self, data, *args, **kwargs):
    self.currencyRate = None
    if 'rate' in kwargs:
        self.currencyRate = kwargs['rate']

def __call__(self, data, *args, **kwargs):
    bar = [data.lines[i][0] for i in range(data.size())]

    if len(self.currencyRate):            
        rate = self.currencyRate[0]

        # adjust data with new rate 
        bar[data.Open] = bar[data.Open] / rate
        bar[data.Close] = bar[data.Close] / rate          
        bar[data.High] = bar[data.High] / rate
        bar[data.Low] = bar[data.Low] / rate

        # update stream
        data.backwards(force=True)  # remove the copied bar from stream
        data._add2stack(bar)  

        # bar is ready to be proceesed 
        return False

    # bar couldn't be converted due to rate data havn't been initialized
    # can happen if data feed for currency:
    # 1. was not passed at all
    # 2. was pass for future date (no rate reference was given yet)
    # 3. wasn't initialize by the system (can happen in first iteration)
    #data.backwards()  # remove bar from data stack
    #return True  # tell outer data loop to fetch a new bar

    # unfortunately removing bars cause the sync between data feeds to fail so data need to be removed manualy
    bar[data.Open] = 0
    bar[data.Close] = 0         
    bar[data.High] = 0
    bar[data.Low] = 0
    bar[data.Volume] = 0

    # update stream
    data.backwards(force=True) 
    data._add2stack(bar)

    return False