Arbitrage Strategy Test with Minute Data

vivalavida1982

I am new to python and backtrader. I want to test an arbitrage strategy with two different instruments. I need to calculate the price spread and then decide at which spread level I need to put my order on these two instruments. Here are the codes.

In class Arbitrage()
def init(self):

    self.dataclose = self.datas[0].close

    self.order = None
    self.buyprice = None
    self.buycomm = None


def notify_order(self, order):

    if order.status in [bt.Order.Submitted, bt.Order.Accepted]:
        return

    if order.status == order.Completed:
        if order.isbuy():
            self.log(
                'BUY EXECUTED, Price: %.2f, Cost: %.2f, Comm %.2f' %
                (order.executed.price,
                 order.executed.value,
                 order.executed.comm))

            self.buyprice = order.executed.price
            self.buycomm = order.executed.comm

        else:  # Sell
            self.log('SELL EXECUTED, Price: %.2f, Cost: %.2f, Comm %.2f' %
                     (order.executed.price,
                      order.executed.value,
                      order.executed.comm))

    elif order.status in [order.Canceled, order.Margin, order.Rejected]:
        self.log('Order Canceled/Margin/Rejected')

    self.order = None

def notify_trade(self,trade):
    if not trade.isclosed:
        return

def next(self):

    #self.log('Close, %.2f' % self.dataclose[0])

    # check if order is pending, if yes, we cannot send 2nd order
    if self.order:
        return

    # check if we are in market
    if not self.position: # not yet in the market

        for i in self.data1.close:
            for j in self.data2.close:
                diff = data2.close - data1.close

                if diff > 3.2:
                    self.log('Buy T1706, Sell TF1706')
                    self.buy(self.data1, size=1)
                    self.sell(self.data2,size=2)

                if diff < 2.95:
                    self.log('Sell T1706, Buy TF1706')
                    self.buy(self.data2, size=2)
                    self.sell(self.data1,size=1)

In strategy conduct:
if name == 'main':

# create a cerebro
cerebro = bt.Cerebro()


# create the 1st data
dataframe1 = pd.read_csv('C:/Users/Vicky/Desktop/pythontest/T1706.csv', index_col=0)
data1 = btfeeds.YahooFinanceCSVData(
    dataname = dataframe1,
    fromdate=datetime.datetime(2017, 3, 1),
    todate=datetime.datetime(2017, 3, 2)
)
cerebro.adddata(data1)

# create the 2nd data
dataframe2 = pd.read_csv('C:/Users/Vicky/Desktop/pythontest/TF1706.csv', index_col=0)
data2 = btfeeds.YahooFinanceCSVData(
    dataname=dataframe2,
    fromdate=datetime.datetime(2017, 3, 1),
    todate=datetime.datetime(2017, 3, 2)
)


cerebro.adddata(data2)

cerebro.broker.setcash(100000.0)

cerebro.broker.setcommission(commission=0.001)

print('Starting Portfolio Value: %.2f' % cerebro.broker.getvalue())

cerebro.run()

and the errors:
Traceback (most recent call last):
File "C:/Users/Vicky/Desktop/pythontest/Arbitrage.py", line 109, in <module>
todate=datetime.datetime(2017, 3, 2)
File "C:\Python\lib\site-packages\backtrader\metabase.py", line 89, in call
_obj, args, kwargs = cls.dopostinit(_obj, *args, **kwargs)
File "C:\Python\lib\site-packages\backtrader\feed.py", line 632, in dopostinit
_obj._name, _ = os.path.splitext(os.path.basename(_obj.p.dataname))
File "C:\Python\lib\ntpath.py", line 235, in basename
return split(p)[1]
File "C:\Python\lib\ntpath.py", line 204, in split
p = os.fspath(p)
TypeError: expected str, bytes or os.PathLike object, not DataFrame

I run with pycharm, python 3.6

backtrader

@vivalavida1982 said in Arbitrage Strategy Test with Minute Data:

I am new to python and backtrader

We have all been new to something. This is not a critic, rather advice from experience. The best way to put the new label behind is reading everything and mostly what one has before his/her eyes.

For example at the top of the page:

For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

@vivalavida1982 said in Arbitrage Strategy Test with Minute Data:

TypeError: expected str, bytes or os.PathLike object, not DataFrame

The error says that you are passing a pandas.DataFrame to something which is expecting a string (or similar)

dataframe1 = pd.read_csv('C:/Users/Vicky/Desktop/pythontest/T1706.csv', index_col=0)
data1 = btfeeds.YahooFinanceCSVData(

You create it a pandas.DataFrame and give it to YahooFinanceCSVData, which is obviously (given the name) not expecting a pandas.DataFrame. This is obviously not an example you may have read in the documentation and/or blogs posts or samples in the sources.

If you want to use pandas.DataFrame as your source you probably want to use bt.feeds.PandasData as the data feed. See for example:

In order to progress and before developing any logic for the strategy (which in this case involves also multiple data feeds), it would be advisable to have a small snippet that guarantees the data is loading and is loading as intended (for example by having a strategy whose only purpose is to print out the values of the data feeds)

vivalavida1982

I used Pandas.Data, and the list index is out of range, and then I tried YahooFinanceCSVData. I used the same file with another strategy, it worked perfectly.

the change of code:
# create the 1st data
dataframe1 = pd.read_csv('C:/Users/Vicky/Desktop/pythontest/T1706.csv', index_col=0)
data1 = btfeeds.PandasData(
dataname = dataframe1,
fromdate=datetime.datetime(2017, 3, 1),
todate=datetime.datetime(2017, 3, 2)
)
cerebro.adddata(data1)

# create the 2nd data
dataframe2 = pd.read_csv('C:/Users/Vicky/Desktop/pythontest/TF1706.csv', index_col=0)
data2 = btfeeds.PandasData(
    dataname=dataframe2,
    fromdate=datetime.datetime(2017, 3, 1),
    todate=datetime.datetime(2017, 3, 2)
)

Traceback (most recent call last):
File "C:/Users/Vicky/Desktop/pythontest/Arbitrage.py", line 105, in <module>
todate=datetime.datetime(2017, 3, 2)
File "C:\Python\lib\site-packages\backtrader\metabase.py", line 88, in call
_obj, args, kwargs = cls.doinit(_obj, *args, **kwargs)
File "C:\Python\lib\site-packages\backtrader\metabase.py", line 78, in doinit
_obj.init(*args, **kwargs)
File "C:\Python\lib\site-packages\backtrader\feeds\pandafeed.py", line 192, in init
self._colmapping[datafield] = colnames[i]
IndexError: list index out of range

0_1497421744636_6003bf6e-7d88-4047-85f0-7610c8c275ea-image.png
Date Open High Low Close Volume
3/1/2017 09:33 95.62 95.62 95.595 95.595 150.1
3/1/2017 09:34 95.595 95.61 95.585 95.61 278.19
3/1/2017 09:35 95.615 95.615 95.6 95.615 76.49
3/1/2017 09:36 95.615 95.625 95.61 95.62 71.71
3/1/2017 09:37 95.62 95.635 95.62 95.635 100.41
3/1/2017 09:38 95.635 95.645 95.625 95.645 218.05
3/1/2017 09:39 95.64 95.645 95.63 95.63 69.82
3/1/2017 09:40 95.635 95.645 95.63 95.64 102.33
3/1/2017 09:41 95.64 95.655 95.64 95.64 241.03
3/1/2017 09:42 95.64 95.65 95.64 95.64 79.39
3/1/2017 09:43 95.645 95.645 95.6 95.6 183.59
3/1/2017 09:44 95.6 95.625 95.6 95.62 135.77
3/1/2017 09:45 95.62 95.62 95.585 95.6 354.67

backtrader

For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

backtrader

@vivalavida1982 said in Arbitrage Strategy Test with Minute Data:

I used Pandas.Data, and the list index is out of range

@vivalavida1982 said in Arbitrage Strategy Test with Minute Data:

data2 = btfeeds.PandasData(
    dataname=dataframe2,
    fromdate=datetime.datetime(2017, 3, 1),
    todate=datetime.datetime(2017, 3, 2)
)

You are not telling PandasData which fields are there and where they are located. See:

vivalavida1982

Thanks for replying, but the problem is not the function, the problem is the format of date in the csv. I found that csv sometimes loses information while editing. It is better change or reedit the data under excel format and then save as csv format.

But i have another problem, the system doesn't read the 2nd series of data.

class Arbitrage(bt.Strategy):

def log(self, txt, dt=None):

    if self.p.printout:
        dt = dt or self.data.datetime[0]
        dt = bt.num2date(dt)
        print('%s, %s' % (dt.isoformat(), txt))

def __init__(self):

    self.dataclose = self.datas[0].close

    self.order = None
    self.buyprice = None
    self.buycomm = None


def notify_order(self, order):

    if order.status in [bt.Order.Submitted, bt.Order.Accepted]:
        return

    if order.status == order.Completed:
        if order.isbuy():
            self.log(
                'BUY EXECUTED, Price: %.2f, Cost: %.2f, Comm %.2f' %
                (order.executed.price,
                 order.executed.value,
                 order.executed.comm))

            self.buyprice = order.executed.price
            self.buycomm = order.executed.comm

        else:  # Sell
            self.log('SELL EXECUTED, Price: %.2f, Cost: %.2f, Comm %.2f' %
                     (order.executed.price,
                      order.executed.value,
                      order.executed.comm))

    elif order.status in [order.Canceled, order.Margin, order.Rejected]:
        self.log('Order Canceled/Margin/Rejected')

    self.order = None

def notify_trade(self,trade):
    if not trade.isclosed:
        return

def next(self):

    #self.log('Close, %.2f' % self.dataclose[0])

    # check if order is pending, if yes, we cannot send 2nd order
    if self.order:
        return

    # check if we are in market
    if not self.position: # not yet in the market

        for i in self.data1['Date'] and self.data2['Date']:

            diff = self.data2.close - self.data1.close

            if diff > 3.0:
                self.log('Buy T1706, Sell TF1706')
                self.buy(self.data1, size=1)
                self.sell(self.data2,size=2)

            if diff < 2.8:
                self.log('Sell T1706, Buy TF1706')
                self.buy(self.data2, size=2)
                self.sell(self.data1,size=1)

and the errors:

C:\Python\python.exe C:/Users/Vicky/Desktop/pythontest/Arbitrage.py
Starting Portfolio Value: 100000.00
Traceback (most recent call last):
File "C:/Users/Vicky/Desktop/pythontest/Arbitrage.py", line 132, in <module>
cerebro.run()
File "C:\Python\lib\site-packages\backtrader\cerebro.py", line 1045, in run
runstrat = self.runstrategies(iterstrat)
File "C:\Python\lib\site-packages\backtrader\cerebro.py", line 1202, in runstrategies
self._runonce(runstrats)
File "C:\Python\lib\site-packages\backtrader\cerebro.py", line 1597, in _runonce
strat._oncepost(dt0)
File "C:\Python\lib\site-packages\backtrader\strategy.py", line 288, in _oncepost
self.nextstart() # only called for the 1st value
File "C:\Python\lib\site-packages\backtrader\lineiterator.py", line 329, in nextstart
self.next()
File "C:/Users/Vicky/Desktop/pythontest/Arbitrage.py", line 72, in next
for i in self.data1['Date'] and self.data2['Date']:
File "C:\Python\lib\site-packages\backtrader\lineseries.py", line 435, in getitem
return self.lines[0][key]
File "C:\Python\lib\site-packages\backtrader\linebuffer.py", line 163, in getitem
return self.array[self.idx + ago]
TypeError: unsupported operand type(s) for +: 'int' and 'str'

Process finished with exit code 1

backtrader

A sincere recommendation again: read what you have before your eyes.

First:

For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

Or else your posts are unreadable.

Second:

@vivalavida1982 said in Arbitrage Strategy Test with Minute Data:

TypeError: unsupported operand type(s) for +: 'int' and 'str'

You are obviously passing an str where an int is expected.

Third.

@vivalavida1982 said in Arbitrage Strategy Test with Minute Data:

for i in self.data1['Date'] and self.data2['Date']:

This seems to be a pandas-like syntax expectation. backtrader is not pandas. The expectation of your statement there is really a mistery here. Data feeds (like other lines objects) support integer indexing. See Docs - Platform Concepts