Backtrader Community

samk

@backtrader

I am trying:

X = ret_1m_mat.join(ret_3m_mat).join(ret_6m_mat).join(ret_12m_mat).dropna()

However, getting error saying:

AttributeError: 'list' object has no attribute 'join'

Thanks for the help in advance as always..........

samk

Also, I want to set the name of this giant Matrix that is combined from these 4 lines of code to 'X'............

samk

@backtrader said in How can I construct matrix of returns for 4 stocks:

ret_1m_mat = [x[0] for x in self.rets_1m]

Hello there,

Thanks for the response. Much appreciated. Ok so for the second part, to create the individual matrices, I continued your logic and have:

 # combine 1 month, 3 month, 6 month, and 12 month returns into one matrix
        ret_1m_mat = [x[0] for x in self.rets_1m]
        ret_3m_mat = [x[0] for x in self.rets_3m]
        ret_6m_mat = [x[0] for x in self.rets_6m]
        ret_12m_mat = [x[0] for x in self.rets_12m]

My question is, how would I now go about in combining all of the 4 lines of code above into one giant matrix by joining them all together?

Thanks in advance for all the help as always!

Sam

samk

@Ed-Bartosh

I did that. This is what I get:

C:\Users\Sam>python -c "import ccxt; print(getattr(ccxt, 'gemini'))"
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: No module named ccxt

Keep in mind, this is after this was completed too:

C:\Users\Sam>pip install git+https://github.com/bartosh/backtrader.git@ccxt
Collecting git+https://github.com/bartosh/backtrader.git@ccxt
Cloning https://github.com/bartosh/backtrader.git (to ccxt) to c:\users\sam\appdata\local\temp\pip-merrrk-build
Installing collected packages: backtrader
Running setup.py install for backtrader ... done
Successfully installed backtrader-1.9.59.122

Which is very odd because I have already pip installed and imported ccxt.......

samk

@Ed-Bartosh

So I went ahead again and gave a crack at it trying to install using:

pip install git+https://github.com/bartosh/backtrader.git@ccxt

Here is what went into command line:

C:\Users\Sam>pip install git+https://github.com/bartosh/backtrader.git@ccxt
Collecting git+https://github.com/bartosh/backtrader.git@ccxt
Cloning https://github.com/bartosh/backtrader.git (to ccxt) to c:\users\sam\appdata\local\temp\pip-merrrk-build
Installing collected packages: backtrader
Running setup.py install for backtrader ... done
Successfully installed backtrader-1.9.59.122

Now, my question is, looks like it was cloned to the directory:

c:\users\sam\appdata\local\temp\pip-merrrk-build

However, I work my python environment in:

c:\users\sam\anaconda64best\

Am I having an issue Ed because I need to copy everything from the first directory to my Python working directory? Then it should work?

Thanks for the help.........

samk

@Ed-Bartosh

Thank you I found it. Instead of cloning github, which obviously caused me many issues, I have went ahead and saved the files from your github manually. These files being 'ccxtbroker.py' (which holds the class CCXTBroker) and 'ccxt.py' (which holds the class CCXT ).

I am receiving an error however saying:

***Traceback (most recent call last):
File "C:/Users/Sam/PycharmProjects/Test/.ipynb_checkpoints/Crypto Backtrader Algorithm Using CCXT Broker & Gemini.py", line 71, in <module>
sys.exit(runstrategy(sys.argv))
File "C:/Users/Sam/PycharmProjects/Test/.ipynb_checkpoints/Crypto Backtrader Algorithm Using CCXT Broker & Gemini.py", line 51, in runstrategy
broker = ccxtbroker.CCXTBroker(exchange='gemini', currency='USD', config=broker_config)
File "C:\Users\Sam\Anaconda64Best\lib\site-packages\backtrader\metabase.py", line 88, in call
_obj, args, kwargs = cls.doinit(_obj, *args, **kwargs)
File "C:\Users\Sam\Anaconda64Best\lib\site-packages\backtrader\metabase.py", line 78, in doinit
_obj.init(args, kwargs)
File "C:\Users\Sam\PycharmProjects\Test.ipynb_checkpoints\ccxtbroker.py", line 42, in init
self.exchange = getattr(ccxt, exchange)(config)
AttributeError: module 'ccxt' has no attribute 'gemini'

Curious on why this may be happening? My full code is pasted below for reference. Thanks Ed.

Code:

from __future__ import (absolute_import, division, print_function,
                        unicode_literals)

from datetime import datetime, timedelta
import os.path  # To manage paths
import sys  # To find out the script name (in argv[0])

import ccxtbroker
import ccxt
import backtrader as bt

# Import the backtrader platform
import backtrader as bt
import backtrader.feeds as btfeeds
import datetime  # For datetime objects
import argparse
import time
import backtrader.analyzers as btan
from backtrader import Analyzer
import math
import numpy as np
import datetime

class TestStrategy(bt.Strategy):
    def next(self):
        for data in self.datas:
            print('*' * 5, 'NEXT:', bt.num2date(data.datetime[0]), data._name, data.open[0], data.high[0],
                  data.low[0], data.close[0], data.volume[0],
                  bt.TimeFrame.getname(data._timeframe), len(data))
            if not self.getposition(data):
                order = self.buy(data, exectype=bt.Order.Limit, size=10, price=data.close[0])
            else:
                order = self.sell(data, exectype=bt.Order.Limit, size=10, price=data.close[0])

    def notify_order(self, order):
        print('*' * 5, "NOTIFY ORDER", order)

def runstrategy(argv):
    # Create a cerebro
    cerebro = bt.Cerebro()

    # Create broker
    broker_config = {'urls': {'api': 'https://api.sandbox.gemini.com'},
                     'apiKey': '<42o4kPgIwkYTyqKdrMD4>',
                     'secret': '<2hi9U3Md3ahMVeTDqfRPXzCebS38>',
                     'nonce': lambda: str(int(time.time() * 1000))
                    }
    broker = ccxtbroker.CCXTBroker(exchange='gemini', currency='USD', config=broker_config)
    cerebro.setbroker(broker)

    # Create data feeds
    data_ticks = ccxt.CCXT(exchange='gdax', symbol='BTC/USD', name="btc_usd_tick",
                               timeframe=bt.TimeFrame.Ticks, compression=1)
    cerebro.adddata(data_ticks)

    hist_start_date = datetime.utcnow() - timedelta(minutes=30)
    data_min = ccxt.CCXT(exchange="gdax", symbol="BTC/USD", name="btc_usd_min",
                             timeframe=bt.TimeFrame.Minutes, fromdate=hist_start_date)
    cerebro.adddata(data_min)

    # Add the strategy
    cerebro.addstrategy(TestStrategy)

    # Run the strategy
    cerebro.run()

if __name__ == '__main__':
    sys.exit(runstrategy(sys.argv))

samk

@Ed-Bartosh

I am on your page here:

https://github.com/bartosh/backtrader/tree/master/backtrader/brokers

And I do not see any file called ccxtbroker.py, in order to identify the CCXTBroker class? I only see IB, oanda, VC, and bboker.

Is there somewhere on your git page where I can find the ccxtbroker.py?

Thanks......

samk

@Søren-Pallesen @Ed-Bartosh

Guys,

So I followed Soren's method. Created a new environment, added git, then did pip install ccxt and pip install git+https://github.com/bartosh/backtrader.git@ccxt

Both loaded just fine.

However when I am running the script, still getting an error:

ModuleNotFoundError: No module named 'backtrader.brokers.ccxtbroker'

As if it is still not reading it in my Pycharm interface. Even though everything was said to have installed ok!

I am very confused here. the statement I am using in the file to import CCXT is:

from backtrader.brokers.ccxtbroker import CCXTBroker

Is this not correct maybe? Or another import line of code I am supposed to use?

Please let me know. I really want to get this going so I can help collaborate with you guys by applying some created ML and deep learning models I have used on equities, equity options, and futures on the crypto currencies specifically. Think there may be substantial value there...........

samk

@Ed-Bartosh

So I use conda to install git and it worked great. then I did pip install as you mentioned:

C:\Users\Sam>pip install git+https://github.com/bartosh/backtrader.git@ccxt
Collecting git+https://github.com/bartosh/backtrader.git@ccxt
Cloning https://github.com/bartosh/backtrader.git (to ccxt) to c:\users\sam\appdata\local\temp\pip-yljmoz-build
Installing collected packages: backtrader
Running setup.py install for backtrader ... done
Successfully installed backtrader-1.9.59.122

So looks like it was a success. However when I run your file I am getting same error as @soren:

C:\Users\Sam\Anaconda64Best\python.exe "C:/Users/Sam/PycharmProjects/Test/.ipynb_checkpoints/Backtrader Bitcoin Algorithm.py"
Traceback (most recent call last):
File "C:/Users/Sam/PycharmProjects/Test/.ipynb_checkpoints/Backtrader Bitcoin Algorithm.py", line 68, in <module>
sys.exit(runstrategy(sys.argv))
File "C:/Users/Sam/PycharmProjects/Test/.ipynb_checkpoints/Backtrader Bitcoin Algorithm.py", line 48, in runstrategy
broker = bt.brokers.CCXTBroker(exchange='gemini', currency='USD', config=broker_config)
AttributeError: module 'backtrader.brokers' has no attribute 'CCXTBroker'

Do we need to import something at the top? Because using:

import backtrader.brokers.ccxtbroker

seems to do nothing. Please let me know.......Thanks Ed..

samk

@Ed-Bartosh

I even tried to run

pip install git+https://github.com/bartosh/backtrader.git@ccxt

after changing to directory that I saved git in which is: C:\Users\Sam\Anaconda64Best

But still getting the same error.......

C:\Users\Sam\Anaconda64Best\Lib\site-packages\backtrader>cd C:\Users\Sam\Anaconda64Best

*************C:\Users\Sam\Anaconda64Best>pip install git+https://github.com/bartosh/backtrader.git@ccxt

Collecting git+https://github.com/bartosh/backtrader.git@ccxt
Cloning https://github.com/bartosh/backtrader.git (to ccxt) to c:\users\sam\appdata\local\temp\pip-al_ctv-build
Error [Error 2] The system cannot find the file specified while executing command git clone -q https://github.com/bartosh/backtrader.git c:\users\sam\appdata\local\temp\pip-al_ctv-build
Cannot find command 'git'*************

samk

@Ed-Bartosh said in Anyone use backtrader to do live trading on Bitcoin exchange?:

pip install git+https://github.com/bartosh/backtrader.git@ccxt

I installed git through your link just fine and installed it in: C:\Users\Sam\Anaconda64Best

I already have backtrader installed on my windows pc and it is saved in:

C:\Users\Sam\Anaconda64Best\Lib\site-packages\backtrader

Thus, I went and tried:

C:\Users\Sam>cd C:\Users\Sam\Anaconda64Best\Lib\site-packages\backtrader (to change directory to backtrader first).
Then I tried: pip install git+https://github.com/bartosh/backtrader.git@ccxt

But I am getting the error:

Collecting git+https://github.com/bartosh/backtrader.git@ccxt
Cloning https://github.com/bartosh/backtrader.git (to ccxt) to c:\users\sam\appdata\local\temp\pip-kh91l0-build
Error [Error 2] The system cannot find the file specified while executing command git clone -q https://github.com/bartosh/backtrader.git c:\users\sam\appdata\local\temp\pip-kh91l0-build
Cannot find command 'git'

Any idea what I am doing wrong here? Thanks in advance Ed......

samk

@Søren-Pallesen @Ed-Bartosh I am getting the same error.

samk

@Søren-Pallesen @Ed-Bartosh

Hi Guys,

Think I made some progress, yet still stuck here. As mentioned, my python directory with all created modules and files is:

C:\Users\Sam\Anaconda64Best\Scripts\

Thus, for part 1, I went ahead and tried:

C:\Users\Sam\Anaconda64Best\Scripts\conda.exe -forge git clone https://github.com/bartosh/backtrader.git

And it seemed to work fine as I got no errors. I then attempted to change directory to where backtrader is, which was the command:

cd C:\Users\Sam\Anaconda64Best\Lib\site-packages\backtrader

That worked fine too. But the last part is still not working. I am attempting to run command:

C:\Users\Sam\Anaconda64Best\Lib\site-packages\backtrader> git checkout -b ccxt origin/ccxt

However, still getting error that says: 'git' is not recognized as an internal or external command,
operable program or batch file.

Any idea what is going wrong here? Appreciate the help.

samk

@Søren-Pallesen @Ed-Bartosh

Hello,

Thanks very much for the share on this. I am trying to get this going (ccxt integrated with backtrader that is), and having a hard time. I have been using backtrader for a while to run some ML algos, but new to gemini and ccxt. I have successfully installed ccxt using pip install. But now stuck on part 2, which is to 'Clone backtrader with ccxt support and reset source tree to ccxt branch'. I am not sure how to do this. I know you guys say:

git clone https://github.com/bartosh/backtrader.git
cd backtrader
git checkout -b ccxt origin/ccxt

However, I am not really sure how to do the 3 lines mentioned above. Exactly, what do I need to type into my command line to do the 3 steps above? My working directory path for python is:

C:\Users\Sam\Anaconda64Best\Scripts\

However, when I type into the command line:

C:\Users\Sam>C:\Users\Sam\Anaconda64Best\Scripts\conda.exe install git clone https://github.com/bartosh/backtrader.git

I get the following error:

PackageNotFoundError: Packages missing in current channels:

clone

Curious on what I am doing wrong here? Is it a syntax issue with how I am trying to run git clone? And need to change the way I am typing it out?

Please let me know what you think when you guys have a chance. I very much appreciate the assistance and hope to speak soon.

Best,

Sam

samk

Hello,

Have a question and hoping someone can help. Basically, I am taking the file with headings and values that looks like this:

What I have done in Quantopian before is take this spreadsheet of asset values and create 4 different matrix of returns, being 1 month returns, 3 month returns, 6 month returns, and 12 month returns. Then after creating these different matrices, combining them all into one final matrix. I have saved the file in my code as 'spy_ief_uup_gld'.

The code I used in quantopian was similar to:

 #Use returns history of the 4 tickers with different look back period as factor, and combine into 1 matrix
        self.returns_mat_1m = self.spy_ief_uup_gld[1:4].PercentChange(20)
        self.returns_mat_1m.columns = ['SPY_1m', 'IEF_1m', 'UUP_1m', 'GLD_1m']

        self.returns_mat_3m = self.spy_ief_uup_gld[1:4].PercentChange(60)
        self.returns_mat_3m.columns = ['SPY_3m', 'IEF_3m', 'UUP_3m', 'GLD_3m']

        self.returns_mat_6m = self.spy_ief_uup_gld[1:4].PercentChange(120)
        self.returns_mat_6m.columns = ['SPY_6m', 'IEF_6m', 'UUP_6m', 'GLD_6m']

        self.returns_mat_12m = self.spy_ief_uup_gld[1:4].PercentChange(240)
        self.returns_mat_12m.columns = ['SPY_12m', 'IEF_12m', 'UUP_12m', 'GLD_12m']

        # combine 1 month, 3 month, 6 month, and 12 month returns into one matrix
        X = self.returns_mat_1m.join(self.returns_mat_3m).join(self.returns_mat_6m).join(self.returns_mat_12m)
        # X = X.dropna(axis=1)

Howver, having a hard time changing the code synax above to suit Backtrader syntax. I was wondering if anyone had any idea on how to construct this matrix with a similar code format to fit Backtrader syntax? Appreciate the guidance and help as always and thanks in advance.

Sam

samk

@mpskowron

Thanks a lot. I was not aware of this option at all! It definitely helped me. Looks like I was not allocating the shares equally amongst the two stocks (calculation was off). Anyhow, what you mentioned helped me see what was going on a lot and will definitely be using it for more algo. Thanks!

Sam

samk

@ab_trader

Sorry sent the wrong revised code. The correct one with print statements shown below.....

from __future__ import (absolute_import, division, print_function,
                        unicode_literals)

import argparse
import datetime

# The above could be sent to an independent module
import backtrader as bt
import backtrader.feeds as btfeeds
import backtrader.indicators as btind
import backtrader.analyzers as btan


class PairTradingStrategy(bt.Strategy):
    params = dict(
        period=10,
        stake=10,
        qty1=0,
        qty2=0,
        printout=True,
        upper=2.1,
        lower=-2.1,
        up_medium=0.5,
        low_medium=-0.5,
        status=0,
        portfolio_value=100000,
    )

    def log(self, txt, dt=None):
        if self.p.printout:
            dt = dt or self.data.datetime[0] # set date time
            dt = bt.num2date(dt)
            print('%s, %s' % (dt.isoformat(), txt))

    def notify_order(self, order):
        if order.status in [bt.Order.Submitted, bt.Order.Accepted]:
            return  # Await further notifications

        if order.status == order.Completed:
            if order.isbuy():
                buytxt = 'BUY COMPLETE, %.2f' % order.executed.price
                self.log(buytxt, order.executed.dt)
            else:
                selltxt = 'SELL COMPLETE, %.2f' % order.executed.price
                self.log(selltxt, order.executed.dt)

        elif order.status in [order.Expired, order.Canceled, order.Margin]:
            self.log('%s ,' % order.Status[order.status])
            pass  # Simply log

        # Allow new orders
        self.orderid = None

    def __init__(self):
        # To control operation entries
        self.orderid = None
        self.qty1 = self.p.qty1
        self.qty2 = self.p.qty2
        self.upper_limit = self.p.upper
        self.lower_limit = self.p.lower
        self.up_medium = self.p.up_medium
        self.low_medium = self.p.low_medium
        self.status = self.p.status
        self.portfolio_value = self.p.portfolio_value

        # Signals performed with PD.OLS :                                   # signal performed using ordinary least regression
        self.transform = btind.OLS_TransformationN(self.data0, self.data1,
                                                   period=self.p.period)
        self.zscore = self.transform.zscore         # calculate z score ( - mean / std deviation)

        # Checking signals built with StatsModel.API :
        # self.ols_transfo = btind.OLS_Transformation(self.data0, self.data1,
        #                                             period=self.p.period,
        #                                             plot=True)

    def next(self):

        if self.orderid:
            return  # if an order is active, no new orders are allowed

        if self.p.printout:
            print('Self  len:', len(self))
            print('Data0 len:', len(self.data0)) # length of time series for data 0
            print('Data1 len:', len(self.data1)) # length of time series for data 1
            print('Data0 len == Data1 len:',
                  len(self.data0) == len(self.data1)) # set lengths equal to each other (ensure they are)

            print('Data0 dt:', self.data0.datetime.datetime())
            print('Data1 dt:', self.data1.datetime.datetime())

        print('status is', self.status)
        print('zscore is', self.zscore[0])

        # Step 2: Check conditions for SHORT the spread & place the order
        # Checking the condition for SHORT
        if (self.zscore[0] > self.upper_limit) and (self.status != 1):  # get short if z score is above upper limit and not short the spread already (!=1)

            # Calculating the number of shares for each stock
            value = 0.5 * self.portfolio_value  # Divide the cash equally
            x = int(value / (self.data0.close))  # Find the number of shares for Stock1
            y = int(value / (self.data1.close))  # Find the number of shares for Stock2
            print('x + self.qty1 is', x + self.qty1)
            print('y + self.qty2 is', y + self.qty2)

            # Placing the order
            self.log('SELL CREATE %s, price = %.2f, qty = %d' % ("DG", self.data0.close[0], x + self.qty1))    # sell DG and assign to x
            self.sell(data=self.data0, size=(x + self.qty1))  # Place an order for buying y + qty2 shares
            self.log('BUY CREATE %s, price = %.2f, qty = %d' % ("DLTR", self.data1.close[0], y + self.qty2))    # buy DLTR and assign to y
            self.buy(data=self.data1, size=(y + self.qty2))  # Place an order for selling x + qty1 shares (DG)

            # Updating the counters with new value
            self.qty1 = x  # The new open position quantity for Stock1 is x shares
            self.qty2 = y  # The new open position quantity for Stock2 is y shares

            self.status = 1  # The current status is "short the spread"

            # Step 3: Check conditions for LONG the spread & place the order
            # Checking the condition for LONG
        elif (self.zscore[0] < self.lower_limit) and (self.status != 2): # get long if z score is below lower limit and not long the spread already (!=2)

            # Calculating the number of shares for each stock
            value = 0.5 * self.portfolio_value  # Divide the cash equally
            x = int(value / (self.data0.close))  # Find the number of shares for Stock1
            y = int(value / (self.data1.close))  # Find the number of shares for Stock2
            print('x + self.qty1 is', x + self.qty1)
            print('y + self.qty2 is', y + self.qty2)

            # Place the order
            self.log('BUY CREATE %s, price = %.2f, qty = %d' % ("DG", self.data0.close[0], x + self.qty1)) # buy DG and assigned to x
            self.buy(data=self.data0, size=(x + self.qty1))  # Place an order for buying x + qty1 shares
            self.log('SELL CREATE %s, price = %.2f, qty = %d' % ("DLTR", self.data1.close[0], y + self.qty2)) # sell DLTR and assign to y
            self.sell(data=self.data1, size=(y + self.qty2))  # Place an order for selling y + qty2

            # Updating the counters with new value
            self.qty1 = x  # The new open position quantity for Stock1 is x shares
            self.qty2 = y  # The new open position quantity for Stock2 is y shares
            self.status = 2  # The current status is "long the spread"


            # Step 4: Check conditions for No Trade
            # If the z-score is within the two bounds, close all

        elif (self.zscore[0] < self.up_medium and self.zscore[0] > self.low_medium):
            self.log('CLOSE LONG %s, price = %.2f' % ("DG", self.data0.close[0]))
            self.close(self.data0)
            self.log('CLOSE SHORT %s, price = %.2f' % ("DLTR", self.data1.close[0]))
            self.close(self.data1)

        elif (self.zscore[0] < self.up_medium and self.zscore[0] > self.low_medium):
            self.log('CLOSE SHORT %s, price = %.2f' % ("DG", self.data0.close[0]))
            self.close(self.data0)
            self.log('CLOSE LONG %s, price = %.2f' % ("DLTR", self.data1.close[0]))
            self.close(self.data1)


    def stop(self):
        print('==================================================')
        print('Starting Value - %.2f' % self.broker.startingcash)
        print('Ending   Value - %.2f' % self.broker.getvalue())
        print('==================================================')


def runstrategy():
    args = parse_args()

    # Create a cerebro
    cerebro = bt.Cerebro()


    # Get the dates from the args
    fromdate = datetime.datetime.strptime("1/4/2010", "%m/%d/%Y")
    todate = datetime.datetime.strptime("10/17/2017", "%m/%d/%Y")

    # Create the 1st data
    data0 = btfeeds.GenericCSVData( # DG Data
        dataname='C:\\Users\\Sam\\PycharmProjects\\Test\\.ipynb_checkpoints\\DG.csv',

        fromdate=fromdate,
        todate=todate,

        nullvalue=0.0,  # missing values to be replaced with 0

        dtformat=('%m/%d/%Y'),

        datetime=0,
        time=-1,
        open=-1,
        high=-1,
        low=-1,
        close=4,
        adjclose=-1,
        volume=-1,
        openinterest=-1,

    )

    # Add the 1st data to cerebro
    cerebro.adddata(data0)

    # Create the 2nd data
    data1 = btfeeds.GenericCSVData(  # DLTR Data
        dataname='C:\\Users\\Sam\\PycharmProjects\\Test\\.ipynb_checkpoints\\DLTR.csv',

        fromdate=fromdate,
        todate=todate,

        nullvalue=0.0,  # missing values to be replaced with 0

        dtformat=('%m/%d/%Y'),

        datetime=0,
        time=-1,
        open=-1,
        high=-1,
        low=-1,
        close=4,
        adjclose=-1,
        volume=-1,
        openinterest=-1,

    )

    # Add the 2nd data to cerebro
    cerebro.adddata(data1)

    # Add the strategy
    cerebro.addstrategy(PairTradingStrategy,
                        period=args.period,
                        stake=args.stake)

    # Add the commission - only stocks like a for each operation
    cerebro.broker.setcash(args.cash)

    # Add the commission - only stocks like a for each operation
    cerebro.broker.setcommission(commission=args.commperc)

    # And run it
    cerebro.run(runonce=not args.runnext,
                preload=not args.nopreload,
                oldsync=args.oldsync)

    # Plot if requested
    if args.plot:
        cerebro.plot(numfigs=args.numfigs, volume=False, zdown=False)



def parse_args():
    parser = argparse.ArgumentParser(description='MultiData Strategy')

    parser.add_argument('--data0', '-d0',
                        default='C:\\Users\\Sam\\PycharmProjects\\Test\\.ipynb_checkpoints\\DG.csv',
                        help='1st data into the system')

    parser.add_argument('--data1', '-d1',
                        default='C:\\Users\\Sam\PycharmProjects\\Test\\.ipynb_checkpoints\\DLTR.csv',
                        help='2nd data into the system')

    parser.add_argument('--fromdate', '-f',
                        default='01/04/2010',
                        help='Starting date in %m/%d/%Y format')

    parser.add_argument('--todate', '-t',
                        default='10/17/2017',
                        help='Starting date in %m/%d/%Y format')

    parser.add_argument('--period', default=10, type=int,
                        help='Period to apply to the Simple Moving Average')

    parser.add_argument('--cash', default=100000, type=int,
                        help='Starting Cash')

    parser.add_argument('--runnext', action='store_true',
                        help='Use next by next instead of runonce')

    parser.add_argument('--nopreload', action='store_true',
                        help='Do not preload the data')

    parser.add_argument('--oldsync', action='store_true',
                        help='Use old data synchronization method')

    parser.add_argument('--commperc', default=0.005, type=float,
                        help='Percentage commission (0.005 is 0.5%%')

    parser.add_argument('--stake', default=10, type=int,
                        help='Stake to apply in each operation')

    parser.add_argument('--plot', '-p', default=True, action='store_true',
                        help='Plot the read data')

    parser.add_argument('--numfigs', '-n', default=1,
                        help='Plot using numfigs figures')

    return parser.parse_args()


if __name__ == '__main__':
    runstrategy()

samk

Hi @ab_trader ,

Thank you very much for the response. I have created multiple print statements as you suggested, however just am having a hard time seeing where it is flushing in negative balance territory, or why it is for that matter. Any help in terms of what I can be missing here would be greatly appreciated. I have pasted new adjusted code with print statements below. Thanks for the help. I am very new to backtrader still, so could jut be missing something very silly here....

from __future__ import (absolute_import, division, print_function,
                        unicode_literals)

import argparse
import datetime

# The above could be sent to an independent module
import backtrader as bt
import backtrader.feeds as btfeeds
import backtrader.indicators as btind
import backtrader.analyzers as btan


class PairTradingStrategy(bt.Strategy):
    params = dict(
        period=10,
        stake=10,
        qty1=0,
        qty2=0,
        printout=True,
        upper=2.1,
        lower=-2.1,
        up_medium=0.5,
        low_medium=-0.5,
        status=0,
        portfolio_value=100000,
    )

    def log(self, txt, dt=None):
        if self.p.printout:
            dt = dt or self.data.datetime[0] # set date time
            dt = bt.num2date(dt)
            print('%s, %s' % (dt.isoformat(), txt))

    def notify_order(self, order):
        if order.status in [bt.Order.Submitted, bt.Order.Accepted]:
            return  # Await further notifications

        if order.status == order.Completed:
            if order.isbuy():
                buytxt = 'BUY COMPLETE, %.2f' % order.executed.price
                self.log(buytxt, order.executed.dt)
            else:
                selltxt = 'SELL COMPLETE, %.2f' % order.executed.price
                self.log(selltxt, order.executed.dt)

        elif order.status in [order.Expired, order.Canceled, order.Margin]:
            self.log('%s ,' % order.Status[order.status])
            pass  # Simply log

        # Allow new orders
        self.orderid = None

    def __init__(self):
        # To control operation entries
        self.orderid = None
        self.qty1 = self.p.qty1
        self.qty2 = self.p.qty2
        self.upper_limit = self.p.upper
        self.lower_limit = self.p.lower
        self.up_medium = self.p.up_medium
        self.low_medium = self.p.low_medium
        self.status = self.p.status
        self.portfolio_value = self.p.portfolio_value

        # Signals performed with PD.OLS :                                   # signal performed using ordinary least regression
        self.transform = btind.OLS_TransformationN(self.data0, self.data1,
                                                   period=self.p.period)
        self.zscore = self.transform.zscore         # calculate z score ( - mean / std deviation)

        # Checking signals built with StatsModel.API :
        # self.ols_transfo = btind.OLS_Transformation(self.data0, self.data1,
        #                                             period=self.p.period,
        #                                             plot=True)

    def next(self):

        if self.orderid:
            return  # if an order is active, no new orders are allowed

        if self.p.printout:
            print('Self  len:', len(self))
            print('Data0 len:', len(self.data0)) # length of time series for data 0
            print('Data1 len:', len(self.data1)) # length of time series for data 1
            print('Data0 len == Data1 len:',
                  len(self.data0) == len(self.data1)) # set lengths equal to each other (ensure they are)

            print('Data0 dt:', self.data0.datetime.datetime())
            print('Data1 dt:', self.data1.datetime.datetime())

        print('status is', self.status)
        print('zscore is', self.zscore[0])

        # Step 2: Check conditions for SHORT the spread & place the order
        # Checking the condition for SHORT
        if (self.zscore[0] > self.upper_limit) and (self.status != 1):  # get short if z score is above upper limit and not short the spread already (!=1)

            # Calculating the number of shares for each stock
            value = 0.5 * self.portfolio_value  # Divide the cash equally
            x = int(value / (self.data0.close))  # Find the number of shares for Stock1
            y = int(value / (self.data1.close))  # Find the number of shares for Stock2
            print('x + self.qty1 is', x + self.qty1)
            print('y + self.qty2 is', y + self.qty2)

            # Placing the order
            self.log('SELL CREATE %s, price = %.2f, qty = %d' % ("DG", self.data0.close[0], x + self.qty1))    # sell DG and assign to x
            self.sell(data=self.data0, size=(x + self.qty1))  # Place an order for buying y + qty2 shares
            self.log('BUY CREATE %s, price = %.2f, qty = %d' % ("DLTR", self.data1.close[0], y + self.qty2))    # buy DLTR and assign to y
            self.buy(data=self.data1, size=(y + self.qty2))  # Place an order for selling x + qty1 shares (DG)

            # Updating the counters with new value
            self.qty1 = x  # The new open position quantity for Stock1 is x shares
            self.qty2 = y  # The new open position quantity for Stock2 is y shares

            self.status = 1  # The current status is "short the spread"

            # Step 3: Check conditions for LONG the spread & place the order
            # Checking the condition for LONG
        elif (self.zscore[0] < self.lower_limit) and (self.status != 2): # get long if z score is below lower limit and not long the spread already (!=2)

            # Calculating the number of shares for each stock
            value = 0.5 * self.portfolio_value  # Divide the cash equally
            x = int(value / (self.data0.close))  # Find the number of shares for Stock1
            y = int(value / (self.data1.close))  # Find the number of shares for Stock2
            print('x + self.qty1 is', x + self.qty1)
            print('y + self.qty2 is', y + self.qty2)

            # Place the order
            self.log('BUY CREATE %s, price = %.2f, qty = %d' % ("DG", self.data0.close[0], x + self.qty1)) # buy DG and assigned to x
            self.buy(data=self.data0, size=(x + self.qty1))  # Place an order for buying x + qty1 shares
            self.log('SELL CREATE %s, price = %.2f, qty = %d' % ("DLTR", self.data1.close[0], y + self.qty2)) # sell DLTR and assign to y
            self.sell(data=self.data1, size=(y + self.qty2))  # Place an order for selling y + qty2

            # Updating the counters with new value
            self.qty1 = x  # The new open position quantity for Stock1 is x shares
            self.qty2 = y  # The new open position quantity for Stock2 is y shares
            self.status = 2  # The current status is "long the spread"


            # Step 4: Check conditions for No Trade
            # If the z-score is within the two bounds, close all

        elif (self.zscore[0] < self.up_medium and self.zscore[0] > self.low_medium):
            self.log('CLOSE LONG %s, price = %.2f' % ("DG", self.data0.close[0]))
            self.close(self.data0)
            self.log('CLOSE LONG %s, price = %.2f' % ("DLTR", self.data1.close[0]))
            self.close(self.data1)


    def stop(self):
        print('==================================================')
        print('Starting Value - %.2f' % self.broker.startingcash)
        print('Ending   Value - %.2f' % self.broker.getvalue())
        print('==================================================')


def runstrategy():
    args = parse_args()

    # Create a cerebro
    cerebro = bt.Cerebro()


    # Get the dates from the args
    fromdate = datetime.datetime.strptime("1/4/2010", "%m/%d/%Y")
    todate = datetime.datetime.strptime("10/17/2017", "%m/%d/%Y")

    # Create the 1st data
    data0 = btfeeds.GenericCSVData( # DG Data
        dataname='C:\\Users\\Sam\\PycharmProjects\\Test\\.ipynb_checkpoints\\DG.csv',

        fromdate=fromdate,
        todate=todate,

        nullvalue=0.0,  # missing values to be replaced with 0

        dtformat=('%m/%d/%Y'),

        datetime=0,
        time=-1,
        open=-1,
        high=-1,
        low=-1,
        close=4,
        adjclose=-1,
        volume=-1,
        openinterest=-1,

    )

    # Add the 1st data to cerebro
    cerebro.adddata(data0)

    # Create the 2nd data
    data1 = btfeeds.GenericCSVData(  # DLTR Data
        dataname='C:\\Users\\Sam\\PycharmProjects\\Test\\.ipynb_checkpoints\\DLTR.csv',

        fromdate=fromdate,
        todate=todate,

        nullvalue=0.0,  # missing values to be replaced with 0

        dtformat=('%m/%d/%Y'),

        datetime=0,
        time=-1,
        open=-1,
        high=-1,
        low=-1,
        close=4,
        adjclose=-1,
        volume=-1,
        openinterest=-1,

    )

    # Add the 2nd data to cerebro
    cerebro.adddata(data1)

    # Add the strategy
    cerebro.addstrategy(PairTradingStrategy,
                        period=args.period,
                        stake=args.stake)

    # Add the commission - only stocks like a for each operation
    cerebro.broker.setcash(args.cash)

    # Add the commission - only stocks like a for each operation
    cerebro.broker.setcommission(commission=args.commperc)

    # And run it
    cerebro.run(runonce=not args.runnext,
                preload=not args.nopreload,
                oldsync=args.oldsync)

    # Plot if requested
    if args.plot:
        cerebro.plot(numfigs=args.numfigs, volume=False, zdown=False)



def parse_args():
    parser = argparse.ArgumentParser(description='MultiData Strategy')

    parser.add_argument('--data0', '-d0',
                        default='C:\\Users\\Sam\\PycharmProjects\\Test\\.ipynb_checkpoints\\DG.csv',
                        help='1st data into the system')

    parser.add_argument('--data1', '-d1',
                        default='C:\\Users\\Sam\PycharmProjects\\Test\\.ipynb_checkpoints\\DLTR.csv',
                        help='2nd data into the system')

    parser.add_argument('--fromdate', '-f',
                        default='01/04/2010',
                        help='Starting date in %m/%d/%Y format')

    parser.add_argument('--todate', '-t',
                        default='10/17/2017',
                        help='Starting date in %m/%d/%Y format')

    parser.add_argument('--period', default=10, type=int,
                        help='Period to apply to the Simple Moving Average')

    parser.add_argument('--cash', default=100000, type=int,
                        help='Starting Cash')

    parser.add_argument('--runnext', action='store_true',
                        help='Use next by next instead of runonce')

    parser.add_argument('--nopreload', action='store_true',
                        help='Do not preload the data')

    parser.add_argument('--oldsync', action='store_true',
                        help='Use old data synchronization method')

    parser.add_argument('--commperc', default=0.005, type=float,
                        help='Percentage commission (0.005 is 0.5%%')

    parser.add_argument('--stake', default=10, type=int,
                        help='Stake to apply in each operation')

    parser.add_argument('--plot', '-p', default=True, action='store_true',
                        help='Plot the read data')

    parser.add_argument('--numfigs', '-n', default=1,
                        help='Plot using numfigs figures')

    return parser.parse_args()


if __name__ == '__main__':
    runstrategy()

samk

@backtrader @administrators

Any idea what is going on here by any chance?

samk

Hello,

I am attempting to create a pair trading algorithm between DG and DLTR as tickers. While the algorithm is running, there seems to be some odd error I have not been able to figure out. While I start balance at 100,000, ending balance is - 3,513,102; which obviously does not make sense.

Wondering if anyone can please take a quick glance at my code and see where the problem could be lying here. Very much appreciate the assistance and thanks in advance. Full code below:

from __future__ import (absolute_import, division, print_function,
                        unicode_literals)

import argparse
import datetime

# The above could be sent to an independent module
import backtrader as bt
import backtrader.feeds as btfeeds
import backtrader.indicators as btind


class PairTradingStrategy(bt.Strategy):
    params = dict(
        period=10,
        stake=10,
        qty1=0,
        qty2=0,
        printout=True,
        upper=2.1,
        lower=-2.1,
        up_medium=0.5,
        low_medium=-0.5,
        status=0,
        portfolio_value=100000,
    )

    def log(self, txt, dt=None):
        if self.p.printout:
            dt = dt or self.data.datetime[0] # set date time
            dt = bt.num2date(dt)
            print('%s, %s' % (dt.isoformat(), txt))

    def notify_order(self, order):
        if order.status in [bt.Order.Submitted, bt.Order.Accepted]:
            return  # Await further notifications

        if order.status == order.Completed:
            if order.isbuy():
                buytxt = 'BUY COMPLETE, %.2f' % order.executed.price
                self.log(buytxt, order.executed.dt)
            else:
                selltxt = 'SELL COMPLETE, %.2f' % order.executed.price
                self.log(selltxt, order.executed.dt)

        elif order.status in [order.Expired, order.Canceled, order.Margin]:
            self.log('%s ,' % order.Status[order.status])
            pass  # Simply log

        # Allow new orders
        self.orderid = None

    def __init__(self):
        # To control operation entries
        self.orderid = None
        self.qty1 = self.p.qty1
        self.qty2 = self.p.qty2
        self.upper_limit = self.p.upper
        self.lower_limit = self.p.lower
        self.up_medium = self.p.up_medium
        self.low_medium = self.p.low_medium
        self.status = self.p.status
        self.portfolio_value = self.p.portfolio_value

        # Signals performed with PD.OLS :                                   # signal performed using ordinary least regression
        self.transform = btind.OLS_TransformationN(self.data0, self.data1,
                                                   period=self.p.period)
        self.zscore = self.transform.zscore         # calculate z score ( - mean / std deviation)

        # Checking signals built with StatsModel.API :
        # self.ols_transfo = btind.OLS_Transformation(self.data0, self.data1,
        #                                             period=self.p.period,
        #                                             plot=True)

    def next(self):

        if self.orderid:
            return  # if an order is active, no new orders are allowed

        if self.p.printout:
            print('Self  len:', len(self))
            print('Data0 len:', len(self.data0)) # length of time series for data 0
            print('Data1 len:', len(self.data1)) # length of time series for data 1
            print('Data0 len == Data1 len:',
                  len(self.data0) == len(self.data1)) # set lengths equal to each other (ensure they are)

            print('Data0 dt:', self.data0.datetime.datetime())
            print('Data1 dt:', self.data1.datetime.datetime())

        print('status is', self.status)
        print('zscore is', self.zscore[0])

        # Step 2: Check conditions for SHORT the spread & place the order
        # Checking the condition for SHORT
        if (self.zscore[0] > self.upper_limit) and (self.status != 1):  # get short if z score is above upper limit and not short the spread already (!=1)

            # Calculating the number of shares for each stock
            value = 0.5 * self.portfolio_value  # Divide the cash equally
            x = int(value / (self.data0.close))  # Find the number of shares for Stock1
            y = int(value / (self.data1.close))  # Find the number of shares for Stock2
            print('x + self.qty1 is', x + self.qty1)
            print('y + self.qty2 is', y + self.qty2)

            # Placing the order
            self.log('SELL CREATE %s, price = %.2f, qty = %d' % ("DG", self.data0.close[0], x + self.qty1))    # sell DG and assign to x
            self.sell(data=self.data0, size=(x + self.qty1))  # Place an order for buying y + qty2 shares
            self.log('BUY CREATE %s, price = %.2f, qty = %d' % ("DLTR", self.data1.close[0], y + self.qty2))    # buy DLTR and assign to y
            self.buy(data=self.data1, size=(y + self.qty2))  # Place an order for selling x + qty1 shares (DG)

            # Updating the counters with new value
            self.qty1 = x  # The new open position quantity for Stock1 is x shares
            self.qty2 = y  # The new open position quantity for Stock2 is y shares

            self.status = 1  # The current status is "short the spread"

            # Step 3: Check conditions for LONG the spread & place the order
            # Checking the condition for LONG
        elif (self.zscore[0] < self.lower_limit) and (self.status != 2): # get long if z score is below lower limit and not long the spread already (!=2)

            # Calculating the number of shares for each stock
            value = 0.5 * self.portfolio_value  # Divide the cash equally
            x = int(value / (self.data0.close))  # Find the number of shares for Stock1
            y = int(value / (self.data1.close))  # Find the number of shares for Stock2
            print('x + self.qty1 is', x + self.qty1)
            print('y + self.qty2 is', y + self.qty2)

            # Place the order
            self.log('BUY CREATE %s, price = %.2f, qty = %d' % ("DG", self.data0.close[0], x + self.qty1)) # buy DG and assigned to x
            self.buy(data=self.data0, size=(x + self.qty1))  # Place an order for buying x + qty1 shares
            self.log('SELL CREATE %s, price = %.2f, qty = %d' % ("DLTR", self.data1.close[0], y + self.qty2)) # sell DLTR and assign to y
            self.sell(data=self.data1, size=(y + self.qty2))  # Place an order for selling y + qty2

            # Updating the counters with new value
            self.qty1 = x  # The new open position quantity for Stock1 is x shares
            self.qty2 = y  # The new open position quantity for Stock2 is y shares
            self.status = 2  # The current status is "long the spread"


            # Step 4: Check conditions for No Trade
            # If the z-score is within the two bounds, close all

        elif (self.zscore[0] < self.up_medium and self.zscore[0] > self.low_medium):
            self.log('CLOSE LONG %s, price = %.2f' % ("DG", self.data0.close[0]))
            self.close(self.data0)
            self.log('CLOSE LONG %s, price = %.2f' % ("DLTR", self.data1.close[0]))
            self.close(self.data1)


    def stop(self):
        print('==================================================')
        print('Starting Value - %.2f' % self.broker.startingcash)
        print('Ending   Value - %.2f' % self.broker.getvalue())
        print('==================================================')


def runstrategy():
    args = parse_args()

    # Create a cerebro
    cerebro = bt.Cerebro()


    # Get the dates from the args
    fromdate = datetime.datetime.strptime("1/4/2010", "%m/%d/%Y")
    todate = datetime.datetime.strptime("10/17/2017", "%m/%d/%Y")

    # Create the 1st data
    data0 = btfeeds.GenericCSVData( # DG Data
        dataname='C:\\Users\\Sam\\PycharmProjects\\Test\\.ipynb_checkpoints\\DG.csv',

        fromdate=fromdate,
        todate=todate,

        nullvalue=0.0,  # missing values to be replaced with 0

        dtformat=('%m/%d/%Y'),

        datetime=0,
        time=-1,
        open=-1,
        high=-1,
        low=-1,
        close=4,
        adjclose=-1,
        volume=-1,
        openinterest=-1,

    )

    # Add the 1st data to cerebro
    cerebro.adddata(data0)

    # Create the 2nd data
    data1 = btfeeds.GenericCSVData(  # DLTR Data
        dataname='C:\\Users\\Sam\\PycharmProjects\\Test\\.ipynb_checkpoints\\DLTR.csv',

        fromdate=fromdate,
        todate=todate,

        nullvalue=0.0,  # missing values to be replaced with 0

        dtformat=('%m/%d/%Y'),

        datetime=0,
        time=-1,
        open=-1,
        high=-1,
        low=-1,
        close=4,
        adjclose=-1,
        volume=-1,
        openinterest=-1,

    )

    # Add the 2nd data to cerebro
    cerebro.adddata(data1)

    # Add the strategy
    cerebro.addstrategy(PairTradingStrategy,
                        period=args.period,
                        stake=args.stake)

    # Add the commission - only stocks like a for each operation
    cerebro.broker.setcash(args.cash)

    # Add the commission - only stocks like a for each operation
    cerebro.broker.setcommission(commission=args.commperc)

    # And run it
    cerebro.run(runonce=not args.runnext,
                preload=not args.nopreload,
                oldsync=args.oldsync)

    # Plot if requested
    if args.plot:
        cerebro.plot(numfigs=args.numfigs, volume=False, zdown=False)



def parse_args():
    parser = argparse.ArgumentParser(description='MultiData Strategy')

    parser.add_argument('--data0', '-d0',
                        default='C:\\Users\\Sam\\PycharmProjects\\Test\\.ipynb_checkpoints\\DG.csv',
                        help='1st data into the system')

    parser.add_argument('--data1', '-d1',
                        default='C:\\Users\\Sam\PycharmProjects\\Test\\.ipynb_checkpoints\\DLTR.csv',
                        help='2nd data into the system')

    parser.add_argument('--fromdate', '-f',
                        default='01/04/2010',
                        help='Starting date in %m/%d/%Y format')

    parser.add_argument('--todate', '-t',
                        default='10/17/2017',
                        help='Starting date in %m/%d/%Y format')

    parser.add_argument('--period', default=10, type=int,
                        help='Period to apply to the Simple Moving Average')

    parser.add_argument('--cash', default=100000, type=int,
                        help='Starting Cash')

    parser.add_argument('--runnext', action='store_true',
                        help='Use next by next instead of runonce')

    parser.add_argument('--nopreload', action='store_true',
                        help='Do not preload the data')

    parser.add_argument('--oldsync', action='store_true',
                        help='Use old data synchronization method')

    parser.add_argument('--commperc', default=0.005, type=float,
                        help='Percentage commission (0.005 is 0.5%%')

    parser.add_argument('--stake', default=10, type=int,
                        help='Stake to apply in each operation')

    parser.add_argument('--plot', '-p', default=True, action='store_true',
                        help='Plot the read data')

    parser.add_argument('--numfigs', '-n', default=1,
                        help='Plot using numfigs figures')

    return parser.parse_args()


if __name__ == '__main__':
    runstrategy()

Posts made by samk