Anyone use backtrader to do live trading on Bitcoin exchange?

Søren Pallesen

@Ed-Bartosh did the pip install directly abd get the same AttributeError: module 'backtrader.brokers' has no attribute 'CCXTBroker'

samk

@Ed-Bartosh

I even tried to run

pip install git+https://github.com/bartosh/backtrader.git@ccxt

after changing to directory that I saved git in which is: C:\Users\Sam\Anaconda64Best

But still getting the same error.......

C:\Users\Sam\Anaconda64Best\Lib\site-packages\backtrader>cd C:\Users\Sam\Anaconda64Best

*************C:\Users\Sam\Anaconda64Best>pip install git+https://github.com/bartosh/backtrader.git@ccxt

Collecting git+https://github.com/bartosh/backtrader.git@ccxt
Cloning https://github.com/bartosh/backtrader.git (to ccxt) to c:\users\sam\appdata\local\temp\pip-al_ctv-build
Error [Error 2] The system cannot find the file specified while executing command git clone -q https://github.com/bartosh/backtrader.git c:\users\sam\appdata\local\temp\pip-al_ctv-build
Cannot find command 'git'*************

Ed Bartosh

Guys, you both don't have git installed. Please, install it from here: https://git-scm.com/download/win, run it and you'll end up in the command line, where you will be able to install backtrader with this command:

pip install git+https://github.com/bartosh/backtrader.git@ccxt

P.S. you don't need to go to C:\Users\Sam\Anaconda64Best\Lib\site-packages\backtrader and do anything there. That's the area pip installs packages to. Leave its job to it :)

Ed Bartosh

@Søren-Pallesen said in Anyone use backtrader to do live trading on Bitcoin exchange?:

@Ed-Bartosh Im getting:

(C:\Users\Soren\Anaconda3\envs\CCXTwBT2) C:\Users\Soren>python -c 'from backtrader.brokers.ccxtbroker import CCXTBroker'
  File "<string>", line 1
    'from
        ^
SyntaxError: EOL while scanning string literal```

that's interesting. Here is how it runs on Windows computer usually:

PS C:\Users\Administrator> python -c 'import sys'
PS C:\Users\Administrator> python -c 'import backtrader'
PS C:\Users\Administrator> python -c 'import backtrader.brokers'
PS C:\Users\Administrator> python -c 'import backtrader.brokers.ccxtbroker'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named ccxtbroker

Ed Bartosh

how is it going?

I've just found a way to install git in anaconda environment:

conda install -c anaconda git

This may be easier for you than installing git separately. Can you try this?

samk

@Ed-Bartosh

So I use conda to install git and it worked great. then I did pip install as you mentioned:

C:\Users\Sam>pip install git+https://github.com/bartosh/backtrader.git@ccxt
Collecting git+https://github.com/bartosh/backtrader.git@ccxt
Cloning https://github.com/bartosh/backtrader.git (to ccxt) to c:\users\sam\appdata\local\temp\pip-yljmoz-build
Installing collected packages: backtrader
Running setup.py install for backtrader ... done
Successfully installed backtrader-1.9.59.122

So looks like it was a success. However when I run your file I am getting same error as @soren:

C:\Users\Sam\Anaconda64Best\python.exe "C:/Users/Sam/PycharmProjects/Test/.ipynb_checkpoints/Backtrader Bitcoin Algorithm.py"
Traceback (most recent call last):
File "C:/Users/Sam/PycharmProjects/Test/.ipynb_checkpoints/Backtrader Bitcoin Algorithm.py", line 68, in <module>
sys.exit(runstrategy(sys.argv))
File "C:/Users/Sam/PycharmProjects/Test/.ipynb_checkpoints/Backtrader Bitcoin Algorithm.py", line 48, in runstrategy
broker = bt.brokers.CCXTBroker(exchange='gemini', currency='USD', config=broker_config)
AttributeError: module 'backtrader.brokers' has no attribute 'CCXTBroker'

Do we need to import something at the top? Because using:

import backtrader.brokers.ccxtbroker

seems to do nothing. Please let me know.......Thanks Ed..

Ed Bartosh

@samk Can you try to import it this way?

python -c 'from backtrader.brokers.ccxtbroker import CCXTBroker'

Søren Pallesen

Its resolved. The key is to install Git in the virtual environment via Conda. Very easy to do using Anaconda Navigator, which i highly recommend for managing environments. My step were:

Make and activate new environment (I do this in Anaconda Navigator)
Install Conda package Git into the environment (I do this in Anaconda Navigator)
(optional) Conda package Jupyter into the environment (I do this in Anaconda Navigator)
Open terminal (command prompt) in the environment
pip install ccxt
pip install git+https://github.com/bartosh/backtrader.git@ccxt

Done :-)

Now the work can really begin - look like a very promising project. And thx for the support in getting it up and running.

samk

@Søren-Pallesen @Ed-Bartosh

Guys,

So I followed Soren's method. Created a new environment, added git, then did pip install ccxt and pip install git+https://github.com/bartosh/backtrader.git@ccxt

Both loaded just fine.

However when I am running the script, still getting an error:

ModuleNotFoundError: No module named 'backtrader.brokers.ccxtbroker'

As if it is still not reading it in my Pycharm interface. Even though everything was said to have installed ok!

I am very confused here. the statement I am using in the file to import CCXT is:

from backtrader.brokers.ccxtbroker import CCXTBroker

Is this not correct maybe? Or another import line of code I am supposed to use?

Please let me know. I really want to get this going so I can help collaborate with you guys by applying some created ML and deep learning models I have used on equities, equity options, and futures on the crypto currencies specifically. Think there may be substantial value there...........

Ed Bartosh

@Søren-Pallesen Try to uninstall backtrader(pip uninstall backtrader) and install it again.

Søren Pallesen

@Ed-Bartosh Everything works perfectly - so im good - thx for your help
@samk I ran an update to Windows 10 just before i did the install - the environment runs with Python 3.6 - not sure what else i can help with ...maybe reinstall Anaconda to latest version - im interested in ML models as well maybe we could share some thoughts at a point in time

Ed Bartosh

@theref said in Anyone use backtrader to do live trading on Bitcoin exchange?:

But it appears that CCXT feeds are only making one request. Eg:
hist_start_date = datetime.utcnow() - timedelta(days=50)
data_min = bt.feeds.CCXT(exchange="gdax", symbol="BTC/USD", name="btc_usd_min",
                         timeframe=bt.TimeFrame.Minutes, fromdate=hist_start_date, historical=True)
This only returns the first 200 minutes, starting from 50 days ago. I may have completely misunderstood what's occurring, but I'd love to help out so feel free to point me in the right direction :)

Reimplemented fetching historical ohlcv data: https://github.com/bartosh/backtrader/commit/4d43bf8c0f159ad3b819218be113b4515a4dc5e5

Please, get the latest code from ccxt branch and try if it works for you.

I was able to successfully get 50 days of price data from geminy in 1 min. time frame. This might not work for all exchanges due to the limitations I pointed out above in this thread.

It took around 5 minutes to fetch the data on my machine.

Søren Pallesen

@Ed-Bartosh have you thought about merging in the refactoring @sutpen did (the backtrader 'store' paradigm) or is it not worth it?

Also, would you recommend just running this again to update with the latest code that supports historical data? :
pip install git+https://github.com/bartosh/backtrader.git@ccxt

Ed Bartosh

@Søren-Pallesen said in Anyone use backtrader to do live trading on Bitcoin exchange?:

@Ed-Bartosh have you thought about merging in the refactoring @sutpen did (the backtrader 'store' paradigm) or is it not worth it?

I have mixed feelings about it. From one side it's good to have the code similar to other data feeds. From other point of view the code is more complex. I don't like complexity and trying to avoid it as much as I can. ccxt is already a wrapper around many APIs. Making yet another layer(store) around it feels unneeded.

I actually asked this question (do i need to implement a store) from backtrader author and got an answer that it's not mandatory:
https://community.backtrader.com/topic/642/iqfeed-store-and-feed-help-needed/7

Also, would you recommend just running this again to update with the latest code that supports historical data? :
pip install git+https://github.com/bartosh/backtrader.git@ccxt

Just to be on the safe side I'd suggest to uninstall it first (pip uninstall backtrader) and then install again.

Søren Pallesen

@Ed-Bartosh Agree - like the simplicity if its not needed

samk

@Ed-Bartosh

I am on your page here:

https://github.com/bartosh/backtrader/tree/master/backtrader/brokers

And I do not see any file called ccxtbroker.py, in order to identify the CCXTBroker class? I only see IB, oanda, VC, and bboker.

Is there somewhere on your git page where I can find the ccxtbroker.py?

Thanks......

Ed Bartosh

@samk you're looking at the master branch. ccxt code is in ccxt branch: https://github.com/bartosh/backtrader/tree/ccxt/backtrader/brokers

Mikk Laos

There is a slight problem with ccxt broker. It does not account for exchange granularity. When I try to use poloniex exchange where minimum granularity is 5 m it still uses 1m when I remove granularity from feeds\ccxt.py it works fine

    for ohlcv in self.exchange.fetch_ohlcv(self.symbol, timeframe=granularity
                                                   since=since, limit=limit)[::-1]:

samk

@Ed-Bartosh

Thank you I found it. Instead of cloning github, which obviously caused me many issues, I have went ahead and saved the files from your github manually. These files being 'ccxtbroker.py' (which holds the class CCXTBroker) and 'ccxt.py' (which holds the class CCXT ).

I am receiving an error however saying:

***Traceback (most recent call last):
File "C:/Users/Sam/PycharmProjects/Test/.ipynb_checkpoints/Crypto Backtrader Algorithm Using CCXT Broker & Gemini.py", line 71, in <module>
sys.exit(runstrategy(sys.argv))
File "C:/Users/Sam/PycharmProjects/Test/.ipynb_checkpoints/Crypto Backtrader Algorithm Using CCXT Broker & Gemini.py", line 51, in runstrategy
broker = ccxtbroker.CCXTBroker(exchange='gemini', currency='USD', config=broker_config)
File "C:\Users\Sam\Anaconda64Best\lib\site-packages\backtrader\metabase.py", line 88, in call
_obj, args, kwargs = cls.doinit(_obj, *args, **kwargs)
File "C:\Users\Sam\Anaconda64Best\lib\site-packages\backtrader\metabase.py", line 78, in doinit
_obj.init(args, kwargs)
File "C:\Users\Sam\PycharmProjects\Test.ipynb_checkpoints\ccxtbroker.py", line 42, in init
self.exchange = getattr(ccxt, exchange)(config)
AttributeError: module 'ccxt' has no attribute 'gemini'

Curious on why this may be happening? My full code is pasted below for reference. Thanks Ed.

Code:

from __future__ import (absolute_import, division, print_function,
                        unicode_literals)

from datetime import datetime, timedelta
import os.path  # To manage paths
import sys  # To find out the script name (in argv[0])

import ccxtbroker
import ccxt
import backtrader as bt

# Import the backtrader platform
import backtrader as bt
import backtrader.feeds as btfeeds
import datetime  # For datetime objects
import argparse
import time
import backtrader.analyzers as btan
from backtrader import Analyzer
import math
import numpy as np
import datetime

class TestStrategy(bt.Strategy):
    def next(self):
        for data in self.datas:
            print('*' * 5, 'NEXT:', bt.num2date(data.datetime[0]), data._name, data.open[0], data.high[0],
                  data.low[0], data.close[0], data.volume[0],
                  bt.TimeFrame.getname(data._timeframe), len(data))
            if not self.getposition(data):
                order = self.buy(data, exectype=bt.Order.Limit, size=10, price=data.close[0])
            else:
                order = self.sell(data, exectype=bt.Order.Limit, size=10, price=data.close[0])

    def notify_order(self, order):
        print('*' * 5, "NOTIFY ORDER", order)

def runstrategy(argv):
    # Create a cerebro
    cerebro = bt.Cerebro()

    # Create broker
    broker_config = {'urls': {'api': 'https://api.sandbox.gemini.com'},
                     'apiKey': '<42o4kPgIwkYTyqKdrMD4>',
                     'secret': '<2hi9U3Md3ahMVeTDqfRPXzCebS38>',
                     'nonce': lambda: str(int(time.time() * 1000))
                    }
    broker = ccxtbroker.CCXTBroker(exchange='gemini', currency='USD', config=broker_config)
    cerebro.setbroker(broker)

    # Create data feeds
    data_ticks = ccxt.CCXT(exchange='gdax', symbol='BTC/USD', name="btc_usd_tick",
                               timeframe=bt.TimeFrame.Ticks, compression=1)
    cerebro.adddata(data_ticks)

    hist_start_date = datetime.utcnow() - timedelta(minutes=30)
    data_min = ccxt.CCXT(exchange="gdax", symbol="BTC/USD", name="btc_usd_min",
                             timeframe=bt.TimeFrame.Minutes, fromdate=hist_start_date)
    cerebro.adddata(data_min)

    # Add the strategy
    cerebro.addstrategy(TestStrategy)

    # Run the strategy
    cerebro.run()

if __name__ == '__main__':
    sys.exit(runstrategy(sys.argv))

Ed Bartosh

@Mikk-Laos said in Anyone use backtrader to do live trading on Bitcoin exchange?:

There is a slight problem with ccxt broker. It does not account for exchange granularity. When I try to use poloniex exchange where minimum granularity is 5 m it still uses 1m when I remove granularity from feeds\ccxt.py it works fine

Thanks for pointing out to this. Can you show how you initialise your datafeed?
The current code should pick up 5m granularity if you use minute time frame and compression 5. You don't need to change anything in ccxt.py