Anyone use backtrader to do live trading on Bitcoin exchange?

Mikk Laos

I would also like to contribute with development. If anyone can give some tips, it would be appriciated

Ed Bartosh

As people show some interest I decided to get started.

Here is a simplified code just for the start.

I intentionally made it as simple as possible to make it understandable and easy to play with. Please, consider this only as an invitation to collaborate.

I followed suggestions and examples from this guide

This test script:

#!/usr/bin/env python
# -*- coding: utf-8; py-indent-offset:4 -*-
###############################################################################
#
# Copyright (C) 2017 Ed Bartosh
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program.  If not, see <http://www.gnu.org/licenses/>.
#
###############################################################################
from __future__ import (absolute_import, division, print_function,
                        unicode_literals)

import sys

import backtrader as bt

class TestStrategy(bt.Strategy):
    def notify_data(self, data, status, *args, **kwargs):
        print('*' * 5, 'DATA NOTIF:', data._getstatusname(status))

    def next(self):
        print('*' * 5, 'NEXT:', bt.num2date(self.data0.datetime[0]), self.data0._name, self.data0.open[0],
              bt.TimeFrame.getname(self.data0._timeframe), len(self.data0))

def runstrategy(argv):
    # Create a cerebro
    cerebro = bt.Cerebro()

    data = bt.feeds.CCXT(exchange='gdax', symbol='BTC/USD', timeframe=bt.TimeFrame.Ticks, compression=1)
    cerebro.resampledata(data, timeframe=bt.TimeFrame.Seconds)
    #cerebro.adddata(data)
    
    # Add the strategy
    cerebro.addstrategy(TestStrategy)

    # Run the strategy
    cerebro.run()

if __name__ == '__main__':
    sys.exit(runstrategy(sys.argv))

produces this output:

$ ./gdaxtest.py 
loaded tick time: 2017-10-28 15:39:39.726000, price: 5715.01, size: 0.01757807
loaded tick time: 2017-10-28 15:39:39.726000, price: 5715.01, size: 0.01757807
loaded tick time: 2017-10-28 15:39:42.513000, price: 5715.01, size: 1.74e-06
***** NEXT: 2017-10-28 15:39:40  5715.01 Second 1
loaded tick time: 2017-10-28 15:39:42.513000, price: 5715.01, size: 1.74e-06
loaded tick time: 2017-10-28 15:39:43.848000, price: 5715.01, size: 0.14919127
***** NEXT: 2017-10-28 15:39:43  5715.01 Second 2
loaded tick time: 2017-10-28 15:39:43.848000, price: 5715.01, size: 0.14919127
loaded tick time: 2017-10-28 15:39:43.848000, price: 5715.01, size: 0.14919127
loaded tick time: 2017-10-28 15:39:43.848000, price: 5715.01, size: 0.14919127
loaded tick time: 2017-10-28 15:39:43.978000, price: 5715.01, size: 0.01624613
loaded tick time: 2017-10-28 15:39:43.978000, price: 5715.01, size: 0.01624613
loaded tick time: 2017-10-28 15:39:43.978000, price: 5715.01, size: 0.01624613
loaded tick time: 2017-10-28 15:39:43.978000, price: 5715.01, size: 0.01624613
loaded tick time: 2017-10-28 15:39:43.978000, price: 5715.01, size: 0.01624613
loaded tick time: 2017-10-28 15:39:43.978000, price: 5715.01, size: 0.01624613
loaded tick time: 2017-10-28 15:39:43.978000, price: 5715.01, size: 0.01624613
loaded tick time: 2017-10-28 15:39:45.516000, price: 5715.01, size: 0.06103097
***** NEXT: 2017-10-28 15:39:44  5715.01 Second 3
loaded tick time: 2017-10-28 15:39:45.516000, price: 5715.01, size: 0.46466909
loaded tick time: 2017-10-28 15:39:49.408000, price: 5715.01, size: 0.17441143
***** NEXT: 2017-10-28 15:39:46  5715.01 Second 4
loaded tick time: 2017-10-28 15:39:49.408000, price: 5715.01, size: 0.17441143
loaded tick time: 2017-10-28 15:39:49.408000, price: 5715.01, size: 0.17441143
loaded tick time: 2017-10-28 15:39:49.408000, price: 5715.01, size: 0.17441143
loaded tick time: 2017-10-28 15:39:49.408000, price: 5715.01, size: 0.17441143
loaded tick time: 2017-10-28 15:40:02.295000, price: 5715.01, size: 1.74e-06
***** NEXT: 2017-10-28 15:39:50  5715.01 Second 5
loaded tick time: 2017-10-28 15:40:02.295000, price: 5715.01, size: 1.74e-06
loaded tick time: 2017-10-28 15:40:02.295000, price: 5715.01, size: 1.74e-06
loaded tick time: 2017-10-28 15:40:02.295000, price: 5715.01, size: 1.74e-06
loaded tick time: 2017-10-28 15:40:02.295000, price: 5715.01, size: 1.74e-06
loaded tick time: 2017-10-28 15:40:02.295000, price: 5715.01, size: 1.74e-06
loaded tick time: 2017-10-28 15:40:02.295000, price: 5715.01, size: 1.74e-06
loaded tick time: 2017-10-28 15:40:11.930000, price: 5715.01, size: 1.74e-06
***** NEXT: 2017-10-28 15:40:03  5715.01 Second 6
loaded tick time: 2017-10-28 15:40:15.836000, price: 5715.0, size: 0.1853
***** NEXT: 2017-10-28 15:40:12  5715.01 Second 7
loaded tick time: 2017-10-28 15:40:15.836000, price: 5715.0, size: 0.1853
loaded tick time: 2017-10-28 15:40:15.836000, price: 5715.0, size: 0.1853

Next I'm going to implement loading historical ohlc data using fetchOHLCV ccxt API.

Any suggestions and help are welcome.

CptanPanic

@Ed-Bartosh Nice job, looks great.

Ed Bartosh

Implemented support for all ccxt time frames using fetchOHLCV. The code is in ccxt branch of my github repo.

Chapachan

@Ed-Bartosh will you implement both backtesting and live? That would be great!

Ed Bartosh

@Chapachan @Chapachan Correct me if I'm wrong, but the only difference between backtesting and live mode is 'historical=True' parameter for the feed. If this is true then yes, I'll implement both.

Paska Houso

@Ed-Bartosh

Some posts (can't find them) reference the islive method in the feeds. This was apparently intended to tell the platform to run in step-by-step mode.

See here for example:

https://github.com/mementum/backtrader/blob/f6e7c6dfb4151c7d5d554e2bce5a2ced8daa85c5/backtrader/feeds/oanda.py#L165-L168

historical seems to be a switch to stop actions when a feed supports live data, but not really related to running live.

Ed Bartosh

@Paska-Houso

historical seems to be a switch to stop actions when a feed supports live data, but not really related to running live.

I'm not sure it's not related. Here is an example: https://github.com/mementum/backtrader/blob/f6e7c6dfb4151c7d5d554e2bce5a2ced8daa85c5/backtrader/feeds/ibdata.py#L253-L256

Paska Houso

It seems obvious that if you are going to only do a historical download, you may report the not historical to understand if the feed is going to deliver live data.

But there may be other circumstances that could let you report False. A quick idea:

You are downloading the historical data for an asset for which your subscription doesn't allow live data.

Ed Bartosh

@Paska-Houso

I'm probably missing something, but with the current ccxt feed code It looks like it works more or less expected way in both modes. To switch between modes I only change 'historical' parameter for the feed.

The current code is in cctx branch

I'd appreciate if you can point me out where I'm wrong in my implementation.

Ed Bartosh

Here is my testing code just in case if anybody will be willing to help me with testing:

#!/usr/bin/env python
# -*- coding: utf-8; py-indent-offset:4 -*-
###############################################################################
#
# Copyright (C) 2017 Ed Bartosh
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program.  If not, see <http://www.gnu.org/licenses/>.
#
###############################################################################
from __future__ import (absolute_import, division, print_function,
                        unicode_literals)

import sys

from datetime import datetime, timedelta

import backtrader as bt

class TestStrategy(bt.Strategy):
    def next(self):
        for data in self.datas:
            print('*' * 5, 'NEXT:', bt.num2date(data.datetime[0]), data._name, data.open[0], data.high[0],
                  data.low[0], data.close[0], data.volume[0],
                  bt.TimeFrame.getname(data._timeframe), len(data))

def runstrategy(argv):
    # Create a cerebro
    cerebro = bt.Cerebro()

    data_ticks = bt.feeds.CCXT(exchange='gdax', symbol='BTC/USD', name="btc_usd_tick",
                             timeframe=bt.TimeFrame.Ticks)
    cerebro.adddata(data_ticks)

    hist_start_date = datetime.utcnow() - timedelta(minutes=30)
    data_min = bt.feeds.CCXT(exchange="gdax", symbol="BTC/USD", name="btc_usd_min",
                             timeframe=bt.TimeFrame.Minutes, fromdate=hist_start_date) #, historical=True)
    cerebro.adddata(data_min)

    # Add the strategy
    cerebro.addstrategy(TestStrategy)

    # Run the strategy
    cerebro.run()

if __name__ == '__main__':
    sys.exit(runstrategy(sys.argv))

remroc

historical option is for downloading historical data in order to feed your strategy. Sometimes you need those data to compute a moving averages or other indicator before actually trading with LIVE data...

Hope it makes sense...

kroitor

@Ed-Bartosh awesome! Let us know if you have any difficulties or questions on ccxt. Thx for your involvement!

Ed Bartosh

Started to implement basic support of ordering.

Testing code uses gdax exchange for its data feed and gemini for ordering:

from __future__ import (absolute_import, division, print_function,
                        unicode_literals)

import sys
import time

from datetime import datetime, timedelta

import backtrader as bt

from pandas import bdate_range

class TestStrategy(bt.Strategy):
    def next(self):
        for data in self.datas:
            print('*' * 5, 'NEXT:', bt.num2date(data.datetime[0]), data._name, data.open[0], data.high[0],
                  data.low[0], data.close[0], data.volume[0],
                  bt.TimeFrame.getname(data._timeframe), len(data))
            if not self.getposition(data):
                order = self.buy(data, exectype=bt.Order.Limit, size=10, price=data.close[0])
            else:
                order = self.sell(data, exectype=bt.Order.Limit, size=10, price=data.close[0])

    def notify_order(self, order):
        print('*' * 5, "NOTIFY ORDER", order)

def runstrategy(argv):
    # Create a cerebro
    cerebro = bt.Cerebro()

    # Create broker
    broker_config = {'urls': {
                         'logo': 'https://user-images.githubusercontent.com/1294454/27816857-ce7be644-6096-11e7-82d6-3c257263229c.jpg',
                         'api': 'https://api.sandbox.gemini.com',
                         'www': 'https://gemini.com',
                         'doc': 'https://docs.gemini.com/rest-api',},
                     'apiKey': 'put your api key here',
                     'secret': 'put your secret here',
                     'nonce': lambda: str(int(time.time() * 1000))
                    }
    broker = bt.brokers.CCXTBroker(exchange='gemini', currency='USD', config=broker_config)
    cerebro.setbroker(broker)

    # Create data feeds
    data_ticks = bt.feeds.CCXT(exchange='gdax', symbol='BTC/USD', name="btc_usd_tick",
                             timeframe=bt.TimeFrame.Ticks)
    #cerebro.resampledata(data_sec, timeframe=bt.TimeFrame.Seconds)
    cerebro.adddata(data_ticks)

    #hist_start_date = bdate_range(end=datetime.now(), periods=1)[0].to_pydatetime()
    #hist_start_date = datetime.utcnow() - timedelta(minutes=30)
    #data_min = bt.feeds.CCXT(exchange="gdax", symbol="BTC/USD", name="btc_usd_min",
    #                         timeframe=bt.TimeFrame.Minutes, fromdate=hist_start_date)
    #cerebro.adddata(data_min)

    # Add the strategy
    cerebro.addstrategy(TestStrategy)

    # Run the strategy
    cerebro.run()

if __name__ == '__main__':
    sys.exit(runstrategy(sys.argv))

output:

$ ./ccxttest.py 
btc_usd_tick: loaded tick: time: 2017-11-12 12:08:21.737000, price: 6251.54, size: 1.59e-06
***** NEXT: 2017-11-12 12:08:21.736998 btc_usd_tick 6251.54 6251.54 6251.54 6251.54 1.59e-06 Tick 1
btc_usd_tick: loaded tick: time: 2017-11-12 12:08:55.247000, price: 6278.9, size: 0.01797351
***** NOTIFY ORDER Ref: 1
OrdType: 1
OrdType: Sell
Status: 0
Status: Created
Size: -10.0
Price: None
Price Limit: None
TrailAmount: None
TrailPercent: None
ExecType: 0
ExecType: Market
CommInfo: None
End of Session: 736646.0
Info: AutoOrderedDict()
Broker: None
Alive: True
***** NEXT: 2017-11-12 12:08:55.246996 btc_usd_tick 6278.9 6278.9 6278.9 6278.9 0.01797351 Tick 2
btc_usd_tick: loaded tick: time: 2017-11-12 12:08:58.981000, price: 6278.89, size: 0.02526792
***** NOTIFY ORDER Ref: 2
OrdType: 1
OrdType: Sell
Status: 0
Status: Created
Size: -10.0
Price: None
Price Limit: None
TrailAmount: None
TrailPercent: None
ExecType: 0
ExecType: Market
CommInfo: None
End of Session: 736646.0
Info: AutoOrderedDict()
Broker: None
Alive: True

perelin

@Ed-Bartosh said in Anyone use backtrader to do live trading on Bitcoin exchange?:

Started to implement basic support of ordering.
Testing code uses gdax exchange for its data feed and gemini for ordering:

Awesome! Anyway to try that out? Would be glad to help with development :)

Ed Bartosh

@perelin Sure, you can just clone the code from my repository and run test script I posted here. Let me know if you need more detailed instructions.

kicking_potato88

Hi guys, I managed to successfully run both test scripts Ed wrote. Just note initially running the first TestStrategy I had trouble running it in Python 3, but Ed has updated the branch to be compatible with both Python 2 and 3.

For the second TestStrategy (with Gemini orders placed), I was getting errors relating to "fundmode", searching the backtrader forum seems a solution is adding stdstats=False to cerebro.run, i.e:

cerebro.run(stdstats=False)

https://community.backtrader.com/topic/542/broker-and-fundmode

kicking_potato88

Hi, extending the 2nd strategy, I tried to build a simple 200 hour moving average system (buy when price closes above 200HMA, and sell when price closes below 200HMA).

However, using the data feed this way, I tend to obtain timeout errors:

*ExchangeNotAvailable: gemini POST https://api.sandbox.gemini.com/v1/balances <urlopen error _ssl.c:761: The handshake operation timed out>

RequestTimeout: gdax GET https://api.gdax.com/products/BTC-USD/candles?granularity=3600 request timeout*

Any thoughts on how to make my data feed more stable? Could I modify the feed such that historical data is loaded via a CSV file, and subsequent live data is fed in separately?

Thanks!

import time
import configparser

from datetime import datetime, timedelta

import backtrader as bt
import backtrader.indicators as btind
import backtrader.feeds as btfeeds

from pandas import bdate_range

class GeminiBTC200MAH1Strategy(bt.Strategy):
    
    def __init__(self):
        
        self.sma200 = btind.SimpleMovingAverage(self.data, period=200)
    
    def next(self):
        for data in self.datas:
            print('*' * 5, 'NEXT:', bt.num2date(data.datetime[0]), data._name, data.open[0], data.high[0],
                  data.low[0], data.close[0], data.volume[0],
                  bt.TimeFrame.getname(data._timeframe), len(data))
            if not self.getposition(data) and data.close[0] > self.sma200[0]:
                order = self.buy(data, exectype=bt.Order.Market, size=10)
            elif self.getposition(data) and data.close[0] < self.sma200[0]:
                order = self.sell(data, exectype=bt.Order.Market, size=10)

    def notify_order(self, order):
        print('*' * 5, "NOTIFY ORDER", order)

def runstrategy(argv):
    # Create a cerebro
    cerebro = bt.Cerebro()
    
    #Handle config issues
    config = configparser.ConfigParser()
    config.read('gemini.cfg')
    gemini_api_key = config['gemini']['gemini_api_key']
    gemini_api_secret = config['gemini']['gemini_api_secret']
    
    # Create broker
    broker_config = {'urls': {
                         'logo': 'https://user-images.githubusercontent.com/1294454/27816857-ce7be644-6096-11e7-82d6-3c257263229c.jpg',
                         'api': 'https://api.sandbox.gemini.com',
                         'www': 'https://gemini.com',
                         'doc': 'https://docs.gemini.com/rest-api',},
                     'apiKey': gemini_api_key,
                     'secret': gemini_api_secret,
                     'nonce': lambda: str(int(time.time() * 1000))
                    }
    broker = bt.brokers.CCXTBroker(exchange='gemini', currency='USD', config=broker_config)
    cerebro.setbroker(broker)

    # Create data feeds
    hist_start_date = bdate_range(end=(datetime.now() - timedelta(days=10)), periods=1)[0].to_pydatetime()
    #hist_start_date = datetime.utcnow() - timedelta(minutes=30)
    data_hour = bt.feeds.CCXT(exchange="gdax", symbol="BTC/USD", name="btc_usd_1h",                         timeframe=bt.TimeFrame.Minutes, compression=60, fromdate=hist_start_date)
    cerebro.adddata(data_hour)

    # Add the strategy
    cerebro.addstrategy(GeminiBTC200MAH1Strategy)

    # Run the strategy
    cerebro.run(stdstats=False)

if __name__ == '__main__':
    sys.exit(runstrategy(sys.argv))

xord37

Hey,
When I use the time frame of minutes, it just hangs and doesn't do anything -

hist_start_date = datetime.utcnow() - timedelta(minutes=30)
data_min = bt.feeds.CCXT(exchange="gdax", symbol="BTC/USD", name="btc_usd_min",
                         timeframe=bt.TimeFrame.Minutes, fromdate=hist_start_date)
cerebro.adddata(data_min)

When I kill it, it seems to get stuck in some waiting routine -

Traceback (most recent call last):
  File "crypt-test.py", line 63, in <module>
    sys.exit(runstrategy(sys.argv))
  File "crypt-test.py", line 60, in runstrategy
    cerebro.run()
  File "C:\Python27\lib\site-packages\backtrader\cerebro.py", line 1127, in run
    runstrat = self.runstrategies(iterstrat)
  File "C:\Python27\lib\site-packages\backtrader\cerebro.py", line 1295, in runstrategies
    self._runnext(runstrats)
  File "C:\Python27\lib\site-packages\backtrader\cerebro.py", line 1538, in _runnext
    drets.append(d.next(ticks=False))
  File "C:\Python27\lib\site-packages\backtrader\feed.py", line 404, in next
    ret = self.load()
  File "C:\Python27\lib\site-packages\backtrader\feed.py", line 476, in load
    _loadret = self._load()
  File "C:\Python27\lib\site-packages\backtrader\feeds\ccxt.py", line 115, in _load
    self._fetch_ohlcv()
  File "C:\Python27\lib\site-packages\backtrader\feeds\ccxt.py", line 151, in _fetch_ohlcv
    sleep(self.exchange.rateLimit / 1000) # time.sleep wants seconds
KeyboardInterrupt

Why does it sleep there?

xord37

About my last message - It seems that there was a bug in the ccxt feed, I fixed it and made a pull request