For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

Anyone use backtrader to do live trading on Bitcoin exchange?



  • I would also like to contribute with development. If anyone can give some tips, it would be appriciated



  • As people show some interest I decided to get started.

    Here is a simplified code just for the start.

    I intentionally made it as simple as possible to make it understandable and easy to play with. Please, consider this only as an invitation to collaborate.

    I followed suggestions and examples from this guide

    This test script:

    #!/usr/bin/env python
    # -*- coding: utf-8; py-indent-offset:4 -*-
    ###############################################################################
    #
    # Copyright (C) 2017 Ed Bartosh
    #
    # This program is free software: you can redistribute it and/or modify
    # it under the terms of the GNU General Public License as published by
    # the Free Software Foundation, either version 3 of the License, or
    # (at your option) any later version.
    #
    # This program is distributed in the hope that it will be useful,
    # but WITHOUT ANY WARRANTY; without even the implied warranty of
    # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
    # GNU General Public License for more details.
    #
    # You should have received a copy of the GNU General Public License
    # along with this program.  If not, see <http://www.gnu.org/licenses/>.
    #
    ###############################################################################
    from __future__ import (absolute_import, division, print_function,
                            unicode_literals)
    
    import sys
    
    import backtrader as bt
    
    class TestStrategy(bt.Strategy):
        def notify_data(self, data, status, *args, **kwargs):
            print('*' * 5, 'DATA NOTIF:', data._getstatusname(status))
    
        def next(self):
            print('*' * 5, 'NEXT:', bt.num2date(self.data0.datetime[0]), self.data0._name, self.data0.open[0],
                  bt.TimeFrame.getname(self.data0._timeframe), len(self.data0))
    
    def runstrategy(argv):
        # Create a cerebro
        cerebro = bt.Cerebro()
    
        data = bt.feeds.CCXT(exchange='gdax', symbol='BTC/USD', timeframe=bt.TimeFrame.Ticks, compression=1)
        cerebro.resampledata(data, timeframe=bt.TimeFrame.Seconds)
        #cerebro.adddata(data)
        
        # Add the strategy
        cerebro.addstrategy(TestStrategy)
    
        # Run the strategy
        cerebro.run()
    
    if __name__ == '__main__':
        sys.exit(runstrategy(sys.argv))
    

    produces this output:

    $ ./gdaxtest.py 
    loaded tick time: 2017-10-28 15:39:39.726000, price: 5715.01, size: 0.01757807
    loaded tick time: 2017-10-28 15:39:39.726000, price: 5715.01, size: 0.01757807
    loaded tick time: 2017-10-28 15:39:42.513000, price: 5715.01, size: 1.74e-06
    ***** NEXT: 2017-10-28 15:39:40  5715.01 Second 1
    loaded tick time: 2017-10-28 15:39:42.513000, price: 5715.01, size: 1.74e-06
    loaded tick time: 2017-10-28 15:39:43.848000, price: 5715.01, size: 0.14919127
    ***** NEXT: 2017-10-28 15:39:43  5715.01 Second 2
    loaded tick time: 2017-10-28 15:39:43.848000, price: 5715.01, size: 0.14919127
    loaded tick time: 2017-10-28 15:39:43.848000, price: 5715.01, size: 0.14919127
    loaded tick time: 2017-10-28 15:39:43.848000, price: 5715.01, size: 0.14919127
    loaded tick time: 2017-10-28 15:39:43.978000, price: 5715.01, size: 0.01624613
    loaded tick time: 2017-10-28 15:39:43.978000, price: 5715.01, size: 0.01624613
    loaded tick time: 2017-10-28 15:39:43.978000, price: 5715.01, size: 0.01624613
    loaded tick time: 2017-10-28 15:39:43.978000, price: 5715.01, size: 0.01624613
    loaded tick time: 2017-10-28 15:39:43.978000, price: 5715.01, size: 0.01624613
    loaded tick time: 2017-10-28 15:39:43.978000, price: 5715.01, size: 0.01624613
    loaded tick time: 2017-10-28 15:39:43.978000, price: 5715.01, size: 0.01624613
    loaded tick time: 2017-10-28 15:39:45.516000, price: 5715.01, size: 0.06103097
    ***** NEXT: 2017-10-28 15:39:44  5715.01 Second 3
    loaded tick time: 2017-10-28 15:39:45.516000, price: 5715.01, size: 0.46466909
    loaded tick time: 2017-10-28 15:39:49.408000, price: 5715.01, size: 0.17441143
    ***** NEXT: 2017-10-28 15:39:46  5715.01 Second 4
    loaded tick time: 2017-10-28 15:39:49.408000, price: 5715.01, size: 0.17441143
    loaded tick time: 2017-10-28 15:39:49.408000, price: 5715.01, size: 0.17441143
    loaded tick time: 2017-10-28 15:39:49.408000, price: 5715.01, size: 0.17441143
    loaded tick time: 2017-10-28 15:39:49.408000, price: 5715.01, size: 0.17441143
    loaded tick time: 2017-10-28 15:40:02.295000, price: 5715.01, size: 1.74e-06
    ***** NEXT: 2017-10-28 15:39:50  5715.01 Second 5
    loaded tick time: 2017-10-28 15:40:02.295000, price: 5715.01, size: 1.74e-06
    loaded tick time: 2017-10-28 15:40:02.295000, price: 5715.01, size: 1.74e-06
    loaded tick time: 2017-10-28 15:40:02.295000, price: 5715.01, size: 1.74e-06
    loaded tick time: 2017-10-28 15:40:02.295000, price: 5715.01, size: 1.74e-06
    loaded tick time: 2017-10-28 15:40:02.295000, price: 5715.01, size: 1.74e-06
    loaded tick time: 2017-10-28 15:40:02.295000, price: 5715.01, size: 1.74e-06
    loaded tick time: 2017-10-28 15:40:11.930000, price: 5715.01, size: 1.74e-06
    ***** NEXT: 2017-10-28 15:40:03  5715.01 Second 6
    loaded tick time: 2017-10-28 15:40:15.836000, price: 5715.0, size: 0.1853
    ***** NEXT: 2017-10-28 15:40:12  5715.01 Second 7
    loaded tick time: 2017-10-28 15:40:15.836000, price: 5715.0, size: 0.1853
    loaded tick time: 2017-10-28 15:40:15.836000, price: 5715.0, size: 0.1853
    

    Next I'm going to implement loading historical ohlc data using fetchOHLCV ccxt API.

    Any suggestions and help are welcome.



  • @Ed-Bartosh Nice job, looks great.



  • Implemented support for all ccxt time frames using fetchOHLCV. The code is in ccxt branch of my github repo.



  • @Ed-Bartosh will you implement both backtesting and live? That would be great!



  • @Chapachan @Chapachan Correct me if I'm wrong, but the only difference between backtesting and live mode is 'historical=True' parameter for the feed. If this is true then yes, I'll implement both.



  • @Ed-Bartosh

    Some posts (can't find them) reference the islive method in the feeds. This was apparently intended to tell the platform to run in step-by-step mode.

    See here for example:

    https://github.com/mementum/backtrader/blob/f6e7c6dfb4151c7d5d554e2bce5a2ced8daa85c5/backtrader/feeds/oanda.py#L165-L168

    historical seems to be a switch to stop actions when a feed supports live data, but not really related to running live.



  • @Paska-Houso

    historical seems to be a switch to stop actions when a feed supports live data, but not really related to running live.

    I'm not sure it's not related. Here is an example: https://github.com/mementum/backtrader/blob/f6e7c6dfb4151c7d5d554e2bce5a2ced8daa85c5/backtrader/feeds/ibdata.py#L253-L256



  • It seems obvious that if you are going to only do a historical download, you may report the not historical to understand if the feed is going to deliver live data.

    But there may be other circumstances that could let you report False. A quick idea:

    • You are downloading the historical data for an asset for which your subscription doesn't allow live data.


  • @Paska-Houso

    I'm probably missing something, but with the current ccxt feed code It looks like it works more or less expected way in both modes. To switch between modes I only change 'historical' parameter for the feed.

    The current code is in cctx branch

    I'd appreciate if you can point me out where I'm wrong in my implementation.



  • Here is my testing code just in case if anybody will be willing to help me with testing:

    #!/usr/bin/env python
    # -*- coding: utf-8; py-indent-offset:4 -*-
    ###############################################################################
    #
    # Copyright (C) 2017 Ed Bartosh
    #
    # This program is free software: you can redistribute it and/or modify
    # it under the terms of the GNU General Public License as published by
    # the Free Software Foundation, either version 3 of the License, or
    # (at your option) any later version.
    #
    # This program is distributed in the hope that it will be useful,
    # but WITHOUT ANY WARRANTY; without even the implied warranty of
    # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
    # GNU General Public License for more details.
    #
    # You should have received a copy of the GNU General Public License
    # along with this program.  If not, see <http://www.gnu.org/licenses/>.
    #
    ###############################################################################
    from __future__ import (absolute_import, division, print_function,
                            unicode_literals)
    
    import sys
    
    from datetime import datetime, timedelta
    
    import backtrader as bt
    
    class TestStrategy(bt.Strategy):
        def next(self):
            for data in self.datas:
                print('*' * 5, 'NEXT:', bt.num2date(data.datetime[0]), data._name, data.open[0], data.high[0],
                      data.low[0], data.close[0], data.volume[0],
                      bt.TimeFrame.getname(data._timeframe), len(data))
    
    def runstrategy(argv):
        # Create a cerebro
        cerebro = bt.Cerebro()
    
        data_ticks = bt.feeds.CCXT(exchange='gdax', symbol='BTC/USD', name="btc_usd_tick",
                                 timeframe=bt.TimeFrame.Ticks)
        cerebro.adddata(data_ticks)
    
        hist_start_date = datetime.utcnow() - timedelta(minutes=30)
        data_min = bt.feeds.CCXT(exchange="gdax", symbol="BTC/USD", name="btc_usd_min",
                                 timeframe=bt.TimeFrame.Minutes, fromdate=hist_start_date) #, historical=True)
        cerebro.adddata(data_min)
    
        # Add the strategy
        cerebro.addstrategy(TestStrategy)
    
        # Run the strategy
        cerebro.run()
    
    if __name__ == '__main__':
        sys.exit(runstrategy(sys.argv))
    


  • historical option is for downloading historical data in order to feed your strategy. Sometimes you need those data to compute a moving averages or other indicator before actually trading with LIVE data...

    Hope it makes sense...



  • @Ed-Bartosh awesome! Let us know if you have any difficulties or questions on ccxt. Thx for your involvement!



  • Started to implement basic support of ordering.

    Testing code uses gdax exchange for its data feed and gemini for ordering:

    from __future__ import (absolute_import, division, print_function,
                            unicode_literals)
    
    import sys
    import time
    
    from datetime import datetime, timedelta
    
    import backtrader as bt
    
    from pandas import bdate_range
    
    class TestStrategy(bt.Strategy):
        def next(self):
            for data in self.datas:
                print('*' * 5, 'NEXT:', bt.num2date(data.datetime[0]), data._name, data.open[0], data.high[0],
                      data.low[0], data.close[0], data.volume[0],
                      bt.TimeFrame.getname(data._timeframe), len(data))
                if not self.getposition(data):
                    order = self.buy(data, exectype=bt.Order.Limit, size=10, price=data.close[0])
                else:
                    order = self.sell(data, exectype=bt.Order.Limit, size=10, price=data.close[0])
    
        def notify_order(self, order):
            print('*' * 5, "NOTIFY ORDER", order)
    
    def runstrategy(argv):
        # Create a cerebro
        cerebro = bt.Cerebro()
    
        # Create broker
        broker_config = {'urls': {
                             'logo': 'https://user-images.githubusercontent.com/1294454/27816857-ce7be644-6096-11e7-82d6-3c257263229c.jpg',
                             'api': 'https://api.sandbox.gemini.com',
                             'www': 'https://gemini.com',
                             'doc': 'https://docs.gemini.com/rest-api',},
                         'apiKey': 'put your api key here',
                         'secret': 'put your secret here',
                         'nonce': lambda: str(int(time.time() * 1000))
                        }
        broker = bt.brokers.CCXTBroker(exchange='gemini', currency='USD', config=broker_config)
        cerebro.setbroker(broker)
    
        # Create data feeds
        data_ticks = bt.feeds.CCXT(exchange='gdax', symbol='BTC/USD', name="btc_usd_tick",
                                 timeframe=bt.TimeFrame.Ticks)
        #cerebro.resampledata(data_sec, timeframe=bt.TimeFrame.Seconds)
        cerebro.adddata(data_ticks)
    
        #hist_start_date = bdate_range(end=datetime.now(), periods=1)[0].to_pydatetime()
        #hist_start_date = datetime.utcnow() - timedelta(minutes=30)
        #data_min = bt.feeds.CCXT(exchange="gdax", symbol="BTC/USD", name="btc_usd_min",
        #                         timeframe=bt.TimeFrame.Minutes, fromdate=hist_start_date)
        #cerebro.adddata(data_min)
    
        # Add the strategy
        cerebro.addstrategy(TestStrategy)
    
        # Run the strategy
        cerebro.run()
    
    if __name__ == '__main__':
        sys.exit(runstrategy(sys.argv))
    

    output:

    $ ./ccxttest.py 
    btc_usd_tick: loaded tick: time: 2017-11-12 12:08:21.737000, price: 6251.54, size: 1.59e-06
    ***** NEXT: 2017-11-12 12:08:21.736998 btc_usd_tick 6251.54 6251.54 6251.54 6251.54 1.59e-06 Tick 1
    btc_usd_tick: loaded tick: time: 2017-11-12 12:08:55.247000, price: 6278.9, size: 0.01797351
    ***** NOTIFY ORDER Ref: 1
    OrdType: 1
    OrdType: Sell
    Status: 0
    Status: Created
    Size: -10.0
    Price: None
    Price Limit: None
    TrailAmount: None
    TrailPercent: None
    ExecType: 0
    ExecType: Market
    CommInfo: None
    End of Session: 736646.0
    Info: AutoOrderedDict()
    Broker: None
    Alive: True
    ***** NEXT: 2017-11-12 12:08:55.246996 btc_usd_tick 6278.9 6278.9 6278.9 6278.9 0.01797351 Tick 2
    btc_usd_tick: loaded tick: time: 2017-11-12 12:08:58.981000, price: 6278.89, size: 0.02526792
    ***** NOTIFY ORDER Ref: 2
    OrdType: 1
    OrdType: Sell
    Status: 0
    Status: Created
    Size: -10.0
    Price: None
    Price Limit: None
    TrailAmount: None
    TrailPercent: None
    ExecType: 0
    ExecType: Market
    CommInfo: None
    End of Session: 736646.0
    Info: AutoOrderedDict()
    Broker: None
    Alive: True
    


  • @Ed-Bartosh said in Anyone use backtrader to do live trading on Bitcoin exchange?:

    Started to implement basic support of ordering.
    Testing code uses gdax exchange for its data feed and gemini for ordering:

    Awesome! Anyway to try that out? Would be glad to help with development :)



  • @perelin Sure, you can just clone the code from my repository and run test script I posted here. Let me know if you need more detailed instructions.



  • Hi guys, I managed to successfully run both test scripts Ed wrote. Just note initially running the first TestStrategy I had trouble running it in Python 3, but Ed has updated the branch to be compatible with both Python 2 and 3.

    For the second TestStrategy (with Gemini orders placed), I was getting errors relating to "fundmode", searching the backtrader forum seems a solution is adding stdstats=False to cerebro.run, i.e:

    cerebro.run(stdstats=False)
    

    https://community.backtrader.com/topic/542/broker-and-fundmode



  • Hi, extending the 2nd strategy, I tried to build a simple 200 hour moving average system (buy when price closes above 200HMA, and sell when price closes below 200HMA).

    However, using the data feed this way, I tend to obtain timeout errors:

    *ExchangeNotAvailable: gemini POST https://api.sandbox.gemini.com/v1/balances <urlopen error _ssl.c:761: The handshake operation timed out>

    RequestTimeout: gdax GET https://api.gdax.com/products/BTC-USD/candles?granularity=3600 request timeout*

    Any thoughts on how to make my data feed more stable? Could I modify the feed such that historical data is loaded via a CSV file, and subsequent live data is fed in separately?

    Thanks!

    import time
    import configparser
    
    from datetime import datetime, timedelta
    
    import backtrader as bt
    import backtrader.indicators as btind
    import backtrader.feeds as btfeeds
    
    from pandas import bdate_range
    
    class GeminiBTC200MAH1Strategy(bt.Strategy):
        
        def __init__(self):
            
            self.sma200 = btind.SimpleMovingAverage(self.data, period=200)
        
        def next(self):
            for data in self.datas:
                print('*' * 5, 'NEXT:', bt.num2date(data.datetime[0]), data._name, data.open[0], data.high[0],
                      data.low[0], data.close[0], data.volume[0],
                      bt.TimeFrame.getname(data._timeframe), len(data))
                if not self.getposition(data) and data.close[0] > self.sma200[0]:
                    order = self.buy(data, exectype=bt.Order.Market, size=10)
                elif self.getposition(data) and data.close[0] < self.sma200[0]:
                    order = self.sell(data, exectype=bt.Order.Market, size=10)
    
        def notify_order(self, order):
            print('*' * 5, "NOTIFY ORDER", order)
    
    def runstrategy(argv):
        # Create a cerebro
        cerebro = bt.Cerebro()
        
        #Handle config issues
        config = configparser.ConfigParser()
        config.read('gemini.cfg')
        gemini_api_key = config['gemini']['gemini_api_key']
        gemini_api_secret = config['gemini']['gemini_api_secret']
        
        # Create broker
        broker_config = {'urls': {
                             'logo': 'https://user-images.githubusercontent.com/1294454/27816857-ce7be644-6096-11e7-82d6-3c257263229c.jpg',
                             'api': 'https://api.sandbox.gemini.com',
                             'www': 'https://gemini.com',
                             'doc': 'https://docs.gemini.com/rest-api',},
                         'apiKey': gemini_api_key,
                         'secret': gemini_api_secret,
                         'nonce': lambda: str(int(time.time() * 1000))
                        }
        broker = bt.brokers.CCXTBroker(exchange='gemini', currency='USD', config=broker_config)
        cerebro.setbroker(broker)
    
        # Create data feeds
        hist_start_date = bdate_range(end=(datetime.now() - timedelta(days=10)), periods=1)[0].to_pydatetime()
        #hist_start_date = datetime.utcnow() - timedelta(minutes=30)
        data_hour = bt.feeds.CCXT(exchange="gdax", symbol="BTC/USD", name="btc_usd_1h",                         timeframe=bt.TimeFrame.Minutes, compression=60, fromdate=hist_start_date)
        cerebro.adddata(data_hour)
    
        # Add the strategy
        cerebro.addstrategy(GeminiBTC200MAH1Strategy)
    
        # Run the strategy
        cerebro.run(stdstats=False)
    
    if __name__ == '__main__':
        sys.exit(runstrategy(sys.argv))
    


  • Hey,
    When I use the time frame of minutes, it just hangs and doesn't do anything -

    hist_start_date = datetime.utcnow() - timedelta(minutes=30)
    data_min = bt.feeds.CCXT(exchange="gdax", symbol="BTC/USD", name="btc_usd_min",
                             timeframe=bt.TimeFrame.Minutes, fromdate=hist_start_date)
    cerebro.adddata(data_min)
    

    When I kill it, it seems to get stuck in some waiting routine -

    Traceback (most recent call last):
      File "crypt-test.py", line 63, in <module>
        sys.exit(runstrategy(sys.argv))
      File "crypt-test.py", line 60, in runstrategy
        cerebro.run()
      File "C:\Python27\lib\site-packages\backtrader\cerebro.py", line 1127, in run
        runstrat = self.runstrategies(iterstrat)
      File "C:\Python27\lib\site-packages\backtrader\cerebro.py", line 1295, in runstrategies
        self._runnext(runstrats)
      File "C:\Python27\lib\site-packages\backtrader\cerebro.py", line 1538, in _runnext
        drets.append(d.next(ticks=False))
      File "C:\Python27\lib\site-packages\backtrader\feed.py", line 404, in next
        ret = self.load()
      File "C:\Python27\lib\site-packages\backtrader\feed.py", line 476, in load
        _loadret = self._load()
      File "C:\Python27\lib\site-packages\backtrader\feeds\ccxt.py", line 115, in _load
        self._fetch_ohlcv()
      File "C:\Python27\lib\site-packages\backtrader\feeds\ccxt.py", line 151, in _fetch_ohlcv
        sleep(self.exchange.rateLimit / 1000) # time.sleep wants seconds
    KeyboardInterrupt
    

    Why does it sleep there?



  • About my last message - It seems that there was a bug in the ccxt feed, I fixed it and made a pull request


 

});