Anyone use backtrader to do live trading on Bitcoin exchange?

Ed Bartosh

@samk just a side note. In my opinion Anaconda is evil and should be avoided at any cost. It causes a lot more problems than it solves in my opinion. This is a simple install of two python packages and we're fighting with it for several days.

Mikk Laos

@Ed-Bartosh
Actually I just removed granularities from feeds/ccxt.pyˇ

data_min = bt.feeds.CCXT(exchange=exchange, symbol=my_symbol, name="btc_usd_min",
                         timeframe=bt.TimeFrame.Minutes, fromdate=hist_start_date)

my test script is the same that I pasted

cerebro = bt.Cerebro()

    my_symbol = "BTC_USD"
    exchange = "poloniex"
    # Create broker
    broker_config = {
        'apiKey': '####',
        'secret': '#######',
        'nonce': lambda: str(int(time.time() * 1000))
    }
    broker = bt.brokers.CCXTBroker(exchange=exchange, currency='USDT', config=broker_config)
    cerebro.setbroker(broker)

    my_symbol = "BTC/USDT"

    # Create data feeds
    data_ticks = bt.feeds.CCXT(exchange=exchange, symbol=my_symbol, name="getmarketsummary",
                               timeframe=bt.TimeFrame.Ticks, compression=1)
    cerebro.adddata(data_ticks)

    hist_start_date = datetime.utcnow() - timedelta(minutes=30)
    data_min = bt.feeds.CCXT(exchange=exchange, symbol=my_symbol, name="btc_usd_min",
                             timeframe=bt.TimeFrame.Minutes, fromdate=hist_start_date)

   cerebro.adddata(data_min)

    # Add the strategy
    cerebro.addstrategy(TestStrategy)

    # Run the strategy
    cerebro.run()

Dont understand. What should I put for input if I want to use poloniex then . I cheked feeds/ccxt.py init does not contain timeframe at all. Should I leave timeframe blank then?

Ed Bartosh

@Mikk-Laos said in Anyone use backtrader to do live trading on Bitcoin exchange?:

Dont understand. What should I put for input if I want to use poloniex then . I cheked feeds/ccxt.py init does not contain timeframe at all. Should I leave timeframe blank then?

You should use minute timeframe with compression=5 if your broker only supports 5 minutes timeframe:

data_min = bt.feeds.CCXT(exchange=exchange, symbol=my_symbol,
                         name="btc_usd_min",
                         timeframe=bt.TimeFrame.Minutes,
                         compression=5,
                         fromdate=hist_start_date)

This is the only way I know to tell backtrader to use 5 minutes time frame.

What I still don't understand is how you made it working by removing lower granularities from ccxt feed code. In my opinion it should just fail getting supported granularity for 1 minute time frame. Can you show the changes you've made in ccxt feed code?

Søren Pallesen

@Ed-Bartosh said in Anyone use backtrader to do live trading on Bitcoin exchange?:

@Søren-Pallesen I didn't merge that PR yet. I will when it's fixed.

I think the config for the feed should be in the same format as for the broker:

config = {'urls': {'api': 'https://api.sandbox.gemini.com'},
                     'apiKey': 'XXXXX',
                     'secret': 'XXXXX',
                     'nonce': lambda: str(int(time.time() * 1000))
                    }
broker = bt.brokers.CCXTBroker(exchange='gemini',
                               currency='USD', config=config)
cerebro.setbroker(broker)

# Create data feeds
data_ticks = bt.feeds.CCXT(exchange='geminy', symbol='BTC/USD',
                          name="btc_usd_tick",
                          timeframe=bt.TimeFrame.Ticks,
                          compression=1, config=config)
cerebro.adddata(data_ticks)

I see you merged the PR - thx.

I would like BOTH the broker and the feed to point to a (different) sandbox URL, so my question is:

Is it enough to set the sandbox URL only ONCE in the config object when instantiating the broker for the feed also to point to that sandbox URL?

Or must i set it in config object BOTH when instantiating the broker AND the feed?

Thx in advance & Merry X-Mas :-)

Ed Bartosh

@kazi said in [Anyone use backtrader to do live trading on Bitcoin exchange?]

the only issue being http timeouts on both the ticker and balance functions eg. 502 etc errors. I would like to know if there is away to handle these errors "gracefully"

I've implemented retrying queries if network errors (502, 504, etc) occur:
https://github.com/bartosh/backtrader/commit/ff650e4be548f819dec08ec7303f7c4d613535d9

Unfortunately it required to move all ccxt-related code to the separate module(ccxtstore) to avoid code duplication. It's quite big change, so something might break because of this. Let me know if it does. I'll try to fix it as fast as I can.

Ed Bartosh

@Søren-Pallesen said in Anyone use backtrader to do live trading on Bitcoin exchange?:

@Ed-Bartosh said in Anyone use backtrader to do live trading on Bitcoin exchange?:

I would like BOTH the broker and the feed to point to a (different) sandbox URL, so my question is:

Is it enough to set the sandbox URL only ONCE in the config object when instantiating the broker for the feed also to point to that sandbox URL?

No, it's not enough. The broker and the feed are totally independent entities.

Or must i set it in config object BOTH when instantiating the broker AND the feed?
You must set it for both. If you don't set it for the feed then default ccxt configuration for the specified exchange will be used.

Ed Bartosh

@Mikk-Laos said in Anyone use backtrader to do live trading on Bitcoin exchange?:

What I still don't understand is how you made it working by removing lower granularities from ccxt feed code. In my opinion it should just fail getting supported granularity for 1 minute time frame. Can you show the changes you've made in ccxt feed code?

Can you show your changes, please? I'm still wondering what did you change to make the feed working for you in 5 minute time frame when you configure it to work in 1 minute time frame.

BTW, I've implemented additional time frame check:
https://github.com/bartosh/backtrader/commit/30516b83592c3a5ccdcc14162cbf605199734195
Please, try if it works for you.

kazi

Thanks @Ed-Bartosh, I look forward to testing, I am not sure as to how to install the tree, could you please provide the commands?

Many thanks!

Ed Bartosh

@kazi said in Anyone use backtrader to do live trading on Bitcoin exchange?:

Thanks @Ed-Bartosh, I look forward to testing, I am not sure as to how to install the tree, could you please provide the commands?

pip install git+https://github.com/bartosh/backtrader.git@ccxt

kazi

@Ed-Bartosh Ahh, I thought the changes you made were tied to a specific tree. Thank you for your hard work! I will get back to you if there any bugs.

Nguyễn Tài Nguyên

Hi,
I use below code to get data from Binance exchange but it keep running forever. Can you show me what wrong I did ? Thank you !

from __future__ import(absolute_import, division, print_function, unicode_literals)
from datetime import datetime
import os.path
import sys
import time

import backtrader as bt

if __name__ == '__main__':
    cerebro = bt.Cerebro()

    my_symbol = "BTC/USDT"
    exchange = "binance"
    # Create broker
    broker_config = {
        'apiKey': 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX',
        'secret': 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx',
        'nonce': lambda: str(int(time.time() * 1000))
    }

    broker = bt.brokers.CCXTBroker(exchange=exchange, currency='USDT', config=broker_config)
    cerebro.setbroker(broker)

    # Create data feeds
    data_ticks = bt.feeds.CCXT(exchange=exchange, symbol=my_symbol, name="getmarketsummary",
                               timeframe=bt.TimeFrame.Ticks, compression=5)
    cerebro.adddata(data_ticks)

    hist_start_date = datetime.utcnow()
    data_min = bt.feeds.CCXT(exchange=exchange, symbol=my_symbol, name="btc_usd_min",
                             timeframe=bt.TimeFrame.Minutes, fromdate=hist_start_date)

cerebro.adddata(data_min)

#

# Run the strategy
cerebro.run()

Ed Bartosh

@Nguyễn-Tài-Nguyên said in Anyone use backtrader to do live trading on Bitcoin exchange?:

I use below code to get data from Binance exchange but it keep running forever. Can you show me what wrong I did ? Thank you !

You probably forgot to add strategy object, no?

Nguyễn Tài Nguyên

Yes, I think so :D.
I will update the code

Søren Pallesen

@Ed-Bartosh Having a basic problem getting live feeds from Bitmex running. im doing:

# -*- coding: utf-8; py-indent-offset:4 -*-
###############################################################################
#
# Copyright (C) 2017 Ed Bartosh
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program.  If not, see <http://www.gnu.org/licenses/>.
#
###############################################################################
from __future__ import (absolute_import, division, print_function,
                        unicode_literals)

import sys
import time
from datetime import datetime, timedelta

import backtrader as bt

class TestStrategy(bt.Strategy):
    def next(self):
        for data in self.datas:
            print('*' * 5, 'NEXT:', bt.num2date(data.datetime[0]), data._name, data.open[0], data.high[0],
                  data.low[0], data.close[0], data.volume[0],
                  bt.TimeFrame.getname(data._timeframe), len(data))
            if not self.getposition(data):
                order = self.buy(data, exectype=bt.Order.Limit, size=10, price=data.close[0])
            else:
                order = self.sell(data, exectype=bt.Order.Limit, size=10, price=data.close[0])

    def notify_order(self, order):
        print('*' * 5, "NOTIFY ORDER", order)

def runstrategy(argv):
    # Create a cerebro
    cerebro = bt.Cerebro()

    hist_start_date = datetime.utcnow() - timedelta(minutes=30)
    data_min = bt.feeds.CCXT(exchange="bitmex", symbol="BTC/USD", name="btc_usd_min",
                             timeframe=bt.TimeFrame.Minutes, fromdate=hist_start_date)
    cerebro.adddata(data_min)

    # Add the strategy
    cerebro.addstrategy(TestStrategy)

    # Run the strategy
    cerebro.run()

if __name__ == '__main__':
    runstrategy(sys.argv)

and just getting:

***** NEXT: 2017-12-30 19:07:00 btc_usd_min 12975.0 12980.0 12975.0 12980.0 24132.0 Minute 1

That is one bar... i must be missing some thing basic - please help? :-)

Nguyễn Tài Nguyên

@Søren-Pallesen
you haven't set broker

Søren Pallesen

@Nguyễn-Tài-Nguyên I wanted just live feeds from Bitmex not live trading - so im trading against the default broker. So i dont believe you are correct. But thx for the input anyway

@Nguyễn-Tài-Nguyên is it working for you? If yes, how?

Søren Pallesen

@Ed-Bartosh Another simpler example with Gdax (however i specifically need to use Bitmex):

#!/usr/bin/env python
# -*- coding: utf-8; py-indent-offset:4 -*-

from __future__ import (absolute_import, division, print_function,
                        unicode_literals)
import time
from datetime import datetime, timedelta
import backtrader as bt

class TestStrategy(bt.Strategy):
    def next(self):
        print('*' * 5, 'NEXT:', bt.num2date(self.data.datetime[0]), self.data._name, self.data.open[0], self.data.high[0],
              self.data.low[0], self.data.close[0], self.data.volume[0],
              bt.TimeFrame.getname(self.data._timeframe), len(self.data))

if __name__ == '__main__':
    cerebro = bt.Cerebro()
    
    hist_start_date = datetime.utcnow() - timedelta(minutes=10)
    data_min = bt.feeds.CCXT(exchange='gdax', symbol="BTC/USD", name="btc_usd_min", fromdate=hist_start_date,
                             timeframe=bt.TimeFrame.Minutes)
    cerebro.adddata(data_min)
    cerebro.addstrategy(TestStrategy)
    cerebro.run()

This does get me 10 historical bars:

***** NEXT: 2017-12-31 12:00:00 btc_usd_min 13140.07 13140.07 13100.0 13100.0 1.2353579000000001 Minute 1
***** NEXT: 2017-12-31 12:01:00 btc_usd_min 13099.99 13100.0 13052.65 13100.0 6.11072207 Minute 2
***** NEXT: 2017-12-31 12:02:00 btc_usd_min 13100.0 13149.52 13099.99 13127.09 12.04309168 Minute 3
***** NEXT: 2017-12-31 12:03:00 btc_usd_min 13127.1 13127.1 13100.06 13100.08 1.3129460699999997 Minute 4
***** NEXT: 2017-12-31 12:04:00 btc_usd_min 13100.09 13100.09 13079.0 13079.02 1.07406138 Minute 5
***** NEXT: 2017-12-31 12:05:00 btc_usd_min 13079.02 13100.0 13079.01 13098.97 2.3448107200000003 Minute 6
***** NEXT: 2017-12-31 12:06:00 btc_usd_min 13098.98 13098.98 13098.97 13098.98 0.8174808400000002 Minute 7
***** NEXT: 2017-12-31 12:07:00 btc_usd_min 13098.98 13100.0 13098.98 13099.99 1.3718365200000002 Minute 8
***** NEXT: 2017-12-31 12:08:00 btc_usd_min 13100.0 13100.0 13099.98 13099.98 6.12858836 Minute 9
***** NEXT: 2017-12-31 12:09:00 btc_usd_min 13098.98 13098.98 13098.98 13098.98 0.01 Minute 10

But also here execution just stops and no further feed is loaded.

I see that you now have implemented a CCXT Store - is that related? Maybe you could post an updated sample strategy if there is any difference in the usage?

Thx :-) ...and Happy New Year !!!

Nguyễn Tài Nguyên

@Søren-Pallesen said in Anyone use backtrader to do live trading on Bitcoin exchange?:

@Nguyễn-Tài-Nguyên I wanted just live feeds from Bitmex not live trading - so im trading against the default broker. So i dont believe you are correct. But thx for the input anyway

@Nguyễn-Tài-Nguyên is it working for you? If yes, how?

You are right. That is my wrong :D. Sorry about that.

Nguyễn Tài Nguyên

@Søren-Pallesen When I use other exchange ( for example: binance), it only shows 1 row. How to get the full data ?

    #!/usr/bin/env python
# -*- coding: utf-8; py-indent-offset:4 -*-

from __future__ import (absolute_import, division, print_function,
						unicode_literals)
import time
from datetime import datetime, timedelta
import backtrader as bt

class TestStrategy(bt.Strategy):
	def next(self):
		print('*' * 5, 'NEXT:', bt.num2date(self.data.datetime[0]), self.data._name, self.data.open[0], self.data.high[0],
			  self.data.low[0], self.data.close[0], self.data.volume[0],
			  bt.TimeFrame.getname(self.data._timeframe), len(self.data))

if __name__ == '__main__':
	cerebro = bt.Cerebro()
	
	hist_start_date = datetime.utcnow() - timedelta(minutes=10)
	data_min = bt.feeds.CCXT(exchange='binance', symbol="BTC/USD", name="btc_usd_min", fromdate=hist_start_date,
							 timeframe=bt.TimeFrame.Minutes)
	cerebro.adddata(data_min)
	cerebro.addstrategy(TestStrategy)
	cerebro.run()

Søren Pallesen

@Nguyễn-Tài-Nguyên I have the same issue. Its a quedtion for @Ed-Bartosh that I hope can spare some time soon to help us out.