Anyone use backtrader to do live trading on Bitcoin exchange?

[sirhc]

@thatblokedave
Definitively coming to the same conclusion and probably the same solution or a solution using a smart ratelimiter.
But If we don't need those broker values intensively, we can cheat.

You can check my thought here : https://github.com/bartosh/backtrader/issues/22

Even for getposition(). A strategy can store and update his own size with orders notifications.

[monejava]

Hi everyone,

First of all thanks @Ed-Bartosh for implementing the CCXT broker.

I was wandering what is the official repo for this and if there is any plan to contribute this back to backtrader or to develop as a standalone module that people could use.

I'm starting to use it, and I'd be happy to contribute to it but so far the only thing I have found is this fork: https://github.com/bartosh/backtrader/tree/ccxt

Thanks
Simone

[Sandro Böhme]

I think the most up to date repo is this one:
https://github.com/Dave-Vallance/bt-ccxt-store

[pfederra]

Thanks @sandro-böhme , the repo that you mention seems to be the latest one and is working fine with Binance as per my testings. I have a question in regards to the order status. I do place orders correctly and can see the order status in the dashboard. However, in the notify_order() when it comes to printing exections details like :
self.log (order.executed.price,order.executed.value,order.executed.comm)

It always prints 0 for all 3 values, I am not sure why I cant see the order execution details. Does it work for you? Also, is there anyone actively looking at fixing the reported issues?

Thanks

[Noh]

Hey, Im having a problem when trying to run the following code.
I get the error: AttributeError: module 'backtrader.feeds' has no attribute 'CCXT'
I'm running the latest version of python on the latest version of ubuntu and when I check pip3 freeze, I have backtrader and ccxt installed.

import backtrader as bt
import ccxt
import pandas as pd

class Strategy():
def init(self, client, timeframe='5m'):
self.client = client
self.timeframe = timeframe

def getSignal(self):
	cerebro = bt.Cerebro()
	data = bt.feeds.CCXT(exchange='bitmex', symbol='BTC/USD', timeframe=bt.TimeFrame.Minutes, compression=1)
	cerebro.addstrategy(bt.Strategy)
	cerebro.adddata(data)
	print(data)
	cerebro.run()
	cerebro.plot(style="candlesticks")

any help would be appreciated

[pfederra]

@noh I did have a similar issue which resolved it. You need to make sure to create a new environment with correct version of the BT fork for CCXT. See below the steps I followed to fix my issue:

conda create --name New-env
source activate New-env
conda install -c anaconda git
pip install ccxt
pip install git+https://github.com/Dave-Vallance/bt-ccxt-store.git

This should resolve your issue.

[Rodrigo Brito]

I published a simple example with Binance integration: https://github.com/rodrigo-brito/backtrader-binance-bot

[pfederra]

@rodrigo-brito This looks great thanks for sharing!

[CooleRnax]

@rodrigo-brito
how do you achieve data persistence between restarts?

[Rodrigo Brito]

@coolernax I don't persist the feed or strategies positions. The feed loads a delayed history when started. About the trade position, I'm start always with no funds in crypto, the first operation should be a buy. Maybe it can be improved with a simple flag in the script.

[pfederra]

@rodrigo-brito -

1. When running your script in live mode does the notify_order() return correct order details? when i place order I am unable to get filled price back from order.executed.price,order.executed.value,order.executed.comm all the fileds returning 0. Is this something that anyone looking at fixing?
2. How stable do you think the CCXT Live broker for BT is at the moment? Do you have high confidence in it's stability? Are you running it live?
3. Also, the Dave-wallace fork seems to have a new function self.getposition() whch does not seem to work for Binance as i receive empty object where I should get a list of my open positions.

[Rodrigo Brito]

Hi, @pfederra

1. I have the same issue in notify_order(), it works only in simulations to me.
2. I think the fork has a lot of issues, but work very well for the main features, like order and feed.
3. The get positions does not work for me very well, it has a precision issue. I'm using a function to get my wallet ballance and settting a custom sizer. I'm also defining a global variable with the last operation (BUY/SELL) to control the next operation. It is a working arrond, but working in live.

[DiamondDogs95]

@rodrigo-brito
How do we proceed to test with another dataset ? (ex: from 01/01/2019 to 05/05/2019)
It seems that your scripts do not incorporate params to generate a csv file with from some date to another.

Best regards,
Diamond

[xiaoliangbuaa]

Hi there. In each loop (calling _load()), maybe more than one trade is appended in the "for trade in trades" block, but only one trade is popped, ie. trade = self.trades.popleft(). Will this cause delay? Will this datafeed be much slower than the real world?

[TheRoque]

Hi guys. I am using this CCXTbt library, and to be honnest, it's kinda hard getting it to work properly. I have a simple question: How ot have a fetch_ohlcv function work on an optimised time frame ? The thing is, I see this function running all the time, spamming the exchange with no reason. I think that instead, it should be called at a fixed time, like every minute, or every granularity, more generally speaking.

I am perfectly aware of the rateLimit parameter on the exchange, but I don't want to set a timeout between my API calls: I don't want any timeout between the time I fetch a ticker, and place an order, because a small wait can result in losing a bid or a sell.

[d416]

@theroque said in Anyone use backtrader to do live trading on Bitcoin exchange?:

Hi guys. I am using this CCXTbt library, and to be honnest, it's kinda hard getting it to work properly. I have a simple question: How ot have a fetch_ohlcv function work on an optimised time frame ? The thing is, I see this function running all the time, spamming the exchange with no reason. I think that instead, it should be called at a fixed time, like every minute, or every granularity, more generally speaking.

I am perfectly aware of the rateLimit parameter on the exchange, but I don't want to set a timeout between my API calls: I don't want any timeout between the time I fetch a ticker, and place an order, because a small wait can result in losing a bid or a sell.

Try this thread:
https://community.backtrader.com/topic/401/optimizing-with-ibstore-causes-redundant-connections-downloads/5

A similar thing happens with IB on optimization. BT has the best answer at the end.

[kaya]

Hey guys awesome work you are doing here.
Is this the best fork to start with? https://github.com/bartosh/backtrader/tree/ccxt
I'm looking to trade bitmex, polonix, btcmarkets.

[backtrader]

You probably want to use a Store, rather than a fork (which had no good reason to be). Someone took the right approach extracting the code from the fork.

• https://www.backtrader.com/recipes/storesbrokersdata/bt-ccxt-store/bt-ccxt-store.html

[pepelepew25]

Has anyone managed to use this on Bitmex ?
Using Rodrigo's code it starts ok and fetch the market data but after gets into a loop while trying to fetch open orders until I kill it ?

[  __init__] init
2019-06-20 14:16:40.232891 - fetch_open_orders - Attempt 0
[     fetch] GET https://testnet.bitmex.com/api/v1/order?filter=%7B%22open%22%3Atrue%7D, Request: {'Content-Type': 'application/json', 'api-key': 'xxx', 'api-nonce': '1561036602233', 'api-signature': '30b4be8ae34bcda5190f5cc74e21f738c8e8337459587a2a2efec238f88bb412', 'Accept-Encoding': 'gzip, deflate'} None
[_make_request] https://testnet.bitmex.com:443 "GET /api/v1/order?filter=%7B%22open%22%3Atrue%7D HTTP/1.1" 200 2
[     fetch] GET https://testnet.bitmex.com/api/v1/order?filter=%7B%22open%22%3Atrue%7D, Response: 200 {'Date': 'Thu, 20 Jun 2019 13:16:42 GMT', 'Content-Type': 'application/json; charset=utf-8', 'Content-Length': '2', 'Connection': 'keep-alive', 'X-RateLimit-Limit': '60', 'X-RateLimit-Remaining': '59', 'X-RateLimit-Reset': '1561036603', 'X-Powered-By': 'Profit', 'ETag': 'W/"2-l9Fw4VUO7kr8CvBlt4zaMCqXZ0w"', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains'} []
2019-06-20 14:16:42.288010 - fetch_open_orders - Attempt 0
[     fetch] GET https://testnet.bitmex.com/api/v1/order?filter=%7B%22open%22%3Atrue%7D, Request: {'Content-Type': 'application/json', 'api-key': 'xxx', 'api-nonce': '1561036604289', 'api-signature': '077c17b5124f62b164c568bd30315277ca7f779f862b32c4ebb06730a1e8a539', 'Accept-Encoding': 'gzip, deflate'} None
[_make_request] https://testnet.bitmex.com:443 "GET /api/v1/order?filter=%7B%22open%22%3Atrue%7D HTTP/1.1" 200 2
[     fetch] GET https://testnet.bitmex.com/api/v1/order?filter=%7B%22open%22%3Atrue%7D, Response: 200 {'Date': 'Thu, 20 Jun 2019 13:16:44 GMT', 'Content-Type': 'application/json; charset=utf-8', 'Content-Length': '2', 'Connection': 'keep-alive', 'X-RateLimit-Limit': '60', 'X-RateLimit-Remaining': '59', 'X-RateLimit-Reset': '1561036605', 'X-Powered-By': 'Profit', 'ETag': 'W/"2-l9Fw4VUO7kr8CvBlt4zaMCqXZ0w"', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains'} []
2019-06-20 14:16:44.331896 - fetch_open_orders - Attempt 0
[     fetch] GET https://testnet.bitmex.com/api/v1/order?filter=%7B%22open%22%3Atrue%7D, Request: {'Content-Type': 'application/json', 'api-key': 'xxx', 'api-nonce': '1561036606332', 'api-signature': '8bc9b7d98138af4a248631260534a61d77fef496066da3701a222c4af5a9f1af', 'Accept-Encoding': 'gzip, deflate'} None
[_make_request] https://testnet.bitmex.com:443 "GET /api/v1/order?filter=%7B%22open%22%3Atrue%7D HTTP/1.1" 200 2
[     fetch] GET https://testnet.bitmex.com/api/v1/order?filter=%7B%22open%22%3Atrue%7D, Response: 200 {'Date': 'Thu, 20 Jun 2019 13:16:46 GMT', 'Content-Type': 'application/json; charset=utf-8', 'Content-Length': '2', 'Connection': 'keep-alive', 'X-RateLimit-Limit': '60', 'X-RateLimit-Remaining': '59', 'X-RateLimit-Reset': '1561036607', 'X-Powered-By': 'Profit', 'ETag': 'W/"2-l9Fw4VUO7kr8CvBlt4zaMCqXZ0w"', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains'} []
2019-06-20 14:16:46.383414 - fetch_open_orders - Attempt 0
[     fetch] GET https://testnet.bitmex.com/api/v1/order?filter=%7B%22open%22%3Atrue%7D, Request: {'Content-Type': 'application/json', 'api-key': 'xxx', 'api-nonce': '1561036608384', 'api-signature': 'b49377a1603d3557b58c13d688226b3d6117b62bdc09c10527c629b61ec104a9', 'Accept-Encoding': 'gzip, deflate'} None
[_make_request] https://testnet.bitmex.com:443 "GET /api/v1/order?filter=%7B%22open%22%3Atrue%7D HTTP/1.1" 200 2
[     fetch] GET https://testnet.bitmex.com/api/v1/order?filter=%7B%22open%22%3Atrue%7D, Response: 200 {'Date': 'Thu, 20 Jun 2019 13:16:48 GMT', 'Content-Type': 'application/json; charset=utf-8', 'Content-Length': '2', 'Connection': 'keep-alive', 'X-RateLimit-Limit': '60', 'X-RateLimit-Remaining': '59', 'X-RateLimit-Reset': '1561036609', 'X-Powered-By': 'Profit', 'ETag': 'W/"2-l9Fw4VUO7kr8CvBlt4zaMCqXZ0w"', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains'} []
2019-06-20 14:16:48.433256 - fetch_open_orders - Attempt 0
^X^Cfinished.

[mr-m0nst3r]

@ed-bartosh
Hi Bartosh,

Thank you for the implementation of ccxt in backtrader.

I'm trying to add 1 minute data to feed, and then resample it to 2 minutes data, so I can get MACD of 1 minute and MACD of 2 minutes in the Strategy.

But I'm keeping getting the error of:

ValueError: min() arg is an empty sequence

Here's the code:

# !/usr/bin/env python
# -*- coding: utf-8; py-indent-offset:4 -*-
from __future__ import (absolute_import, division, print_function,
                        unicode_literals)
import sys
import time
from datetime import datetime, timedelta
from datetime import datetime, timedelta

import backtrader as bt
import ccxt

class TestStrategy(bt.Strategy):
    params = (
        ('printlog', True),
    )

    def log(self, txt, dt=None, doprint=False):
        ''' Logging function fot this strategy'''
        if self.params.printlog or doprint:
            dt = dt or bt.num2date(self.data.datetime[0])
            print('%s, %s' % (dt, txt))

    def start(self):
        self.counter = 0
        print('START')

    def prenext(self):
        self.counter += 1
        print('prenext len %d - counter %d' % (len(self), self.counter))

    def __init__(self):
        self.macd = bt.indicators.MACDHisto(self.datas[0])
        self.macd2 = bt.indicators.MACDHisto(self.datas[1])


    def next(self):
        self.counter += 1
        price_txt = "Counter: " + str(self.counter) + " Open/Close/High/Low/Volume: " + str(self.data0.open[0]) + " - "+ str(self.data0.close[0]) + " - " + str(self.data0.high[0]) + " - " + str(self.data0.low[0])  + " - " + str(self.data0.volume[0]) + " Len: "+ str(len(self.data0))# + " Time Frame:" + bt.TimeFrame.getname(self.data0._timeframe) + " Len: "+ str(len(self.data0))
        self.log(price_txt)
        macd_txt = "MACD: {:.2f}, Histo: {:.2f}".format(self.macd.macd[0],self.macd.histo[0])
        self.log("MACD#1: " + macd_txt)
        macd2_txt = "MACD: {:.2f}, Histo: {:.2f}".format(self.macd2.macd[0],self.macd2.histo[0])
        self.log("MACD#2: " + macd2_txt)

if __name__ == '__main__':
    cerebro = bt.Cerebro()

    #exchange = sys.argv[1] if len(sys.argv) > 1 else 'gdax'
    exchange = sys.argv[1] if len(sys.argv) > 1 else 'okex'
    symbol = sys.argv[2] if len(sys.argv) > 2 else 'ETH/USDT'

    hist_start_date = datetime.utcnow() - timedelta(minutes=10)
    print('UTC NOW: ', datetime.utcnow())
    print('hist_start_data: ', hist_start_date)
    print('Using symbol: ', symbol)

    # Create data feeds
    data_1m = bt.feeds.CCXT(exchange=exchange, symbol=symbol, name="1m",
                               timeframe=bt.TimeFrame.Minutes, fromdate=hist_start_date,compression=1)
    cerebro.adddata(data_1m)

    cerebro.resampledata(data_1m, timeframe=bt.TimeFrame.Minutes, compression=2)
    
    cerebro.addstrategy(TestStrategy)
    cerebro.run()

The output is:

python stage1.py
UTC NOW:  2019-06-21 05:41:48.784742
hist_start_data:  2019-06-21 05:31:48.784520
Using symbol:  ETH/USDT
START
prenext len 1 - counter 1
prenext len 2 - counter 2
prenext len 3 - counter 3
prenext len 4 - counter 4
prenext len 5 - counter 5
prenext len 6 - counter 6
prenext len 7 - counter 7
prenext len 8 - counter 8
prenext len 9 - counter 9
prenext len 10 - counter 10
prenext len 11 - counter 11
prenext len 12 - counter 12
Traceback (most recent call last):
  File "stage1.py", line 66, in <module>
    cerebro.run()
  File "/Users/michael/.venv/backtrader-ccxt/lib/python3.6/site-packages/backtrader/cerebro.py", line 1127, in run
    runstrat = self.runstrategies(iterstrat)
  File "/Users/michael/.venv/backtrader-ccxt/lib/python3.6/site-packages/backtrader/cerebro.py", line 1298, in runstrategies
    self._runnext(runstrats)
  File "/Users/michael/.venv/backtrader-ccxt/lib/python3.6/site-packages/backtrader/cerebro.py", line 1557, in _runnext
    dt0 = min((d for i, d in enumerate(dts)
ValueError: min() arg is an empty sequence

What am I doing wrong here?

Thank you.