For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

Anyone use backtrader to do live trading on Bitcoin exchange?



  • I tried to run for data on 2hr timeframe with below code:

    data = bt.feeds.CCXT(exchange='bitmex',
                         symbol='BTC/USD',
                         timeframe=bt.TimeFrame.Minutes,
                         fromdate = datetime(2018,2,10),
                         todate = datetime(2018,4,30),
                        )
    cerebro.resampledata(data, timeframe=bt.TimeFrame.Minutes, compression=120)
    

    But got this error:

    ValueError: 'BitMEX' exchange doesn't support fetching OHLCV data for 2h time frame
    

    Also when I rather download 1 Min into a CSV file and then resample, it works fine. Can we not run strategies on timeframes not supported by the exchange?



  • I tried to run for data on 2hr timeframe with below code:

    data = bt.feeds.CCXT(exchange='bitmex',
                         symbol='BTC/USD',
                         timeframe=bt.TimeFrame.Minutes,
                         fromdate = datetime(2018,2,10),
                         todate = datetime(2018,4,30),
                        )
    cerebro.resampledata(data, timeframe=bt.TimeFrame.Minutes, compression=120)
    

    But got this error:

    ValueError: 'BitMEX' exchange doesn't support fetching OHLCV data for 2h time frame
    

    Rather when I download 1 Min into a CSV file and then resample, it works fine. Can we not run strategies on timeframes not supported by the exchange?

    Also how do we calculate the pnl for inverse contracts like the above?



  • @3pm said in [Anyone use backtrader to do live trading on Bitcoin exchange?]I

    import ccxt which I imported on line 3 of my script; e.i.

    import backtrader as bt
    #import ccxt
    from datetime import datetime
    

    Doing this and running the script gives the following error:

    <ipython-input-1-c57c0f04533a> in <module>()
         30 #Get  data
         31 symbol = 'BTC/USDT'
    ---> 32 data = bt.feeds.CCXT(exchange='bittrex',
         33                      config={'rateLimit': 2000},
         34                      symbol=symbol,
    
    AttributeError: module 'backtrader.feeds' has no attribute 'CCXT'
    

    Uncommenting the import makes everything work again...

    Thanks for the detailed info! I'll try to reproduce it and fix.



  • Hey I am having an issue with dynamic datafeeds, I am not able to pin it on the ccxt branch or backtrader itself, here is the code:

    class TestStrategy(bt.Strategy):
    
       def start(self):
           self.counter = 0
    
           print('START')
    
       def next(self):
           try:
               self.counter +=1
               print(self.data0.open[0],self.data1.open[0])
               print("lalalal")
           except:
               traceback.print_exc()
               pass
           #sio.emit('logs', 'fries', namespace='/test', broadcast=True)
    
    
    
    #io.on('connect', namespace='/test')
    #ARBY = TestStrategy
    
    
    def d(sid, environ):
       print('connect + thats sid', sid)
    
    
    #@sio.on('arbitrage', namespace='/test')
    def check_oppor(sid, DATA):
       #sio.emit('logs', 'STARTING BOT...', namespace='/test', broadcast=True)
       print(DATA['API_KEY'], DATA['API_SECRET'], DATA['EXCHANGES'])
       global addD, datas
       global broker, makers, takers, withdrawls, configs, exchanges, keys, secrets
       exchanges = DATA['EXCHANGES']
       keys = DATA['API_KEY']
       secrets = DATA['API_SECRET']
       exchanges = [item for items in exchanges for item in items.split(",")]
       keys = [item for items in keys for item in items.split(",")]
       secrets = [item for items in secrets for item in items.split(",")]
    
    
    
       for key, secret in zip(keys, secrets):
           config = {
               'apiKey': key,
               'secret': secret,
               'nonce': lambda: str(int(time.time() * 1000))
           }
           configs.append(config)
    
       datas = []
       cerebro = bt.Cerebro()
    
       hist_start_date = datetime.utcnow() - timedelta(minutes=3)
    
       indicators = []
       print(len(configs))
       d = exchanges
    
       for index, c in zip(range(len(configs)), configs):
           try:
               print(index, "this is the index")
               Obj = getattr(ccxt, exchanges[index])
               Obj = Obj(c)
    
               indicators.append(Obj)
    
               data = bt.feeds.CCXT(exchange=exchanges[index], config=c,
                                symbol='ETH/BTC',
                                timeframe=bt.TimeFrame.Minutes,
                                fromdate=hist_start_date, compression=1, ohlcv_limit=999,retries=100)
    
               data1 = bt.feeds.CCXT(exchange=exchanges[index], config=c,
                                 symbol='BCH/BTC',
                                 timeframe=bt.TimeFrame.Minutes,
                                 fromdate=hist_start_date, compression=1, ohlcv_limit=999,retries=100)
    
               data2 = bt.feeds.CCXT(exchange=exchanges[index], config=c,
                                 symbol='LTC/BTC',
                                 timeframe=bt.TimeFrame.Minutes,
                                 fromdate=hist_start_date, compression=1, ohlcv_limit=999,retries=100)
    
    
               datas.append([data,data1,data2])
               print(datas)
           except:
               traceback.print_exc()
               pass
       print (datas)
       allDatas = [elem for sublist in datas for elem in sublist]
       print (allDatas)
       for i in indicators:
           fees = i.describe()['fees']
           maker = fees['trading']['maker']
           taker = fees['trading']['taker']
           makers.append(maker)
           takers.append(taker)
    
           withdrawal = fees['funding']['withdraw']
           withdrawls.append(withdrawal)
           print(withdrawal)
    
       broker = bt.brokers.CCXTBroker(exchange='kucoin',
                                      currency='BTC', config=configs[1])
    
       addD = cerebro.adddata
       print (allDatas)
       for ind, d in enumerate(allDatas, start=0):
           try:
               addD(d, name="data" + str(ind))
               print(ind)
           except:
               traceback.print_exc()
               pass
    
       print(allDatas)
    
       cerebro.addstrategy(TestStrategy)
       cerebro.run()
    

    However, if we only add one datafeed per iteration, it works fine:

    class TestStrategy(bt.Strategy):
    
       def start(self):
           self.counter = 0
    
           print('START')
    
       def next(self):
           try:
               self.counter +=1
               print(self.data0.open[0],self.data1.open[0])
               print("lalalal")
           except:
               traceback.print_exc()
               pass
           #sio.emit('logs', 'fries', namespace='/test', broadcast=True)
    
    
    
    #io.on('connect', namespace='/test')
    #ARBY = TestStrategy
    
    
    def d(sid, environ):
       print('connect + thats sid', sid)
    
    
    #@sio.on('arbitrage', namespace='/test')
    def check_oppor(sid, DATA):
       #sio.emit('logs', 'STARTING BOT...', namespace='/test', broadcast=True)
       print(DATA['API_KEY'], DATA['API_SECRET'], DATA['EXCHANGES'])
       global addD, datas
       global broker, makers, takers, withdrawls, configs, exchanges, keys, secrets
       exchanges = DATA['EXCHANGES']
       keys = DATA['API_KEY']
       secrets = DATA['API_SECRET']
       exchanges = [item for items in exchanges for item in items.split(",")]
       keys = [item for items in keys for item in items.split(",")]
       secrets = [item for items in secrets for item in items.split(",")]
    
    
    
       for key, secret in zip(keys, secrets):
           config = {
               'apiKey': key,
               'secret': secret,
               'nonce': lambda: str(int(time.time() * 1000))
           }
           configs.append(config)
    
       datas = []
       cerebro = bt.Cerebro()
    
       hist_start_date = datetime.utcnow() - timedelta(minutes=3)
    
       indicators = []
       print(len(configs))
       d = exchanges
    
       for index, c in zip(range(len(configs)), configs):
           try:
               print(index, "this is the index")
               Obj = getattr(ccxt, exchanges[index])
               Obj = Obj(c)
    
               indicators.append(Obj)
    
               data = bt.feeds.CCXT(exchange=exchanges[index], config=c,
                                symbol='ETH/BTC',
                                timeframe=bt.TimeFrame.Minutes,
                                fromdate=hist_start_date, compression=1, ohlcv_limit=999,retries=100)
    
               data1 = bt.feeds.CCXT(exchange=exchanges[index], config=c,
                                 symbol='BCH/BTC',
                                 timeframe=bt.TimeFrame.Minutes,
                                 fromdate=hist_start_date, compression=1, ohlcv_limit=999,retries=100)
    
               data2 = bt.feeds.CCXT(exchange=exchanges[index], config=c,
                                 symbol='LTC/BTC',
                                 timeframe=bt.TimeFrame.Minutes,
                                 fromdate=hist_start_date, compression=1, ohlcv_limit=999,retries=100)
    
    
               datas.append([data])
               print(datas)
           except:
               traceback.print_exc()
               pass
       print (datas)
       allDatas = [elem for sublist in datas for elem in sublist]
       print (allDatas)
       for i in indicators:
           fees = i.describe()['fees']
           maker = fees['trading']['maker']
           taker = fees['trading']['taker']
           makers.append(maker)
           takers.append(taker)
    
           withdrawal = fees['funding']['withdraw']
           withdrawls.append(withdrawal)
           print(withdrawal)
    
       broker = bt.brokers.CCXTBroker(exchange='kucoin',
                                      currency='BTC', config=configs[1])
    
       addD = cerebro.adddata
       print (allDatas)
       for ind, d in enumerate(allDatas, start=0):
           try:
               addD(d, name="data" + str(ind))
               print(ind)
           except:
               traceback.print_exc()
               pass
    
       print(allDatas)
    
       cerebro.addstrategy(TestStrategy)
       cerebro.run()
    


  • @jogus said in Anyone use backtrader to do live trading on Bitcoin exchange?:

    Hi guys, first..u´ve done an amazing job with this project! I know that it´s still under development but u guys are doing great results.. it is fantastic to see how everybody can interact and give his help in order to improve this tool.

    I´m new in this community, I found out this awesome tool a few days ago. So, hello and thanks!

    I have the following problem when I was trying to set up everything and replicate the code that I saw in the last posts:

     [pylint] E1123:Unexpected keyword argument 'timeframe' in constructor call
     [pylint] E1123:Unexpected keyword argument 'fromdate' in constructor call
     [pylint] E1123:Unexpected keyword argument 'todate' in constructor call
     bt: backtrader
    

    I don´t know what to do with this..maybe I´ve cloned badly the package, I don´t know..Can u help me?

     data = bt.feeds.CCXT(exchange='bittrex',
                          symbol=symbol,
                          timeframe=bt.TimeFrame.Minutes,
                          fromdate = datetime.datetime(2018,1,1),
                          todate = datetime(2018,4,1)
                         )
    

    Anyone can help with this issue? Another question..what kind of data admits cerebro to operate the strategy I mean what kind of data needs on "adddata" function...because maybe I can use ccxt directly instead of using bt.feeds instance. Thanks



  • @siv said in Anyone use backtrader to do live trading on Bitcoin exchange?:

    Hey I am having an issue with dynamic datafeeds, I am not able to pin it on the ccxt branch or backtrader itself, here is the code:

    class TestStrategy(bt.Strategy):
    
       def start(self):
           self.counter = 0
    
           print('START')
    
       def next(self):
           try:
               self.counter +=1
               print(self.data0.open[0],self.data1.open[0])
               print("lalalal")
           except:
               traceback.print_exc()
               pass
           #sio.emit('logs', 'fries', namespace='/test', broadcast=True)
    
    
    
    #io.on('connect', namespace='/test')
    #ARBY = TestStrategy
    
    
    def d(sid, environ):
       print('connect + thats sid', sid)
    
    
    #@sio.on('arbitrage', namespace='/test')
    def check_oppor(sid, DATA):
       #sio.emit('logs', 'STARTING BOT...', namespace='/test', broadcast=True)
       print(DATA['API_KEY'], DATA['API_SECRET'], DATA['EXCHANGES'])
       global addD, datas
       global broker, makers, takers, withdrawls, configs, exchanges, keys, secrets
       exchanges = DATA['EXCHANGES']
       keys = DATA['API_KEY']
       secrets = DATA['API_SECRET']
       exchanges = [item for items in exchanges for item in items.split(",")]
       keys = [item for items in keys for item in items.split(",")]
       secrets = [item for items in secrets for item in items.split(",")]
    
    
    
       for key, secret in zip(keys, secrets):
           config = {
               'apiKey': key,
               'secret': secret,
               'nonce': lambda: str(int(time.time() * 1000))
           }
           configs.append(config)
    
       datas = []
       cerebro = bt.Cerebro()
    
       hist_start_date = datetime.utcnow() - timedelta(minutes=3)
    
       indicators = []
       print(len(configs))
       d = exchanges
    
       for index, c in zip(range(len(configs)), configs):
           try:
               print(index, "this is the index")
               Obj = getattr(ccxt, exchanges[index])
               Obj = Obj(c)
    
               indicators.append(Obj)
    
               data = bt.feeds.CCXT(exchange=exchanges[index], config=c,
                                symbol='ETH/BTC',
                                timeframe=bt.TimeFrame.Minutes,
                                fromdate=hist_start_date, compression=1, ohlcv_limit=999,retries=100)
    
               data1 = bt.feeds.CCXT(exchange=exchanges[index], config=c,
                                 symbol='BCH/BTC',
                                 timeframe=bt.TimeFrame.Minutes,
                                 fromdate=hist_start_date, compression=1, ohlcv_limit=999,retries=100)
    
               data2 = bt.feeds.CCXT(exchange=exchanges[index], config=c,
                                 symbol='LTC/BTC',
                                 timeframe=bt.TimeFrame.Minutes,
                                 fromdate=hist_start_date, compression=1, ohlcv_limit=999,retries=100)
    
    
               datas.append([data,data1,data2])
               print(datas)
           except:
               traceback.print_exc()
               pass
       print (datas)
       allDatas = [elem for sublist in datas for elem in sublist]
       print (allDatas)
       for i in indicators:
           fees = i.describe()['fees']
           maker = fees['trading']['maker']
           taker = fees['trading']['taker']
           makers.append(maker)
           takers.append(taker)
    
           withdrawal = fees['funding']['withdraw']
           withdrawls.append(withdrawal)
           print(withdrawal)
    
       broker = bt.brokers.CCXTBroker(exchange='kucoin',
                                      currency='BTC', config=configs[1])
    
       addD = cerebro.adddata
       print (allDatas)
       for ind, d in enumerate(allDatas, start=0):
           try:
               addD(d, name="data" + str(ind))
               print(ind)
           except:
               traceback.print_exc()
               pass
    
       print(allDatas)
    
       cerebro.addstrategy(TestStrategy)
       cerebro.run()
    

    However, if we only add one datafeed per iteration, it works fine:

    class TestStrategy(bt.Strategy):
    
       def start(self):
           self.counter = 0
    
           print('START')
    
       def next(self):
           try:
               self.counter +=1
               print(self.data0.open[0],self.data1.open[0])
               print("lalalal")
           except:
               traceback.print_exc()
               pass
           #sio.emit('logs', 'fries', namespace='/test', broadcast=True)
    
    
    
    #io.on('connect', namespace='/test')
    #ARBY = TestStrategy
    
    
    def d(sid, environ):
       print('connect + thats sid', sid)
    
    
    #@sio.on('arbitrage', namespace='/test')
    def check_oppor(sid, DATA):
       #sio.emit('logs', 'STARTING BOT...', namespace='/test', broadcast=True)
       print(DATA['API_KEY'], DATA['API_SECRET'], DATA['EXCHANGES'])
       global addD, datas
       global broker, makers, takers, withdrawls, configs, exchanges, keys, secrets
       exchanges = DATA['EXCHANGES']
       keys = DATA['API_KEY']
       secrets = DATA['API_SECRET']
       exchanges = [item for items in exchanges for item in items.split(",")]
       keys = [item for items in keys for item in items.split(",")]
       secrets = [item for items in secrets for item in items.split(",")]
    
    
    
       for key, secret in zip(keys, secrets):
           config = {
               'apiKey': key,
               'secret': secret,
               'nonce': lambda: str(int(time.time() * 1000))
           }
           configs.append(config)
    
       datas = []
       cerebro = bt.Cerebro()
    
       hist_start_date = datetime.utcnow() - timedelta(minutes=3)
    
       indicators = []
       print(len(configs))
       d = exchanges
    
       for index, c in zip(range(len(configs)), configs):
           try:
               print(index, "this is the index")
               Obj = getattr(ccxt, exchanges[index])
               Obj = Obj(c)
    
               indicators.append(Obj)
    
               data = bt.feeds.CCXT(exchange=exchanges[index], config=c,
                                symbol='ETH/BTC',
                                timeframe=bt.TimeFrame.Minutes,
                                fromdate=hist_start_date, compression=1, ohlcv_limit=999,retries=100)
    
               data1 = bt.feeds.CCXT(exchange=exchanges[index], config=c,
                                 symbol='BCH/BTC',
                                 timeframe=bt.TimeFrame.Minutes,
                                 fromdate=hist_start_date, compression=1, ohlcv_limit=999,retries=100)
    
               data2 = bt.feeds.CCXT(exchange=exchanges[index], config=c,
                                 symbol='LTC/BTC',
                                 timeframe=bt.TimeFrame.Minutes,
                                 fromdate=hist_start_date, compression=1, ohlcv_limit=999,retries=100)
    
    
               datas.append([data])
               print(datas)
           except:
               traceback.print_exc()
               pass
       print (datas)
       allDatas = [elem for sublist in datas for elem in sublist]
       print (allDatas)
       for i in indicators:
           fees = i.describe()['fees']
           maker = fees['trading']['maker']
           taker = fees['trading']['taker']
           makers.append(maker)
           takers.append(taker)
    
           withdrawal = fees['funding']['withdraw']
           withdrawls.append(withdrawal)
           print(withdrawal)
    
       broker = bt.brokers.CCXTBroker(exchange='kucoin',
                                      currency='BTC', config=configs[1])
    
       addD = cerebro.adddata
       print (allDatas)
       for ind, d in enumerate(allDatas, start=0):
           try:
               addD(d, name="data" + str(ind))
               print(ind)
           except:
               traceback.print_exc()
               pass
    
       print(allDatas)
    
       cerebro.addstrategy(TestStrategy)
       cerebro.run()
    

    Having this same issue. Was able to pin it on the ccxt branch after replacing the CCXT feeds with generic csvs in the same spot and it worked. I will try to debug further but any ideas @Ed-Bartosh



  • Update: @Ed-Bartosh, seem to have focused it down to this function in the feeds/ccxt.py. Added the print for debugging. This could use better error-handling no? Anyways, I will likely be able to solve from here and will submit a PR if applicable.

    def _load_ohlcv(self):
            try:
                ohlcv = self._data.popleft()
            except IndexError:
                print('error loading ohlcv')
                return None # no data in the queue
    
            tstamp, open_, high, low, close, volume = ohlcv
    
            dtime = datetime.utcfromtimestamp(tstamp // 1000)
    
            self.lines.datetime[0] = bt.date2num(dtime)
            self.lines.open[0] = open_
            self.lines.high[0] = high
            self.lines.low[0] = low
            self.lines.close[0] = close
            self.lines.volume[0] = volume
    
            return True
    

    EDIT: seems issue is actually coming from fetch_ohlcv() with the error originating in _load_ohlcv().



  • @fivo getting the exact results. Did you find the solution?



  • @imartynov @siv I don't quite understand the issue. Can you sketch some sample code for me to run?



  • Hi,

    Latest project update:

    • rebased on top of the latest backtrader release 1.9.64.122
    • briefly tested with the latest ccxt
    • implemented get_value and get_cash methods for CCXTBroker

    Please, install it from ccxt branch of my github repo:

    pip install pip install --upgrade git+https://github.com/bartosh/backtrader.git@ccxt
    

    and install latest ccxt :

    pip install ccxt
    

    PS: Tried to reproduce issues mentioned here, but failed. Please, provide the code that I can run to see the issue if you want it to be fixed. Thanks.

    Regards,
    Ed



  • Hi,

    When using CCXT as data feeds, how to let CCXT to fetch_ohlcv for different contract_type, for example, 'next_week', 'quarter'? Thanks.

    data_ticks = bt.feeds.CCXT(
        exchange='okex',
        symbol='LTC/USD',
        name='LTC_USD',
        timeframe=bt.TimeFrame.Minutes,
        fromdate=hist_start_date,
        historical='False',
        compression=1)


  • I add a CCXTBroker, data feed from CCXT, the code is like below:

    broker_config = {
                    #'verbose': True,
                    'apiKey': 'xxxxx',
                    'secret': 'xxxxx',
                    }
    broker = bt.brokers.CCXTBroker(exchange='okex', currency='USD', config=broker_config)
    cerebro.setbroker(broker)
    
    hist_start_date = datetime.utcnow() - timedelta(minutes=15)
    data_ticks = bt.feeds.CCXT(
        exchange='okex',
        symbol='LTC/USD',
        name='LTC_USD',
        timeframe=bt.TimeFrame.Minutes,
        fromdate=hist_start_date,
        historical='False',
        compression=1)
    
    # Add the Data Feed to Cerebro
    cerebro.adddata(data_ticks)
    

    It reports the error below:

    Traceback (most recent call last):
    File "d:/My Documents/Python Scripts/liveStrategyEngine/bt_ccxt_test2.py", line 217, in <module>
    cerebro.run()
    File "D:\Continuum\anaconda3\lib\site-packages\backtrader\cerebro.py", line 1127, in run
    runstrat = self.runstrategies(iterstrat)
    File "D:\Continuum\anaconda3\lib\site-packages\backtrader\cerebro.py", line 1315, in runstrategies
    self.stop_writers(runstrats)
    File "D:\Continuum\anaconda3\lib\site-packages\backtrader\cerebro.py", line 1347, in stop_writers
    stratinfos[stname] = strat.getwriterinfo()
    File "D:\Continuum\anaconda3\lib\site-packages\backtrader\strategy.py", line 450, in getwriterinfo
    ainfo.Value.Begin = self.broker.startingcash
    AttributeError: 'CCXTBroker' object has no attribute 'startingcash'.

    What's going on here?



  • @Ed-Bartosh first of all great work on creating this bridge between ccxt and backtrader, really awesome!!! hope to see it merge to the main backtrader stream.

    I'm familiar with ccxt project and was looking at the your implemented code.
    I have a question about the ccxtstore.

    def fetch_trades(self, symbol):
        return self.exchange.fetch_trades(symbol)
    

    you used 'fetch_trades' which is public. I would have used 'fetch_my_trades' which is the private version. with the private version you can connect between your orders and your filled trades. were as the public version holds (in most exchanges) general trade info

    does it make sense or i'm missing something?



  • ccxt is not a normal broker in a sense that we can operate on more then one exchange. we are basically working here with broker of brokers.

    I haven't tested it yet but i'm curious of how it works when we want to use more than one exchange at the same time. like in arbitrage strategy for example

    can backtrader handle multi-broker scenario?



  • @jacob I've thought about the same problem, and I don't think it can do it 。
    sadly~ hope I am wrong



  • @jacob Thank you!

    I would have used 'fetch_my_trades' which is the private version.

    I prefer using public APIs. There is no guarantee that private APIs don't disappear or change any time.

    can backtrader handle multi-broker scenario?

    To my limited knowledge backtrader doesn't support this. I be happy to be wrong here.



  • @Ed-Bartosh Thank you for the great job!
    Could you please tell me whether this project can support the real trade with ccxt now?



  • @jianlong-zhu I simply don't know as I'm not trading crypto. I guess it depends on exchange. You can see more info in this thread.



  • @ed-bartosh said in Anyone use backtrader to do live trading on Bitcoin exchange?:

    I would have used 'fetch_my_trades' which is the private version.

    I prefer using public APIs. There is no guarantee that private APIs don't disappear or change any time.

    The public version is very different than the private one. it is not just the matter of 'filtering' your own trades.
    only in the private version there is a guarantee of connecting between the orders you made and the trades that happened.
    The public version can't hold orders id as this is private information (at least on most exchanges). from my experience, using the public version can cause false trade identification.

    I'm not sure why you think those private API can disappear as they are valuable information for trading. but you can check 'has' for if 'fetchMyTrades' is true and if not use the public version



  • @ed-bartosh said in Anyone use backtrader to do live trading on Bitcoin exchange?:

    can backtrader handle multi-broker scenario?

    To my limited knowledge backtrader doesn't support this. I be happy to be wrong here.

    multi-broker is not supported.
    but what if we change the store.exchange at runtime?


Log in to reply
 

});