Navigation

    Backtrader Community

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

    Anyone use backtrader to do live trading on Bitcoin exchange?

    General Discussion
    pairs trading crypto
    79
    325
    143058
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • E
      Ed Bartosh @sfkiwi last edited by

      @sfkiwi yes, that issue was reported some time ago and there are 2 PRs about it: https://github.com/bartosh/backtrader/pull/13 and https://github.com/bartosh/backtrader/pull/6

      Unfortunately they're both outdated and require rebase. Can you check if any of them fixes the issue for you?

      1 Reply Last reply Reply Quote 0
      • Jacob
        Jacob @Ed Bartosh last edited by

        @ed-bartosh

        Currently backfill_from is not supported. I'll be happy to see this implemented. I hope it's not that hard to implement considering working example from ibdata.py.

        will add it to my backlog developments :)

        can you maybe open the option for 'issues' in your repo. I have a couple more ideas for developments and maybe other people also would like to participate. ccxt topic is very active and using this one super long thread is messy

        E 1 Reply Last reply Reply Quote 0
        • E
          Ed Bartosh @Jacob last edited by

          @jacob > can you maybe open the option for 'issues' in your repo.

          done

          1 Reply Last reply Reply Quote 0
          • T
            TOTORO last edited by

            Has anyone managed to fix sync/ gathering partial candle data issue mentioned above?
            If the solution is here https://github.com/bartosh/backtrader/pull/6, why cannot we merge ? (github noob here, sorry :p). Btw, thank you @Ed-Bartosh for this fabulous work :)

            E T 2 Replies Last reply Reply Quote 0
            • dedeco
              dedeco @Ed Bartosh last edited by

              Hi @ed-bartosh!

              I tried to run on bitmex exchange live:

              data = bt.feeds.CCXT(exchange='bitmex', symbol='BTC/USD', timeframe=bt.TimeFrame.Ticks, compression=1)

              When I tried to run live using Bitmex exchange I got some errors:
              0_1535874007208_Screen Shot 2018-09-02 at 04.37.15.png

              I open a issue (#20) and fixed on my repo fork.

              Cloud you accept a pull request? My user is @dedeco on github. If not, the fix is on issue description.

              Thanks!!

              E 1 Reply Last reply Reply Quote 0
              • E
                Ed Bartosh @dedeco last edited by

                @dedeco > Cloud you accept a pull request?

                Absolutely.

                BTW, You don't need to ask permissions for this, just submit PR. In worse case scenario it'll be rejected :)

                1 Reply Last reply Reply Quote 0
                • E
                  Ed Bartosh @TOTORO last edited by

                  @totoro said in Anyone use backtrader to do live trading on Bitcoin exchange?:

                  if the solution is here https://github.com/bartosh/backtrader/pull/6, why cannot we merge ?

                  I believe comments in that PR explain why it's not merged.

                  Btw, thank you @Ed-Bartosh for this fabulous work :)

                  You're welcome. It's actually a community work, not just mine. Many people contributed to it.
                  BTW, you have chance to do it too :)

                  1 Reply Last reply Reply Quote 0
                  • P
                    petens last edited by

                    Hello all, and thanks @Ed-Bartosh and others for your phenomenal work.

                    I am having a problem trading using backtrader on the Bittrex exchange (I have not tried other exchanges yet). The code below, slightly modified from Soren Pallesen's example 01-09-2018, places a limit order far from the current price to simply check the trading functionality:

                    from __future__ import (absolute_import, division, print_function,
                                            unicode_literals)
                    import sys
                    import time
                    from datetime import datetime, timedelta
                    
                    import backtrader as bt
                    import ccxt
                    
                    class TestStrategy(bt.Strategy):
                    
                        def start(self):
                            self.counter = 0
                            print('START')
                    
                        def prenext(self):
                            self.counter += 1
                            print('prenext len %d - counter %d' % (len(self), self.counter))
                    
                        def __init__(self):
                                self.dmi = bt.indicators.DMI(self.data, period=3)
                                self.order = None
                    
                        def next(self):
                            print('------ next len %d - counter %d' % (len(self), self.counter))
                    
                            print(self.dmi.plusDI[0], self.dmi.minusDI[0], self.dmi.adx[0])
                    
                            self.counter += 1
                    
                            print('*' * 5, 'NEXT:', bt.num2date(self.data0.datetime[0]),
                                  self.data0._name, self.data0.open[0], self.data0.high[0],
                                  self.data0.low[0], self.data0.close[0], self.data0.volume[0],
                                  bt.TimeFrame.getname(self.data0._timeframe), len(self.data0))
                    
                            if not self.getposition(data) and self.order==None:
                                print('\nBUYING')
                                self.order = self.buy(data, exectype=bt.Order.Limit, size=0.5,
                                                      price=data.close[0]*0.5)
                                print(self.order)
                            else:
                                print('\nSELLING')
                                self.order = self.sell(data, exectype=bt.Order.Limit, size=0.5,
                                                       price=data.close[0]*1.5)
                                print(self.order)
                    
                    
                        def notify_order(self, order):
                            print('*' * 5, "NOTIFY ORDER", order)
                    
                    if __name__ == '__main__':
                        cerebro = bt.Cerebro()
                    
                        exchange = 'bittrex'
                        symbol = 'NEO/BTC'
                    
                        # Create broker
                        broker_config = {'apiKey': 'xxxxxxxxxx',
                                         'secret': 'xxxxxxxxxx',
                                         'nonce': lambda: str(int(time.time() * 1000))
                                        }
                        broker = bt.brokers.CCXTBroker(exchange='bittrex',
                                                       currency='BTC',
                                                       config=broker_config)
                        cerebro.setbroker(broker)
                    
                        # Create data feed
                        hist_start_date = datetime.utcnow() - timedelta(minutes=30)
                        data = bt.feeds.CCXT(exchange=exchange,
                                             symbol=symbol,
                                             timeframe=bt.TimeFrame.Minutes,
                                             fromdate=hist_start_date,
                                             compression=1)
                    
                        cerebro.adddata(data)
                        cerebro.addstrategy(TestStrategy)
                        cerebro.run()
                    

                    This successfully places an order on the exchange. However, the script crashes immediately afterward producing the following output:

                    START
                    prenext len 1 - counter 1
                    prenext len 2 - counter 2
                    prenext len 3 - counter 3
                    prenext len 4 - counter 4
                    prenext len 5 - counter 5
                    ------ next len 6 - counter 5
                    45.64666519054272 24.016532585430657 32.95714573240973
                    ***** NEXT: 2018-09-03 15:40:00  0.00310706 0.00310706 0.00310706 0.00310706 0.5496506 Minute 6
                    
                    BUYING
                    Traceback (most recent call last):
                      File "C:/Users/pns21/Documents/Python Files/Beginner and Tutuorial files/Backtrader - Bartosh/Peter Test 2018.09.03.py", line 83, in <module>
                        cerebro.run()
                      File "C:\Users\pns21\AppData\Local\Programs\Python\Python36\lib\site-packages\backtrader\cerebro.py", line 1127, in run
                        runstrat = self.runstrategies(iterstrat)
                      File "C:\Users\pns21\AppData\Local\Programs\Python\Python36\lib\site-packages\backtrader\cerebro.py", line 1298, in runstrategies
                        self._runnext(runstrats)
                      File "C:\Users\pns21\AppData\Local\Programs\Python\Python36\lib\site-packages\backtrader\cerebro.py", line 1630, in _runnext
                        strat._next()
                      File "C:\Users\pns21\AppData\Local\Programs\Python\Python36\lib\site-packages\backtrader\strategy.py", line 325, in _next
                        super(Strategy, self)._next()
                      File "C:\Users\pns21\AppData\Local\Programs\Python\Python36\lib\site-packages\backtrader\lineiterator.py", line 268, in _next
                        self.nextstart()  # only called for the 1st value
                      File "C:\Users\pns21\AppData\Local\Programs\Python\Python36\lib\site-packages\backtrader\lineiterator.py", line 342, in nextstart
                        self.next()
                      File "C:/Users/pns21/Documents/Python Files/Beginner and Tutuorial files/Backtrader - Bartosh/Peter Test 2018.09.03.py", line 45, in next
                        price=data.close[0]*0.5)
                      File "C:\Users\pns21\AppData\Local\Programs\Python\Python36\lib\site-packages\backtrader\strategy.py", line 917, in buy
                        **kwargs)
                      File "C:\Users\pns21\AppData\Local\Programs\Python\Python36\lib\site-packages\backtrader\brokers\ccxtbroker.py", line 86, in buy
                        return self._submit(owner, data, exectype, 'buy', size, price, kwargs)
                      File "C:\Users\pns21\AppData\Local\Programs\Python\Python36\lib\site-packages\backtrader\brokers\ccxtbroker.py", line 78, in _submit
                        order = CCXTOrder(owner, data, _order)
                      File "C:\Users\pns21\AppData\Local\Programs\Python\Python36\lib\site-packages\backtrader\metabase.py", line 88, in __call__
                        _obj, args, kwargs = cls.doinit(_obj, *args, **kwargs)
                      File "C:\Users\pns21\AppData\Local\Programs\Python\Python36\lib\site-packages\backtrader\metabase.py", line 78, in doinit
                        _obj.__init__(*args, **kwargs)
                      File "C:\Users\pns21\AppData\Local\Programs\Python\Python36\lib\site-packages\backtrader\stores\ccxtstore.py", line 39, in __init__
                        self.size = float(ccxt_order['amount'])
                    TypeError: float() argument must be a string or a number, not 'NoneType'
                    

                    Thanks in advance for any help solving this issue!!

                    E 1 Reply Last reply Reply Quote 0
                    • E
                      Ed Bartosh @petens last edited by

                      @petens said in Anyone use backtrader to do live trading on Bitcoin exchange?:

                      size=0.5

                      Does it work with size=1 ?

                      P 1 Reply Last reply Reply Quote 0
                      • P
                        petens @Ed Bartosh last edited by

                        @ed-bartosh sadly it has the same result. The minimum order size on the bittrex exchange is 50,000 Satoshi or 0.0005 BTC, and either amount is over that for this symbol. Regardless the limit order is being successfully placed on the exchange. My inkling is that processing of the return values is causing this error, but i'm not experienced enough of a programmer to tell.

                        P E 2 Replies Last reply Reply Quote 0
                        • P
                          petens @petens last edited by

                          @Ed-Bartosh and others, I did some digging and I think I found the cause: the Bittrex API doesn't return any amount info.
                          In the process of making an order ccxtstore.py calls the ccxt method "create_order" (at line 119 in the current version) :

                          order = self.exchange.create_order(symbol=symbol, type=order_type, side=side,
                                                                     amount=amount, price=price, params=params)
                          

                          This works and places an order on the exchange, and returns a dictionary to "order" like this:

                          print('order:', order)
                          
                          order: {'info': {'success': True, 'message': '', 'result': {'uuid': 'd9765e24-5d48-47b2-a343-4d487e679040'}}, 'id': 'd9765e24-5d48-47b2-a343-4d487e679040', 'symbol': 'NEO/BTC', 'type': 'limit', 'side': 'buy', 'status': 'open'}
                          

                          This is parsed with the ccxt method "parse_order" in the next command and returns a dict:

                          parsed_order = self.exchange.parse_order(order['info'])
                          print('parsed_order:', parsed_order)
                          
                          parsed_order: {'info': {'success': True, 'message': '', 'result': {'uuid': 'd9765e24-5d48-47b2-a343-4d487e679040'}}, 'id': None, 'timestamp': None, 'datetime': None, 'lastTradeTimestamp': None, 'symbol': None, 'type': 'limit', 'side': None, 'price': None, 'cost': None, 'average': None, 'amount': None, 'filled': None, 'remaining': None, 'status': None, 'fee': None}
                          

                          It's obvious bittrex doesn't return much info about the order. This dict is passed to "ccxt_order" on the call of the CCXTOrder class, and the script crashes while running:

                                  self.size = float(ccxt_order['amount'])
                          

                          from ccxtstore.py, because of the None value for the 'amount' key.
                          Bittrex doesn't give the necessary info but there's other ways to get it with their API. Any ideas on how to fix this? Should this be handled in the ccxt method "parse_order"?

                          On another note, what other crypto exchanges work well for people using this fork of backtrader?

                          1 Reply Last reply Reply Quote 0
                          • T
                            ThatBlokeDave @TOTORO last edited by

                            @totoro

                            Regarding the partial candle data, even though the PR wasn't merged, you can pick up fork of the Ed's and the others awesome work here:

                            https://github.com/Dave-Vallance/bt-ccxt-store

                            The repo contains made a few changes I made including an attempt to address the partial candle issue (although the solution is different to the PR). These changes work for me on the few exchanges I am looking at but your mileage may vary and I don't really plan on supporting it :)

                            There are some other changes such as speeding up backfilling and allowing you to map some broker keys and strings if the defaults don't work amongst other things. (for example, since CCXT does not have a unified stop/limit order implementation, Kraken expects you to create a stoploss with 'stop-loss', whereas bitmex expects just 'stop'.

                            For a full list of changes you can checkout the readme.

                            1 Reply Last reply Reply Quote 0
                            • E
                              Ed Bartosh @petens last edited by

                              @petens Can you check if this commit fixes the issue for you?

                              P 1 Reply Last reply Reply Quote 0
                              • sirhc
                                sirhc last edited by

                                Hi,

                                I am trying to understand backtrader/ccxt/bitmex live behaviours.

                                I am running a test script : simple strat with bitmex data feed in 1 min bar and bitmex broker.
                                It computes a sma(period=2), waiting for this sma to be calculated and then send one single limit order.

                                There is a delay I don't undertand:
                                If I load 4 bars from bitmex, without setting a broker, historical data are loading quickly and strategy run quickly.
                                With a broker it takes 1 min to run the strategy on 4 bars.
                                If I need few days of backfill, it will take forever. Is it normal ?

                                @ThatBlokeDave : Is that the kind of problem you add and corrected when you say " speeding up backfilling" ?

                                Ouput without broker : = 5 sec to init

                                time is  2018-09-10 10:54:33.984922  ## start
                                init pausetrade =  True
                                ***** DATA NOTIF: DELAYED
                                time is  2018-09-10 10:54:38.298118  ## next
                                ***** NEXT: 2018-09-10 08:51:00  6295.0 6295.0 6292.0 6292.5 365509.0 Minute 1
                                time is  2018-09-10 10:54:38.299116  ## next
                                ***** NEXT: 2018-09-10 08:52:00  6292.5 6292.5 6290.5 6290.5 496485.0 Minute 2
                                time is  2018-09-10 10:54:38.300116  ## next
                                ***** NEXT: 2018-09-10 08:53:00  6290.5 6290.5 6282.0 6282.0 2219366.0 Minute 3
                                time is  2018-09-10 10:54:38.300116  ## next
                                ***** NEXT: 2018-09-10 08:54:00  6282.0 6284.0 6282.0 6284.0 2435976.0 Minute 4
                                ***** DATA NOTIF: LIVE
                                time is  2018-09-10 10:55:03.080312  ## next
                                ***** NEXT: 2018-09-10 08:55:00  6282.0 6282.0 6282.0 6282.0 0.0 Minute 5
                                time is  2018-09-10 10:55:03.080312  ## SEND BUY LIMIT: 1 @ 5283.00
                                Shutdown requested...exiting
                                

                                With a broker : = 44 sec to init !!

                                time is  2018-09-10 12:13:18.374900  ## start
                                init pausetrade =  True
                                ***** DATA NOTIF: DELAYED
                                time is  2018-09-10 12:13:31.147005  ## next
                                ***** NEXT: 2018-09-10 10:10:00  6279.0 6279.0 6278.5 6279.0 546301.0 Minute 1
                                time is  2018-09-10 12:13:41.502875  ## next
                                ***** NEXT: 2018-09-10 10:11:00  6279.0 6279.5 6278.5 6279.5 335856.0 Minute 2
                                time is  2018-09-10 12:13:51.868567  ## next
                                ***** NEXT: 2018-09-10 10:12:00  6279.5 6279.5 6279.0 6279.0 141525.0 Minute 3
                                time is  2018-09-10 12:14:02.329758  ## next
                                ***** NEXT: 2018-09-10 10:13:00  6279.0 6279.5 6279.0 6279.0 21171.0 Minute 4
                                ***** DATA NOTIF: LIVE
                                time is  2018-09-10 12:14:16.795837  ## next
                                ***** NEXT: 2018-09-10 10:14:00  6279.5 6279.5 6279.0 6279.0 20132.0 Minute 5
                                time is  2018-09-10 12:14:20.980734  ## SEND BUY LIMIT: 1.0 @ 5279.00
                                ***** NOTIFY ORDER 0
                                time is  2018-09-10 12:15:08.587889  ## next
                                ***** NEXT: 2018-09-10 10:15:00  6279.0 6279.0 6279.0 6279.0 0.0 Minute 6
                                Shutdown requested...exiting
                                

                                Here is my code for this test :

                                from datetime import datetime, timedelta
                                import ccxt
                                import backtrader as bt
                                import os
                                from pandas import bdate_range
                                #get CCXT LOGS
                                #import logging
                                #logging.basicConfig(level=logging.DEBUG)
                                
                                class mmTest(bt.Strategy):
                                    params = (('pfast', 5), ('pslow', 30),)
                                    order = None
                                    pausetrade = True
                                
                                    def log(self, txt, dt=None):
                                        ''' Logging function for this strategy'''
                                        dt = dt or self.datas[0].datetime.datetime(0)
                                        print('%s, %s' % (str(dt), txt))
                                
                                    def __init__(self):
                                        # Keep a reference to the "close" line in the data[0] dataseries
                                        self.pausetrade = not self.data.haslivedata()
                                        print( "init pausetrade = " , self.pausetrade)
                                        self.dataclose = self.datas[0].close
                                        self.mm = bt.ind.MovingAverageSimple(self.data.close,period=2)
                                        self.order = None
                                
                                    def notify_order(self, order):
                                        print('*' * 5, "NOTIFY ORDER", order.status )
                                        
                                    def notify_data(self, data, status, *args, **kwargs):
                                       self.pausetrade = (status != data.LIVE)
                                       print('*' * 5, 'DATA NOTIF:', data._getstatusname(status), *args )
                                               
                                    def sendBuyOrder(self):
                                        if self.pausetrade:
                                            return
                                        if self.mm[0] >0:
                                            limit=self.mm[0] - 1000
                                            ticksize = 0.5
                                            limit=round(limit/ticksize)*ticksize
                                            self.order =self.buy(price= limit, exectype=bt.Order.Limit,transmit=True)
                                            print('time is ',datetime.now(),' ## SEND BUY LIMIT: {:s} @ {:.2f}'.format(str(self.order.size),limit ))
                                        else:
                                            print('MM = %f' % (self.mm[0]))
                                 
                                    def next(self):
                                        print('time is ',datetime.now(),' ## next')
                                        for data in self.datas:
                                            print('*' * 5, 'NEXT:', bt.num2date(data.datetime[0]), data._name, data.open[0], data.high[0],data.low[0], data.close[0], data.volume[0], bt.TimeFrame.getname(data._timeframe), len(data))
                                        if not self.order :
                                            self.sendBuyOrder()
                                        
                                try:
                                    print('time is ',datetime.now(),' ## start')
                                    
                                    cerebro = bt.Cerebro()
                                    
                                    feedconf = {'apiKey': '8KoTAFQZ3G1zkgRPSjj-LZi3', 'secret': ''}
                                    
                                    data = bt.feeds.CCXT(exchange="bitmex", symbol="BTC/USD",fromdate=datetime.now()+timedelta(hours = -2,minutes=-4), timeframe=bt.TimeFrame.Minutes,compression=1,config=feedconf,backfill_start=True)
                                
                                    broker_config = {'apiKey': '8KoTAFQZ3G1zkgRPSjj-LZi3',
                                                         'secret': '****',}
                                    
                                    broker = bt.brokers.CCXTBroker(exchange='bitmex',
                                                                       currency='XBt',
                                                                       config=broker_config)
                                    cerebro.setbroker(broker)
                                    
                                    cerebro.adddata(data)
                                    cerebro.addstrategy(mmTest)
                                    cerebro.run()
                                    
                                    #cerebro.plot()
                                
                                except KeyboardInterrupt:
                                    print("Shutdown requested...exiting")
                                #except Exception:
                                #    traceback.print_exc(file=sys.stdout)
                                quit
                                
                                
                                T 1 Reply Last reply Reply Quote 0
                                • P
                                  petens @Ed Bartosh last edited by

                                  Hey @ed-bartosh, that does seem to do the trick, a million thanks! I'll keep looking for issues and fixes to contribute as I go along using this.

                                  1 Reply Last reply Reply Quote 0
                                  • sirhc
                                    sirhc last edited by

                                    @søren-pallesen
                                    Hi,
                                    Did you get correct speed for backfilling few minutes bars when you use Bitmex feed AND Bitmex broker?
                                    It is seems that 1-minute bars are received in one block but a simple strategy with 1 indicator needs at least 10sec per bar to run/compute the "next" function.

                                    May be, it is the normal behaviour of Backtrader running live?

                                    1 Reply Last reply Reply Quote 0
                                    • T
                                      thomaslee last edited by

                                      Hi, not sure if this is the right place to ask but how do you input backfeed data from ccxt into backtrader? Do I have to convert the ccxt output to CSV and proceed from there?

                                      import ccxt
                                      
                                      bitmex = ccxt.bitmex()
                                      
                                      # params:
                                      symbol = 'BTC/USD'
                                      timeframe = '1d'
                                      limit = 750
                                      params = {'partial': False}  # ←--------  no reversal
                                      since = None
                                      
                                      candles = bitmex.fetch_ohlcv(symbol, timeframe, since, limit, params)
                                      

                                      Here I want to use the data in candles as feed for backtesting (not live trading). Thank you!

                                      sirhc 1 Reply Last reply Reply Quote 0
                                      • sirhc
                                        sirhc @thomaslee last edited by sirhc

                                        @thomaslee
                                        You can probably convert candles into a Pandas dataframe with datetime, open, high, low , close, volume.
                                        And then use this feed :

                                        data0=bt.feeds.PandasDirectData(dataname=dataframe,**kwargs,datetime=0,time=1,open=2,high=3,low=4,close=5,volume=6,openinterest=-1,timeframe=bt.TimeFrame.Days,compression=1)
                                        
                                        1 Reply Last reply Reply Quote 1
                                        • T
                                          ThatBlokeDave @sirhc last edited by

                                          @sirhc

                                          Is that the kind of problem you add and corrected when you say " speeding up backfilling" ?

                                          Yes - That is correct. The reason it is slow is because the broker will try to check your cash/balance on every call of next(). (If I remember correctly it makes 2 or 3 checks in total between each call of next). Since, the store will make use rest API calls to do these checks, this can add quite a few seconds delay between each call of next(). (Because of the rest API rate limit).

                                          So in my fork, I cheat a bit. The cash and balance checks return the last values received from the last rest call. They do not make another call. Instead, the user must manually check the balance if they need up to date balance information before entering a position. (at which point the latest values will be saved).

                                          Not an ideal solution but it works for me.

                                          sirhc 1 Reply Last reply Reply Quote 0
                                          • R
                                            ramoslin02 @Ed Bartosh last edited by

                                            @ed-bartosh said in Anyone use backtrader to do live trading on Bitcoin exchange?:

                                            guide

                                            Do you ever experience the problem of self.order = self.buy(size=1.5) or in the buy method ,the param size can not set to a float value ?

                                            1 Reply Last reply Reply Quote 0
                                            • 1
                                            • 2
                                            • 12
                                            • 13
                                            • 14
                                            • 15
                                            • 16
                                            • 17
                                            • 14 / 17
                                            • First post
                                              Last post
                                            Copyright © 2016, 2017, 2018, 2019, 2020, 2021 NodeBB Forums | Contributors