Navigation

    Backtrader Community

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

    Trying to use Binance Data

    General Discussion
    3
    4
    988
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Jeff Doyle
      Jeff Doyle last edited by

      I am trying to define a process to backtest Data pulled from the Binance API. I originally tried to pull the data into the MACD settings article found at https://www.backtrader.com/blog/posts/2016-07-30-macd-settings/macd-settings/ and decided to go to a simpler file.

      Everytime I try to change the way that Backtrader can pull the data I simply run into a new bug. Can anyone please guide me on how to use Backtrader to test Binance data. I am providing the simple code below.

      import backtrader as bt
      import backtrader.feeds as btfeeds
      import os
      import sys
      import datetime
      
      
      class TestStrategy(bt.Strategy):
      
          def log(self, txt, dt=None):
              dt = dt or self.datas[0].datetime.date(0)
              print('%s, %s' % (dt.isoformat(), txt))
      
          def __init__(self):
              self.dataclose = self.datas[0].close
              self.order = None
              self.buyprice = None
              self.buycomm = None
      
              bt.indicators.ExponentialMovingAverage(self.datas[0], period=25)
      
          def notify_order(self, order):
              if order.status in [order.Submitted, order.Accepted]:
                  # Buy/Sell order submitted/accepted to/by broker - Nothing to do
                  return
      
              if order.status in [order.Completed]:
                  if order.isbuy():
                      self.log(
                          'BUY EXECUTED, Price: %.2f, Cost: %.2f, Comm %.2f' %
                          (order.executed.price,
                           order.executed.value,
                           order.executed.comm))
      
                      self.buyprice = order.executed.price
                      self.buycomm = order.executed.comm
                  else:
                      self.log('SELL EXECUTED, Price: %.2f, Cost: %.2f, Comm %.2f' %
                               (order.executed.price,
                                order.executed.value,
                                order.executed.comm))
      
                  self.bar_executed = len(self)
      
              elif order.status in [order.Canceled, order.Margin, order.Rejected]:
                  self.log('Order Canceled/Margin/Rejected')
      
              self.order = None
      
          def notify_trade(self, trade):
              if not trade.isclosed:
                  return
      
              self.log('OPERATION PROFIT, GROSS %.2f, NET %.2f' %
                       (trade.pnl, trade.pnlcomm))
      
          def next(self):
              self.log('Close, %.2f' % self.dataclose[0])
      
              if self.order:
                  return
      
              if not self.position:
      
                  if self.dataclose[0] - self.dataclose[-1] > 500:
                      self.log('BUY CREATE, %.2f' % self.dataclose[0])
                      self.order = self.buy()
      
              else:
      
                  if self.dataclose[0] - self.dataclose[-1] < -500:
                      self.log('SELL CREATE, %.2f' % self.dataclose[0])
                      self.order = self.sell()
      
          modpath = os.path.dirname(os.path.abspath(sys.argv[0]))
          datapath = os.path.join(modpath, 'C:/Users/Jeff2/BTCUSDT_MinuteBars.csv')
      
      
      data = btfeeds.GenericCSVData(
          datapath = os.path.join(modpath, 'C:/Users/Jeff2/BTCUSDT_MinuteBars.csv')
      
          fromdate=datetime.datetime(2020, 4, 6),
          todate=datetime.datetime(2020, 4, 7),
      
          nullvalue=0.0,
      
          dtformat=('%Y-%m-%d'),
          tmformat=('%H:%M:%S'),
      
          Timestamp=0,
          Open=1,
          High=2,
          Low=3,
          Close=4,
          Volume=5,
          Close_time=6,
          Quote_AV=7,
          Trades=8,
          TB_Base_AV=9,
          TB_Quote_AV=10,
          Ignore=-1,
          timeframe=bt.TimeFrame.Minutes,)
      
      cerebro.adddata(data)
      
      cerebro.broker.setcash(1000.0)
      
      cerebro.addsizer(bt.sizers.FixedSize, stake=0.05)
      
      cerebro.broker.setcommission(commission=0.005)
      
      print('Starting Balance: %.2f' % cerebro.broker.getvalue())
      
      cerebro.run()
      
      print('Final Balance: %.2f' % cerebro.broker.getvalue())
      
      cerebro.plot()
      

      Data

      640f529e-f7a5-4b88-a56b-f7655d7d9dc9-image.png

      1 Reply Last reply Reply Quote 0
      • Yaroslav Horyslavets
        Yaroslav Horyslavets last edited by

        backtrader data feeds contain the usual industry standard feeds, which are the ones to be filled. Namely:

        datetime

        open

        high

        low

        close

        volume

        openinterest

        Just delete others.

        1 Reply Last reply Reply Quote 0
        • A
          alpha last edited by

          As Yaroslav mentioned you can simply use any of the backtrader input options, I don't recall anything special about Binance data. I usually load my data with Pandas first (some data sources come from DB, some from CSV) for easier pre-processing, and then I do something like this:

          bt_data = bt.feeds.PandasDirectData(dataname=crypt_df, datetime=crypt_df.columns.get_loc('timestamp')+1, 
              open=crypt_df.columns.get_loc('open')+1, high=crypt_df.columns.get_loc('max')+1, low=crypt_df.columns.get_loc('min')+1, 
              close=crypt_df.columns.get_loc('close')+1,volume=crypt_df.columns.get_loc('volume')+1,openinterest=-1)
          

          Then we use this later on when initializing Cerebro:

              cerebro.adddata(bt_data)
          

          Not sure what bugs you're referring to, I haven't had any issues that I recall in data loading, although some of the behavior is undocumented and requires some review of the BT code (for example, the "+1" offsets in the PandasDirectData class that I'm using.

          1 Reply Last reply Reply Quote 0
          • Jeff Doyle
            Jeff Doyle last edited by

            It looks like I was missing the "main" section of the code. The following code works minus the plotting function.

            import backtrader as bt
            import backtrader.feeds as btfeeds
            import os
            import sys
            import datetime
            
            
            class TestStrategy(bt.Strategy):
            
                def log(self, txt, dt=None):
                    dt = dt or self.datas[0].datetime.date(0)
                    print('%s, %s' % (dt.isoformat(), txt))
            
                def __init__(self):
                    self.dataclose = self.datas[0].close
                    self.order = None
                    self.buyprice = None
                    self.buycomm = None
            
                    bt.indicators.ExponentialMovingAverage(self.datas[0], period=25)
            
                def notify_order(self, order):
                    if order.status in [order.Submitted, order.Accepted]:
                        # Buy/Sell order submitted/accepted to/by broker - Nothing to do
                        return
            
                    if order.status in [order.Completed]:
                        if order.isbuy():
                            self.log(
                                'BUY EXECUTED, Price: %.2f, Cost: %.2f, Comm %.2f' %
                                (order.executed.price,
                                 order.executed.value,
                                 order.executed.comm))
            
                            self.buyprice = order.executed.price
                            self.buycomm = order.executed.comm
                        else:
                            self.log('SELL EXECUTED, Price: %.2f, Cost: %.2f, Comm %.2f' %
                                     (order.executed.price,
                                      order.executed.value,
                                      order.executed.comm))
            
                        self.bar_executed = len(self)
            
                    elif order.status in [order.Canceled, order.Margin, order.Rejected]:
                        self.log('Order Canceled/Margin/Rejected')
            
                    self.order = None
            
                def notify_trade(self, trade):
                    if not trade.isclosed:
                        return
            
                    self.log('OPERATION PROFIT, GROSS %.2f, NET %.2f' %
                             (trade.pnl, trade.pnlcomm))
            
                def next(self):
                    self.log('Close, %.2f' % self.dataclose[0])
            
                    if self.order:
                        return
            
                    if not self.position:
            
                        if self.dataclose[0] - self.dataclose[-1] > 500:
                            self.log('BUY CREATE, %.2f' % self.dataclose[0])
                            self.order = self.buy()
            
                    else:
            
                        if self.dataclose[0] - self.dataclose[-1] < -500:
                            self.log('SELL CREATE, %.2f' % self.dataclose[0])
                            self.order = self.sell()
            
            
            if __name__ == '__main__':
                cerebro = bt.Cerebro()
                cerebro.addstrategy(TestStrategy)
                modpath = os.path.dirname(os.path.abspath(sys.argv[0]))
                datapath = os.path.join(modpath, 'C:/Users/Jeff2/BTCUSDT_MinuteBars.csv')
            
            # Create a Data Feed
            data = btfeeds.GenericCSVData(
                dataname=datapath,
                fromdate=datetime.datetime(2018, 1, 1),
                dtformat=('%Y-%m-%d %H:%M:%S'),
                datetime=0,
                high=2,
                low=3,
                open=1,
                close=4,
                volume=5,
                openinterest=-1,
                timeframe=bt.TimeFrame.Minutes,
                compression=30
            )
            
            cerebro.adddata(data)
            
            cerebro.broker.setcash(1000.0)
            
            cerebro.addsizer(bt.sizers.FixedSize, stake=0.05)
            
            cerebro.broker.setcommission(commission=0.005)
            
            print('Starting Balance: %.2f' % cerebro.broker.getvalue())
            
            cerebro.run()
            
            print('Final Balance: %.2f' % cerebro.broker.getvalue())
            
            cerebro.plot()
            

            Alpha - Are you using the above method to load in multiple assets?

            I am trying to bring the above data format into the MACDsettings article's code found at https://www.backtrader.com/blog/posts/2016-07-30-macd-settings/macd-settings/#usage-of-the-sample and I keep running into the same error of: Traceback (most recent call last):
            File ".\BinanceBTCUSDT_MACDStrat.py", line 305, in <module>
            runstrat()
            File ".\BinanceBTCUSDT_MACDStrat.py", line 175, in runstrat
            compression=30
            File "C:\Users\Jeff2\Anaconda3\lib\site-packages\backtrader\metabase.py", line 88, in call
            _obj, args, kwargs = cls.doinit(_obj, *args, **kwargs)
            File "C:\Users\Jeff2\Anaconda3\lib\site-packages\backtrader\metabase.py", line 78, in doinit
            _obj.init(*args, **kwargs)
            TypeError: init() got an unexpected keyword argument 'dtformat'

            The code can be found below. Any ideas on what might be causing this issue?

            #!/usr/bin/env python
            # -*- coding: utf-8; py-indent-offset:4 -*-
            ###############################################################################
            #
            # Copyright (C) 2015, 2016 Daniel Rodriguez
            #
            # This program is free software: you can redistribute it and/or modify
            # it under the terms of the GNU General Public License as published by
            # the Free Software Foundation, either version 3 of the License, or
            # (at your option) any later version.
            #
            # This program is distributed in the hope that it will be useful,
            # but WITHOUT ANY WARRANTY; without even the implied warranty of
            # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
            # GNU General Public License for more details.
            #
            # You should have received a copy of the GNU General Public License
            # along with this program.  If not, see <http://www.gnu.org/licenses/>.
            #
            ###############################################################################
            from __future__ import (absolute_import, division, print_function,
                                    unicode_literals)
            
            
            import argparse
            import datetime
            import random
            
            import backtrader as bt
            
            BTVERSION = tuple(int(x) for x in bt.__version__.split('.'))
            
            
            class FixedPerc(bt.Sizer):
                '''This sizer simply returns a fixed size for any operation
            
                Params:
                  - ``perc`` (default: ``0.20``) Perc of cash to allocate for operation
                '''
            
                params = (
                    ('perc', 0.20),  # perc of cash to use for operation
                )
            
                def _getsizing(self, comminfo, cash, data, isbuy):
                    cashtouse = self.p.perc * cash
                    if BTVERSION > (1, 7, 1, 93):
                        size = comminfo.getsize(data.close[0], cashtouse)
                    else:
                        size = cashtouse // data.close[0]
                    return size
            
            
            class TheStrategy(bt.Strategy):
                '''
                This strategy is loosely based on some of the examples from the Van
                K. Tharp book: *Trade Your Way To Financial Freedom*. The logic:
            
                  - Enter the market if:
                    - The MACD.macd line crosses the MACD.signal line to the upside
                    - The Simple Moving Average has a negative direction in the last x
                      periods (actual value below value x periods ago)
            
                 - Set a stop price x times the ATR value away from the close
            
                 - If in the market:
            
                   - Check if the current close has gone below the stop price. If yes,
                     exit.
                   - If not, update the stop price if the new stop price would be higher
                     than the current
                '''
            
                params = (
                    # Standard MACD Parameters
                    ('macd1', 12),
                    ('macd2', 26),
                    ('macdsig', 9),
                    ('atrperiod', 14),  # ATR Period (standard)
                    ('atrdist', 3.0),   # ATR distance for stop price
                    ('smaperiod', 30),  # SMA Period (pretty standard)
                    ('dirperiod', 10),  # Lookback period to consider SMA trend direction
                )
            
            
            
                def notify_order(self, order):
                    if order.status == order.Completed:
                        pass
            
                    if not order.alive():
                        self.order = None  # indicate no order is pending
            
                def __init__(self):
                    self.macd = bt.indicators.MACD(self.data,
                                                   period_me1=self.p.macd1,
                                                   period_me2=self.p.macd2,
                                                   period_signal=self.p.macdsig)
            
                    # Cross of macd.macd and macd.signal
                    self.mcross = bt.indicators.CrossOver(self.macd.macd, self.macd.signal)
            
                    # To set the stop price
                    self.atr = bt.indicators.ATR(self.data, period=self.p.atrperiod)
            
                    # Control market trend
                    self.sma = bt.indicators.SMA(self.data, period=self.p.smaperiod)
                    self.smadir = self.sma - self.sma(-self.p.dirperiod)
            
                def start(self):
                    self.order = None  # sentinel to avoid operrations on pending order
            
                def next(self):
                    if self.order:
                        return  # pending order execution
            
                    if not self.position:  # not in the market
                        if self.mcross[0] > 0.0 and self.smadir < 0.0:
                            self.order = self.buy()
                            pdist = self.atr[0] * self.p.atrdist
                            self.pstop = self.data.close[0] - pdist
            
                    else:  # in the market
                        pclose = self.data.close[0]
                        pstop = self.pstop
            
                        if pclose < pstop:
                            self.close()  # stop met - get out
                        else:
                            pdist = self.atr[0] * self.p.atrdist
                            # Update only if greater than
                            self.pstop = max(pstop, pclose - pdist)
            
            
            DATASETS = {
                'BTCUSDT': 'C:/Users/Jeff2/Algro project/Binance/BTCUSDT_MinuteBars.csv'
            }
            
            
            def runstrat(args=None):
                args = parse_args(args)
            
                cerebro = bt.Cerebro()
                cerebro.broker.set_cash(args.cash)
                comminfo = bt.commissions.CommInfo_Stocks_Perc(commission=args.commperc,
                                                               percabs=True)
            
                cerebro.broker.addcommissioninfo(comminfo)
            
                dkwargs = dict()
                if args.fromdate is not None:
                    fromdate = datetime.datetime.strptime(args.fromdate, '%d %b %Y')
                    dkwargs['fromdate'] = fromdate
            
                if args.todate is not None:
                    todate = datetime.datetime.strptime(args.todate, '%d %b %Y')
                    dkwargs['todate'] = todate
            
                # if dataset is None, args.data has been given
                dataname = DATASETS.get(args.dataset, args.data)
                data0 = bt.feeds.GenericCSV(dataname=dataname,
                fromdate=datetime.datetime(2018, 1, 1),
                dtformat=('%Y-%m-%d %H:%M:%S'),
                datetime=0,
                high=2,
                low=3,
                open=1,
                close=4,
                volume=5,
                openinterest=-1,
                timeframe=bt.TimeFrame.Minutes,
            )
            
                cerebro.adddata(data0)
            
                cerebro.addstrategy(TheStrategy,
                                    macd1=args.macd1, macd2=args.macd2,
                                    macdsig=args.macdsig,
                                    atrperiod=args.atrperiod,
                                    atrdist=args.atrdist,
                                    smaperiod=args.smaperiod,
                                    dirperiod=args.dirperiod)
            
                cerebro.addsizer(FixedPerc, perc=args.cashalloc)
            
                # Add TimeReturn Analyzers for self and the benchmark data
                cerebro.addanalyzer(bt.analyzers.TimeReturn, _name='alltime_roi',
                                    timeframe=bt.TimeFrame.NoTimeFrame)
            
                cerebro.addanalyzer(bt.analyzers.TimeReturn, data=data0, _name='benchmark',
                                    timeframe=bt.TimeFrame.NoTimeFrame)
            
                # Add TimeReturn Analyzers fot the annuyl returns
                cerebro.addanalyzer(bt.analyzers.TimeReturn, timeframe=bt.TimeFrame.Years)
                # Add a SharpeRatio
                cerebro.addanalyzer(bt.analyzers.SharpeRatio, timeframe=bt.TimeFrame.Years,
                                    riskfreerate=args.riskfreerate)
            
                # Add SQN to qualify the trades
                cerebro.addanalyzer(bt.analyzers.SQN)
                cerebro.addobserver(bt.observers.DrawDown)  # visualize the drawdown evol
            
                results = cerebro.run()
                st0 = results[0]
            
                for alyzer in st0.analyzers:
                    alyzer.print()
            
                if args.plot:
                    pkwargs = dict(style='bar')
                    if args.plot is not True:  # evals to True but is not True
                        npkwargs = eval('dict(' + args.plot + ')')  # args were passed
                        pkwargs.update(npkwargs)
            
                    cerebro.plot(**pkwargs)
            
            
            def parse_args(pargs=None):
            
                parser = argparse.ArgumentParser(
                    formatter_class=argparse.ArgumentDefaultsHelpFormatter,
                    description='Sample for Tharp example with MACD')
            
                group1 = parser.add_mutually_exclusive_group(required=True)
                group1.add_argument('--data', required=False, default=None,
                                    help='Specific data to be read in')
            
                group1.add_argument('--dataset', required=False, action='store',
                                    default=None, choices=DATASETS.keys(),
                                    help='Choose one of the predefined data sets')
            
                parser.add_argument('--fromdate', required=False,
                                    default='01 Jan 2016',
                                    help='Starting date in %d %b %Y %H:%M:%S format')
            
                parser.add_argument('--todate', required=False,
                                    default=None,
                                    help='Ending date in %d %b %Y %H:%M:%S format')
            
                parser.add_argument('--cash', required=False, action='store',
                                    type=float, default=50000,
                                    help=('Cash to start with'))
            
                parser.add_argument('--cashalloc', required=False, action='store',
                                    type=float, default=0.20,
                                    help=('Perc (abs) of cash to allocate for ops'))
            
                parser.add_argument('--commperc', required=False, action='store',
                                    type=float, default=0.0033,
                                    help=('Perc (abs) commision in each operation. '
                                          '0.001 -> 0.1%%, 0.01 -> 1%%'))
            
                parser.add_argument('--macd1', required=False, action='store',
                                    type=int, default=12,
                                    help=('MACD Period 1 value'))
            
                parser.add_argument('--macd2', required=False, action='store',
                                    type=int, default=26,
                                    help=('MACD Period 2 value'))
            
                parser.add_argument('--macdsig', required=False, action='store',
                                    type=int, default=9,
                                    help=('MACD Signal Period value'))
            
                parser.add_argument('--atrperiod', required=False, action='store',
                                    type=int, default=14,
                                    help=('ATR Period To Consider'))
            
                parser.add_argument('--atrdist', required=False, action='store',
                                    type=float, default=3.0,
                                    help=('ATR Factor for stop price calculation'))
            
                parser.add_argument('--smaperiod', required=False, action='store',
                                    type=int, default=30,
                                    help=('Period for the moving average'))
            
                parser.add_argument('--dirperiod', required=False, action='store',
                                    type=int, default=10,
                                    help=('Period for SMA direction calculation'))
            
                parser.add_argument('--riskfreerate', required=False, action='store',
                                    type=float, default=0.01,
                                    help=('Risk free rate in Perc (abs) of the asset for '
                                          'the Sharpe Ratio'))
                # Plot options
                parser.add_argument('--plot', '-p', nargs='?', required=False,
                                    metavar='kwargs', const=True,
                                    help=('Plot the read data applying any kwargs passed\n'
                                          '\n'
                                          'For example:\n'
                                          '\n'
                                          '  --plot style="candle" (to plot candles)\n'))
            
                if pargs is not None:
                    return parser.parse_args(pargs)
            
                return parser.parse_args()
            
            
            if __name__ == '__main__':
                runstrat()
            
            1 Reply Last reply Reply Quote 0
            • 1 / 1
            • First post
              Last post
            Copyright © 2016, 2017, 2018 NodeBB Forums | Contributors
            $(document).ready(function () { app.coldLoad(); }); }