For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

Trying to use Binance Data



  • I am trying to define a process to backtest Data pulled from the Binance API. I originally tried to pull the data into the MACD settings article found at https://www.backtrader.com/blog/posts/2016-07-30-macd-settings/macd-settings/ and decided to go to a simpler file.

    Everytime I try to change the way that Backtrader can pull the data I simply run into a new bug. Can anyone please guide me on how to use Backtrader to test Binance data. I am providing the simple code below.

    import backtrader as bt
    import backtrader.feeds as btfeeds
    import os
    import sys
    import datetime
    
    
    class TestStrategy(bt.Strategy):
    
        def log(self, txt, dt=None):
            dt = dt or self.datas[0].datetime.date(0)
            print('%s, %s' % (dt.isoformat(), txt))
    
        def __init__(self):
            self.dataclose = self.datas[0].close
            self.order = None
            self.buyprice = None
            self.buycomm = None
    
            bt.indicators.ExponentialMovingAverage(self.datas[0], period=25)
    
        def notify_order(self, order):
            if order.status in [order.Submitted, order.Accepted]:
                # Buy/Sell order submitted/accepted to/by broker - Nothing to do
                return
    
            if order.status in [order.Completed]:
                if order.isbuy():
                    self.log(
                        'BUY EXECUTED, Price: %.2f, Cost: %.2f, Comm %.2f' %
                        (order.executed.price,
                         order.executed.value,
                         order.executed.comm))
    
                    self.buyprice = order.executed.price
                    self.buycomm = order.executed.comm
                else:
                    self.log('SELL EXECUTED, Price: %.2f, Cost: %.2f, Comm %.2f' %
                             (order.executed.price,
                              order.executed.value,
                              order.executed.comm))
    
                self.bar_executed = len(self)
    
            elif order.status in [order.Canceled, order.Margin, order.Rejected]:
                self.log('Order Canceled/Margin/Rejected')
    
            self.order = None
    
        def notify_trade(self, trade):
            if not trade.isclosed:
                return
    
            self.log('OPERATION PROFIT, GROSS %.2f, NET %.2f' %
                     (trade.pnl, trade.pnlcomm))
    
        def next(self):
            self.log('Close, %.2f' % self.dataclose[0])
    
            if self.order:
                return
    
            if not self.position:
    
                if self.dataclose[0] - self.dataclose[-1] > 500:
                    self.log('BUY CREATE, %.2f' % self.dataclose[0])
                    self.order = self.buy()
    
            else:
    
                if self.dataclose[0] - self.dataclose[-1] < -500:
                    self.log('SELL CREATE, %.2f' % self.dataclose[0])
                    self.order = self.sell()
    
        modpath = os.path.dirname(os.path.abspath(sys.argv[0]))
        datapath = os.path.join(modpath, 'C:/Users/Jeff2/BTCUSDT_MinuteBars.csv')
    
    
    data = btfeeds.GenericCSVData(
        datapath = os.path.join(modpath, 'C:/Users/Jeff2/BTCUSDT_MinuteBars.csv')
    
        fromdate=datetime.datetime(2020, 4, 6),
        todate=datetime.datetime(2020, 4, 7),
    
        nullvalue=0.0,
    
        dtformat=('%Y-%m-%d'),
        tmformat=('%H:%M:%S'),
    
        Timestamp=0,
        Open=1,
        High=2,
        Low=3,
        Close=4,
        Volume=5,
        Close_time=6,
        Quote_AV=7,
        Trades=8,
        TB_Base_AV=9,
        TB_Quote_AV=10,
        Ignore=-1,
        timeframe=bt.TimeFrame.Minutes,)
    
    cerebro.adddata(data)
    
    cerebro.broker.setcash(1000.0)
    
    cerebro.addsizer(bt.sizers.FixedSize, stake=0.05)
    
    cerebro.broker.setcommission(commission=0.005)
    
    print('Starting Balance: %.2f' % cerebro.broker.getvalue())
    
    cerebro.run()
    
    print('Final Balance: %.2f' % cerebro.broker.getvalue())
    
    cerebro.plot()
    

    Data

    640f529e-f7a5-4b88-a56b-f7655d7d9dc9-image.png



  • backtrader data feeds contain the usual industry standard feeds, which are the ones to be filled. Namely:

    datetime

    open

    high

    low

    close

    volume

    openinterest

    Just delete others.



  • As Yaroslav mentioned you can simply use any of the backtrader input options, I don't recall anything special about Binance data. I usually load my data with Pandas first (some data sources come from DB, some from CSV) for easier pre-processing, and then I do something like this:

    bt_data = bt.feeds.PandasDirectData(dataname=crypt_df, datetime=crypt_df.columns.get_loc('timestamp')+1, 
        open=crypt_df.columns.get_loc('open')+1, high=crypt_df.columns.get_loc('max')+1, low=crypt_df.columns.get_loc('min')+1, 
        close=crypt_df.columns.get_loc('close')+1,volume=crypt_df.columns.get_loc('volume')+1,openinterest=-1)
    

    Then we use this later on when initializing Cerebro:

        cerebro.adddata(bt_data)
    

    Not sure what bugs you're referring to, I haven't had any issues that I recall in data loading, although some of the behavior is undocumented and requires some review of the BT code (for example, the "+1" offsets in the PandasDirectData class that I'm using.



  • It looks like I was missing the "main" section of the code. The following code works minus the plotting function.

    import backtrader as bt
    import backtrader.feeds as btfeeds
    import os
    import sys
    import datetime
    
    
    class TestStrategy(bt.Strategy):
    
        def log(self, txt, dt=None):
            dt = dt or self.datas[0].datetime.date(0)
            print('%s, %s' % (dt.isoformat(), txt))
    
        def __init__(self):
            self.dataclose = self.datas[0].close
            self.order = None
            self.buyprice = None
            self.buycomm = None
    
            bt.indicators.ExponentialMovingAverage(self.datas[0], period=25)
    
        def notify_order(self, order):
            if order.status in [order.Submitted, order.Accepted]:
                # Buy/Sell order submitted/accepted to/by broker - Nothing to do
                return
    
            if order.status in [order.Completed]:
                if order.isbuy():
                    self.log(
                        'BUY EXECUTED, Price: %.2f, Cost: %.2f, Comm %.2f' %
                        (order.executed.price,
                         order.executed.value,
                         order.executed.comm))
    
                    self.buyprice = order.executed.price
                    self.buycomm = order.executed.comm
                else:
                    self.log('SELL EXECUTED, Price: %.2f, Cost: %.2f, Comm %.2f' %
                             (order.executed.price,
                              order.executed.value,
                              order.executed.comm))
    
                self.bar_executed = len(self)
    
            elif order.status in [order.Canceled, order.Margin, order.Rejected]:
                self.log('Order Canceled/Margin/Rejected')
    
            self.order = None
    
        def notify_trade(self, trade):
            if not trade.isclosed:
                return
    
            self.log('OPERATION PROFIT, GROSS %.2f, NET %.2f' %
                     (trade.pnl, trade.pnlcomm))
    
        def next(self):
            self.log('Close, %.2f' % self.dataclose[0])
    
            if self.order:
                return
    
            if not self.position:
    
                if self.dataclose[0] - self.dataclose[-1] > 500:
                    self.log('BUY CREATE, %.2f' % self.dataclose[0])
                    self.order = self.buy()
    
            else:
    
                if self.dataclose[0] - self.dataclose[-1] < -500:
                    self.log('SELL CREATE, %.2f' % self.dataclose[0])
                    self.order = self.sell()
    
    
    if __name__ == '__main__':
        cerebro = bt.Cerebro()
        cerebro.addstrategy(TestStrategy)
        modpath = os.path.dirname(os.path.abspath(sys.argv[0]))
        datapath = os.path.join(modpath, 'C:/Users/Jeff2/BTCUSDT_MinuteBars.csv')
    
    # Create a Data Feed
    data = btfeeds.GenericCSVData(
        dataname=datapath,
        fromdate=datetime.datetime(2018, 1, 1),
        dtformat=('%Y-%m-%d %H:%M:%S'),
        datetime=0,
        high=2,
        low=3,
        open=1,
        close=4,
        volume=5,
        openinterest=-1,
        timeframe=bt.TimeFrame.Minutes,
        compression=30
    )
    
    cerebro.adddata(data)
    
    cerebro.broker.setcash(1000.0)
    
    cerebro.addsizer(bt.sizers.FixedSize, stake=0.05)
    
    cerebro.broker.setcommission(commission=0.005)
    
    print('Starting Balance: %.2f' % cerebro.broker.getvalue())
    
    cerebro.run()
    
    print('Final Balance: %.2f' % cerebro.broker.getvalue())
    
    cerebro.plot()
    

    Alpha - Are you using the above method to load in multiple assets?

    I am trying to bring the above data format into the MACDsettings article's code found at https://www.backtrader.com/blog/posts/2016-07-30-macd-settings/macd-settings/#usage-of-the-sample and I keep running into the same error of: Traceback (most recent call last):
    File ".\BinanceBTCUSDT_MACDStrat.py", line 305, in <module>
    runstrat()
    File ".\BinanceBTCUSDT_MACDStrat.py", line 175, in runstrat
    compression=30
    File "C:\Users\Jeff2\Anaconda3\lib\site-packages\backtrader\metabase.py", line 88, in call
    _obj, args, kwargs = cls.doinit(_obj, *args, **kwargs)
    File "C:\Users\Jeff2\Anaconda3\lib\site-packages\backtrader\metabase.py", line 78, in doinit
    _obj.init(*args, **kwargs)
    TypeError: init() got an unexpected keyword argument 'dtformat'

    The code can be found below. Any ideas on what might be causing this issue?

    #!/usr/bin/env python
    # -*- coding: utf-8; py-indent-offset:4 -*-
    ###############################################################################
    #
    # Copyright (C) 2015, 2016 Daniel Rodriguez
    #
    # This program is free software: you can redistribute it and/or modify
    # it under the terms of the GNU General Public License as published by
    # the Free Software Foundation, either version 3 of the License, or
    # (at your option) any later version.
    #
    # This program is distributed in the hope that it will be useful,
    # but WITHOUT ANY WARRANTY; without even the implied warranty of
    # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
    # GNU General Public License for more details.
    #
    # You should have received a copy of the GNU General Public License
    # along with this program.  If not, see <http://www.gnu.org/licenses/>.
    #
    ###############################################################################
    from __future__ import (absolute_import, division, print_function,
                            unicode_literals)
    
    
    import argparse
    import datetime
    import random
    
    import backtrader as bt
    
    BTVERSION = tuple(int(x) for x in bt.__version__.split('.'))
    
    
    class FixedPerc(bt.Sizer):
        '''This sizer simply returns a fixed size for any operation
    
        Params:
          - ``perc`` (default: ``0.20``) Perc of cash to allocate for operation
        '''
    
        params = (
            ('perc', 0.20),  # perc of cash to use for operation
        )
    
        def _getsizing(self, comminfo, cash, data, isbuy):
            cashtouse = self.p.perc * cash
            if BTVERSION > (1, 7, 1, 93):
                size = comminfo.getsize(data.close[0], cashtouse)
            else:
                size = cashtouse // data.close[0]
            return size
    
    
    class TheStrategy(bt.Strategy):
        '''
        This strategy is loosely based on some of the examples from the Van
        K. Tharp book: *Trade Your Way To Financial Freedom*. The logic:
    
          - Enter the market if:
            - The MACD.macd line crosses the MACD.signal line to the upside
            - The Simple Moving Average has a negative direction in the last x
              periods (actual value below value x periods ago)
    
         - Set a stop price x times the ATR value away from the close
    
         - If in the market:
    
           - Check if the current close has gone below the stop price. If yes,
             exit.
           - If not, update the stop price if the new stop price would be higher
             than the current
        '''
    
        params = (
            # Standard MACD Parameters
            ('macd1', 12),
            ('macd2', 26),
            ('macdsig', 9),
            ('atrperiod', 14),  # ATR Period (standard)
            ('atrdist', 3.0),   # ATR distance for stop price
            ('smaperiod', 30),  # SMA Period (pretty standard)
            ('dirperiod', 10),  # Lookback period to consider SMA trend direction
        )
    
    
    
        def notify_order(self, order):
            if order.status == order.Completed:
                pass
    
            if not order.alive():
                self.order = None  # indicate no order is pending
    
        def __init__(self):
            self.macd = bt.indicators.MACD(self.data,
                                           period_me1=self.p.macd1,
                                           period_me2=self.p.macd2,
                                           period_signal=self.p.macdsig)
    
            # Cross of macd.macd and macd.signal
            self.mcross = bt.indicators.CrossOver(self.macd.macd, self.macd.signal)
    
            # To set the stop price
            self.atr = bt.indicators.ATR(self.data, period=self.p.atrperiod)
    
            # Control market trend
            self.sma = bt.indicators.SMA(self.data, period=self.p.smaperiod)
            self.smadir = self.sma - self.sma(-self.p.dirperiod)
    
        def start(self):
            self.order = None  # sentinel to avoid operrations on pending order
    
        def next(self):
            if self.order:
                return  # pending order execution
    
            if not self.position:  # not in the market
                if self.mcross[0] > 0.0 and self.smadir < 0.0:
                    self.order = self.buy()
                    pdist = self.atr[0] * self.p.atrdist
                    self.pstop = self.data.close[0] - pdist
    
            else:  # in the market
                pclose = self.data.close[0]
                pstop = self.pstop
    
                if pclose < pstop:
                    self.close()  # stop met - get out
                else:
                    pdist = self.atr[0] * self.p.atrdist
                    # Update only if greater than
                    self.pstop = max(pstop, pclose - pdist)
    
    
    DATASETS = {
        'BTCUSDT': 'C:/Users/Jeff2/Algro project/Binance/BTCUSDT_MinuteBars.csv'
    }
    
    
    def runstrat(args=None):
        args = parse_args(args)
    
        cerebro = bt.Cerebro()
        cerebro.broker.set_cash(args.cash)
        comminfo = bt.commissions.CommInfo_Stocks_Perc(commission=args.commperc,
                                                       percabs=True)
    
        cerebro.broker.addcommissioninfo(comminfo)
    
        dkwargs = dict()
        if args.fromdate is not None:
            fromdate = datetime.datetime.strptime(args.fromdate, '%d %b %Y')
            dkwargs['fromdate'] = fromdate
    
        if args.todate is not None:
            todate = datetime.datetime.strptime(args.todate, '%d %b %Y')
            dkwargs['todate'] = todate
    
        # if dataset is None, args.data has been given
        dataname = DATASETS.get(args.dataset, args.data)
        data0 = bt.feeds.GenericCSV(dataname=dataname,
        fromdate=datetime.datetime(2018, 1, 1),
        dtformat=('%Y-%m-%d %H:%M:%S'),
        datetime=0,
        high=2,
        low=3,
        open=1,
        close=4,
        volume=5,
        openinterest=-1,
        timeframe=bt.TimeFrame.Minutes,
    )
    
        cerebro.adddata(data0)
    
        cerebro.addstrategy(TheStrategy,
                            macd1=args.macd1, macd2=args.macd2,
                            macdsig=args.macdsig,
                            atrperiod=args.atrperiod,
                            atrdist=args.atrdist,
                            smaperiod=args.smaperiod,
                            dirperiod=args.dirperiod)
    
        cerebro.addsizer(FixedPerc, perc=args.cashalloc)
    
        # Add TimeReturn Analyzers for self and the benchmark data
        cerebro.addanalyzer(bt.analyzers.TimeReturn, _name='alltime_roi',
                            timeframe=bt.TimeFrame.NoTimeFrame)
    
        cerebro.addanalyzer(bt.analyzers.TimeReturn, data=data0, _name='benchmark',
                            timeframe=bt.TimeFrame.NoTimeFrame)
    
        # Add TimeReturn Analyzers fot the annuyl returns
        cerebro.addanalyzer(bt.analyzers.TimeReturn, timeframe=bt.TimeFrame.Years)
        # Add a SharpeRatio
        cerebro.addanalyzer(bt.analyzers.SharpeRatio, timeframe=bt.TimeFrame.Years,
                            riskfreerate=args.riskfreerate)
    
        # Add SQN to qualify the trades
        cerebro.addanalyzer(bt.analyzers.SQN)
        cerebro.addobserver(bt.observers.DrawDown)  # visualize the drawdown evol
    
        results = cerebro.run()
        st0 = results[0]
    
        for alyzer in st0.analyzers:
            alyzer.print()
    
        if args.plot:
            pkwargs = dict(style='bar')
            if args.plot is not True:  # evals to True but is not True
                npkwargs = eval('dict(' + args.plot + ')')  # args were passed
                pkwargs.update(npkwargs)
    
            cerebro.plot(**pkwargs)
    
    
    def parse_args(pargs=None):
    
        parser = argparse.ArgumentParser(
            formatter_class=argparse.ArgumentDefaultsHelpFormatter,
            description='Sample for Tharp example with MACD')
    
        group1 = parser.add_mutually_exclusive_group(required=True)
        group1.add_argument('--data', required=False, default=None,
                            help='Specific data to be read in')
    
        group1.add_argument('--dataset', required=False, action='store',
                            default=None, choices=DATASETS.keys(),
                            help='Choose one of the predefined data sets')
    
        parser.add_argument('--fromdate', required=False,
                            default='01 Jan 2016',
                            help='Starting date in %d %b %Y %H:%M:%S format')
    
        parser.add_argument('--todate', required=False,
                            default=None,
                            help='Ending date in %d %b %Y %H:%M:%S format')
    
        parser.add_argument('--cash', required=False, action='store',
                            type=float, default=50000,
                            help=('Cash to start with'))
    
        parser.add_argument('--cashalloc', required=False, action='store',
                            type=float, default=0.20,
                            help=('Perc (abs) of cash to allocate for ops'))
    
        parser.add_argument('--commperc', required=False, action='store',
                            type=float, default=0.0033,
                            help=('Perc (abs) commision in each operation. '
                                  '0.001 -> 0.1%%, 0.01 -> 1%%'))
    
        parser.add_argument('--macd1', required=False, action='store',
                            type=int, default=12,
                            help=('MACD Period 1 value'))
    
        parser.add_argument('--macd2', required=False, action='store',
                            type=int, default=26,
                            help=('MACD Period 2 value'))
    
        parser.add_argument('--macdsig', required=False, action='store',
                            type=int, default=9,
                            help=('MACD Signal Period value'))
    
        parser.add_argument('--atrperiod', required=False, action='store',
                            type=int, default=14,
                            help=('ATR Period To Consider'))
    
        parser.add_argument('--atrdist', required=False, action='store',
                            type=float, default=3.0,
                            help=('ATR Factor for stop price calculation'))
    
        parser.add_argument('--smaperiod', required=False, action='store',
                            type=int, default=30,
                            help=('Period for the moving average'))
    
        parser.add_argument('--dirperiod', required=False, action='store',
                            type=int, default=10,
                            help=('Period for SMA direction calculation'))
    
        parser.add_argument('--riskfreerate', required=False, action='store',
                            type=float, default=0.01,
                            help=('Risk free rate in Perc (abs) of the asset for '
                                  'the Sharpe Ratio'))
        # Plot options
        parser.add_argument('--plot', '-p', nargs='?', required=False,
                            metavar='kwargs', const=True,
                            help=('Plot the read data applying any kwargs passed\n'
                                  '\n'
                                  'For example:\n'
                                  '\n'
                                  '  --plot style="candle" (to plot candles)\n'))
    
        if pargs is not None:
            return parser.parse_args(pargs)
    
        return parser.parse_args()
    
    
    if __name__ == '__main__':
        runstrat()
    

Log in to reply
 

});