For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

Pyfolio Analyzer doesn't get the right positions



  • Hi, Backtraders!

    I facing an issue retrieving the positions from a backtesting in Backtrader. I'm using Pyfolio analyzer to try to achieve this but it doesn't work properly.

    This is what I got in positions for my strategy:
    0_1537295944079_Screenshot from 2018-09-18 13-37-19.png

    I tried the following function of Pyfolio but it doesn't work either:
    0_1537296049508_Screenshot from 2018-09-18 13-37-31.png

    The code I am using in backtrader to run the strategy is this one:

        cerebro = bt.Cerebro()
    
        # Add pyfolio analyzer
        cerebro.addanalyzer(bt.analyzers.PyFolio, _name='pyfolio')
    
        # Data feed --------------------------------
        for equity in equity_list:
            data = get_data(
                hp.cd_dospuntos(os.getcwd(), 2) + '/input_gral/backtest_data/' + equity + '.csv',
                from_date,
                to_date
            )
            cerebro.adddata(data, name=equity)
    
        # Set our desired cash start
        cerebro.broker.setcash(100000.0)
    
        # Comissions
        cerebro.broker.setcommission(commission=0.001)
    
        # Configuramos el slipage a 0.5%
        cerebro.broker.set_slippage_perc(perc=0.005,
                                         slip_open=True, slip_limit=True,
                                         slip_match=True, slip_out=False)
    
        # When working with large percentages it would be good to disable checksubmit in the broker. The rationale being
        # that if the current total portfolio is large, the broker might reject some orders if no cash is left.
        # You could order the orders in such a way that this doesn't happen, but disabling checksubmitwould be easier.
        cerebro.broker.set_checksubmit(checksubmit=False)
    
        # Strategy
        cerebro.addstrategy(
            TestStrategy,
            buy_and_hold=buy_and_hold,
            sellAll_date=sellAll_date,
        )
    
        start = time.time()
        print('Starting Portfolio Value: %.2f' % cerebro.broker.getvalue())
    
        # Execute
        results = cerebro.run(runonce=False)
    
        print('Final Portfolio Value: %.2f' % cerebro.broker.getvalue())
        end = time.time()
    
        # Obtiene los resultados para el posterior análisis con pyfolio
        strat = results[0]
        pyfoliozer = strat.analyzers.getbyname('pyfolio')
        returns, positions, transactions, gross_lev = pyfoliozer.get_pf_items()
    
        # Grabamos los resultados a un archivo pickle
        filepath = 'out/BTesting_results.pkl'
        if buy_and_hold:
            filepath = 'out/BTesting_results_buy_and_hold.pkl'
        file = open(filepath, 'wb')
        pikle_dict = {
            'returns': returns,
            'positions': positions,
            'transactions': transactions,
            'gross_lev': gross_lev,
        }
        pickle.dump(pikle_dict, file)
    

    Any help is welcome. Thanks in advance.



  • It seems to be a bug in pyfolio.py:

            pss = self.rets['positions']
            ps = [[k] + v[-2:] for k, v in iteritems(pss)]
            cols = ps.pop(0)  # headers are in the first entry
            positions = DF.from_records(ps, index=cols[0], columns=cols)
            positions.index = pandas.to_datetime(positions.index)
            positions.index = positions.index.tz_localize('UTC')
    

    Line 130 is ps = [[k] + v[-2:] for k, v in iteritems(pss)] but it should be ps = [[k] + v[:] for k, v in iteritems(pss)]. With that change now I get the right positions in my strategy:

    0_1537418741682_Screenshot from 2018-09-19 23-45-07.png

    If the bug could be confirmed, it would be great to get fixed in a future release.

    Happy trading!


  • administrators

    See this commit from 2017-03-12: https://github.com/backtrader/backtrader/commit/e407a583d9cf9e873104c90aa5ee8d155bffa307

            -ps = [[k] + v for k, v in iteritems(pss)]
            +ps = [[k] + v[-2:] for k, v in iteritems(pss)]
    

    Your proposal is to turn this

    @j-javier-gálvez said in Pyfolio Analyzer doesn't get the right positions:

    ps = [[k] + v[-2:] for k, v in iteritems(pss)]
    

    into

    @j-javier-gálvez said in Pyfolio Analyzer doesn't get the right positions:

    ps = [[k] + v[:] for k, v in iteritems(pss)]
    

    which literally undo that commit. Back then the commit was done to track the changes in the PyFolio API. After a given point in time it was deemed not sensible to try to track it, given that PyFolio is a tool which will be changed to answer to the demands of the platforms related to it.

    See here: Docs - PyFolio Overview

    Suggestions:

    • You keep your changes as an external analyzer (you put the entire analyzer in your project9
    • You monkey patch the method which contains the offending (in your case) code

    The only alternative within backtrader would be to add a parameter which would let you select that behavior as opposed to the current one.



  • @backtrader Thank you very much for your useful answer. Greetings!