Data Analysis all wrong?



  • Noob questions (once again!)

    1. can someone explain "value" and "broker cash" in the chart that's plotted after cerebro has run?

    If the chart says 593k in "broker cash" and 91k in "value", does that mean

    See screenshot: https://i.screenshot.net/0058dc0

    1. I am having trouble getting draw down and sharpe ratio observers to work

    Code:

    cerebro.addanalyzer(btanalyzers.SharpeRatio, _name='mysharpe')
    
    thestrats = cerebro.run()
    thestrat = thestrats[0]
    
    print('Sharpe Ratio:', thestrat.analyzers.mysharpe.get_analysis())
    

    However, the result show up with nada:

    Sharpe Ratio: OrderedDict([('sharperatio', None)])
    
    1. Assuming my interpretation of #1 is correct... Pyfolio stats and the values reported in are WAY off. The chart stats show healthy profits, yeah pyfolio is showing that it's net negative

    Is my implementation correct? Here's the code

    cerebro.addanalyzer(bt.analyzers.PyFolio, _name='pyfolio')
    pyfoliozer = strat.analyzers.getbyname('pyfolio')
    returns, positions, transactions, gross_lev = pyfoliozer.get_pf_items()
    
    # print(pyfoliozer.get_pf_items())
    
    pyfolio.create_full_tear_sheet(
        returns,
        estimate_intraday=False,
        positions=positions,
        transactions=transactions,
        #gross_lev=gross_lev,
        #live_start_date='2017-01-01',
        round_trips=True)
    

    Yes, pyfolio is installed system wide via pip3.


  • administrators

    1. can someone explain "value" and "broker cash" in the chart that's plotted after cerebro has run?

    Chart is unreachable. The forum supports direct pasting of charts though.

    Short positions add cash to the system. But even with the chart in hand one cannot know what you are doing, just by looking at the chart.

    cash is the actual free cash. value is the value of the assets plus the cash. Short positions are negative (which is compensated by the addition of cash, because you have sold something). In real-life that cash isn't free, because you pay interest on the shorts.

    1. I am having trouble getting draw down and sharpe ratio observers to work

    That's isolated code. Nobody can make a diagnosis out of it.

    1. Assuming my interpretation of #1 is correct... Pyfolio stats and the values reported in are WAY off. The chart stats show healthy profits, yeah pyfolio is showing that it's net negative

    There is nothing there that shows what pyfolio itself shows but ... as stated above you seem to be entering short positions. Having that much cash doesn't imply you are making a profit. It simply means you have sold a lot of things, but you are probably still losing lots of money (add also the real life interest)

    Since your final value is 91k, an educated guess is that you have bootstrapped your backtesting with 100k and have therefore lost 9k, even if you have lots of cash in the hand (which you eventually have to returns to your creditors or else face the consequences)

    If you don't like that view (which should be what happens in real life) you can turn things upside down by setting the shortcash parameter in the broker to False. This won't add cash to your system when selling and will consider the positions to be all positive. See Docs - Broker



  • ok here's the code in entirety:

    from __future__ import (absolute_import, division, print_function,
                                unicode_literals)
    
    
        import argparse
        import datetime
        import random
    
        import backtrader as bt
        import backtrader.indicators as btind
        import backtrader.feeds as btfeeds
    
    
        class SMA_CrossOver(bt.Strategy):
    
            params = (('fast', 10), ('slow', 30))
    
            def __init__(self):
    
                sma_fast = btind.SMA(period=self.p.fast)
                sma_slow = btind.SMA(period=self.p.slow)
    
                self.buysig = btind.CrossOver(sma_fast, sma_slow)
    
            def next(self):
                if self.position.size:
                    if self.buysig < 0:
                        self.sell()
    
                elif self.buysig > 0:
                    self.buy()
    
    
    
    
        cerebro = bt.Cerebro()
        cerebro.broker.set_cash(100000)
    
    
    
        data = btfeeds.GenericCSVData(
            dataname='sample.csv',
    
            fromdate=datetime.datetime(2011, 1, 2),
            todate=datetime.datetime(2011, 1, 21),
    
            nullvalue=0.0,
    
            dtformat=('%Y-%m-%d %H:%M:%S'),
    
            datetime=0  ,
            open=1,
            high=2,
            low=3,
            close=4,
            volume=5,
            openinterest=6
        )
    
    
        cerebro.adddata(data)
    
        cerebro.addstrategy(SMA_CrossOver)
        cerebro.addanalyzer(bt.analyzers.PyFolio, _name='pyfolio')
    
        results = cerebro.run()
        strat = results[0]
        pyfoliozer = strat.analyzers.getbyname('pyfolio')
    
        returns, positions, transactions, gross_lev = pyfoliozer.get_pf_items()
    
        print('-- RETURNS')
        print(returns)
        print("\n\n")
        print('-- POSITIONS')
        print(positions)
        print("\n\n")
        print('-- TRANSACTIONS')
        print(transactions)
        print("\n\n")
        print('-- GROSS LEVERAGE')
        print(gross_lev)
    
        import pyfolio as pf
        pf.create_full_tear_sheet(
            returns,
            positions=positions,
            transactions=transactions,
            gross_lev=gross_lev,
            live_start_date='2011-01-02',
            round_trips=True)
    
    
        #cerebro.plot(style=args.plot_style)
    

    Here is the csv file

    Problem #1

    TypeError: create_full_tear_sheet() got an unexpected keyword argument 'gross_lev'
    

    Problem #2 - After I comment out gross_lev, i still get

    TypeError: Empty 'DataFrame': no numeric data to plot

  • administrators

    @Taewoo-Kim said in Data Analysis all wrong?:

    TypeError: create_full_tear_sheet() got an unexpected keyword argument 'gross_lev'

    See here for the update on pyfolio: Blog - Pyfolio Integration

    Since your data seems to be 1-minute based, you should configure the data feed with appropriate timeframe and compression.

    In any case, the code above doesn't seem to be the one that was executed before:

    • SharpeRatio is not present
    • pyfolio was reporting negative values before, but it is producing an (expected) error now


  • See here for the update on pyfolio: Blog - Pyfolio Integration

    Ah, didn't see that.Thanks

    Since your data seems to be 1-minute based, you should configure the data feed with appropriate timeframe and compression.

    I changed the data feed by adding timeframe & compression

    data = btfeeds.GenericCSVData(
        dataname='sample.csv',
        fromdate=datetime.datetime(2011, 1, 2),
        todate=datetime.datetime(2011, 1, 21),
        nullvalue=0.0,
        dtformat=('%Y-%m-%d %H:%M:%S'),
        datetime=0  ,
        open=1,
        high=2,
        low=3,
        close=4,
        volume=5,
        openinterest=6,
        timeframe=bt.TimeFrame.Minutes,
        compression=1
    )
    

    Yet i still get

    TypeError: Empty 'DataFrame': no numeric data to plot
    

    Sharpe ratio... sorry about that. Here's the code I used to get the sharpe ratio error:

    from __future__ import (absolute_import, division, print_function,
                            unicode_literals)
    
    
    import argparse
    import datetime
    import random
    import sys
    import backtrader as bt
    import backtrader.indicators as btind
    import backtrader.feeds as btfeeds
    import backtrader.analyzers as btanalyzers
    
    class SMA_CrossOver(bt.Strategy):
    
        params = (('fast', 10), ('slow', 30))
    
        def __init__(self):
    
            sma_fast = btind.SMA(period=self.p.fast)
            sma_slow = btind.SMA(period=self.p.slow)
    
            self.buysig = btind.CrossOver(sma_fast, sma_slow)
    
        def next(self):
            if self.position.size:
                if self.buysig < 0:
                    self.sell()
    
            elif self.buysig > 0:
                self.buy()
    
    
    cerebro = bt.Cerebro()
    cerebro.broker.set_cash(100000)
    
    
    data = btfeeds.GenericCSVData(
        dataname='sample.csv',
    
        fromdate=datetime.datetime(2011, 1, 2),
        todate=datetime.datetime(2011, 1, 21),
    
        nullvalue=0.0,
    
        dtformat=('%Y-%m-%d %H:%M:%S'),
    
        datetime=0  ,
        open=1,
        high=2,
        low=3,
        close=4,
        volume=5,
        openinterest=6
    )
    
    
    cerebro.adddata(data)
    
    cerebro.addstrategy(SMA_CrossOver)
    cerebro.addanalyzer(bt.analyzers.PyFolio, _name='pyfolio')
    
    
    # Analyzer
    cerebro.addanalyzer(btanalyzers.SharpeRatio, _name='mysharpe')
    
    thestrats = cerebro.run()
    thestrat = thestrats[0]
    
    print('Sharpe Ratio:', thestrat.analyzers.mysharpe.get_analysis())
    

    FYI: I have the latest backtrader

    >>> import backtrader
    >>> print(backtrader.__version__)
    1.9.41.116

  • administrators

    @Taewoo-Kim said in Data Analysis all wrong?:

    TypeError: Empty 'DataFrame': no numeric data to plot

    The timeframe and compression are not for pyfolio. The pyfolio API has changed as indicated in the update to the Blog post quoted above and the analyzer doesn't work with the latest version of pyfolio. You may try installing pyfolio-0.5.1 which is known to work.

    The posts are getting more complicated. Let's try to get things summarized:

    • You stated that you had a problem with over 500k in cash and yet 91kin value

      Which script did you use to get into such a problem? (which seems simply a case of opening many short positions)

    • Is the same script as the one generating your None for the SharpeRatio?

      The script for the SharpeRatio above still contains no timeframe and compression for the data feed.

      In any case the defaults for the SharpeRatio is to work with Years. See: Docs - Analyzers Reference. Your data contains roughly 20 days. It is unlikely the analyzer can calculate anything with that data feed.



    • You stated that you had a problem with over 500k in cash and yet 91kin value

    This is fixed. You were right. There was a bug that kep opening tons of short positions. Sorry about that.

    • Is the same script as the one generating your None for the SharpeRatio?

    I should've separated the thread but your forum software is limiting # of posts I can per day, so i try to batch them in one shot.

    I used the same code base to test both sharpe ratio issue as well as pyfolio issue.

    The script for the `SharpeRatio` above still contains no `timeframe` and `compression` for the  data feed.
    

    Ok, will test w/ full years worth as well as specifying timeframe/compression

    The timeframe and compression are not for pyfolio. The pyfolio API has changed as indicated in the update to the Blog post quoted above and the analyzer doesn't work with the latest version of pyfolio. You may try installing pyfolio-0.5.1 which is known to work.

    Ok, downgraded to pyfolio 0.5.1

    >>> import pyfolio
    >>> print(pyfolio.__version__)
    v0.5.1
    

    Now i get this error (on the same code base)

    Out-of-Sample Months: 0
    Backtest Months: 0
    Traceback (most recent call last):
      File "test_pyfolio.py", line 95, in <module>
        round_trips=True)
      File "/usr/local/lib/python3.5/dist-packages/pyfolio/tears.py", line 170, in create_full_tear_sheet
        set_context=set_context)
      File "/usr/local/lib/python3.5/dist-packages/pyfolio/plotting.py", line 49, in call_w_context
        return func(*args, **kwargs)
      File "/usr/local/lib/python3.5/dist-packages/pyfolio/tears.py", line 265, in create_returns_tear_sheet
        live_start_date=live_start_date)
      File "/usr/local/lib/python3.5/dist-packages/pyfolio/plotting.py", line 556, in show_perf_stats
        factor_returns=factor_returns)
      File "/usr/local/lib/python3.5/dist-packages/pyfolio/timeseries.py", line 784, in perf_stats
        stats[stat_func.__name__] = stat_func(returns)
      File "/usr/local/lib/python3.5/dist-packages/pyfolio/timeseries.py", line 486, in stability_of_timeseries
        cum_log_returns.values)[2]
      File "/usr/local/lib/python3.5/dist-packages/scipy/stats/_stats_mstats_common.py", line 72, in linregress
        raise ValueError("Inputs must not be empty.")
    ValueError: Inputs must not be empty.

  • administrators

    @Taewoo-Kim said in Data Analysis all wrong?:

    limiting # of posts I can per day

    Spam anti-measures limit # of posts within short time spans (not per day)

    Now i get this error (on the same code base)

    Probably because downgrading to pyfolio-0.5.1 doesn't downgrade the packages with which it works. In this case and due to the changes in the pyfolio API, it will take sometime to get it working again.


Log in to reply
 

Looks like your connection to Backtrader Community was lost, please wait while we try to reconnect.