Data Analysis all wrong?

Taewoo Kim

Noob questions (once again!)

can someone explain "value" and "broker cash" in the chart that's plotted after cerebro has run?

If the chart says 593k in "broker cash" and 91k in "value", does that mean

broker cash = cash position ? (i.e. starting cash - P&L from the trades - that's what I got from reading fom the observer reference- https://www.backtrader.com/docu/observers-reference.html)
value = total sum of uncloses positions?

See screenshot: https://i.screenshot.net/0058dc0

I am having trouble getting draw down and sharpe ratio observers to work

Code:

cerebro.addanalyzer(btanalyzers.SharpeRatio, _name='mysharpe')

thestrats = cerebro.run()
thestrat = thestrats[0]

print('Sharpe Ratio:', thestrat.analyzers.mysharpe.get_analysis())

However, the result show up with nada:

Sharpe Ratio: OrderedDict([('sharperatio', None)])

Assuming my interpretation of #1 is correct... Pyfolio stats and the values reported in are WAY off. The chart stats show healthy profits, yeah pyfolio is showing that it's net negative

Is my implementation correct? Here's the code

cerebro.addanalyzer(bt.analyzers.PyFolio, _name='pyfolio')
pyfoliozer = strat.analyzers.getbyname('pyfolio')
returns, positions, transactions, gross_lev = pyfoliozer.get_pf_items()

# print(pyfoliozer.get_pf_items())

pyfolio.create_full_tear_sheet(
    returns,
    estimate_intraday=False,
    positions=positions,
    transactions=transactions,
    #gross_lev=gross_lev,
    #live_start_date='2017-01-01',
    round_trips=True)

Yes, pyfolio is installed system wide via pip3.

backtrader

can someone explain "value" and "broker cash" in the chart that's plotted after cerebro has run?

Chart is unreachable. The forum supports direct pasting of charts though.

Short positions add cash to the system. But even with the chart in hand one cannot know what you are doing, just by looking at the chart.

cash is the actual free cash. value is the value of the assets plus the cash. Short positions are negative (which is compensated by the addition of cash, because you have sold something). In real-life that cash isn't free, because you pay interest on the shorts.

I am having trouble getting draw down and sharpe ratio observers to work

That's isolated code. Nobody can make a diagnosis out of it.

Assuming my interpretation of #1 is correct... Pyfolio stats and the values reported in are WAY off. The chart stats show healthy profits, yeah pyfolio is showing that it's net negative

There is nothing there that shows what pyfolio itself shows but ... as stated above you seem to be entering short positions. Having that much cash doesn't imply you are making a profit. It simply means you have sold a lot of things, but you are probably still losing lots of money (add also the real life interest)

Since your final value is 91k, an educated guess is that you have bootstrapped your backtesting with 100k and have therefore lost 9k, even if you have lots of cash in the hand (which you eventually have to returns to your creditors or else face the consequences)

If you don't like that view (which should be what happens in real life) you can turn things upside down by setting the shortcash parameter in the broker to False. This won't add cash to your system when selling and will consider the positions to be all positive. See Docs - Broker

Taewoo Kim

ok here's the code in entirety:

from __future__ import (absolute_import, division, print_function,
                            unicode_literals)


    import argparse
    import datetime
    import random

    import backtrader as bt
    import backtrader.indicators as btind
    import backtrader.feeds as btfeeds


    class SMA_CrossOver(bt.Strategy):

        params = (('fast', 10), ('slow', 30))

        def __init__(self):

            sma_fast = btind.SMA(period=self.p.fast)
            sma_slow = btind.SMA(period=self.p.slow)

            self.buysig = btind.CrossOver(sma_fast, sma_slow)

        def next(self):
            if self.position.size:
                if self.buysig < 0:
                    self.sell()

            elif self.buysig > 0:
                self.buy()




    cerebro = bt.Cerebro()
    cerebro.broker.set_cash(100000)



    data = btfeeds.GenericCSVData(
        dataname='sample.csv',

        fromdate=datetime.datetime(2011, 1, 2),
        todate=datetime.datetime(2011, 1, 21),

        nullvalue=0.0,

        dtformat=('%Y-%m-%d %H:%M:%S'),

        datetime=0  ,
        open=1,
        high=2,
        low=3,
        close=4,
        volume=5,
        openinterest=6
    )


    cerebro.adddata(data)

    cerebro.addstrategy(SMA_CrossOver)
    cerebro.addanalyzer(bt.analyzers.PyFolio, _name='pyfolio')

    results = cerebro.run()
    strat = results[0]
    pyfoliozer = strat.analyzers.getbyname('pyfolio')

    returns, positions, transactions, gross_lev = pyfoliozer.get_pf_items()

    print('-- RETURNS')
    print(returns)
    print("\n\n")
    print('-- POSITIONS')
    print(positions)
    print("\n\n")
    print('-- TRANSACTIONS')
    print(transactions)
    print("\n\n")
    print('-- GROSS LEVERAGE')
    print(gross_lev)

    import pyfolio as pf
    pf.create_full_tear_sheet(
        returns,
        positions=positions,
        transactions=transactions,
        gross_lev=gross_lev,
        live_start_date='2011-01-02',
        round_trips=True)


    #cerebro.plot(style=args.plot_style)

Here is the csv file

Problem #1

TypeError: create_full_tear_sheet() got an unexpected keyword argument 'gross_lev'

Problem #2 - After I comment out gross_lev, i still get

TypeError: Empty 'DataFrame': no numeric data to plot

backtrader

@Taewoo-Kim said in Data Analysis all wrong?:

TypeError: create_full_tear_sheet() got an unexpected keyword argument 'gross_lev'

See here for the update on pyfolio: Blog - Pyfolio Integration

Since your data seems to be 1-minute based, you should configure the data feed with appropriate timeframe and compression.

In any case, the code above doesn't seem to be the one that was executed before:

SharpeRatio is not present
pyfolio was reporting negative values before, but it is producing an (expected) error now

Taewoo Kim

See here for the update on pyfolio: Blog - Pyfolio Integration

Ah, didn't see that.Thanks

Since your data seems to be 1-minute based, you should configure the data feed with appropriate timeframe and compression.

I changed the data feed by adding timeframe & compression

data = btfeeds.GenericCSVData(
    dataname='sample.csv',
    fromdate=datetime.datetime(2011, 1, 2),
    todate=datetime.datetime(2011, 1, 21),
    nullvalue=0.0,
    dtformat=('%Y-%m-%d %H:%M:%S'),
    datetime=0  ,
    open=1,
    high=2,
    low=3,
    close=4,
    volume=5,
    openinterest=6,
    timeframe=bt.TimeFrame.Minutes,
    compression=1
)

Yet i still get

TypeError: Empty 'DataFrame': no numeric data to plot

Sharpe ratio... sorry about that. Here's the code I used to get the sharpe ratio error:

from __future__ import (absolute_import, division, print_function,
                        unicode_literals)


import argparse
import datetime
import random
import sys
import backtrader as bt
import backtrader.indicators as btind
import backtrader.feeds as btfeeds
import backtrader.analyzers as btanalyzers

class SMA_CrossOver(bt.Strategy):

    params = (('fast', 10), ('slow', 30))

    def __init__(self):

        sma_fast = btind.SMA(period=self.p.fast)
        sma_slow = btind.SMA(period=self.p.slow)

        self.buysig = btind.CrossOver(sma_fast, sma_slow)

    def next(self):
        if self.position.size:
            if self.buysig < 0:
                self.sell()

        elif self.buysig > 0:
            self.buy()


cerebro = bt.Cerebro()
cerebro.broker.set_cash(100000)


data = btfeeds.GenericCSVData(
    dataname='sample.csv',

    fromdate=datetime.datetime(2011, 1, 2),
    todate=datetime.datetime(2011, 1, 21),

    nullvalue=0.0,

    dtformat=('%Y-%m-%d %H:%M:%S'),

    datetime=0  ,
    open=1,
    high=2,
    low=3,
    close=4,
    volume=5,
    openinterest=6
)


cerebro.adddata(data)

cerebro.addstrategy(SMA_CrossOver)
cerebro.addanalyzer(bt.analyzers.PyFolio, _name='pyfolio')


# Analyzer
cerebro.addanalyzer(btanalyzers.SharpeRatio, _name='mysharpe')

thestrats = cerebro.run()
thestrat = thestrats[0]

print('Sharpe Ratio:', thestrat.analyzers.mysharpe.get_analysis())

FYI: I have the latest backtrader

>>> import backtrader
>>> print(backtrader.__version__)
1.9.41.116

backtrader

@Taewoo-Kim said in Data Analysis all wrong?:

TypeError: Empty 'DataFrame': no numeric data to plot

The timeframe and compression are not for pyfolio. The pyfolio API has changed as indicated in the update to the Blog post quoted above and the analyzer doesn't work with the latest version of pyfolio. You may try installing pyfolio-0.5.1 which is known to work.

The posts are getting more complicated. Let's try to get things summarized:

You stated that you had a problem with over 500k in cash and yet 91kin value

Which script did you use to get into such a problem? (which seems simply a case of opening many short positions)
Is the same script as the one generating your None for the SharpeRatio?

The script for the SharpeRatio above still contains no timeframe and compression for the data feed.

In any case the defaults for the SharpeRatio is to work with Years. See: Docs - Analyzers Reference. Your data contains roughly 20 days. It is unlikely the analyzer can calculate anything with that data feed.

Taewoo Kim

You stated that you had a problem with over 500k in cash and yet 91kin value

This is fixed. You were right. There was a bug that kep opening tons of short positions. Sorry about that.

Is the same script as the one generating your None for the SharpeRatio?

I should've separated the thread but your forum software is limiting # of posts I can per day, so i try to batch them in one shot.

I used the same code base to test both sharpe ratio issue as well as pyfolio issue.

The script for the `SharpeRatio` above still contains no `timeframe` and `compression` for the  data feed.

Ok, will test w/ full years worth as well as specifying timeframe/compression

The timeframe and compression are not for pyfolio. The pyfolio API has changed as indicated in the update to the Blog post quoted above and the analyzer doesn't work with the latest version of pyfolio. You may try installing pyfolio-0.5.1 which is known to work.

Ok, downgraded to pyfolio 0.5.1

>>> import pyfolio
>>> print(pyfolio.__version__)
v0.5.1

Now i get this error (on the same code base)

Out-of-Sample Months: 0
Backtest Months: 0
Traceback (most recent call last):
  File "test_pyfolio.py", line 95, in <module>
    round_trips=True)
  File "/usr/local/lib/python3.5/dist-packages/pyfolio/tears.py", line 170, in create_full_tear_sheet
    set_context=set_context)
  File "/usr/local/lib/python3.5/dist-packages/pyfolio/plotting.py", line 49, in call_w_context
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/pyfolio/tears.py", line 265, in create_returns_tear_sheet
    live_start_date=live_start_date)
  File "/usr/local/lib/python3.5/dist-packages/pyfolio/plotting.py", line 556, in show_perf_stats
    factor_returns=factor_returns)
  File "/usr/local/lib/python3.5/dist-packages/pyfolio/timeseries.py", line 784, in perf_stats
    stats[stat_func.__name__] = stat_func(returns)
  File "/usr/local/lib/python3.5/dist-packages/pyfolio/timeseries.py", line 486, in stability_of_timeseries
    cum_log_returns.values)[2]
  File "/usr/local/lib/python3.5/dist-packages/scipy/stats/_stats_mstats_common.py", line 72, in linregress
    raise ValueError("Inputs must not be empty.")
ValueError: Inputs must not be empty.

backtrader

@Taewoo-Kim said in Data Analysis all wrong?:

limiting # of posts I can per day

Spam anti-measures limit # of posts within short time spans (not per day)

Now i get this error (on the same code base)

Probably because downgrading to pyfolio-0.5.1 doesn't downgrade the packages with which it works. In this case and due to the changes in the pyfolio API, it will take sometime to get it working again.