Get mininum and maximum dates in backtrader DataFeed

gabrielfior

I am trying to simulate a BuyAndHold strategy, following the example given in the BackTrader tutorial. My problem is that my strategy does not run for the full time period that I defined for my data feeds. One possibility would be that the products queried by the DataFeed (I am using YahooFinanceData) were not yet available.

Is it possible to identify the max and min dates that are available in the DataFeed? For instance, the product VGEK.DE is not available before 24.09.2019, hence I would assume that the strategy cannot be back-tested before that data.

Please find my code below.

# Defining Data Feed
start=datetime.datetime(2017,1,1)
end=datetime.datetime(2020,7,15)
pacific_feed = bt.feeds.YahooFinanceData(dataname='VGEK.DE',period='d', fromdate=start,
                                todate=end)

# Defining strategy
class BuyAndHold_More_Fund_Pacific(bt.Strategy):
    params = dict(
        monthly_cash=700.0,  # amount of cash to buy every month
    )

    def start(self):
        # Activate the fund mode and set the default value at 100
        self.broker.set_fundmode(fundmode=True, fundstartval=100.00)
        self.cash_start = self.broker.get_cash()
        self.val_start = 100.0

        # Add a timer which will be called on the 1st trading day of the month
        self.add_timer(
            bt.timer.SESSION_END,  # when it will be called
            monthdays=[1],  # called on the 1st day of the month
            monthcarry=True,  # called on the 2nd day if the 1st is holiday
        )

        
    def notify_timer(self, timer, when, *args, **kwargs):
        # Add the influx of monthly cash to the broker
        self.broker.add_cash(self.p.monthly_cash)

        # buy available cash
        target_value = self.broker.getcash() + self.p.monthly_cash      
        data_feed = self.getdatabyname(name='VGEK.DE')
        size = round(target_value/self.data.close,2)
        order = self.buy(exectype=bt.Order.Market,size=size,data=data_feed)        
        
    def stop(self):
        # calculate the actual returns
        self.roi = (self.broker.get_value() / self.cash_start) - 1.0
        self.froi = self.broker.get_fundvalue() - self.val_start
        print('ROI:        {:.2f}%'.format(100.0 * self.roi))
        print('Fund Value: {:.2f}%'.format(self.froi))
        print ('broker fund value {} broker val start {}'.format(self.broker.get_fundvalue(),self.val_start))

# run back testing
    cerebro = bt.Cerebro()
    
    # Add a strategy
    cerebro.addstrategy(BuyAndHold_More_Fund_Pacific)
    cerebro.broker.set_cash(0.000001)
    cerebro.broker.set_fundmode(True)
    
    cerebro.adddata(data_pacific, name='VGEK.DE')

    print('Starting Portfolio Value: %.2f' % cerebro.broker.getvalue())

    # Observers
    cerebro.addobserver(bt.observers.TimeReturn, timeframe=bt.TimeFrame.Days)
    
    # Analyzers
    cerebro.addanalyzer(bt.analyzers.PyFolio, _name='pyfolio', timeframe=bt.TimeFrame.Days)
    cerebro.addanalyzer(bt.analyzers.PositionsValue, headers=True, cash=True, _name='mypositions')
    cerebro.addanalyzer(bt.analyzers.TimeReturn,_name='timereturn')

    results = cerebro.run()

    print('Final Portfolio Value: %.2f' % cerebro.broker.getvalue())

What is also not clear to me is why transactions are not completed every month. If I export the transactions:

transactions.head()
amount price sid symbol value
date
2018-02-02 23:59:59.999989+00:00 81.30 42.74 0 VGEK.DE -3474.7620
2018-05-03 23:59:59.999989+00:00 50.55 41.77 0 VGEK.DE -2111.4735
2018-08-02 23:59:59.999989+00:00 46.09 45.77 0 VGEK.DE -2109.5393
2019-01-03 23:59:59.999989+00:00 83.79 41.33 0 VGEK.DE -3463.0407
2019-08-02 23:59:59.999989+00:00 95.61 50.33 0 VGEK.DE -4812.0513

I see that there is not a transaction being completed every month.

Thanks for the help!

ab_trader

@gabrielfior said in Get mininum and maximum dates in backtrader DataFeed:

I see that there is not a transaction being completed every month.

Without getting deeper into the script - you defined amount of shares to buy based on the close price. bt executes the order based on the next bar open price. If this price is higher than the previous close price, than it is not enough cash to buy and orders are rejected.

Your first issue is not clear. bt works only if data available, if it is not available, than backtest is meaningless.

gabrielfior

@ab_trader said in Get mininum and maximum dates in backtrader DataFeed:

available, if it is not available, than backtest is meaningl

Thanks for the clarification regarding buy. I will adapt my script.

Rephrasing my first issue: is it possible to check before executing cerebro.run(), if the data feeds comprise the simulation period? A concrete example:

I have 2 data feeds, VUSA.DE and VGEK.DE.
I follow the BuyAndHold strategy and buy a defined size of both products on the 1st day of each month of the simulation
I want to backtest my strategy between 2015-01-01 and 2020-07-01. If either one of the products is not available previously, I believe that the order is not going to be executed, thus the portfolio will not reflect my strategy correctly (due to the missing data feed). Thus I would like to check beforehand (e.g. by downloading the DataFeed from yahoo to a local file, for instance) if the data feed has data for the whole simulated period.

run-out

You can pre-scrub your data using pandas, then if it passes your test, load the data into the backtest.

ab_trader

So, if I understand correctly, than you want to have backtest goes only if all data feeds are available. And if one of them is missing, than don't do anything. If this is what you want, than you can try to do the following:

in the strategy init or start set self.do_test = False
in the strategy nextstart set self.do_test = True
in the strategy notify_timer do actions only if self.do_test = True

I didn't test it, but it should work.

Or you can pre-process your data feeds as @run-out proposed.

run-out

One other option might be to check the array length of the datas in init or start.
For example if I put in four different stocks with different start dates:

diff_secs = [
    ("FB", datetime.datetime(2019, 5, 1)),
    ("TSLA", datetime.datetime(2019, 6, 1)),
    ("NVDA", datetime.datetime(2019, 7, 1)),
    ("MSFT", datetime.datetime(2019, 8, 1)),
]

for co in diff_secs:
    data = bt.feeds.YahooFinanceData(
        dataname=co[0],
        fromdate=co[1],
        todate=datetime.datetime(2020, 7, 8),
    )
    cerebro.adddata(data)

Then you can count the lenght of array:

def __init__(self):
    for d in self.datas:
        print(f"stock: {d._name}, length: {len(d.array)}")

Resulting in:

stock: FB, length: 299
stock: TSLA, length: 277
stock: NVDA, length: 257
stock: MSFT, length: 235

gabrielfior

@run-out @ab_trader Thanks for the examples and the help.
I will try those approaches in the next days.

Thanks