How to use pandas series with time index as in backtrader?

Mislav Šagovac

I want to trade only on time (minute) when daily volatility is higher than some number. There is a nice function in mlfinlab package to calculate daily volatility:
https://github.com/hudson-and-thames/mlfinlab/blob/master/mlfinlab/util/volatility.py

The problem is that function has pd series as first argument where index is time (in my case every minute). Backtrader as default track close as 'lines'. It doesn't track time index. I ma not sure how can pass close with time index as argument to get_daily_vol function.

Here is the simple sample code:

import backtrader as bt
import pandas as pd
import mlfinlab as ml

class RandomForestStrategy(bt.Strategy):

    def __init__(self):
        
        self.closes = np.array([])
        self.daily_vol = None

    def next(self):

        close_ = data["SPY"].Close
        self.closes = np.append(self.closes, close_)
        self.daily_vol = ml.util.get_daily_vol(
            pd.Series(self.closes, # index=how t odefine this?,
            lookback=50)

Probably one solution is to somehow rewrite the original function from mlfinlab but I would like to original function if possible because I will use more functions from this repo.

run-out

Backtrader is not pandas, so one cannot just copy the code over and have it work. You have a couple of options:

Create a custom indicator from the formulas in your library, or
Just calculate the result on your data, add it in as an extra column in your dataframe, and then import the dataframe with the extra column. You'll need to add the extra column to backtrader so it knows its there.

Mislav Šagovac

That's the main problem. I have ML model that have hundred of inputs (say indicators) plus some variables like the one in the question.

If I have to define every indicator with backtrader indicator function I would need 10 times more time do define the indicators for backtrader than anything else in the pipeline.

I am afraid backtrader is not suitable for that kind of investing, when you have hundreds of inputs, sampling of bars when to trade etc. At least I didn't find solution after reading the documentation. The backtesting can be done through big pandas df which already contains all necessary columns but not sure for live trading.

ab_trader

If I understand correctly - every price bar you want to calculate volatility using external function and than issue orders based on the results. That function uses pandas dataframe as input.

I believe that you will be able to make it. Develop an indicator which in the next() call get prices and date-time as arrays from bt data feed using self.data.get. These arrays can be easily converted to pandas dataframe and you can call that external function.

Check out the following -

Docs - Platform Concepts
Community - FAQ
Community - How to get datetime list from an indicator

Mislav Šagovac

But again I have ti construct indicator using bt.Indicator. Which is ok for one indicator, but I need to construct more than 300 indicators (features). Usually, using pd.DataFrame I just have to call one function which automatically produces all features... For example backetsting.py package has this option but it is really slow and have other problems.

ab_trader

@Mislav-Šagovac said in How to use pandas series with time index as in backtrader?:

But again I have ti construct indicator using bt.Indicator.

If you already have them in that external function, than you don't need to do it.