For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

How to use pandas series with time index as in backtrader?



  • I want to trade only on time (minute) when daily volatility is higher than some number. There is a nice function in mlfinlab package to calculate daily volatility:
    https://github.com/hudson-and-thames/mlfinlab/blob/master/mlfinlab/util/volatility.py

    The problem is that function has pd series as first argument where index is time (in my case every minute). Backtrader as default track close as 'lines'. It doesn't track time index. I ma not sure how can pass close with time index as argument to get_daily_vol function.

    Here is the simple sample code:

    import backtrader as bt
    import pandas as pd
    import mlfinlab as ml
    
    class RandomForestStrategy(bt.Strategy):
    
        def __init__(self):
            
            self.closes = np.array([])
            self.daily_vol = None
    
        def next(self):
    
            close_ = data["SPY"].Close
            self.closes = np.append(self.closes, close_)
            self.daily_vol = ml.util.get_daily_vol(
                pd.Series(self.closes, # index=how t odefine this?,
                lookback=50)
    

    Probably one solution is to somehow rewrite the original function from mlfinlab but I would like to original function if possible because I will use more functions from this repo.



  • Backtrader is not pandas, so one cannot just copy the code over and have it work. You have a couple of options:

    1. Create a custom indicator from the formulas in your library, or
    2. Just calculate the result on your data, add it in as an extra column in your dataframe, and then import the dataframe with the extra column. You'll need to add the extra column to backtrader so it knows its there.


  • That's the main problem. I have ML model that have hundred of inputs (say indicators) plus some variables like the one in the question.

    If I have to define every indicator with backtrader indicator function I would need 10 times more time do define the indicators for backtrader than anything else in the pipeline.

    I am afraid backtrader is not suitable for that kind of investing, when you have hundreds of inputs, sampling of bars when to trade etc. At least I didn't find solution after reading the documentation. The backtesting can be done through big pandas df which already contains all necessary columns but not sure for live trading.



  • If I understand correctly - every price bar you want to calculate volatility using external function and than issue orders based on the results. That function uses pandas dataframe as input.

    I believe that you will be able to make it. Develop an indicator which in the next() call get prices and date-time as arrays from bt data feed using self.data.get. These arrays can be easily converted to pandas dataframe and you can call that external function.

    Check out the following -

    Docs - Platform Concepts
    Community - FAQ
    Community - How to get datetime list from an indicator



  • But again I have ti construct indicator using bt.Indicator. Which is ok for one indicator, but I need to construct more than 300 indicators (features). Usually, using pd.DataFrame I just have to call one function which automatically produces all features... For example backetsting.py package has this option but it is really slow and have other problems.



  • @Mislav-Šagovac said in How to use pandas series with time index as in backtrader?:

    But again I have ti construct indicator using bt.Indicator.

    If you already have them in that external function, than you don't need to do it.


Log in to reply
 

});