For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

Filter on Replaydata



  • I'd like to use my minute data, replay it at 5s, and then use that for a Renko filter. Is this possible?

    I have a csv:
    data = bt.feeds.GenericCSVData(datafile, timeframe = bt.TimeFrame.Minutes)

    However,
    cerebro.replaydata(data) does not return a data object for which I could use replayedData.addfilter(bt.filters.Renko, **renko_args)

    Is this possible?


  • administrators

    @algoguy235 said in Filter on Replaydata:

    Is this possible?

    Give it a try!



  • I want to "force" my one minute data to either replaydata or resampledata at 5 seconds (the use-case is that the Renko filter works properly this way, able to print multiple bricks for single 1 min candle if necessary). However, I have not been able to figure out how to do this type of resample.

    My next attempt will be to create a "dummy" data feed in pandas that has samples at 5 seconds, and feed that into cerebro. I think that should work. Any other suggestions on how to force this kind of "resampling at higher resolution"?



  • It seems like both resampledata and replaydata are designed for manipulating raw data into a lower resolution/samplerate, but there is nothing built-in the system to replay data to a higher resolution and force cerebro to sample at the high rate. So for the sake of closing out this thread I'll post the code that I used to do this using pandas.

    strat = STRAT_OBJECT
    symbol    = 'symbol_string'
    datafile  = './file_location_string'
    startDate = datetime.datetime(2019,1,1)
    endDate = datetime.datetime(2019,1,2)
    boxsize   = 1
    
    # Sample Size: 5s gives opportunity for 12 renko boxes to print in single minute (which is probably overkill)
    sampSize = 5   
     
    print(' '.join([Dt.now().strftime('%m-%d %H:%M::'),
                    'Running:', symbol, 'at: 1m candles from:',
                    str(startDate)[:-9], 'to:', str(endDate)[:-9]]))
    
    if not 60 % sampSize == 0: raise ValueError('sampSize must be a factor of 60')
    
    # my data provider has csv files with separate fields for date and time
    columns        = ['ddate','ttime','open','high','low','close','volume']
    dfraw          = pd.read_csv(datafile, names = columns)
    dfraw['dtime'] = dfraw.ddate + ' ' + dfraw.ttime
    dfraw.dtime    = pd.to_datetime(dfraw.dtime, format='%m/%d/%Y %H:%M')
    dfraw          = dfraw[(dfraw.dtime >= startDate) & (dfraw.dtime < endDate)]
    
    # this upsample method was chosen for optimized speed and efficiency
    timeserieslist = []
    for t in dfraw.dtime:
        daterange = pd.date_range(t , t + DT.timedelta(seconds = 60-sampSize), periods = 60//sampSize)
        timeserieslist.extend(daterange.to_list())
    df_upsamp = pd.DataFrame(timeserieslist, columns = ['dtime'])
    # print(df_upsamp.head())
    
    # map and forward fill the 1m data to replay across the upsampled datetime field
    dfraw_ri = dfraw.set_index('dtime') # ri = re-indexed (helps w Series.map efficiency)
    df_upsamp['open']  = df_upsamp.dtime.map(dfraw_ri.open)
    df_upsamp['high']  = df_upsamp.dtime.map(dfraw_ri.high)
    df_upsamp['low']   = df_upsamp.dtime.map(dfraw_ri.low)
    df_upsamp['close'] = df_upsamp.dtime.map(dfraw_ri.close)
    df_upsamp.fillna(method='ffill', inplace=True)
    # print(df_upsamp.head())
    
    cerebro = bt.Cerebro()
    cerebro.addstrategy(strat)
    
    data_5s = bt.feeds.PandasData(
                            dataname = df_upsamp,
                            timeframe = bt.TimeFrame.Seconds,
                            compression = sampSize,
                            datetime = 'dtime',
                            )
    data_5s.plotinfo.plot = False
    
    data_1m = bt.feeds.PandasData(
                            dataname = dfraw,
                            timeframe = bt.TimeFrame.Minutes,
                            datetime = 'dtime',
                            )
    
    renko_args = dict(
                    hilo     = False,
                    align    = 1,
                    size     = boxsize)
    
    cerebro.adddata(data_5s, '5s_dummy')
    cerebro.adddata(data_1m, '1m_standard')
        
    data_Rk = data_5s.clone()
    data_Rk.addfilter(bt.filters.Renko, **renko_args)
    cerebro.adddata(data_Rk, 'renko')
    
    thestrat = cerebro.run(tradehistory=False, stdstats=False)[0]
    
    print(Dt.now().strftime('%m-%d %H:%M::'),f'Symbol {symbol} Backtest Complete')
    os.system('say "backtest completed"')
    
    figs = cerebro.plot(style='candle', barup= 'green', volume = False, tickrotation = 15, 
                            fmt_x_ticks = '%m-%d %H:%M', fmt_x_data = '%H:%M:%S')
    


  • @algoguy235 What was a reason for such interesting approach?

    Correct me if I am wrong but 1 min bars were taken, then each 1 min bar was split into twelve 5 sec bars with the open, high, low and close prices equal to the same prices of 1 min bar. Then renko filter is applied to 5 sec bars. So 1 renko bars on 1 min bars and 5 sec bars should be the same, since close price assumed the same for twelve 5 sec bars and one 1 min bar. Why not to apply renko filter to raw 1 min bars?



  • @ab_trader
    The implementation of bt.filters.Renko, only allows for a single brick to print in a given time-slice. If the candle in that time-slice is greater than 2 boxes, the Renko filter still only prints one brick. Then, at the next time slice the Renko filter will look at the new close price, and print a maximum of one brick. It will continue like that until the Renko price eventually "catches up' with the current close price. (this explanation assumes we're using hilo=False filter parameter).

    But this is not a correct Renko implementation. Let's call this problem the "time slice problem" (There are a few other problems with the bt.filters.Renko implementation, which are not discussed here). A proper Renko has the ability to print multiple bricks in a single "time slice". However because of the internals of Backtrader, that is not possible. But, we can simulate the opportunity to print multiple bricks in a single time slice, by upsampling the main data set, and setting the Renko filter on the upsampled data. This effectively solves the "time slice problem".



  • Here is an example of the time slice problem.

    0_1553921637352_6eb29586-4fb6-46be-b039-3f1a2a3c656f-image.png

    You can see that the Renko prints one brick at a time, and eventually catches up with close price.



  • Ok, I see the point. Thank you.


Log in to reply
 

});