Bar filling code worked in 1.2.7.88 - failing in 1.9.19.105 #226

backtrader

From Issue #226

I have enclosed a test program and two data files.
The data in the 5min bars file is sparse ie. could be missing. In a previous version - you helped me write a Bar copy routine to fill in the spare data - This routine

class MySessionFiller(btfilters.SessionFiller):
    def _fillbar(self, data, dtime):
        # Copy the previous bar
        bar = [data.lines[i][-1] for i in range(data.size())]
        bar[data.DateTime] = date2num(dtime)  # Fill datetime
        # Add to the stack of bars to save
        data._add2stack(bar)           
        return True

works in 1.2.7.88 - but fails now in 1.9.19
unzip file and just run testgen.py - and you will see a printout. Comment out the MyFiller filter to verify its the filter thats causing the issue and not the original data.

Are you in the new data handling architecture - automatically doing this by default ie. filling in sparse data by copying ?

Thanks.

backtrader

The issue here (notwithstanding something internally) is that sessionend is not set and the duplicate bars happen at exactly 00:00:00 (probably due to a floating precision error)

Simply do, for example,:

data = bt.feeds.XData(dataname="....", sessionend=datetime.time(17, 30))

partha

@backtrader

I put the sessionend and the duplicate at 00:00 went away.
I am still having issues.
feedtime = num2time(self.datas[0].datetime[0])
print '>> time: ', feed0_date_curr, feedtime
Put the code into the Next function in the sample I already uploaded it should print once per 5min bar - look for example at 12-06-16 around 12:40 time -

time: 2016-12-06 12:40:00
time: 2016-12-06 13:05:00
time: 2016-12-06 12:45:00
time: 2016-12-06 12:50:00
time: 2016-12-06 12:55:00
time: 2016-12-06 13:00:00
time: 2016-12-06 13:25:00
time: 2016-12-06 13:45:00
time: 2016-12-06 13:50:00
time: 2016-12-06 23:10:00
is what I see printed out.
You can see the same issue on 12-07 around 12:40 and on 12-12 around 12:55.

Please advise,
Thanks,

backtrader

This second thing was an issue. It was actually an issue in the old versions which didn't properly check the return code from the filters.

Things were tightened along the way and the filter, because it is modifying the stack, should return True when doing so, rather than a generic False. The filter had to be brought up to modern standards.

A patch has been pushed to the development branch and seems to correct the observed behavior.

partha

@backtrader Thanks.