Multi-timeframe testing but "min() arg is an empty sequence"

wwongdg

i am testing my code snippet in jupyter-notebook in order to get myself familiarized with backtrader

I followed backtrader online document on multi-timeframe (https://www.backtrader.com/docu/data-multitimeframe/data-multitimeframe/)
and came up with my code below :

import backtrader       as bt
import backtrader.feeds as btfeeds

inbidfile = 'in_bid_file'

cerebro = bt.Cerebro(stdstats=False)

data = btfeeds.GenericCSVData(dataname=inbidfile,
                              dtformat=('%Y-%m-%d %H:%M:%S'),
                              datetime=0,
                              open=1,
                              high=2,
                              low=3,
                              close=4,
                              volume=-1,
                              openinterest=-1
                             )
cerebro.adddata(data)

data_resampled = cerebro.resampledata(data, 
                                      timeframe=bt.TimeFrame.Minutes, 
                                      compression=60)
cerebro.adddata(data_resampled)
cerebro.run()

got me this error

ValueError                                Traceback (most recent call last)
<ipython-input-7-6ad0d074b1cb> in <module>
     23                                       compression=60)
     24 cerebro.adddata(data_resampled)
---> 25 cerebro.run()

~/anaconda3/envs/env_backtrader/lib/python3.6/site-packages/backtrader/cerebro.py in run(self, **kwargs)
   1125             # let's skip process "spawning"
   1126             for iterstrat in iterstrats:
-> 1127                 runstrat = self.runstrategies(iterstrat)
   1128                 self.runstrats.append(runstrat)
   1129                 if self._dooptimize:

~/anaconda3/envs/env_backtrader/lib/python3.6/site-packages/backtrader/cerebro.py in runstrategies(self, iterstrat, predata)
   1296                     self._runnext_old(runstrats)
   1297                 else:
-> 1298                     self._runnext(runstrats)
   1299 
   1300             for strat in runstrats:

~/anaconda3/envs/env_backtrader/lib/python3.6/site-packages/backtrader/cerebro.py in _runnext(self, runstrats)
   1555                     dt0 = min((d for d in dts if d is not None))
   1556                 else:
-> 1557                     dt0 = min((d for i, d in enumerate(dts)
   1558                                if d is not None and i not in rsonly))
   1559 

ValueError: min() arg is an empty sequence

you help is most appreciated. I am a newbie to python and backtrader.

wwongdg

My own research into this "resample" topic gave me this:

link text

Dave-Vallance commented on Jun 21, 2019
Hi @mr-m0nst3r

Thanks for the submission and report. Yes, resampling currently does not work from my testing. It is an item to tackle in the future. I have marked this as a bug and will keep it open.

Is it still valid to say backtrader's resample function is still being debugged or even implemented?

wwongdg

so i resort to matplotlib and came up with

%matplotlib notebook

import backtrader       as bt
import backtrader.feeds as btfeeds

m1bidfile = 'm1bidfile'

h1bidfile = 'h1bidfile'

cerebro = bt.Cerebro() 

m1data = btfeeds.GenericCSVData(dataname=m1bidfile,
                              dtformat=('%Y-%m-%d %H:%M:%S'),
                              datetime=0,
                              open=1,
                              high=2,
                              low=3,
                              close=4,
                              volume=-1,
                              openinterest=-1
                             )

h1data = btfeeds.GenericCSVData(dataname=h1bidfile,
                              dtformat=('%Y-%m-%d %H:%M:%S'),
                              datetime=0,
                              open=1,
                              high=2,
                              low=3,
                              close=4,
                              volume=-1,
                              openinterest=-1
                             )

cerebro.adddata(m1data)
cerebro.adddata(h1data)

cerebro.run()
cerebro.plot()

wwongdg

I forget to add:

refer to the above matplotlib plot of M1 and H1 traces. It does not appear perfect as the H1 is entirely missing. Any clue or pointers?

wwongdg

%matplotlib notebook

import datetime
import backtrader as bt
import btoandav20

storekwargs = dict(
    token   = 'your_token',
    account = 'your_account',
    practice= True
    )

StoreOandaV20 = btoandav20.stores.OandaV20Store(**storekwargs)
DataOandaV20  = btoandav20.feeds.OandaV20Data(**storekwargs)

#if __name__ == '__main__':
cerebro = bt.Cerebro()
    
m1data = btoandav20.feeds.OandaV20Data(
         dataname="EUR_USD",
         timeframe=bt.TimeFrame.Minutes,
         compression=1,
         qcheck=0.5,
         fromdate=datetime.datetime(2019, 1, 1),
         todate=datetime.datetime(2019, 1, 5)
        )

cerebro.adddata(m1data)

cerebro.resampledata(m1data, 
                     timeframe=bt.TimeFrame.Minutes, 
                     compression=60)


cerebro.run()
cerebro.plot(style='bar', volume=False) #

my first post using data feed from a CSV file would not show multitimeframe plot but a data feed from Oanda would display no problem, weird!

wwongdg

So after the above testing, my conclusion as of this date is:

single datafeed from CSV M1 file works (plots all 1440 bars it appears to me)
single datafeed from CSV M1 and resample the feed to M60 and plot single line quirks (the plot seemed like just 2 bars instead 24 bars)
dual feed from CSV(M1 and M60) plotted only the the M1 and M60 did not appear despite a space reserved for it
only live feed (from Oanda) add data(M1) and resample data(M1) to M60 plotted fine

the example given in (link text) is the case of option (2) above. It is no wonder I could not reproduce the expected plot. Hope the contributing developer community members can look into this.

ab_trader

@wwongdg said in Multi-timeframe testing but "min() arg is an empty sequence":

Is it still valid to say backtrader's resample function is still being debugged or even implemented?

This link and related text is applicable for the ccxt store you found, not to bt itself.

@wwongdg said in Multi-timeframe testing but "min() arg is an empty sequence":

refer to the above matplotlib plot of M1 and H1 traces. It does not appear perfect as the H1 is entirely missing. Any clue or pointers?

In your m1 and h1 cases you did't tell bt the time frame of the data, so bt treated them as daily bars. That's why you have m1 shown fully in 1440 daily bars and 24 h1 bars are shown along the left side of the diagram you displayed above. That narrow red and grey line.

Check out data feed common parameters in Docs - Data feeds

@wwongdg said in Multi-timeframe testing but "min() arg is an empty sequence":

my first post using data feed from a CSV file would not show multitimeframe plot but a data feed from Oanda would display no problem, weird!

it is not weird. You finally told bt the time frame and compression.

@wwongdg said in Multi-timeframe testing but "min() arg is an empty sequence":

So after the above testing, my conclusion as of this date is:

Maybe these items need to be revised after correct parameters be used.

PS I didn't use resample a lot, but when I used it accurately, it always worked as expected.

wwongdg

plus there is a quirk on this forum capability too - there is no edit function after posting

ab_trader

edit is disabled.

wwongdg

@ab_trader
Thanks for the pointers. From your input I have managed to get the below working:

%matplotlib notebook

import backtrader       as bt
import backtrader.feeds as btfeeds
import pandas           as pd

m1bidfile = 'm1 bid actual file location and name'


cerebro = bt.Cerebro() #stdstats=False

m1data = btfeeds.GenericCSVData(dataname=m1bidfile,
                                dtformat=('%Y-%m-%d %H:%M:%S'),
                                timeframe=btfeeds.TimeFrame.Minutes,
                                compression=1,
                                datetime=0,
                                open=1,
                                high=2,
                                low=3,
                                close=4,
                                volume=-1,
                                openinterest=-1
                               )


cerebro.adddata(m1data)


h1data = cerebro.resampledata(m1data, 
                              timeframe=bt.TimeFrame.Minutes, 
                              compression=60)


cerebro.run()
cerebro.plot(style='bar', volume=False, iplot=False) #

wwongdg

Any comment or tips on the code below with pandas datafeed to backtrader

# https://www.backtrader.com/docu/pandas-datafeed/pandas-datafeed/
#

from __future__ import (absolute_import, division, print_function,
                        unicode_literals)

%matplotlib notebook

import backtrader       as bt
import backtrader.feeds as btfeeds
import pandas           as pd

cerebro = bt.Cerebro() #stdstats=False

m1bidfile = '/location_to/datafile'

pd_m1data = pd.read_csv(m1bidfile,
                        #skiprows=,
                        header=0,
                        parse_dates=True,
                        index_col=0
                       )


bt_m1data = btfeeds.PandasData(dataname=pd_m1data,
                               dtformat=('%Y-%m-%d %H:%M:%S'),
                               timeframe=btfeeds.TimeFrame.Minutes,
                               compression=1,
                              )

cerebro.adddata(bt_m1data)

cerebro.run()

cerebro.plot(style='bar', iplot=False)

i ended up with

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-19-903ec5721243> in <module>
     27                                dtformat=('%Y-%m-%d %H:%M:%S'),
     28                                timeframe=btfeeds.TimeFrame.Minutes,
---> 29                                compression=1,
     30                               )
     31 

~/anaconda3/envs/env_backtrader/lib/python3.6/site-packages/backtrader/metabase.py in __call__(cls, *args, **kwargs)
     86         _obj, args, kwargs = cls.donew(*args, **kwargs)
     87         _obj, args, kwargs = cls.dopreinit(_obj, *args, **kwargs)
---> 88         _obj, args, kwargs = cls.doinit(_obj, *args, **kwargs)
     89         _obj, args, kwargs = cls.dopostinit(_obj, *args, **kwargs)
     90         return _obj

~/anaconda3/envs/env_backtrader/lib/python3.6/site-packages/backtrader/metabase.py in doinit(cls, _obj, *args, **kwargs)
     76 
     77     def doinit(cls, _obj, *args, **kwargs):
---> 78         _obj.__init__(*args, **kwargs)
     79         return _obj, args, kwargs
     80 

TypeError: __init__() got an unexpected keyword argument 'dtformat'

ab_trader

Seems like pandas data feed doesn't have dtformat parameter, it relates to csv data feed.