How to filter large stock universe then run trading system on small group
-
Hello,
I am new to backtrader and have been looking through the forums and docs to try to find a solution to my problem but i couldn't find anything that solves it completely.
I have two different datafeeds avaiable to me. The first being daily OHLC data for 100 stocks and then I have tick data for those same 100 stocks.
My idea is to run each stock through a filter daily, and then use that smaller universe of stocks to run my algo.
So far I have been able to filter the large universe of stocks each day into a dictionary that will contain only a handful of names. The trouble is that I have no idea how to use this new dictionary to run a second strategy as backtrader doesn't seem to be able to support that. I also am not sure how to get the name of ticker that is being triggered by my filters. Without the ticker name, im not sure how I can identify the datas.
Does anyone have a solution?
-
You want to run two different algos and you need to run backtrader twice. As you would need to run other platforms twice.
- You run your first algo on a cerebro and end up with a dictionary
- You use the dictionary to make a 2nd cerebro run with that restricted set of data feeds
@sagittarius19 said in How to filter large stock universe then run trading system on small group:
I also am not sure how to get the name of ticker that is being triggered by my filters
You have created the 1st set of data feeds, so it seems you already have access to the ticker names. We probably won't understand your worries without seeing some code.
-
list item@backtrader thank you for the quick reply.
@sagittarius19 said in How to filter large stock universe then run trading system on small group:
I also am not sure how to get the name of ticker that is being triggered by my filters
You have created the 1st set of data feeds, so it seems you already have access to the ticker names. We probably won't understand your worries without seeing some code.
What i mean to say is, that there is no column in datas that says which ticker I am using. Here is a sample of code.
def __init__(self): self.Universe = [] def next(self): for i, d in enumerate(self.datas): self.day_open = self.datas[i].open self.day_close = self.datas[i].close self.date = self.datas[i].datetime.date(0) if self.day_open[0]> ((self.day_close[-1])*1.05): self.Universe.append(self.date) def stop(self): print ("Stock list: %s" % self.Universe)
This is only able to append the dates of times where the stocks are over 5% up on the day at the open. I send those date values to the list 'self.Universe' but I dont know how to record in the same list or a separate list the ticker that the filter was running since there is no "symbol" or "ticker" column in the dataframe.
-
@sagittarius19 said in How to filter large stock universe then run trading system on small group:
self.date = self.datas[i].datetime.date(0) if self.day_open[0]> ((self.day_close[-1])*1.05): self.Universe.append(self.date)
You are appending a
datetime.date
instance to your universe. It would really be a miracle if you could actually find out to which data feed it belongs.You can add each data feed with a name and use it with
self.dnames
orself.data._name
.See:
-
@backtrader I have updated my code below:
import datetime from datetime import timedelta import os.path import sys import pandas as pd import backtrader.feeds as btfeeds import pandas_datareader.data as web import backtrader as bt import backtrader.indicators as btind import backtrader.analyzers as btanalyzers class Pandasclass(btfeeds.PandasData): lines = ('symbol') params = (('symbol',-1)) datafields = btfeeds.PandasData.datafields + (['symbol']) class Universe_filter(bt.Strategy): Universe_date = [] Universe_data = [] def __init__(self): pass def next(self): for i, d in enumerate(self.datas): self.day_open = self.datas[i].open self.day_close = self.datas[i].close self.date = self.datas[i].datetime.date(0) self.symbol = self.datas[i].symbol if self.day_open[0]> ((self.day_close[-1])*1.05): self.Universe_date.append(self.date[0]) self.Universe_data.append(self.symbol[0]) def stop(self): print ("Stock dates: %s" % self.Universe_date) print ("Stock datas: %s" % self.Universe_data) print (help(btfeeds.PandasData)) if __name__ == '__main__': cerebro = bt.Cerebro() cerebro.addstrategy(Universe_filter) start = datetime.date(2017,4,1) end = datetime.date(2018,3,19) #stocks = ["AAPL","MSFT",'TSLA','FB','AMZN','GOOGL','BABA','PYPL','OSTK'] stocks = ['FB','AAPL'] for i in stocks: DF_1 = web.DataReader([i], "morningstar", start, end) DF_1= pd.DataFrame(DF_1) DF_1.reset_index(level = 'Symbol', inplace=True,drop=False) data1 = Pandasclass(dataname = DF_1) cerebro.adddata(data1) cerebro.run()
I am still getting this error though. Any idea what to do?
Traceback (most recent call last): File "c:\Users\Robert\Desktop\Python docs\Extended pandas import attempt.py", line 12, in <module> class Pandasclass(btfeeds.PandasData): File "C:\pyth27 fold\pyth27path\lib\site-packages\backtrader\lineseries.py", line 350, in __new__ cls = super(MetaLineSeries, meta).__new__(meta, name, bases, dct) File "C:\pyth27 fold\pyth27path\lib\site-packages\backtrader\metabase.py", line 239, in __new__ cls.params = params._derive(name, newparams, morebasesparams) File "C:\pyth27 fold\pyth27path\lib\site-packages\backtrader\metabase.py", line 116, in _derive clsinfo.update(info) File "C:\pyth27 fold\pyth27path\lib\_abcoll.py", line 571, in update for key, value in other: ValueError: too many values to unpack
-
Your
params
definition is wrong. You are defining a tuple of 2 elements and not a tuple in a tuple. -
@backtrader thanks for the help regarding this. I know its prolly a very simple fix. Here is what I updated to, and im still getting an error.
class Pandasclass(btfeeds.PandasData): lines = ('symbol') params = (('symbol', -1),) btfeeds.PandasData.datafields = btfeeds.PandasData.datafields + [u'symbol']
File "c:\Python27_path\Python27 Scritps\Extended pandas import attempt4.py", line 13, in <module> class Pandasclass(btfeeds.PandasData): File "c:\Python36\lib\site-packages\backtrader\lineseries.py", line 364, in __new__ linesoverride, lalias=la) File "c:\Python36\lib\site-packages\backtrader\lineseries.py", line 130, in _derive clslines = baselines + lines TypeError: can only concatenate tuple (not "str") to tuple
When i put another dummy value next to it, I got this error.
class Pandasclass(btfeeds.PandasData): lines = ('symbol','blank') params = (('symbol', -1),('blank', -1),) btfeeds.PandasData.datafields = btfeeds.PandasData.datafields + [u'symbol', u'blank']
File "c:\Python27_path\Python27 Scritps\Extended pandas import attempt4.py", line 49, in <module> cerebro.run() File "c:\Python36\lib\site-packages\backtrader\cerebro.py", line 1127, in run runstrat = self.runstrategies(iterstrat) File "c:\Python36\lib\site-packages\backtrader\cerebro.py", line 1209, in runstrategies data.preload() File "c:\Python36\lib\site-packages\backtrader\feed.py", line 435, in preload while self.load(): File "c:\Python36\lib\site-packages\backtrader\feed.py", line 476, in load _loadret = self._load() File "c:\Python36\lib\site-packages\backtrader\feeds\pandafeed.py", line 255, in _load line[0] = self.p.dataname.iloc[self._idx, colindex] File "c:\Python36\lib\site-packages\backtrader\linebuffer.py", line 222, in __setitem__ self.array[self.idx + ago] = value TypeError: must be real number, not str
Does datas not allow for non numerical values to be in the columns?
-
Update: I ended up just using the _names function as you suggested earlier. It works just as well. I would like to know thought, if i can call on none numerical values in the columns of the datas.
class Universe_filter(bt.Strategy): Universe_date = [] Universe_data = [] def __init__(self): pass def next(self): for i, d in enumerate(self.datas): self.day_open = self.datas[i].open self.day_close = self.datas[i].close self.date = self.datas[i].datetime.date self.symbol = self.datas[i]._name if self.day_open[0]> ((self.day_close[-1])*1.05): self.Universe_date.append(self.date(0)) self.Universe_data.append(self.symbol) def stop(self): print ("Stock dates: %s" % self.Universe_date) print ("Stock datas: %s" % self.Universe_data) if __name__ == '__main__': cerebro = bt.Cerebro() cerebro.addstrategy(Universe_filter) start = datetime.date(2017,4,1) end = datetime.date(2018,3,19) #stocks = ["AAPL","MSFT",'TSLA','FB','AMZN','GOOGL','BABA','PYPL','OSTK'] stocks = ['FB','AAPL'] for i in stocks: DF_1 = web.DataReader([i], "morningstar", start, end) DF_1= pd.DataFrame(DF_1) DF_1.reset_index(level = 'Symbol', inplace=True,drop=False) data1 = btfeeds.PandasData(dataname = DF_1) cerebro.adddata(data1, name = i) cerebro.run()
[Running] python "c:\Python27_path\Python27 Scritps\Extended pandas import attempt4.py" Stock dates: [datetime.date(2017, 7, 27), datetime.date(2017, 8, 2)] Stock datas: ['FB', 'AAPL']
-
@sagittarius19 said in How to filter large stock universe then run trading system on small group:
@backtrader thanks for the help regarding this. I know its prolly a very simple fix. Here is what I updated to, and im still getting an error.
class Pandasclass(btfeeds.PandasData): lines = ('symbol') params = (('symbol', -1),) btfeeds.PandasData.datafields = btfeeds.PandasData.datafields + [u'symbol']
File "c:\Python27_path\Python27 Scritps\Extended pandas import attempt4.py", line 13, in <module> class Pandasclass(btfeeds.PandasData): File "c:\Python36\lib\site-packages\backtrader\lineseries.py", line 364, in __new__ linesoverride, lalias=la) File "c:\Python36\lib\site-packages\backtrader\lineseries.py", line 130, in _derive clslines = baselines + lines TypeError: can only concatenate tuple (not "str") to tuple
When i put another dummy value next to it, I got this error.
class Pandasclass(btfeeds.PandasData): lines = ('symbol','blank') params = (('symbol', -1),('blank', -1),) btfeeds.PandasData.datafields = btfeeds.PandasData.datafields + [u'symbol', u'blank']
File "c:\Python27_path\Python27 Scritps\Extended pandas import attempt4.py", line 49, in <module> cerebro.run() File "c:\Python36\lib\site-packages\backtrader\cerebro.py", line 1127, in run runstrat = self.runstrategies(iterstrat) File "c:\Python36\lib\site-packages\backtrader\cerebro.py", line 1209, in runstrategies data.preload() File "c:\Python36\lib\site-packages\backtrader\feed.py", line 435, in preload while self.load(): File "c:\Python36\lib\site-packages\backtrader\feed.py", line 476, in load _loadret = self._load() File "c:\Python36\lib\site-packages\backtrader\feeds\pandafeed.py", line 255, in _load line[0] = self.p.dataname.iloc[self._idx, colindex] File "c:\Python36\lib\site-packages\backtrader\linebuffer.py", line 222, in __setitem__ self.array[self.idx + ago] = value TypeError: must be real number, not str
Does datas not allow for non numerical values to be in the columns?
All values are floats. This is not a database.