For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

Dynamic stock selection and large number of data sets



  • Recently joined community and still catching up with the documentation. First off, many thanks for the great framework - already had some success with running algorithms with multiple indicators.

    I'm a bit confused as if how can I achieve something this - before every trading day or as of the available balance - i would like to screen a big list of stocks (eg: S&P 500 - including currently delisted ones depending the trading day).

    I'm aware of bt-run-py, but can we use that for dynamic stock section and is there a way to feed entire market data (hundreds and thousands of stocks) rather than a hand full of stocks ? also would like to run it in parallel which should limit my allocation sizes/orders based on available funds.

    Apologies if this was asked before - any pointers in right direction will be much appreciated. Many Thanks.



  • @tracy it was discussed several times, try to search forums with the word multi. Also check the following post in the blog.

    Straight forward way is to load all stocks into bt and do your calculations.


  • administrators

    @tracy said in Dynamic stock selection and large number of data sets:

    bt-run-py, but can we use that for dynamic stock section and is there a way to feed entire market data (hundreds and thousands of stocks) rather than a hand full of stocks ?

    bt-run.py is just a tool which can be used to automate certain tasks. It does, by no means, not cover the entire functionality of the framework and custom cases are for sure better handled with custom scripts.

    The amount of data feeds you manage (be it a handful or thousands) is limited by the equipment you use (mostly RAM) and the available time you have, because Python is not the fastest language ever (the high level of dynamism, introspection et al. has a price)

    @tracy said in Dynamic stock selection and large number of data sets:

    also would like to run it in parallel which should limit my allocation sizes/orders based on available funds.

    Your allocation is always limited to the funds you have decided to configure the broker with. Actually: orders will be rejected if you don't have enough funds. And since many people try a perfect 100% allocation, they tend to fail in the last part of the allocation due to rounding errors or gap in prices due to the misconception about how easy is to enter the market with a price which no longer exists (the current close price being examined)

    Use the pointer provided by @ab_trader as a cornerstone to create your own mini-framework for the analysis your thousands of stocks.



  • Many Thanks @ab_trader , @backtrader I will give it a go as per your comments.



  • Slightly off topic - but i kind of progressed to next level - added multiple data without any issues. However, getting value out of indicators seems to be a different story. Adding multiple indicators seems to be giving 0 as its result - always.

    Im adding data to cerebro as follows:

    for data in q_datas:
         cerebro.adddata(bt.feeds.PandasData(dataname=data),data.ticker[0])
    

    and then later in my strategy:

    def __init__(self):
        self.smas = {data: bt.indicators.SMA(data, period=20)
                     for data in self.datas}
        self.emas = {data: bt.indicators.EMA(data, period=5)
                     for data in self.datas}
    
        self.indicators = list()
        for ema, sma in zip(self.emas, self.smas):
            print("{} -> {}".format(ema._name,sma._name))
            self.indicators.append(bt.indicators.CrossOver(sma,ema))
    

    resulting plot:
    0_1520855520777_crossover.png

    Notice Crossover indicator is always 0 ? Failing to see where am I going wrong. Did a good scroll through the documentation and community questions and couldn't find anything matching my case and hence the question. Thanks


  • administrators

    @tracy said in Dynamic stock selection and large number of data sets:

    for ema, sma in zip(self.emas, self.smas):

    You are zipping the data feeds and not the indicators (which are the values in the dictionaries)



  • ah! silly me. Should have done something like this:

     self.indicators = list()
     for ema, sma in zip(self.emas.values(), self.smas.values()):
           self.indicators.append(bt.indicators.CrossOver(sma,ema))
    

    Thanks again @backtrader