For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

Best way to setup a strategy across a large universe of securities



  • Hi all,

    This may be more of a general programming question but I have been struggling to find the best approach. I am looking for input as to what would be deemed the "right" way to setup a strategy over a universe of securities.

    Currently, I perform the following operation in a cmd window:

    1. change directory to the script
    2. type, for example:
      python ibtest.py --data0 GBP.USD-CASH-IDEALPRO
    3. let the strategy run within the command window.

    I have attempted to create a batch file with a for loop, changing the value of GBP.USD-CASH-IDEALPRO within the loop.

    Unfortunately, my problem currently revolves around creating a new command line instance and then entering the commands within that particular instance that was just opened.

    I was thinking, if I were to run the strategy across a large number of securities (200+), then maybe opening a command window for each strategy might be a bit silly.

    Another way, which does currently work, is to run multiple strategies within the same command window. Which is currently the best approach although interpreting the feed gets a bit messy. If people are interested, I can share this approach. EDIT: this method may not executive trades across different instruments due to the if not self.position: condition.

    Is there a better way to approach this?

    Thank you for your help.



  • UPDATE:

    The following windows batch code (replacing ... with the script directory path) will open multiple cmd windows with differing arguments, which is the solution I was looking for.

    for %%i IN (EUR.USD-CASH-IDEALPRO, GBP.USD-CASH-IDEALPRO) DO (
    start cmd.exe /k "cd c:/users/... & python ibtest.py --data0 %%i")
    


  • bt can operate multiple data feeds within one strategy or multiple strategies. In this case you use single script run. Why don't you use this standard approach?

    Here is the link to blog post - Blog - Multi

    Also number of posts were made here on the forum.



  • @ab_trader The problem with using the standard approach is that the universe is large (e.g. 200 stocks) and variable (e.g. may only want to run on 50 stocks at a time)



  • So 200 command line windows work faster than one script processing 200 stocks? Otherwise it is not clear why command line windows are better.

    On the other hand - you have some other higher level code which defines number of stocks for current session (?) (just a guess based on your variability from 50 to 200 stocks). Why not to join it together with the bt and run bt for actual 50 or 200 stocks?

    Anyway, I can only guess. Based on the small piece of info shared and your answers, I believe that the best option is the option you already chosen. :)


  • administrators

    @hbf said in Best way to setup a strategy across a large universe of securities:

    Currently, I perform the following operation in a cmd window:

    Since you are asking for the best way, I would dare to say that using cmd is some millions lightyears away from being the best way of executing anything. Without having to move away from Windows, use either:

    • Windows Subsystem for Linux (aka Bash on Windows) and get running inside the Linux Subsystem with a proper shell

    or

    • Use Cygwin, which offers you a simulated Posix environment and the same proper shell you find under Linux.

    In any case and as pointed out by @ab_trader, it seems odd that the best way is to run a script 200 times, rather than running one script which internally changes the symbols.

    for symbol in symbols:  # where symbols is an iterable with the 200 symbols ...
    
        cerebro = bt.Cerebro()
        data = bt.fees.MyFeed(symbol)
        cerebro.adddata(data)
    
        results = cerebro.run()
        # do something with the results
    


  • @ab_trader @backtrader Thank you both for your responses.

    A little bit more info:

    Currently, the system is being tested on an IB paper account which is subscribed to data feeds. The idea behind opening multiple cmd windows and running the strategy for each security was so that the constant live feed that is output by the logdata, notify_data, notify_order, notify_trade etc. functions could be interpreted simultaneously through the command windows.

    For backtesting across multiple securities, I have used a for loop like @backtrader has suggested and have output analysers to a csv document. My confusion lies in the connection to the broker and analysing the live data feed. Does this change your answer? If not, how can the output be interpreted whilst the strategy is running?

    Thank you both for your suggestions. I will look into the Windows Subsystem for Linux and Cygwin. I guess my priorities have been to get a system up and running ASAP, without consideration of best practices. These will be implemented as I continue to learn.



  • @hbf said in Best way to setup a strategy across a large universe of securities:

    Does this change your answer? If not, how can the output be interpreted whilst the strategy is running?

    I see, you want to see logs for each stock in the separate window for clarity. Then yes, your choice is probably the simplest. Still doubt that these logs can be interpreted simultaneously with 200 windows open at the same time.

    With one script you will have mix of the logs for different events and different stocks in the single command window. If you are catching relatively rare events like order issuing and executions might be more readable in single window.



  • Changed the approach.

    Now using Windows Subsystem for Linux.

    Have created the following shell script if anyone considers doing the same thing.

    #!/bin/sh
    codelist='EUR.USD-CASH-IDEALPRO GBP.USD-CASH-IDEALPRO'
    
    for code in $codelist
    do
      if [ $(pgrep -f 'python ibtest.py --data '$code) \
      > /dev/null ]
      then
        echo $code ' strategy already running'
      else
        rm output_$code.txt &
        nohup python ibtest.py --data $code \
        > output_$code.txt &
      fi
    done
    

  • administrators