For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See:

progress bar / ETA during optimization

  • While running a large optstrategy it would be useful to be get an ETA or to monitor the percentage of parameter combinations that have been completed so far. I have tried using tqdm package but it get confused by the multiprocesses. tqdm can deal with multiprocessing but it needs to be called by the function that distributes the jobs. I don't think that it is possible to do that without modifying cerebro code.
    I also implemented a simple iteration counter in strategy stop method :

    progress_i = 0
    class MyStrat(bt.Strategy):
        def stop(self):
            global progress_i
            progress_i += 1
            if progress_i % 100 == 0:
                print("%d jobs finished" %progress_i)
    def optimize():
        cerebro.optstrategy(MyStrat, **parameter_ranges)
        optrets =
        return optrets

    The problem here is that if I have 48 processors, then I'll have 48 times (or maybe 47) outputs saying "100 jobs finished". I could tweak things so that progress_i is multiplied by the number of processes but I am not sure that I can specify that only one slave process prints out its process. This means I'll have 47 or 48 lines printed for each progress update, which is not very nice.

    Has anybody any suggestion to make a somewhat prettier output?

  • administrators

    It seems you want to add Optimization Callbacks.

    See: Docs - Cerebro and look for optcallback

  • Thanks. Would you have any example script using callback / optcallback ? I struggle to understand how to use them.

  • administrators

    There isn't, but it is easy:

    • If your optimization is going to end up iterating over 1000 strategies, your callback will be called 1000 times, each time with the strategy that just finished its run.

  • @backtrader Sorry I am still kind of lost. In my main function I understand that I should have

    cerebro.optstrategy(MyStrategy, **param_range,

    But where do I define my_callback? And what should it look like? Will it call some method of the strategies? If yes, which one?

    I was trying to do the dumbest test, i.e. print("finished") after each strategy finishes, but I couldn't get it to work. How would you do that?

  • administrators

    You define it wherever you want, but you obviously need to be able to give it to cerebro before entering run

    From the document linked above:

      - Adds a callback to the list of callbacks that will be called with the optimizations when each of the strategies has been run
      - The signature: cb(strategy)

    You define it wherever you want, but you obviously need to be able to pass it cerebro

  • Is there a convenient way to get the total number of optimization combinations that will be run (besides counting it manually)?
    It would be nice if it would be possible to print the optimization progress as percentage or something like "done x out of y runs".

    As a workaround I am doing this currently:

    opt_count = len(list(itertools.product(*cerebro.strats)))

  • administrators

    @vbs said in progress bar / ETA during optimization:

    As a workaround I am doing this currently:
    opt_count = len(list(itertools.product(*cerebro.strats)))

    Which will explode with an out_of_memory exception as soon as you do anything which is not trivial.

    And no, you cannot, because you can pass something which generates values (a range is not a pure generator, but you could pass a generator) and have an infinite amount of possibilities.

  • Ok thanks. Too bad though. Yes, converting to a list can lead to huge amount of memory being consumed at that moment. But I think it could be rewritten to consume the generator and just count the loops without the intermediate list.

    How does it work having an infinite amount of runs? Would you evaluate each new result in optcallback and then somehow stop the run when it meets certain conditions?

  • @backtrader said in progress bar / ETA during optimization:

    It seems you want to add Optimization Callbacks.

    See: Docs - Cerebro and look for optcallback

    For me this callback does not work when using maxcpu=1. But it works fine for example with maxcpu=8. Is that intended or a bug possibly?
    "Does not work" means that it never gets called.

    The documentation at least does not mention any limitations:

  • administrators

    @vbs said in progress bar / ETA during optimization:


    This disables multiprocessing and runs as a regular backtesting process.

  • Yes, I understand it disables multiprocessing when using maxcpus=1 but it is still executing optimization and gives me optimization results, no?

    My point is that the optcallback is not called during the optimization process when maxcpus=1. It is called though when maxcpus is greater than 1.

  • administrators

    That was understood. The comment pointed out that it is following a different path in the code and using the same code as a non-optimizing run. Hence the lack of callback invocation.

  • Ok, so it is intended behavior. I already assumed it would be related to the code path. I think it would be good if optcallback would be called regardless of the parameter maxcpus.
    I tried simply adding it myself, but I am not sure if it will introduce other unexpected behaviors or side effects. So could it be as simple as this? Any reason to not have it?

            if not self._dooptimize or self.p.maxcpus == 1:
                # If no optimmization is wished ... or 1 core is to be used
                # let's skip process "spawning"
                for iterstrat in iterstrats:
                    runstrat = self.runstrategies(iterstrat)
                    if self._dooptimize:
                        for cb in self.optcbs:
                            cb(runstrat)  # callback receives finished strategy

    (last 3 lines added)

Log in to reply

Looks like your connection to Backtrader Community was lost, please wait while we try to reconnect.