progress bar / ETA during optimization

vbs

@tw00000 said in progress bar / ETA during optimization:

Just bumping this again, it seems strange that there's no simple way to find out:

Estimate of how long your backtest will take

My workaround was to add another variable to backtrader so you can query the total run count for the optimization:
https://github.com/verybadsoldier/backtrader/commit/0018b38c65a5cb25b17c42913ec0838b49feb57d

tw00000

@vbs said in progress bar / ETA during optimization:

@tw00000 said in progress bar / ETA during optimization:

Just bumping this again, it seems strange that there's no simple way to find out:

Estimate of how long your backtest will take

My workaround was to add another variable to backtrader so you can query the total run count for the optimization:
https://github.com/verybadsoldier/backtrader/commit/0018b38c65a5cb25b17c42913ec0838b49feb57d

Got it -- could you explain in a little more detail how I could use this with tqdm or some other method ??

sorry if I'm being daft!

vbs

Create a tqdm object using the total runcount from cerebro

        opt_count_total = cerebro.get_opt_runcount()
        pbar = tqdm(smoothing=0.05, desc='Optimization', total=opt_count_total)
        res = cerebro.run()

And use the optcallback to call update() on the tqdm object:

c.optcallback(optimization_step)

def optimization_step(strategy: bt.Strategy):
    pbar.update()

tw00000

@vbs

I wasn't able to make your code function properly 100%. In my Jupyter runs it was returning some weird stuff.

However, I figured out a simple way to get a progress bar for optimization or non-optimization runs!

from tqdm import tqdm

     def __init__(self):
        self.iteration_progress = tqdm(desc='Total runs', total=(self.datas[0].close.buflen()))
        
    def next(self):
        self.iteration_progress.update()
        self.iteration_progress.set_description("Processing {} out of {}".format(len(self.datas[0].close), self.datas[0].close.buflen()))

NikolaTesla

@backtrader
Is there a way to find out the total number of optimization cases without any manual inputs? From what I understand according to this thread, we can use the optcallback as an iterator function call but how exactly do we figure out the total number of cases in the optimization strategy(cerebro.optstrategy()) since you don't suggest using this

opt_count = len(list(itertools.product(*cerebro.strats)))

NikolaTesla

@NikolaTesla

I figured a way to do that but the progress bar is being printed after every callback (i.e after every iteration of the optimization). Is there a way I could restrict it to one instance when it begins? I believe the reason is that I used a GLOBAL variable so that the function optimizer_callbacks(cb) would be able to recognize it. Is there a way I could pass the pbar to optimizer_callbacks without actually using it as a global variable?

def optimizer_callbacks(cb):
    pbar.update()

def strategy_optim(**kwargs):

    total = np.prod([len(value) for key,value in kwargs.items()])

    csv_file = FDaxCSVData(---data---)
    cerebro = bt.Cerebro()
    cerebro.adddata(csv_file)    
    cerebro.broker.setcash(500000.0)
    cerebro.broker.setcommission(commission=2.0)

    strats = cerebro.optstrategy(strategy_name, printlog = False, **kwargs)

    global pbar
    pbar = tqdm.tqdm(smoothing=0.05, desc='Optimization Runs', total=total)
    cerebro.optcallback(optimizer_callbacks)

    runnings = cerebro.run(optreturn=False, maxcpus=2)  

if __name__=="__main__":
    strategy_optim(periods = [100, 200, 300],  abs_margin= [25, 50, 75], trail_stop=[10, 20, 30, 40])

NikolaTesla

@NikolaTesla said in progress bar / ETA during optimization:

@backtrader @vbs

I figured a way to do that but the progress bar is being printed after every callback (i.e after every iteration of the optimization). Is there a way I could restrict it to one instance when it begins? I believe the reason is that I used a GLOBAL variable so that the function optimizer_callbacks(cb) would be able to recognize it. Is there a way I could pass the pbar to optimizer_callbacks without actually using it as a global variable?
def optimizer_callbacks(cb):
    pbar.update()

def strategy_optim(**kwargs):

    total = np.prod([len(value) for key,value in kwargs.items()])

    csv_file = FDaxCSVData(---data---)
    cerebro = bt.Cerebro()
    cerebro.adddata(csv_file)    
    cerebro.broker.setcash(500000.0)
    cerebro.broker.setcommission(commission=2.0)

    strats = cerebro.optstrategy(strategy_name, printlog = False, **kwargs)

    global pbar
    pbar = tqdm.tqdm(smoothing=0.05, desc='Optimization Runs', total=total)
    cerebro.optcallback(optimizer_callbacks)

    runnings = cerebro.run(optreturn=False, maxcpus=2)  

if __name__=="__main__":
    strategy_optim(periods = [100, 200, 300],  abs_margin= [25, 50, 75], trail_stop=[10, 20, 30, 40])

NikolaTesla

@NikolaTesla said in progress bar / ETA during optimization:

@NikolaTesla said in progress bar / ETA during optimization:
@backtrader @vbs

I figured a way to do that but the progress bar is being printed after every callback (i.e after every iteration of the optimization). Is there a way I could restrict it to one instance when it begins? I believe the reason is that I used a GLOBAL variable so that the function optimizer_callbacks(cb) would be able to recognize it. Is there a way I could pass the pbar to optimizer_callbacks without actually using it as a global variable? My code is as follows, please have a look at it. Thank you for your time.
def optimizer_callbacks(cb):
    pbar.update()

def strategy_optim(**kwargs):

    total = np.prod([len(value) for key,value in kwargs.items()])

    csv_file = FDaxCSVData(---data---)
    cerebro = bt.Cerebro()
    cerebro.adddata(csv_file)    
    cerebro.broker.setcash(500000.0)
    cerebro.broker.setcommission(commission=2.0)

    strats = cerebro.optstrategy(strategy_name, printlog = False, **kwargs)

    global pbar
    pbar = tqdm.tqdm(smoothing=0.05, desc='Optimization Runs', total=total)
    cerebro.optcallback(optimizer_callbacks)

    runnings = cerebro.run(optreturn=False, maxcpus=2)  

if __name__=="__main__":
    strategy_optim(periods = [100, 200, 300],  abs_margin= [25, 50, 75], trail_stop=[10, 20, 30, 40])

NikolaTesla

@tw00000 I guess this is useful if you want a progress bar for a single strategy, the tread has been all about a progress bar in the case of an optimization strategy.

NikolaTesla

@NikolaTesla said in progress bar / ETA during optimization:

@NikolaTesla said in progress bar / ETA during optimization:

I realized the repetition of the status bar in a new line after each iteration was an error related to VS Code, so this code should work just fine.

crazy25000

Just finished implementing tqdm for optimizations. The status bars are inline and stay at the top. This way I can view the ETAs and overview of the opt runs:

FredYellow

@crazy25000 said in progress bar / ETA during optimization:

Just finished implementing tqdm for optimizations. The status bars are inline and stay at the top. This way I can view the ETAs and overview of the opt runs:

Excellent!

Do you have a minimal code example?

crazy25000

@fredyellow I can provide one later after work. Maybe during lunch if I have time.

crazy25000

@fredyellow said in progress bar / ETA during optimization:

@crazy25000 said in progress bar / ETA during optimization:

Just finished implementing tqdm for optimizations. The status bars are inline and stay at the top. This way I can view the ETAs and overview of the opt runs:

Excellent!

Do you have a minimal code example?

Here's a minimal code example in Jupyterlab Notebook:

from datetime import datetime

import backtrader as bt
from tqdm.auto import tqdm

pbar = tqdm(desc='Opt runs', leave=True, position=1, unit='run', colour='violet')


class OptimizeStrategy(bt.Strategy):
    params = (
        ('p1', 9),
        ('p2', 21),
    )

    def __init__(self):
        self.ema1 = bt.talib.EMA(self.data, timeperiod=self.p.p1, plotname='EMA1')
        self.ema2 = bt.talib.EMA(self.data, timeperiod=self.p.p2, plotname='EMA2')
        self.crossover = bt.indicators.CrossOver(self.ema1, self.ema2)

    def next(self):
        if self.crossover > 0:
            if self.position:
                self.close()
            self.buy()
        elif self.crossover < 0:
            if self.position:
                self.close()
            self.sell()


def bt_opt_callback(cb):
    pbar.update()


def runstrat():
    cerebro = bt.Cerebro(maxcpus=1)

    cerebro.optstrategy(OptimizeStrategy, p1=range(5, 20, 5), p2=range(50, 200, 10))

    data = bt.feeds.YahooFinanceData(dataname='AAPL', fromdate=datetime(2016, 1, 1), todate=datetime(2017, 1, 1))
    cerebro.adddata(data)

    cerebro.optcallback(cb=bt_opt_callback)
    stratruns = cerebro.run()


if __name__ == '__main__':
    runstrat()
    print("Done")