For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

Calculate expected optimization memory usage



  • Hello.

    I'm having troubles with calculating the expected optimization memory usage.
    I need it to split big parameters dict during optimization into chunks if memory usage is too high.

    I came up with this function.
    But it doesn't work always.
    Please help!

    inputs:
    params - dict | with strategy parameters for the optimization
    start, end - datetime | start and end of the ticker data
    freq - str | ticker data freq (15Min, 1H, etc)

    def memory_gen(params, start, end, freq, threads=8, log=False):
        period_len = len(pd.PeriodIndex(freq=freq, start=start, end=end))
        params_len = 1
        for param in params:
            if type(params[param]) == list:
                if not params_len == 1:
                    params_len *= len(params[param])
                elif params_len == 1 and len(params[param]) > 1:
                    params_len = len(params[param])
    
            n = 20.0
    
        memory = round((period_len * params_len) * n / 1000000 + 0.200 * threads, 2)
        if log:
            logger.info(freq, memory)
        return memory

Log in to reply
 

});