Error in multi-core optimization

Abhay Jindal

I am getting the following error when I run multi-core optimization.
AttributeError: Can't pickle local object 'GenericCSVData.start.<locals>.<lambda>'
Also, how can I close all my open positions or get the total (realized+unrealized) profit after each optimization run? Currently, I am getting only the realized PnL using the following line in the stop function.
round(self.broker.get_value(), 2)

Anton Kozin

@Abhay-Jindal said in Error in multi-core optimization:

I am getting the following error when I run multi-core optimization.
AttributeError: Can't pickle local object 'GenericCSVData.start.<locals>.<lambda>'

I have the same problem. Could someone help with this issue?

chewbacca

@Anton-Kozin Is the "raw" value a negative number?

vladisld

@Abhay-Jindal said in Error in multi-core optimization:

I am getting the following error when I run multi-core optimization.
AttributeError: Can't pickle local object 'GenericCSVData.start.<locals>.<lambda>'

It would be helpful if you could share your code.

Anton Kozin

@chewbacca said in Error in multi-core optimization:

@Anton-Kozin Is the "raw" value a negative number?

What do you mean by the "raw" value? Could you make it more clear for me, please?

I solved the problem by rewriting GenericCSVData class, i think that the reason why we get an error is that Pickle module has some problems with pickling lambda functions. Maybe its more better to use another pickling module (Dill for example) to deal with it.

vladisld

The lambda function inside the GenericCSVData is only used if the dtformat parameter is set to integer value (1 or 2) indicating that the datetime column contains timestamps instead of strings

Is it really a case ? ( this is the reason I've asked to share the code )

Anton Kozin

@vladisld Yes, my data has timestamps in datetime column

data = bt.feeds.GenericCSVData(
            dataname=datapath,
            fromdate=datetime.datetime(2020, 8, 25),
            todate=datetime.datetime(2020, 9, 8),
            timeframe=bt.TimeFrame.Minutes,
            compression=files[datapath],
            dtformat=2,
            reverse=False)

Here is part of my code with data feed function. In fact, it won't help us much.

vladisld

Issue was opened in backtrader2 project: https://github.com/backtrader2/backtrader/issues/39