Multi core indicator processing
-
I am developing a strategy that tests cointegration for a list of 100 companies. I create an object of the indicator for each pair.
When the strategy is processing the method next for the indicator it processes each indicator at a time.
Is it possible to process them using all cores? -
I seams the answer is no, as I have been looking at the code. How ever if I could multiprocess the lines below in lineiterator.py it would work.
Does anyone knows how I can multiprocess this to paralyze indicator analysis?def _once(self): self.forward(size=self._clock.buflen()) for indicator in self._lineiterators[LineIterator.IndType]: indicator._once()
@ab_trader I have seen you are active in this forum. Sorry to call you directly, but do you know how to solve this?
-
@André-Tavares I don't know
-
@backtrader any thoughts on how to speed up this part?
-
I'm not sure the
multiprocessor
package is the way to go here because of possible data/indicators dependencies that need to be satisfied while calculating the indicator values (in case you are going to provide a fully generic solution).Anyway, this could easily require relatively serious development and re-design (even if you are talking about completely independent indicator/data line calculations).
If you have some design in mind, please share - it could be an interesting proposal.
-
@vladisld I don't have it. I have been trying to change the code to allow multiprocessing but it did not work.
Another thing I could do to achieve my goal would be to instead of passing to the strategy a list of companies, I could tell it 2 companies and make an optimization of the strategy. This way I would achieve multiprocessing. However I would like to have an analysis of the whole strategy with all pairs together, and doing this I will have the results per pair.
Is it possible to merge Analyzers in order to see all pairs results together?