How to implement walk-forward optimization with backtrader
Anyone has hints about implementing a walk-fowrad testing with backtrader.
I planning to follow the ideas shared by Kevin Davey in "Building Algorithmic Trading Systems" to test strategies.
wikipediaas an initial reference:
- In-Sample Data: It is a past segment of market data (historical data) reserved for testing purposes. This data is used for the initial testing and any optimization and is the original parameters of a system under test.
- Out-of-Sample Data: It is the reserved data set (historical data) which is not a part of the in-sample data. It is important as this ensures that the system is tested on another period of historical data not earlier thus removing any bias or influences in the checking of the system's performance.
Unless you have another use case in mind it seems like:
You work on the algorithm using a well-known data set.
Once you are satisfied you run the results on a non-well-known data set which lets you check the validity of the idea and the implementation without having overfitted the parameters and logic to a specific set of data
That would be done with the same code in different passes in backtrader
Is there anything specific, a special use case you would have in mind?
I have the following scenario in mind:
Pass the system the following inputs:
- historical data
- in period duration (5Y)
- out period duration (1Y)
- parameters and ranges
Let's say we have 10 years worth of data, then the ideal processing would be as follows:
- Break the data in multiple chunks according to the periods defined above
- In Period 1: 2000-2005
- Out Period 1: 2005-2006
- In Period 2: 2001:2006
- Out Period 2: 2006-2007
- In Period 3: 2002-2007
- Out Period 3: 2007-2008
Than start executiing the execution.
- Optimize parameters for first In Period (2000-2005) - cerebro.optstrategy()
- Get best parameters from previous run and test on Out Period 1 - cerebro.addstrategy()
- Optimize again on the next In Period
- Test with the parameters from previous step on the next out period
- Optimize again ......
- Test again on the next Out Period .....
At the end plot the combined strategy during all out periods from 2005 to 2010 by taking into consideration that parameters are being changed every year with the optimized values from last 5 years worth of data.
I am open for any ideas for implementation.
So far I am thinking to read the data in pandas, break it and loop through it in chunks and feed it to cerebro consecuently.
I have no idea how to combine the outputs from each cerebro run and how to plot all individual strategy executions into a single figure.