Passing data into data feed one by one
-
Hi, I want to test a single strategy using a series of csv files from a folder. I tried to pass them one by one into the data feed using cerebro. adddata(), but the data just keep adding together, which at a single time, it will be multiple data point.
All I can find online is passing into multiple data feed at the same time, but that is not what I want. I want to test my strategy in one data feed, then move on to next and so on.
Is there a way to put the datas into a queue and pass them into the cerebro one by one?
Here is my code:
if __name__ == '__main__': # Create a cerebro entity cerebro = bt.Cerebro() path = "/mnt/c/users/liang/desktop/backtesting/small_data/*.csv" for datapath in glob.glob(path): data = bt.feeds.GenericCSVData( dataname= datapath, dtformat= ('%Y-%m-%d %H:%M:%S UTC'), nullvalue=0.0, timeframe = TimeFrame.Minutes, datetime=0, time=-1, high=-1, low=-1, open=-1, close=1, #this is actully popularity volume=-1, openinterest=-1) # Add the Data Feed to Cerebro cerebro.adddata(data) cerebro.addstrategy(TestStrategy, Name=(datapath[50:-15])) print('Starting Portfolio Value: %.2f' % cerebro.broker.getvalue()) cerebro.run() print('Final Portfolio Value: %.2f' % cerebro.broker.getvalue())
What I did here is not efficient at all. I restarted cerebro multiple times. Is there a way to avioding this?
Thank you so much for the help!
-
You are doing this correctly and unfortunately cerebro will run a full back test using the data provided. So yes you must run separate backtest for each single security.
The good news is you can use optmization which uses multiprocessing to speed things up.
-
@Quanliang-Xie said in Passing data into data feed one by one:
I want to test a single strategy using a series of csv files from a folder
There are several scenarios:
- Running the same strategy on each csv file separately ( separate broker, account balance ) and sequentially (one after another):
In this case, just put the
cerebro
creating code inside yourfor
loop. In the code above the samecerebro
instance is used to host all the datas and strategies. Since the datas are hosted directly incerebro
instance and not instrategy
- all the added datas are just accumulating in the samecerebro
instance.- Same as above, only now run all the strategies simultaneously
Basically very similar - just put the code under
for
loop inside a separate function which will recieve the csv path as a parameter and try to use the 'multiprocessing' python capabilities to run it.Alternatively, you may try to use the Backtrader
optstrategy
as @run-out has suggested, however, this is more involved and wasn't designed for your particular use case.- Running all the csv files against the same broker account (sharing the account cash)
In this case you just need to add all the datas to the same
cerebro
instance as you are doing currently. However, only a single strategy needs to be added to thecerebro
instance. In addition thecerebro
instance should be run only once as well - and not underfor
loop.