Passing data into data feed one by one
Quanliang Xie last edited by
Hi, I want to test a single strategy using a series of csv files from a folder. I tried to pass them one by one into the data feed using cerebro. adddata(), but the data just keep adding together, which at a single time, it will be multiple data point.
All I can find online is passing into multiple data feed at the same time, but that is not what I want. I want to test my strategy in one data feed, then move on to next and so on.
Is there a way to put the datas into a queue and pass them into the cerebro one by one?
Here is my code:
if __name__ == '__main__': # Create a cerebro entity cerebro = bt.Cerebro() path = "/mnt/c/users/liang/desktop/backtesting/small_data/*.csv" for datapath in glob.glob(path): data = bt.feeds.GenericCSVData( dataname= datapath, dtformat= ('%Y-%m-%d %H:%M:%S UTC'), nullvalue=0.0, timeframe = TimeFrame.Minutes, datetime=0, time=-1, high=-1, low=-1, open=-1, close=1, #this is actully popularity volume=-1, openinterest=-1) # Add the Data Feed to Cerebro cerebro.adddata(data) cerebro.addstrategy(TestStrategy, Name=(datapath[50:-15])) print('Starting Portfolio Value: %.2f' % cerebro.broker.getvalue()) cerebro.run() print('Final Portfolio Value: %.2f' % cerebro.broker.getvalue())
What I did here is not efficient at all. I restarted cerebro multiple times. Is there a way to avioding this?
Thank you so much for the help!
run-out last edited by
You are doing this correctly and unfortunately cerebro will run a full back test using the data provided. So yes you must run separate backtest for each single security.
The good news is you can use optmization which uses multiprocessing to speed things up.
vladisld last edited by
I want to test a single strategy using a series of csv files from a folder
There are several scenarios:
- Running the same strategy on each csv file separately ( separate broker, account balance ) and sequentially (one after another):
In this case, just put the
cerebrocreating code inside your
forloop. In the code above the same
cerebroinstance is used to host all the datas and strategies. Since the datas are hosted directly in
cerebroinstance and not in
strategy- all the added datas are just accumulating in the same
- Same as above, only now run all the strategies simultaneously
Basically very similar - just put the code under
forloop inside a separate function which will recieve the csv path as a parameter and try to use the 'multiprocessing' python capabilities to run it.
Alternatively, you may try to use the Backtrader
optstrategyas @run-out has suggested, however, this is more involved and wasn't designed for your particular use case.
- Running all the csv files against the same broker account (sharing the account cash)
In this case you just need to add all the datas to the same
cerebroinstance as you are doing currently. However, only a single strategy needs to be added to the
cerebroinstance. In addition the
cerebroinstance should be run only once as well - and not under