Passing data into data feed one by one

Quanliang Xie

Hi, I want to test a single strategy using a series of csv files from a folder. I tried to pass them one by one into the data feed using cerebro. adddata(), but the data just keep adding together, which at a single time, it will be multiple data point.

All I can find online is passing into multiple data feed at the same time, but that is not what I want. I want to test my strategy in one data feed, then move on to next and so on.

Is there a way to put the datas into a queue and pass them into the cerebro one by one?

Here is my code:

if __name__ == '__main__':	
	# Create a cerebro entity
	cerebro = bt.Cerebro()


	path = "/mnt/c/users/liang/desktop/backtesting/small_data/*.csv"
	for datapath in glob.glob(path):
		data = bt.feeds.GenericCSVData(
			dataname= datapath,
			dtformat= ('%Y-%m-%d %H:%M:%S UTC'),
			nullvalue=0.0,
			timeframe = TimeFrame.Minutes,
			datetime=0,
			time=-1,
			high=-1,
			low=-1,
			open=-1,
			close=1, #this is actully popularity
			volume=-1,
			openinterest=-1)

		# Add the Data Feed to Cerebro
		cerebro.adddata(data)
		cerebro.addstrategy(TestStrategy, Name=(datapath[50:-15]))
		print('Starting Portfolio Value: %.2f' % cerebro.broker.getvalue())
		cerebro.run()
		print('Final Portfolio Value: %.2f' % cerebro.broker.getvalue())

What I did here is not efficient at all. I restarted cerebro multiple times. Is there a way to avioding this?

Thank you so much for the help!

run-out

You are doing this correctly and unfortunately cerebro will run a full back test using the data provided. So yes you must run separate backtest for each single security.

The good news is you can use optmization which uses multiprocessing to speed things up.

vladisld

@Quanliang-Xie said in Passing data into data feed one by one:

I want to test a single strategy using a series of csv files from a folder

There are several scenarios:

Running the same strategy on each csv file separately ( separate broker, account balance ) and sequentially (one after another):

In this case, just put the cerebro creating code inside your for loop. In the code above the same cerebro instance is used to host all the datas and strategies. Since the datas are hosted directly in cerebro instance and not in strategy - all the added datas are just accumulating in the same cerebro instance.

Same as above, only now run all the strategies simultaneously

Basically very similar - just put the code under for loop inside a separate function which will recieve the csv path as a parameter and try to use the 'multiprocessing' python capabilities to run it.

Alternatively, you may try to use the Backtrader optstrategy as @run-out has suggested, however, this is more involved and wasn't designed for your particular use case.

Running all the csv files against the same broker account (sharing the account cash)

In this case you just need to add all the datas to the same cerebro instance as you are doing currently. However, only a single strategy needs to be added to the cerebro instance. In addition the cerebro instance should be run only once as well - and not under for loop.