Multiple symbols + each symbol multiple time frames issue
I have an strategy that has multiple symbols and each symbols has two simultaneous data feeds (minute and day frequency). So if I have N symbols, I have 2N datafeeds that are added to Cerebro.
I could run the strategy with no problem previously, but after updating backtrader to the last two versions it doesn't work anymore. The same strategy works fine if I only have one symbol (still with two data feeds -- minute and day frequency). Any idea what that can be? I can send you the code if it helps.
@rastegarr The version that I'm using is 22.214.171.124.
Version number is for sure a good start. In case you have been following the latest discussions in the community, the resampling code has been fine tuned across the last 3-4 releases to improve bar delivery and synchronization. The end of session is being used as much as possible to decide where the limit is for subdays and days resampling.
The tests done and the experiences in the other threads seem to confirm the validity of the changes, but some corners could still be there.
In order to understand what happens, it would be good to know:
- What is not working
- How the use case looks like (a simple streamlined script, no logic in
That's how things have positively been worked out, along several sessions, in the other thread working on this.
When I have more than one symbols with two different time frames(min and day), the next function is not called: basically, start and stop are called but not the next.
Here is shortened code:
class TestInstrument(): def __init__(self): self.symbol = None self.order = None self.live = False self.i_macd = None self.i_sma = None self.tradable_today = False self.no_longs = 0 self.no_shorts = 0 self.no_profitable_longs = 0 self.no_profitable_shorts = 0 self.current_position_is_long = None self.profit = 0 class MACDStrategy(bt.Strategy): params = dict( initial_cash = None, macd1 = 12, #12 macd2 = 26, #26 macdsig = 9, smaperiod = 28, #28 exectype=bt.Order.Market, max_investment_per_trade = 500000, max_loss_perc_trade = 3000, max_profit_perc_trade = 4000, printlog = False, savelog = False, start_time = datetime.time(9, 35), end_time = datetime.time(15, 30), #originally 35 not zero enter_short_bound = 0.4, #0.3 exit_short_bound = 0, enter_long_bound = -0.4, output_dir = "d:\\glt\\output\\all\\", ) def __init__(self): self.current_date = None self.market_hours = None self.log_file = None self.instruments = dict((self.datas[indx]._dataname, TestInstrument()) for indx in range(0, len(self.datas), 2)) if self.p.savelog: self.log_file = open(self.p.output_dir + "@enter_short" + str(self.p.enter_short_bound) + "@enter_long" + str(self.p.enter_long_bound) + "@max_loss" + str(self.p.max_loss_perc_trade * self.p.max_investment_per_trade) + "@max_profit" + str(self.p.max_profit_perc_trade * self.p.max_investment_per_trade) + ".txt", "w") self.optim_file = open(self.p.output_dir + 'optimization.csv', 'a') for indx in range(0, len(self.datas), 2): datax = self.datas[indx] datax_day = self.datas[indx+1] instrument_name = datax._dataname self.instruments[instrument_name].i_macd = bt.indicators.MACD(datax, period_me1=self.p.macd1, period_me2=self.p.macd2, period_signal=self.p.macdsig) self.instruments[instrument_name].i_sma = bt.indicators.SMA(datax_day, period=self.p.smaperiod) self.log('Signals are created for: %s' %(instrument_name)) print('******************* Strategy Created *********************') def nextstart(self): self.log('--------------------------------------------------') self.log('nextstart called with len %d' %len(self)) self.log('--------------------------------------------------') super(MACDStrategy, self).nextstart() def next(self, frompre=False): self.log("Next is called", doprint = True) #some logic here def start(self): self.log('Start', doprint = True) pass def stop(self): self.log('Stop', doprint = True) if __name__ == '__main__': mydir='d:\\glt\\sp500energy1516_market' # minute based data all_syms = os.listdir(mydir) mycash = 500000 * len(all_syms) / 4 # leverage 4 cerebro = bt.Cerebro(exactbars = False) cerebro.broker.setcash(mycash) for file in all_syms: # Create a Data Feed data = bt.feeds.GenericCSVData( dataname=mydir + "\\" + file, #(2015, 12, 20, 7, 30)-(2016, 2, 22, 16, 0) -> 2 22 Jan #(2016, 2, 24, 7, 30)-(2016, 4, 25, 16, 0) -> 5 25 April fromdate=datetime.datetime(2015, 1, 1, 7, 30), todate=datetime.datetime(2016, 12, 30, 16, 0), nullvalue=0.0, dtformat=('%Y%m%d'), tmformat = ('%H%M'), datetime=0, time=1, open=2, high=3, low=4, close=5, volume=6, openinterest=7) cerebro.adddata(data, name = file[:-4]) cerebro.resampledata(data, timeframe=bt.TimeFrame.Days, compression = 1, name = file[:-4] + "_d") #Set the commission #cerebro.broker.setcommission(commission=0) cerebro.broker.setcommission(commission=0.00003, margin= None, mult=1.0, commtype=None, percabs=True, stocklike=True, leverage=4.0) #Set the slippage cerebro.broker.set_slippage_perc(0.001, slip_open=True, slip_limit=True, slip_match=True, slip_out=False) cerebro.addanalyzer(bt.analyzers.Returns) # Add a strategy cerebro.optstrategy(MACDStrategy, enter_short_bound = [0.5, 0.45], enter_long_bound = [-0.5, -0.45], max_loss_perc_trade = [0.018, 0.016, 0.014, 0.012], max_profit_perc_trade = [0.018, 0.016, 0.014, 0.012], initial_cash = mycash, max_investment_per_trade = 500000, savelog = True) #Run over everything cerebro.run()
Sorry I cannot format the code correctly here.
```python (and close with ``` on a newline again)
or read the
COMPOSEhelp shown in the editing box. It' standard
The sample even if clearly reduced, seems still overly complicated, although it seems (remark: seems) ok.
This is slight variation of what is usually in place to stress resampling to have two
1-minutedata feeds resampled to days.
from __future__ import (absolute_import, division, print_function, unicode_literals) import backtrader as bt import backtrader.utils.flushfile # for early flush under Win32 class St(bt.Strategy): params = (('p', 0),) def prenext(self): self.next() def next(self): print(','.join(str(x) for x in [ 'Strategy', len(self), self.datetime.datetime().strftime('%Y-%m-%dT%H:%M:%S')]) ) for i, d in enumerate(d for d in self.datas if len(d)): out = ['Data' + str(i), len(d), d.datetime.datetime().strftime('%Y-%m-%dT%H:%M:%S'), d.open, d.high, d.low, d.close] print(', '.join(str(x) for x in out)) if __name__ == '__main__': cerebro = bt.Cerebro() data0 = bt.feeds.BacktraderCSVData(dataname='../2006-min-001.txt') cerebro.adddata(data0) cerebro.resampledata(data0, timeframe=bt.TimeFrame.Days) data1 = bt.feeds.BacktraderCSVData(dataname='../2006-data1-min-001.txt') cerebro.adddata(data1) cerebro.resampledata(data1, timeframe=bt.TimeFrame.Days) cerebro.optstrategy(St, p=[0, 1]) cerebro.run(maxcpus=1)
And some of the output
... Strategy,1995,2006-01-04T19:41:00 Data0, 1988, 2006-01-04T19:41:00, 2791.0, 2792.0, 2791.0, 2792.0 Data1, 1, 2006-01-02T23:59:59, 2748.0, 2765.0, 2743.0, 2759.0 Data2, 139, 2006-01-04T19:30:00, 5538.0, 5545.5, 5536.0, 5543.5 Data3, 1, 2006-01-02T23:59:59, 5448.0, 5483.0, 5441.5, 5475.5 ...
maxcpus=None(the default) will use multiple cores and that will interleave the output, but will also work.
@backtrader Thanks for your reply. Yes, it works with some data files, but not with others. Is there anyway I can attach two files here as examples?
You can directly upload files into the thread. Not much input would be needed.
An alternative is to enclose the data (which is apparently
csv) as a code block (with no
pythonspecifyer) and it will allow simple copy & paste.