Multiple symbols + each symbol multiple time frames issue



  • I have an strategy that has multiple symbols and each symbols has two simultaneous data feeds (minute and day frequency). So if I have N symbols, I have 2N datafeeds that are added to Cerebro.

    I could run the strategy with no problem previously, but after updating backtrader to the last two versions it doesn't work anymore. The same strategy works fine if I only have one symbol (still with two data feeds -- minute and day frequency). Any idea what that can be? I can send you the code if it helps.



  • @rastegarr The version that I'm using is 1.9.27.105.


  • administrators

    Version number is for sure a good start. In case you have been following the latest discussions in the community, the resampling code has been fine tuned across the last 3-4 releases to improve bar delivery and synchronization. The end of session is being used as much as possible to decide where the limit is for subdays and days resampling.

    The tests done and the experiences in the other threads seem to confirm the validity of the changes, but some corners could still be there.

    In order to understand what happens, it would be good to know:

    • What is not working
    • How the use case looks like (a simple streamlined script, no logic in next)

    That's how things have positively been worked out, along several sessions, in the other thread working on this.



  • @backtrader

    When I have more than one symbols with two different time frames(min and day), the next function is not called: basically, start and stop are called but not the next.

    Here is shortened code:

    class TestInstrument():
        def __init__(self):
            self.symbol = None
            self.order = None
            self.live = False
            self.i_macd = None
            self.i_sma = None
            self.tradable_today = False
            self.no_longs = 0
            self.no_shorts = 0
            self.no_profitable_longs = 0
            self.no_profitable_shorts = 0
            self.current_position_is_long = None
            self.profit = 0
    
    class MACDStrategy(bt.Strategy):
        params = dict(
            initial_cash = None,
            macd1 = 12, #12
            macd2 = 26, #26
            macdsig = 9,
            smaperiod = 28, #28 
            exectype=bt.Order.Market,
            max_investment_per_trade = 500000,
            max_loss_perc_trade = 3000,
            max_profit_perc_trade = 4000,
            printlog = False,
            savelog = False,
            start_time = datetime.time(9, 35),
            end_time = datetime.time(15, 30), #originally 35 not zero
            enter_short_bound = 0.4, #0.3
            exit_short_bound = 0,
            enter_long_bound = -0.4,
            output_dir = "d:\\glt\\output\\all\\",
        )
    
        def __init__(self):
            self.current_date =  None
            self.market_hours = None
            self.log_file = None
            self.instruments = dict((self.datas[indx]._dataname, TestInstrument()) for indx in range(0, len(self.datas), 2)) 
            if self.p.savelog:
                self.log_file = open(self.p.output_dir + "@enter_short" + str(self.p.enter_short_bound) + "@enter_long" + str(self.p.enter_long_bound) + "@max_loss" + str(self.p.max_loss_perc_trade * self.p.max_investment_per_trade) + "@max_profit" + str(self.p.max_profit_perc_trade * self.p.max_investment_per_trade) + ".txt", "w")
                self.optim_file = open(self.p.output_dir + 'optimization.csv', 'a')
            for indx in range(0, len(self.datas), 2):
                datax = self.datas[indx]
                datax_day = self.datas[indx+1]
                instrument_name = datax._dataname
                self.instruments[instrument_name].i_macd = bt.indicators.MACD(datax, period_me1=self.p.macd1, period_me2=self.p.macd2, period_signal=self.p.macdsig)
                self.instruments[instrument_name].i_sma = bt.indicators.SMA(datax_day, period=self.p.smaperiod)
                self.log('Signals are created for: %s' %(instrument_name))
            print('******************* Strategy Created *********************')
    
        def nextstart(self):
            self.log('--------------------------------------------------')
            self.log('nextstart called with len %d' %len(self))
            self.log('--------------------------------------------------')
            super(MACDStrategy, self).nextstart()
    
        def next(self, frompre=False):
            self.log("Next is called", doprint = True)
            #some logic here
                        
        def start(self):
           self.log('Start', doprint = True)
           pass
    
        def stop(self):
            self.log('Stop', doprint = True)
    
    if __name__ == '__main__':
        
        mydir='d:\\glt\\sp500energy1516_market'  # minute based data
        all_syms = os.listdir(mydir)
        mycash = 500000 * len(all_syms) / 4  # leverage 4
        cerebro = bt.Cerebro(exactbars = False)
        cerebro.broker.setcash(mycash)
        
        for file in all_syms:
            # Create a Data Feed
            data = bt.feeds.GenericCSVData(
                dataname=mydir + "\\" + file,
                #(2015, 12, 20, 7, 30)-(2016, 2, 22, 16, 0) -> 2 22 Jan
                #(2016, 2, 24, 7, 30)-(2016, 4, 25, 16, 0) -> 5 25 April
                fromdate=datetime.datetime(2015, 1, 1, 7, 30), 
                todate=datetime.datetime(2016, 12, 30, 16, 0),
                nullvalue=0.0,
                dtformat=('%Y%m%d'),
                tmformat = ('%H%M'),
                datetime=0,
                time=1,
                open=2,
                high=3,
                low=4,
                close=5,
                volume=6,
                openinterest=7)
            cerebro.adddata(data, name = file[:-4])
            cerebro.resampledata(data, timeframe=bt.TimeFrame.Days, compression = 1, name = file[:-4] + "_d")
            
        #Set the commission
        #cerebro.broker.setcommission(commission=0)
        cerebro.broker.setcommission(commission=0.00003, margin= None, mult=1.0, commtype=None, percabs=True, stocklike=True, leverage=4.0)
    
        #Set the slippage
        cerebro.broker.set_slippage_perc(0.001, slip_open=True, slip_limit=True, slip_match=True, slip_out=False)
    
        cerebro.addanalyzer(bt.analyzers.Returns)
        
        # Add a strategy
        cerebro.optstrategy(MACDStrategy, 
                            enter_short_bound = [0.5, 0.45],
                            enter_long_bound = [-0.5, -0.45], 
                            max_loss_perc_trade = [0.018, 0.016, 0.014, 0.012], 
                            max_profit_perc_trade = [0.018, 0.016, 0.014, 0.012], 
                            initial_cash = mycash, max_investment_per_trade = 500000, savelog = True)
        
        #Run over everything
        cerebro.run()
    


  • Sorry I cannot format the code correctly here.


  • administrators

    Use

    ```python (and close with ``` on a newline again)
    

    or read the COMPOSE help shown in the editing box. It' standard markdown.

    The sample even if clearly reduced, seems still overly complicated, although it seems (remark: seems) ok.

    This is slight variation of what is usually in place to stress resampling to have two 1-minute data feeds resampled to days.

    from __future__ import (absolute_import, division, print_function,
                            unicode_literals)
    
    import backtrader as bt
    import backtrader.utils.flushfile  # for early flush under Win32
    
    
    class St(bt.Strategy):
        params = (('p', 0),)
    
        def prenext(self):
            self.next()
    
        def next(self):
            print(','.join(str(x) for x in [
                'Strategy', len(self),
                self.datetime.datetime().strftime('%Y-%m-%dT%H:%M:%S')])
            )
    
            for i, d in enumerate(d for d in self.datas if len(d)):
                out = ['Data' + str(i), len(d),
                       d.datetime.datetime().strftime('%Y-%m-%dT%H:%M:%S'),
                       d.open[0], d.high[0], d.low[0], d.close[0]]
                print(', '.join(str(x) for x in out))
    
    
    if __name__ == '__main__':
        cerebro = bt.Cerebro()
    
        data0 = bt.feeds.BacktraderCSVData(dataname='../2006-min-001.txt')
        cerebro.adddata(data0)
        cerebro.resampledata(data0, timeframe=bt.TimeFrame.Days)
        data1 = bt.feeds.BacktraderCSVData(dataname='../2006-data1-min-001.txt')
        cerebro.adddata(data1)
        cerebro.resampledata(data1, timeframe=bt.TimeFrame.Days)
        cerebro.optstrategy(St, p=[0, 1])
        cerebro.run(maxcpus=1)
    

    And some of the output

    ...
    Strategy,1995,2006-01-04T19:41:00
    Data0, 1988, 2006-01-04T19:41:00, 2791.0, 2792.0, 2791.0, 2792.0
    Data1, 1, 2006-01-02T23:59:59, 2748.0, 2765.0, 2743.0, 2759.0
    Data2, 139, 2006-01-04T19:30:00, 5538.0, 5545.5, 5536.0, 5543.5
    Data3, 1, 2006-01-02T23:59:59, 5448.0, 5483.0, 5441.5, 5475.5
    ...
    

    Changing maxcpus=1 to maxcpus=None (the default) will use multiple cores and that will interleave the output, but will also work.



  • @backtrader Thanks for your reply. Yes, it works with some data files, but not with others. Is there anyway I can attach two files here as examples?


  • administrators

    You can directly upload files into the thread. Not much input would be needed.

    An alternative is to enclose the data (which is apparently csv) as a code block (with no python specifyer) and it will allow simple copy & paste.


Log in to reply
 

Looks like your connection to Backtrader Community was lost, please wait while we try to reconnect.