For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

index of the new data



  • Hello,

    For my strategy I am using multiple datas.
    I am trying to figure out which data among self.datas has just arrived. What is the best practice to obtain the index no of the newly arrived data?

    I was thinking of keeping a tracklist of last datetimes of each data and comparing self.datas against that tracklist in next(). Do you have any better suggestions?

    Best



  • @bb2

    I'm not sure if this answers your question, but here's what the beginning of my next() function looks like:

            def next(self):
    
                for index, data in enumerate(self.datas):
                    datetime, dataname = self.datetime.date(), data._name
                    print("Processing {} from {}".format(self.datas[index].close, dataname)
                  
    

    I comment out this print function when it comes time to run a big test, but when I'm configuring things it's helpful to see which data is being worked with and make sure everything is loading properly.

    In order for data._name to be useful, you need to do something like this when you add the data:

    cerebro.adddata(data, name='name_of_your_data')
    

  • administrators

    Data feeds get the length incremented when new data has arrived. As such

    • Init (for example a list) a data structure with as many 0 as data feeds you add to the system
    • Go over self.datas and the previous data structure and check the len of each. If it has increased, new data is there.
    • Update the structure with the latest values


  • thanks :)



  • @backtrader I have tried the following and it seems the length of data is not being incremented all the time.

    Strategy:

        def __init__(self):
            self.data_length = [len(data) for data in self.datas]
            return
        
        def next(self):
            for idx, length in enumerate(self.data_length):
                if length != len(self.datas[idx]):
                    print(idx, length, len(self.datas[idx]))
            self.data_length = [len(data) for data in self.datas]
    
            print()
            return
    

    Sample printout:

    
    5 1564 1565
    
    5 1565 1566
    
    0 1616 1617
    
    0 1617 1618
    
    5 1566 1567
    
    0 1618 1619
    
    0 1619 1620
    
    0 1620 1621
    6 125 124
    7 26 25
    8 9 8
    9 5 4
    
    0 1621 1622
    
    1 125 124
    2 26 25
    3 9 8
    4 5 4
    5 1567 1568
    6 124 125
    7 25 26
    8 8 9
    9 4 5
    
    5 1568 1569
    
    0 1622 1623
    1 124 125
    2 25 26
    3 8 9
    4 4 5
    
    5 1569 1570
    
    0 1623 1624
    
    0 1624 1625
    6 125 124
    7 26 25
    8 9 8
    9 5 4
    
    0 1625 1626
    
    0 1626 1627
    
    1 125 124
    2 26 25
    3 9 8
    4 5 4
    5 1570 1571
    6 124 125
    7 25 26
    8 8 9
    9 4 5
    
    5 1571 1572
    
    0 1627 1628
    1 124 125
    2 25 26
    3 8 9
    4 4 5
    
    0 1628 1629
    
    5 1572 1573
    
    0 1629 1630
    
    5 1573 1574
    
    1 125 124
    2 26 25
    3 9 8
    4 5 4
    5 1574 1575
    
    5 1575 1576
    
    0 1630 1631
    1 124 125
    2 25 26
    3 8 9
    4 4 5
    
    0 1631 1632
    
    0 1632 1633
    6 125 124
    7 26 25
    8 9 8
    9 5 4
    
    0 1633 1634
    
    0 1634 1635
    
    5 1576 1577
    6 124 125
    7 25 26
    8 8 9
    9 4 5
    
    1 125 124
    2 26 25
    3 9 8
    4 5 4
    5 1577 1578
    
    5 1578 1579
    
    0 1635 1636
    1 124 125
    2 25 26
    3 8 9
    4 4 5
    
    0 1636 1637
    

    the datas with idx 1, 2, 3, 4, 6, 7, 8, 9 are from replaydata, the ones with idx 0, 5 are data feeds from IB. Why do the data lengths of replaydata decrease time to time?


  • administrators

    @bb2 said in index of the new data:

    self.data_length = [len(data) for data in self.datas]
    

    Don't you think that you should be initializing that to 0? It's simply a matter of good style and the data feeds may have initialized themselves in the background and be growing in length.

    @bb2 said in index of the new data:

    the datas with idx 1, 2, 3, 4, 6, 7, 8, 9 are from replaydata, the ones with idx 0, 5 are data feeds from IB. Why do the data lengths of replaydata decrease time to time?

    Unless you show us the code, how can we tell what you are doing?



  • @bb2 said in index of the new data:

    the datas with idx 1, 2, 3, 4, 6, 7, 8, 9 are from replaydata, the ones with idx 0, 5 are data feeds from IB. Why do the data lengths of replaydata decrease time to time?

    Unless you show us the code, how can we tell what you are doing?

    I am observing this behaviour only for replayed datas. Following contains the code which replays 2 datas. Still observing decrementals in len(self.datas[i]).

    printout:

    
    
    0 967 966
    AUD.CAD-CASH-IDEALPRO_tick 2018-07-18 07:26:29.765855 0.9735
    
    1 966 967
    AUD.NZD-CASH-IDEALPRO_tick 2018-07-18 07:26:59.629664 1.08845
    
    0 966 967
    AUD.CAD-CASH-IDEALPRO_tick 2018-07-18 07:26:58.528423 0.97348
    
    1 967 966
    AUD.NZD-CASH-IDEALPRO_tick 2018-07-18 07:26:29.764980 1.08843
    
    0 967 966
    AUD.CAD-CASH-IDEALPRO_tick 2018-07-18 07:26:29.765855 0.9735
    
    1 966 967
    AUD.NZD-CASH-IDEALPRO_tick 2018-07-18 07:26:59.629664 1.08845
    
    0 966 967
    AUD.CAD-CASH-IDEALPRO_tick 2018-07-18 07:26:58.528423 0.97348
    
    1 967 966
    AUD.NZD-CASH-IDEALPRO_tick 2018-07-18 07:26:29.764980 1.08843
    
    0 967 966
    AUD.CAD-CASH-IDEALPRO_tick 2018-07-18 07:26:29.765855 0.9735
    
    1 966 967
    AUD.NZD-CASH-IDEALPRO_tick 2018-07-18 07:26:59.629664 1.08845
    
    0 966 967
    AUD.CAD-CASH-IDEALPRO_tick 2018-07-18 07:26:58.528423 0.97348
    
    1 967 966
    AUD.NZD-CASH-IDEALPRO_tick 2018-07-18 07:26:29.764980 1.08843
    
    0 967 966
    AUD.CAD-CASH-IDEALPRO_tick 2018-07-18 07:26:29.765855 0.9735
    
    1 966 967
    AUD.NZD-CASH-IDEALPRO_tick 2018-07-18 07:26:59.629664 1.08845
    
    0 966 967
    AUD.CAD-CASH-IDEALPRO_tick 2018-07-18 07:26:58.528423 0.97348
    
    1 967 966
    AUD.NZD-CASH-IDEALPRO_tick 2018-07-18 07:26:29.764980 1.08843
    
    0 967 966
    AUD.CAD-CASH-IDEALPRO_tick 2018-07-18 07:26:29.765855 0.9735
    
    1 966 967
    AUD.NZD-CASH-IDEALPRO_tick 2018-07-18 07:26:59.629664 1.08845
    
    0 966 967
    AUD.CAD-CASH-IDEALPRO_tick 2018-07-18 07:26:58.528423 0.97348
    
    1 967 966
    AUD.NZD-CASH-IDEALPRO_tick 2018-07-18 07:26:29.764980 1.08843
    
    

    is the following enough?

        datanames = [
                'AUD.CAD-CASH-IDEALPRO',
                'AUD.NZD-CASH-IDEALPRO',
            ]
            port = 4002
            clientId = 100
            
            cerebro = bt.Cerebro(stdstats=False)
        
            ibstore = bt.stores.IBStore(
                host='127.0.0.1', port=port, clientId=clientId)
        
        
            for dataname in datanames:
                data = ibstore.getdata(dataname=dataname, qcheck=2.0, tz=pytz.timezone(
                cerebro.replaydata(data, name='{}_tick'.format(dataname), timeframe=bt.TimeFrame.Seconds, compression=30)
            
        
            cerebro.broker.setcash(100000.0)
            cerebro.broker.setcommission(commission=0.0)
            cerebro.addanalyzer(bt.analyzers.TradeAnalyzer, _name="ta")
        
            cerebro.addstrategy(Strat)
            cerebro.addsizer(bt.sizers.FixedSize, stake=1)
        
            strats = cerebro.run(tradehistory=True)