For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

Handling of Multiple Data Feeds with different start date



  • Hello,

    I am adding 69 Data Feeds to cerebro, each one with its own start and end date. In this example, mostly of them but not all of them, have 2010-01-01 as start date:

    1 63DU GY Equity DEUTSCHE BOERSE AG-NEW 2011-05-20 2012-02-10
    2 ABI BB Equity ANHEUSER-BUSCH INBEV SA/NV 2010-01-01 2019-12-27
    3 ACA FP Equity CREDIT AGRICOLE SA 2010-01-01 2019-12-27
    4 AD NA Equity KONINKLIJKE AHOLD DELHAIZE N 2010-01-01 2019-12-27
    5 ADS GY Equity ADIDAS AG 2010-01-01 2019-12-27
    6 AGN NA Equity AEGON NV 2010-01-01 2019-12-27
    7 AI FP Equity AIR LIQUIDE SA 2010-01-01 2019-12-27
    8 AIR FP Equity AIRBUS SE 2010-01-01 2019-12-27
    9 ALO FP Equity ALSTOM 2010-01-01 2019-12-27
    10 ALV GY Equity ALLIANZ SE-REG 2010-01-01 2019-12-27
    11 AMS SQ Equity AMADEUS IT GROUP SA 2010-04-30 2019-12-27
    12 ASML NA Equity ASML HOLDING NV 2010-01-01 2019-12-27
    13 BAS GY Equity BASF SE 2010-01-01 2019-12-27
    14 BAYN GY Equity BAYER AG-REG 2010-01-01 2019-12-27
    15 BBVA SQ Equity BANCO BILBAO VIZCAYA ARGENTA 2010-01-01 2019-12-27
    16 BMW GY Equity BAYERISCHE MOTOREN WERKE AG 2010-01-01 2019-12-27
    17 BN FP Equity DANONE 2010-01-01 2019-12-27
    18 BNP FP Equity BNP PARIBAS 2010-01-01 2019-12-27
    19 CA FP Equity CARREFOUR SA 2010-01-01 2019-12-27
    20 CRH ID Equity CRH PLC 2010-01-01 2019-12-27
    21 CS FP Equity AXA SA 2010-01-01 2019-12-27
    22 DAI GY Equity DAIMLER AG-REGISTERED SHARES 2010-01-01 2019-12-27
    23 DB1 GY Equity DEUTSCHE BOERSE AG 2010-01-01 2019-12-27
    24 DBK GY Equity DEUTSCHE BANK AG-REGISTERED 2010-01-01 2019-12-27
    25 DG FP Equity VINCI SA 2010-01-01 2019-12-27
    26 DPW GY Equity DEUTSCHE POST AG-REG 2010-01-01 2019-12-27
    27 DTE GY Equity DEUTSCHE TELEKOM AG-REG 2010-01-01 2019-12-27
    28 EL FP Equity ESSILORLUXOTTICA 2010-01-01 2019-12-27
    29 ENEL IM Equity ENEL SPA 2010-01-01 2019-12-27
    30 ENGI FP Equity ENGIE 2010-01-01 2019-12-27
    31 ENI IM Equity ENI SPA 2010-01-01 2019-12-27
    32 EOAN GY Equity E.ON SE 2010-01-01 2019-12-27
    33 FP FP Equity TOTAL SA 2010-01-01 2019-12-27
    34 FRE GY Equity FRESENIUS SE & CO KGAA 2010-01-01 2019-12-27
    35 G IM Equity ASSICURAZIONI GENERALI 2010-01-01 2019-12-27
    36 GLE FP Equity SOCIETE GENERALE SA 2010-01-01 2019-12-27
    37 IBE SQ Equity IBERDROLA SA 2010-01-01 2019-12-27
    38 INGA NA Equity ING GROEP NV 2010-01-01 2019-12-27
    39 ISP IM Equity INTESA SANPAOLO 2010-01-01 2019-12-27
    40 ITX SQ Equity INDUSTRIA DE DISENO TEXTIL 2010-01-01 2019-12-27
    41 KER FP Equity KERING 2010-01-01 2019-12-27
    42 LIN GY Equity LINDE PLC 2018-11-02 2019-12-27
    43 LINU GY Equity LINDE AG - TENDER 2017-08-25 2018-10-26
    44 MC FP Equity LVMH MOET HENNESSY LOUIS VUI 2010-01-01 2019-12-27
    45 MT NA Equity ARCELORMITTAL 2010-01-01 2019-12-27
    46 MUV2 GY Equity MUENCHENER RUECKVER AG-REG 2010-01-01 2019-12-27
    47 NOKIA FH Equity NOKIA OYJ 2010-01-01 2019-12-27
    48 OR FP Equity L'OREAL 2010-01-01 2019-12-27
    49 ORA FP Equity ORANGE 2010-01-01 2019-12-27
    50 PHIA NA Equity KONINKLIJKE PHILIPS NV 2010-01-01 2019-12-27
    51 REP SQ Equity REPSOL SA 2010-01-01 2019-12-27
    52 RWE GY Equity RWE AG 2010-01-01 2019-12-27
    53 SAF FP Equity SAFRAN SA 2010-01-01 2019-12-27
    54 SAN FP Equity SANOFI 2010-01-01 2019-12-27
    55 SAN SQ Equity BANCO SANTANDER SA 2010-01-01 2019-12-27
    56 SAP GY Equity SAP SE 2010-01-01 2019-12-27
    57 SGO FP Equity COMPAGNIE DE SAINT GOBAIN 2010-01-01 2019-12-27
    58 SIE GY Equity SIEMENS AG-REG 2010-01-01 2019-12-27
    59 SU FP Equity SCHNEIDER ELECTRIC SE 2010-01-01 2019-12-27
    60 TEF SQ Equity TELEFONICA SA 2010-01-01 2019-12-27
    61 TIT IM Equity TELECOM ITALIA SPA 2010-01-01 2019-12-27
    62 UCG IM Equity UNICREDIT SPA 2010-01-01 2019-12-27
    63 UL FP Equity UNIBAIL-RODAMCO SE 2010-01-01 2013-03-01
    64 UL NA Equity UNIBAIL-RODAMCO SE 2010-01-01 2018-06-08
    65 UNA NA Equity UNILEVER NV 2010-01-01 2019-12-27
    66 URW NA Equity UNIBAIL-RODAMCO-WESTFIELD 2010-01-01 2019-12-27
    67 VIV FP Equity VIVENDI 2010-01-01 2019-12-27
    68 VOW GY Equity VOLKSWAGEN AG 2010-01-01 2019-07-12
    69 VOW3 GY Equity VOLKSWAGEN AG-PREF 2010-01-01 2019-12-27

    next and prenext

    If I should only use the next method the strategy will start to generate signals from the most recent common start date: 2018-11-02

    I understand this is by design, so that code never breaks inside next.

    To bypass this wanted behavior and generate signals from the first available start date of each data feed I am using the prenext method as suggested on several topics.

    This is the code of a simple Strategy that does only some logging:

    class Demo(bt.Strategy):
        """Demo Strategy"""
        
        def __init__(self):
            print('Init ', self.__class__.__name__)
            print()
            
        def prenext(self):
            self.next()
        
        def next(self):
            
            print()
            print('---------------------------')
            print('Date: {}'.format(self.datetime.date()))
            print('---------------------------')
            
            for i, s in enumerate(self.datas):
                print('{} {} {}'.format(i + 1, s.datetime.date(), s._name))
    

    When I loop self.datas inside next I expected to have their datetime.date() in sync with self.datetime.date(), the “heartbeat”. I found this is not the case, here’s the log of the first next run:

    Init Demo

    Date: 2010-01-01

    1 2012-02-10 63DU GY Equity
    2 2010-01-01 ABI BB Equity
    3 2010-01-01 ACA FP Equity
    4 2010-01-01 AD NA Equity
    5 2010-01-01 ADS GY Equity
    6 2010-01-01 AGN NA Equity
    7 2010-01-01 AI FP Equity
    8 2010-01-01 AIR FP Equity
    9 2010-01-01 ALO FP Equity
    10 2010-01-01 ALV GY Equity
    11 2019-12-27 AMS SQ Equity
    12 2010-01-01 ASML NA Equity
    13 2010-01-01 BAS GY Equity
    14 2010-01-01 BAYN GY Equity
    15 2010-01-01 BBVA SQ Equity
    16 2010-01-01 BMW GY Equity
    17 2010-01-01 BN FP Equity
    18 2010-01-01 BNP FP Equity
    19 2010-01-01 CA FP Equity
    20 2010-01-01 CRH ID Equity
    21 2010-01-01 CS FP Equity
    22 2010-01-01 DAI GY Equity
    23 2010-01-01 DB1 GY Equity
    24 2010-01-01 DBK GY Equity
    25 2010-01-01 DG FP Equity
    26 2010-01-01 DPW GY Equity
    27 2010-01-01 DTE GY Equity
    28 2010-01-01 EL FP Equity
    29 2010-01-01 ENEL IM Equity
    30 2010-01-01 ENGI FP Equity
    31 2010-01-01 ENI IM Equity
    32 2010-01-01 EOAN GY Equity
    33 2010-01-01 FP FP Equity
    34 2010-01-01 FRE GY Equity
    35 2010-01-01 G IM Equity
    36 2010-01-01 GLE FP Equity
    37 2010-01-01 IBE SQ Equity
    38 2010-01-01 INGA NA Equity
    39 2010-01-01 ISP IM Equity
    40 2010-01-01 ITX SQ Equity
    41 2010-01-01 KER FP Equity
    42 2019-12-27 LIN GY Equity
    43 2018-10-26 LINU GY Equity
    44 2010-01-01 MC FP Equity
    45 2010-01-01 MT NA Equity
    46 2010-01-01 MUV2 GY Equity
    47 2010-01-01 NOKIA FH Equity
    48 2010-01-01 OR FP Equity
    49 2010-01-01 ORA FP Equity
    50 2010-01-01 PHIA NA Equity
    51 2010-01-01 REP SQ Equity
    52 2010-01-01 RWE GY Equity
    53 2010-01-01 SAF FP Equity
    54 2010-01-01 SAN FP Equity
    55 2010-01-01 SAN SQ Equity
    56 2010-01-01 SAP GY Equity
    57 2010-01-01 SGO FP Equity
    58 2010-01-01 SIE GY Equity
    59 2010-01-01 SU FP Equity
    60 2010-01-01 TEF SQ Equity
    61 2010-01-01 TIT IM Equity
    62 2010-01-01 UCG IM Equity
    63 2010-01-01 UL FP Equity
    64 2010-01-01 UL NA Equity
    65 2010-01-01 UNA NA Equity
    66 2010-01-01 URW NA Equity
    67 2010-01-01 VIV FP Equity
    68 2010-01-01 VOW GY Equity
    69 2010-01-01 VOW3 GY Equity

    If there was a simply buy signals on date 2010-01-01 (self.datetime.date()), the strategy would also buy 63DU GY Equity (2012-02-10 == start date), AMS SQ Equity (2019-12-27 == last date), LIN GY Equity ( 2019-12-27 == last date) and LINU GY Equity (2018-10-26 == last date).

    Am I wrong (very likely) coding like this to generate signals on multiple data feeds with different start date and end date?

    If I am not wrong, and this is by design, does it mean that before the logic to generate signals in the next method I should always check that s.datetime.date() == self.datetime.date() ??

    Thanks

    Nicola


Log in to reply
 

});