Backtrader Community
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

    Kalman et al.

    Blog
    4
    18
    5520
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • B
      backtrader administrators last edited by

      Use the link below to go the original post

      Click here to see the full blog post

      1 Reply Last reply Reply Quote 1
      • J
        junajo10 last edited by

        self.P = np.zeros((2, 2)) should be self.C?

        B 1 Reply Last reply Reply Quote 0
        • B
          backtrader administrators last edited by

          May well be. Will be checked.

          1 Reply Last reply Reply Quote 0
          • J
            junajo10 last edited by

            Also, kf = KalmanFilterInd(self.data0, self.data1)

            1 Reply Last reply Reply Quote 0
            • B
              backtrader administrators last edited by

              That was a just a test to see how the prediction to itself would look like.

              If you have a look at KalmanSignals you will see:

              KalmanFilterInd()

              with even no mention of the data feeds. This passes whatever data feeds are in the parent down to KalmanFilterInd and this is equal to KalmanFilterInd(self.data0, self.data1)

              But thank you for spotting it, it is for sure worth some rewriting to make all options more comprehensible.

              1 Reply Last reply Reply Quote 0
              • R
                remroc last edited by

                WOW I was just absent 2 weeks and that BOOM. Thanks DRo and Junajo10. U rock guys !!

                1 Reply Last reply Reply Quote 0
                • B
                  backtrader administrators @junajo10 last edited by

                  @junajo10 said in Kalman et al.:

                  self.P = np.zeros((2, 2)) should be self.C?

                  The original code has indeed self.P = np.zeros((2, 2)). Even if actually meant self.C = np.zeros((2, 2)) it doesn't play any role.

                  The 1st value of self.R is None, which implies that self.R will be set to np.zeros((2, 2)) and self.C will be calculated later for the 1st time (in the 2nd iteration, self.C will already have a value when self.R is no longer None)

                  1 Reply Last reply Reply Quote 0
                  • J
                    junajo10 last edited by backtrader

                    I don´t know the reason but the values of et and sqrt_qt are very different. I am testing the strategy using intraday data for one day.

                    first file

                    datetime,Open,High,Low,Last,volume
2017-02-14 09:14:00,10280.0,10280.0,10280.0,10280.0,814.0
2017-02-14 09:14:10,10280.0,10280.0,10280.0,10280.0,814.0
2017-02-14 09:14:20,10280.0,10280.0,10280.0,10280.0,814.0
2017-02-14 09:14:30,10280.0,10280.0,10280.0,10280.0,814.0
2017-02-14 09:14:40,10280.0,10280.0,10280.0,10280.0,814.0
2017-02-14 09:14:50,10280.0,10280.0,10280.0,10280.0,814.0
2017-02-14 09:15:00,10280.0,10280.0,10270.0,10275.0,154.0
2017-02-14 09:15:10,10271.0,10274.0,10270.0,10272.0,151.0
2017-02-14 09:15:20,10271.0,10275.0,10245.0,10271.0,233.0
2017-02-14 09:15:30,10273.0,10274.0,10250.0,10272.0,175.0
2017-02-14 09:15:40,10272.0,10272.0,10268.0,10269.0,89.0
2017-02-14 09:15:50,10269.0,10272.0,10268.0,10271.0,30.0
2017-02-14 09:16:00,10271.0,10274.0,10270.0,10273.0,58.0
2017-02-14 09:16:10,10273.0,10277.0,10273.0,10277.0,87.0
2017-02-14 09:16:20,10277.0,10277.0,10275.0,10275.0,53.0
2017-02-14 09:16:30,10275.0,10278.0,10275.0,10278.0,104.0
2017-02-14 09:16:40,10278.0,10278.0,10276.0,10276.0,39.0
2017-02-14 09:16:50,10276.0,10276.0,10274.0,10275.0,55.0
2017-02-14 09:17:00,10276.0,10276.0,10269.0,10270.0,68.0
2017-02-14 09:17:10,10270.0,10271.0,10270.0,10270.0,31.0
2017-02-14 09:17:20,10270.0,10271.0,10269.0,10271.0,26.0
2017-02-14 09:17:30,10272.0,10273.0,10271.0,10271.0,14.0
2017-02-14 09:17:40,10271.0,10271.0,10270.0,10270.0,5.0
2017-02-14 09:17:50,10271.0,10271.0,10271.0,10271.0,13.0
2017-02-14 09:18:00,10270.0,10270.0,10268.0,10269.0,29.0
2017-02-14 09:18:10,10269.0,10270.0,10268.0,10270.0,15.0
2017-02-14 09:18:20,10271.0,10271.0,10269.0,10269.0,22.0
2017-02-14 09:18:30,10268.0,10268.0,10263.0,10264.0,55.0
2017-02-14 09:18:40,10265.0,10265.0,10261.0,10265.0,77.0
2017-02-14 09:18:50,10265.0,10266.0,10263.0,10264.0,21.0
2017-02-14 09:19:00,10263.0,10264.0,10262.0,10263.0,75.0
2017-02-14 09:19:10,10265.0,10265.0,10263.0,10265.0,79.0
2017-02-14 09:19:20,10265.0,10265.0,10263.0,10264.0,27.0
2017-02-14 09:19:30,10263.0,10264.0,10262.0,10263.0,37.0
2017-02-14 09:19:40,10264.0,10264.0,10262.0,10264.0,22.0
2017-02-14 09:19:50,10263.0,10264.0,10262.0,10262.0,38.0
2017-02-14 09:20:00,10263.0,10264.0,10260.0,10260.0,60.0
2017-02-14 09:20:10,10260.0,10260.0,10258.0,10259.0,51.0
2017-02-14 09:20:20,10260.0,10260.0,10257.0,10260.0,85.0

                    second file

                    datetime,Open,High,Low,Last,volume
2017-02-14 09:14:00,23650.0,23650.0,23650.0,23650.0,612.0
2017-02-14 09:14:10,23650.0,23650.0,23650.0,23650.0,612.0
2017-02-14 09:14:20,23650.0,23650.0,23650.0,23650.0,612.0
2017-02-14 09:14:30,23650.0,23650.0,23650.0,23650.0,612.0
2017-02-14 09:14:40,23650.0,23650.0,23650.0,23650.0,612.0
2017-02-14 09:14:50,23650.0,23650.0,23650.0,23650.0,612.0
2017-02-14 09:15:00,23651.0,23660.0,23615.0,23639.0,304.0
2017-02-14 09:15:10,23640.0,23641.0,23631.0,23634.0,163.0
2017-02-14 09:15:20,23636.0,23642.0,23630.0,23639.0,131.0
2017-02-14 09:15:30,23641.0,23642.0,23637.0,23642.0,56.0
2017-02-14 09:15:40,23640.0,23642.0,23635.0,23635.0,53.0
2017-02-14 09:15:50,23638.0,23638.0,23633.0,23636.0,50.0
2017-02-14 09:16:00,23637.0,23647.0,23636.0,23645.0,95.0
2017-02-14 09:16:10,23645.0,23655.0,23642.0,23648.0,162.0
2017-02-14 09:16:20,23648.0,23654.0,23645.0,23651.0,59.0
2017-02-14 09:16:30,23652.0,23654.0,23650.0,23650.0,35.0
2017-02-14 09:16:40,23650.0,23654.0,23650.0,23650.0,45.0
2017-02-14 09:16:50,23652.0,23652.0,23646.0,23648.0,50.0
2017-02-14 09:17:00,23648.0,23651.0,23640.0,23640.0,87.0
2017-02-14 09:17:10,23642.0,23645.0,23642.0,23643.0,21.0
2017-02-14 09:17:20,23643.0,23646.0,23641.0,23646.0,33.0
2017-02-14 09:17:30,23644.0,23646.0,23642.0,23642.0,34.0
2017-02-14 09:17:40,23643.0,23645.0,23643.0,23645.0,18.0
2017-02-14 09:17:50,23646.0,23650.0,23646.0,23650.0,27.0
2017-02-14 09:18:00,23649.0,23649.0,23642.0,23644.0,52.0
2017-02-14 09:18:10,23645.0,23648.0,23645.0,23648.0,19.0
2017-02-14 09:18:20,23647.0,23648.0,23646.0,23648.0,24.0
2017-02-14 09:18:30,23646.0,23646.0,23638.0,23640.0,49.0
2017-02-14 09:18:40,23640.0,23642.0,23637.0,23640.0,49.0
2017-02-14 09:18:50,23640.0,23641.0,23639.0,23639.0,26.0
2017-02-14 09:19:00,23639.0,23642.0,23638.0,23642.0,22.0
2017-02-14 09:19:10,23642.0,23647.0,23641.0,23644.0,37.0
2017-02-14 09:19:20,23643.0,23644.0,23641.0,23644.0,21.0
2017-02-14 09:19:30,23644.0,23644.0,23640.0,23641.0,14.0
2017-02-14 09:19:40,23641.0,23641.0,23638.0,23640.0,34.0
2017-02-14 09:19:50,23641.0,23641.0,23639.0,23640.0,18.0
2017-02-14 09:20:00,23638.0,23641.0,23635.0,23635.0,68.0
2017-02-14 09:20:10,23636.0,23636.0,23632.0,23633.0,66.0
2017-02-14 09:20:20,23633.0,23636.0,23627.0,23636.0,158.0

                    My values: As you can see my errors are between 0 and 1 and my sqrt_qt is near to 74, i think these values can not be good

                    date;price1;price2;et;sqrt_qt;beta
2017-02-14T09:14:10;23650.0000;10280.0000;10280.0000;23651.0000;0.0000
2017-02-14T09:14:20;23650.0000;10280.0000;0.0000;74.7883;0.4347
2017-02-14T09:14:30;23650.0000;10280.0000;0.0000;74.7883;0.4347
2017-02-14T09:14:40;23650.0000;10280.0000;0.0000;74.7883;0.4347
2017-02-14T09:14:50;23650.0000;10280.0000;0.0000;74.7883;0.4347
2017-02-14T09:15:00;23639.0000;10275.0000;-0.2188;74.7535;0.4347
2017-02-14T09:15:10;23634.0000;10272.0000;-0.8268;74.7377;0.4346
2017-02-14T09:15:20;23639.0000;10271.0000;-3.1730;74.7535;0.4346
2017-02-14T09:15:30;23642.0000;10272.0000;-0.3034;74.7630;0.4345
2017-02-14T09:15:40;23635.0000;10269.0000;0.0412;74.7409;0.4345
2017-02-14T09:15:50;23636.0000;10271.0000;1.5655;74.7440;0.4345
2017-02-14T09:16:00;23645.0000;10273.0000;-1.9108;74.7725;0.4345
2017-02-14T09:16:10;23648.0000;10277.0000;2.6966;74.7820;0.4344
2017-02-14T09:16:20;23651.0000;10275.0000;-3.3037;74.7914;0.4346
2017-02-14T09:16:30;23650.0000;10278.0000;3.4344;74.7883;0.4344
2017-02-14T09:16:40;23650.0000;10276.0000;-2.0000;74.7883;0.4346
2017-02-14T09:16:50;23648.0000;10275.0000;-0.1310;74.7820;0.4345
2017-02-14T09:17:00;23640.0000;10270.0000;-1.5242;74.7567;0.4345
2017-02-14T09:17:10;23643.0000;10270.0000;-1.3032;74.7662;0.4344
2017-02-14T09:17:20;23646.0000;10271.0000;-0.3031;74.7756;0.4344
2017-02-14T09:17:30;23642.0000;10271.0000;1.7374;74.7630;0.4343
2017-02-14T09:17:40;23645.0000;10270.0000;-2.3033;74.7725;0.4344
2017-02-14T09:17:50;23650.0000;10271.0000;-1.1716;74.7883;0.4343
2017-02-14T09:18:00;23644.0000;10269.0000;0.6056;74.7693;0.4343
2017-02-14T09:18:10;23648.0000;10270.0000;-0.7372;74.7820;0.4343
2017-02-14T09:18:20;23648.0000;10269.0000;-1.0000;74.7820;0.4343
2017-02-14T09:18:30;23640.0000;10264.0000;-1.5262;74.7567;0.4342
2017-02-14T09:18:40;23640.0000;10265.0000;1.0000;74.7567;0.4342
2017-02-14T09:18:50;23639.0000;10264.0000;-0.5658;74.7535;0.4342
2017-02-14T09:19:00;23642.0000;10263.0000;-2.3025;74.7630;0.4342
2017-02-14T09:19:10;23644.0000;10265.0000;1.1318;74.7693;0.4341
2017-02-14T09:19:20;23644.0000;10264.0000;-1.0000;74.7693;0.4341
2017-02-14T09:19:30;23641.0000;10263.0000;0.3023;74.7598;0.4341
2017-02-14T09:19:40;23640.0000;10264.0000;1.4341;74.7567;0.4341
2017-02-14T09:19:50;23640.0000;10262.0000;-2.0000;74.7567;0.4342
2017-02-14T09:20:00;23635.0000;10260.0000;0.1704;74.7409;0.4341
2017-02-14T09:20:10;23633.0000;10259.0000;-0.1318;74.7345;0.4341
2017-02-14T09:20:20;23636.0000;10260.0000;-0.3022;74.7440;0.4341
                    1 Reply Last reply Reply Quote 0
                    • J
                      junajo10 last edited by backtrader

                      I think the problem is here:

                          # Q_t is the variance of the prediction of observations and hence
    # \sqrt{Q_t} is the standard deviation of the predictions
    Qt = F.dot(self.R).dot(F.T) + self.p.vt

                      The post said:"variance of the last observations", i think we have to take n qt , calculate variance and then the sqrt . What do you think?

                      1 Reply Last reply Reply Quote 0
                      • B
                        backtrader administrators last edited by

                        Will recheck the post and the ported code on that area. Thanks

                        1 Reply Last reply Reply Quote 0
                        • J
                          junajo10 last edited by

                          I think the post is wrong

                          1 Reply Last reply Reply Quote 0
                          • B
                            backtrader administrators last edited by

                            The original post in Quantopian by Aidan O'Mahony shows the same calculation.

                            Reference: Quantopian - Ernie Chan's EWA/EWC pair trade with Kalman filter

                            ...
        Q = x.dot(context.R).dot(x.T) + context.Ve
        sqrt_Q = np.sqrt(Q)
...
                            1 Reply Last reply Reply Quote 0
                            • J
                              junajo10 last edited by

                              https://github.com/lukstei/trading-backtest/blob/master/src/main/java/org/lst/trading/main/strategy/kalman/CointegrationTradingStrategy.java

                              1 Reply Last reply Reply Quote 0
                              • J
                                junajo10 last edited by junajo10

                                I think i have got good values, I have changed the init values:
                                ('delta',1e-7 ),
                                ('vt',1e-3),

                                1 Reply Last reply Reply Quote 0
                                • RandyT
                                  RandyT last edited by

                                  @junajo10 thought I would join the conversation here as well.

                                  Noticed that the Java system you reference below, he is using 1e-10 for delta and 1e-7 for the vt parameter.

                                  Any other thoughts there?

                                  1 Reply Last reply Reply Quote 0
                                  • RandyT
                                    RandyT last edited by

                                    What is needed to change the scale of the KalmanFilterInd? I cannot seem to find the plotinfo magic to scale this up a bit.

                                    B 1 Reply Last reply Reply Quote 0
                                    • B
                                      backtrader administrators @RandyT last edited by

                                      @RandyT There is no magic. The problem is that the initial values of the filter are orders of magnitude greater than the ones when the filter has settled.

                                      1 Reply Last reply Reply Quote -1
                                      • RandyT
                                        RandyT last edited by

                                        I see your use of get_ylim() and set_ylim() but could not find a parameter that would take advantage of this. Seems you use it for volume for this exact purpose.

                                        1 Reply Last reply Reply Quote 0
                                        • 1 / 1
                                        • First post
                                          Last post
                                        Copyright © 2016, 2017, 2018, 2019, 2020, 2021 NodeBB Forums | Contributors