I backtest a strategy about futures,but I find there something strange about the profit

tianjixuetu

2018-11-28, Final Portfolio Value: 53245.46
2018-11-28, Final Portfolio cash: 50067.46
2018-11-29, Final Portfolio Value: 53155.46
2018-11-29, Final Portfolio cash: 49967.46
2018-11-30, BUY EXECUTED, Price: 3192.00, Cost: 3539.00, Comm 3.19
2018-11-30, TRADE PROFIT, GROSS 3470.00, NET 3463.27
2018-11-30, Final Portfolio Value: 53463.27
2018-11-30, Final Portfolio cash: 53463.27

Beause of this is future,so, the broker may be using market to market. After the short position opens on 2018-11-01,next days,there is no strange until 2018-11-29.

On the 2018-11-29,the strategy send a close order,so,the next open ,which is the open price in 2018-11-29.

date            open    close 
2018/11/28	3129	3178
2018/11/29	3179	3188
2018/11/30	3191	3136

I cove the short position in the next day on the open price and a shippage ,it is 3192.so 50000+(3539.00-3192.00)*10- 3.19-3.54=53463.27,there is no doubt. I read the sorce code,I know that when I close the position,I use the function above.

but, my key question is that,from 2018-11-29 to 2018-11-30,what happened?

on the 2018-11-29,the cash is 49967.46, the value is 53155.46. I use the close price 3188 to caculate my cash and value.

so, on 2018-11-29,the cash is 49967.46,the margin is 3188,the value is 49967.46+3188=53155.46. so,the next day,when I close the position,my margin becomes 3191, my cash becomes 49967.46-(3191-3188)*10-10-3.19=49924.27,so my value is 3191+49924.27=53155.27. so, I lost 348,why?

backtrader

Sorry, but there are two things I fail to understand.

Your message completely. I do understand that you are not a native english speaker. As such, let me suggest that you break your content (especially when considering calculations that have to do with different days) into bullet points. Reduce the language to math if possible.
What the actual problem may be ... if I assume I have understood what you say.

@tianjixuetu said in I backtest a strategy about futures,but I find there something strange about the profit:

I cove the short position in the next day on the open price and a shippage ,it is 3192.so 50000+(3539.00-3192.00)*10- 3.19-3.54=53463.27,there is no doubt.

So things are being calculated properly.

@tianjixuetu said in I backtest a strategy about futures,but I find there something strange about the profit:

so, on 2018-11-29,the cash is 49967.46,the margin is 3188,the value is 49967.46+3188=53155.46. so,the next day,when I close the position,my margin becomes 3191, my cash becomes 49967.46-(3191-3188)*10-10-3.19=49924.27,so my value is 3191+49924.27=53155.27. so, I lost 348,why?

No idea why you suddenly says 3191 when you have used slippage and the price is 3192 and no idea why you say that you have margin after you have closed a position. Because when a position is closed, there is no margin at all. The log shows that the final value is 53463.27 and not this last figure you are calculating, 53155.27

tianjixuetu

@backtrader well,let me give a more clear explanation.

First of all ,let us use mark-to-market System.

2018-11-29,there is no doubt,the value equals the cash and position value. the cash is 49967.46, the close price is 3188,which I used a ten leverage,and margin ratio is 0.1,and one point slippage .So,the position value is 3188,the total value is 53155.46.
2019-11-30,because I decide to close my short position,and I send a market order at 2018-11-29,so,it will exexute on the open price at 2019-11-30,the price is 3191,and I add a slippage,so,the exexuted price is 3192.There is no doubt. On the point,at the open moment,when I close the short position,you konw,my cash should become 49967.46-(3191-3188)*10-10-3.19= 49924.27,now the posiiton value become 3191,my total value is 49924.27+3191=53115.27. is there something wrong?

secondly,let us use the profit :

I open the short position at the price :3539,I close the short position at 3192,my commission is 3.54+3.19=6.73

so ,my total value is 50000+(3539-3192)*10-6.73= 53463.27, this is just my last total value. there is no doubt.

you know,53463.27-53115.27=348,just close the position,my profit add 348? why? is it strange?

ab_trader

I've ran a test on the futures with the simple commission scheme:

cerebro.broker.setcommission(commission=0.0, margin=2000.0, mult=1000.0)

All calculations at day of exit are done as expected:
End_value = Prev_day_value + Mult x (Exit_price - Prev_day_close) OR
End_value = Prev_day_cash + Margin + Mult x (Exit_price - Prev_day_close)

Here is the output:

Starting Portfolio Value: 10000.00
2019-05-02, 23:59:59.999989, count, open, close, value, cash
2019-04-04, 23:59:59.999989, 3, 62.54, 62.26, 10000.00, 10000.00
2019-04-04, 23:59:59.999989, buy signal
2019-04-05, 23:59:59.999989, BUY EXECUTED, price 62.11, value 2000.00
2019-04-05, 23:59:59.999989, 4, 62.11, 63.24, 11130.00, 9130.00
2019-04-08, 23:59:59.999989, 5, 63.52, 64.56, 12450.00, 10450.00
2019-04-09, 23:59:59.999989, 6, 64.61, 64.14, 12030.00, 10030.00
2019-04-10, 23:59:59.999989, 7, 64.36, 64.77, 12660.00, 10660.00
2019-04-11, 23:59:59.999989, 8, 64.54, 63.74, 11630.00, 9630.00
2019-04-12, 23:59:59.999989, 9, 63.91, 64.05, 11940.00, 9940.00
2019-04-15, 23:59:59.999989, 10, 63.97, 63.56, 11450.00, 9450.00
2019-04-16, 23:59:59.999989, 11, 63.65, 64.19, 12080.00, 10080.00
2019-04-17, 23:59:59.999989, 12, 64.45, 63.87, 11760.00, 9760.00
2019-04-18, 23:59:59.999989, 13, 63.80, 64.07, 11960.00, 9960.00
2019-04-22, 23:59:59.999989, 14, 64.12, 65.55, 13440.00, 11440.00
2019-04-23, 23:59:59.999989, 15, 65.80, 66.30, 14190.00, 12190.00
2019-04-23, 23:59:59.999989, sell signal
2019-04-24, 23:59:59.999989, SELL EXECUTED, 66.02, value 2000.00
2019-04-24, 23:59:59.999989, OPERATION PROFIT, GROSS 3910.00, NET 3910.00
2019-04-24, 23:59:59.999989, 16, 66.02, 65.89, 13910.00, 13910.00

Looks like you have custom commission scheme where you are getting that additional money. :)

tianjixuetu

thank you very much.
you are right, in your case,there is no doubt.
But I use extend the commission,using this code:

class ComminfoFuturesPercent(CommInfoBase):
    '''write by myself,using in the future backtest,it means we should give a percent comminfo to broker'''
    params = (
        ('stocklike', False),
        ('commtype', CommInfoBase.COMM_PERC),
        ('percabs', True)
    )

    def _getcommission(self, size, price, pseudoexec):
        return abs(size) * price * self.p.mult * self.p.commission

    def get_margin(self, price):
        return price * self.p.mult * self.p.margin

during my test,I use this code:

cerebro.broker.set_slippage_fixed(1, slip_open=True)
comm=ComminfoFuturesPercent(commission=0.0001,margin=0.1, mult=10)
cerebro.broker.addcommissioninfo(comm, name='xxxx')

there is something different between your case and my case. you let the margin equals 2000 forever,but in real world,
the margin may be change awlays the price volitility.

can you help me? could you test it using the ComminfoFuturesPercent class ?

thanks a lot!

tianjixuetu

@ab_trader thank you very much.
you are right, in your case,there is no doubt.
But I use extend the commission,using this code:

class ComminfoFuturesPercent(CommInfoBase):
    '''write by myself,using in the future backtest,it means we should give a percent comminfo to broker'''
    params = (
        ('stocklike', False),
        ('commtype', CommInfoBase.COMM_PERC),
        ('percabs', True)
    )

    def _getcommission(self, size, price, pseudoexec):
        return abs(size) * price * self.p.mult * self.p.commission

    def get_margin(self, price):
        return price * self.p.mult * self.p.margin

during my test,I use this code:

cerebro.broker.set_slippage_fixed(1, slip_open=True)
comm=ComminfoFuturesPercent(commission=0.0001,margin=0.1, mult=10)
cerebro.broker.addcommissioninfo(comm, name='xxxx')

there is something different between your case and my case. you let the margin equals 2000 forever,but in real world,
the margin may be change awlays the price volitility.

can you help me? could you test it using the ComminfoFuturesPercent class ?

thanks a lot!

backtrader

The 348 mentioned here because of the difference 3191 => 3539.

The accounting calculates the margin to retain when opening the position and has to give you the difference back when closing the position of what the actual margin would have been with the closing price.

I cannot yet understand where the problem is.

ab_trader

@tianjixuetu said in I backtest a strategy about futures,but I find there something strange about the profit:

there is something different between your case and my case. you let the margin equals 2000 forever,but in real world,
the margin may be change awlays the price volitility.

Real world is different, you may want to educate your self a little bit more on this.
Sometimes margin doesn't fluctuate with the daily prices, even it still depends on volatility.

Also an advice - post the script you use in the very first message, not in the very last. There are no mid readers here, so only you know what is ran. My simplified case shows that bt works as expected, therefore the reason of the discrepancy is in (a) your additions to the bt or (b) in your calculations posted here. Seems it is case (b).

tianjixuetu

@ab_trader thank you very much.

tianjixuetu

@backtrader I think,I know,why it is . I am certain that there is some bug to calculate everyday's the total value .

Starting Portfolio Value: 50000.00
2018-10-30, pre_close:3600.0,now_open:3602.0,now_close:3582.0,total_value:50000.0,cash:50000.0
2018-10-31, pre_close:3582.0,now_open:3581.0,now_close:3538.0,total_value:50000.0,cash:50000.0
2018-11-01, SELL EXECUTED, Price: 3539.00, Cost: 3539.00, Comm 3.54
2018-11-01, pre_close:3538.0,now_open:3540.0,now_close:3490.0,total_value:50437.461,cash:46947.461
2018-11-02, pre_close:3490.0,now_open:3496.0,now_close:3507.0,total_value:50284.461,cash:46777.461
2018-11-05, pre_close:3507.0,now_open:3506.0,now_close:3527.0,total_value:50104.461,cash:46577.461
2018-11-06, pre_close:3527.0,now_open:3532.0,now_close:3472.0,total_value:50599.461,cash:47127.461
2018-11-07, pre_close:3472.0,now_open:3475.0,now_close:3406.0,total_value:51193.461,cash:47787.461
2018-11-08, pre_close:3406.0,now_open:3406.0,now_close:3427.0,total_value:51004.461,cash:47577.461
2018-11-09, pre_close:3427.0,now_open:3427.0,now_close:3367.0,total_value:51544.461,cash:48177.461
2018-11-12, pre_close:3367.0,now_open:3365.0,now_close:3339.0,total_value:51796.461,cash:48457.461
2018-11-13, pre_close:3339.0,now_open:3341.0,now_close:3346.0,total_value:51733.461,cash:48387.461
2018-11-14, pre_close:3346.0,now_open:3345.0,now_close:3381.0,total_value:51418.461,cash:48037.461
2018-11-15, pre_close:3381.0,now_open:3394.0,now_close:3357.0,total_value:51634.461,cash:48277.461
2018-11-16, pre_close:3357.0,now_open:3357.0,now_close:3361.0,total_value:51598.461,cash:48237.461
2018-11-19, pre_close:3361.0,now_open:3355.0,now_close:3373.0,total_value:51490.461,cash:48117.461
2018-11-20, pre_close:3373.0,now_open:3373.0,now_close:3326.0,total_value:51913.461,cash:48587.461
2018-11-21, pre_close:3326.0,now_open:3325.0,now_close:3359.0,total_value:51616.461,cash:48257.461
2018-11-22, pre_close:3359.0,now_open:3362.0,now_close:3312.0,total_value:52039.461,cash:48727.461
2018-11-23, pre_close:3312.0,now_open:3312.0,now_close:3239.0,total_value:52696.461,cash:49457.461
2018-11-26, pre_close:3239.0,now_open:3230.0,now_close:3124.0,total_value:53731.461,cash:50607.461
2018-11-27, pre_close:3124.0,now_open:3130.0,now_close:3135.0,total_value:53632.461,cash:50497.461
2018-11-28, pre_close:3135.0,now_open:3129.0,now_close:3178.0,total_value:53245.461,cash:50067.461
2018-11-29, pre_close:3178.0,now_open:3179.0,now_close:3188.0,total_value:53155.461,cash:49967.461
2018-11-30, BUY EXECUTED, Price: 3192.00, Cost: 3539.00, Comm 3.19
2018-11-30, TRADE PROFIT, GROSS 3470.00, NET 3463.27
2018-11-30, pre_close:3188.0,now_open:3191.0,now_close:3136.0,total_value:53463.269,cash:53463.269
2018-12-03, pre_close:3136.0,now_open:3135.0,now_close:3196.0,total_value:53463.269,cash:53463.269

@ab_trader
according to ab_trader,and our future experience,we know,that,the second fuction is not correct.

End_value = Prev_day_value + Mult x (Exit_price - Prev_day_close) OR
End_value = Prev_day_cash + Margin + Mult x (Exit_price - Prev_day_close)

if you use the new function,the two ways to calculate the value will be same.

End_value = Prev_day_cash + Margin + Mult x (Exit_price - Prev_day_close)+ changes_in_margin.

according to our experience,our cash will be the cash in yesterday,my margin used,and my margin changes.

so, is my opioion right?

tianjixuetu

@tianjixuetu according to our experience,our cash will be the cash in yesterday,the profit ,and my margin changes.

backtrader

@tianjixuetu said in I backtest a strategy about futures,but I find there something strange about the profit:

so, is my opioion right?

Not it is not. As pointed out above:

@backtrader said in I backtest a strategy about futures,but I find there something strange about the profit:

The accounting calculates the margin to retain when opening the position and has to give you the difference back when closing the position of what the actual margin would have been with the closing price.

If you don't like how it works, fine and good.

tianjixuetu

@backtrader ok,thank you very much.I think ,I need to read the source code again and again .