Order Synchronization

T H

Hi. There.
Sometimes the state of the order object and the state of the actual order don't match, is there any way to synchronize this?
Thanks!

vladisld

Could you provide more details:

what live broker is used ( assuming you are talking about live trading )
API logs if available

Backtrader updates its order objects on notifications from the broker API. So the best way to detect such discrepancies is to analyze the API logs.

T H

@vladisld Thank you for your reply.
The Live Broker I use is OANDA.
How can I get API logs?
It only occurs occasionally when there is a big price movement.

T H

And once it goes out of alignment, it will never go back to the way it was...

vladisld

@T-H said in Order Synchronization:

How can I get API logs?

Not using Oanda myself unfortunately. Hope other forum members may help here.

T H

@vladisld It's fine! Thank you for your concern.

@T-H could you provide more details on how you use oanda. also do you open positions on the same account outside of backtrader?

T H

@dasch said in Order Synchronization:

could you provide more details on how you use oanda.

Set the sell_bracket at 5 pips intervals. (tp:-5pips, s/l:+1000pips)
Holds the return value of the order in the List.
The state of the order is monitored in next().

When I check the order that the status of main order is "Accepted" in "OANDA Web Trading Platform", it is already executed.
Moreover, when I close all orders with self.close(), it doesn't close just that one order.

Am I making sense?

also do you open positions on the same account outside of backtrader?

Never.

When initializing the oanda store, set the param notif_transactions=True to get all transactions that were received and post the problematic order / transactions either here or open an issue with that on github.

T H

@dasch Thank you for your advice.
I already set notif_transactions=True, but I don’t understand how to use it.
Please tell me how to get those transactions.

the store will recieve a notification for every transaction. They are available in notify_store(...) in your strategy.

for example this way:

    def notify_store(self, msg, *args, **kwargs):
        print(msg)

T H

@dasch I didn't know that!!!!
I’m so happy! Thank you so so much!
I'll try it, after that I’ll wait for the problem to be reproduced.

T H

Finally, I succeeded in getting the api transactions.
The order I made (tradeid:54) should originally be in Accepted status after filled, but that status is still "Submitted".
In fact, When I check the order (tradeid:54) in "OANDA Web Trading Platform” , the order has been successfully filled.

NEW BRACKET ORDER[tradeid:54]
( type:Sell, size:300, price:137.92799999999986, exectype:2(Limit), tradeid:54, stopprice:147.92799999999986, limitprice:137.87799999999984 )

2020-09-10 14:03:07,051 [    INFO] *****, STORE NOTIF:, {'id': '368', 'time': '1599714154.801444989', 'userID': 10733067, 'accountID': '101-009-10733067-005', 'batchID': '368', 'requestID': '24724085988296074', 'type': 'LIMIT_ORDER', 'instrument': 'GBP_JPY', 'units': '-300.0', 'price': 137.928, 'timeInForce': 'GTC', 'positionFill': 'DEFAULT', 'triggerCondition': 'DEFAULT', 'reason': 'CLIENT_ORDER', 'clientExtensions': {'id': '1599667859.922813-160'}, 'takeProfitOnFill': {'price': 137.878, 'timeInForce': 'GTC', 'clientExtensions': {'id': '1599667859.922813-162'}}, 'stopLossOnFill': {'price': 147.928, 'timeInForce': 'GTC', 'clientExtensions': {'id': '1599667859.922813-161'}}}
    :
    :
2020-09-10 14:18:46,834 [    INFO] *****, STORE NOTIF:, {'id': '369', 'time': '1599715094.455659870', 'userID': 10733067, 'accountID': '101-009-10733067-005', 'batchID': '369', 'requestID': '42738488438139833', 'type': 'ORDER_CANCEL', 'orderID': '368', 'clientOrderID': '1599667859.922813-160', 'reason': 'CLIENT_REQUEST_REPLACED', 'replacedByOrderID': '370'}
2020-09-10 14:18:46,836 [    INFO] *****, STORE NOTIF:, {'id': '370', 'time': '1599715094.455659870', 'userID': 10733067, 'accountID': '101-009-10733067-005', 'batchID': '369', 'requestID': '42738488438139833', 'type': 'LIMIT_ORDER', 'instrument': 'GBP_JPY', 'units': '-300.0', 'price': 137.928, 'timeInForce': 'GTC', 'positionFill': 'DEFAULT', 'triggerCondition': 'DEFAULT', 'reason': 'REPLACEMENT', 'clientExtensions': {'id': '1599667859.922813-160'}, 'takeProfitOnFill': {'price': 137.876, 'timeInForce': 'GTC'}, 'stopLossOnFill': {'price': 147.928, 'timeInForce': 'GTC'}, 'replacesOrderID': '368'}

ORDER STATUS[tradeid:54] (2020-09-10 14:03:06 - 2020-09-10 14:21:51)
Main Order: Accepted
TP Order: Submitted
SL Order: Submitted
MAIN ORDER FILLED[tradeid:54] and NEW BRACKET ORDER[tradeid:55]
(type:Sell, size:300, price:137.97799999999987, exectype:2(Limit), tradeid:55, stopprice:147.97799999999987, limitprice:137.92799999999986)

2020-09-10 14:21:51,437 [    INFO] *****, STORE NOTIF:, {'id': '371', 'time': '1599715278.907071425', 'userID': 12345678, 'accountID': '101-009-12345678-005', 'batchID': '371', 'type': 'ORDER_FILL', 'orderID': '370', 'clientOrderID': '1599667859.922813-160', 'instrument': 'GBP_JPY', 'units': '-300.0', 'gainQuoteHomeConversionFactor': '1.0', 'lossQuoteHomeConversionFactor': '1.0', 'price': 137.928, 'fullVWAP': 137.928, 'fullPrice': {'type': 'PRICE', 'bids': [{'price': 137.928, 'liquidity': '250000'}], 'asks': [{'price': 137.942, 'liquidity': '250000'}], 'closeoutBid': 137.921, 'closeoutAsk': 137.948}, 'reason': 'LIMIT_ORDER', 'pl': '0.0', 'financing': '0.0', 'commission': '0.0', 'guaranteedExecutionFee': '0.0', 'accountBalance': '2999633.0916', 'tradeOpened': {'tradeID': '371', 'units': '-300.0', 'price': 137.928, 'guaranteedExecutionFee': '0.0', 'halfSpreadCost': '2.1', 'initialMarginRequired': '2069.01'}, 'halfSpreadCost': '2.1'}
2020-09-10 14:21:51,437 [    INFO] *****, STORE NOTIF:, {'id': '372', 'time': '1599715278.907071425', 'userID': 12345678, 'accountID': '101-009-12345678-005', 'batchID': '371', 'type': 'TAKE_PROFIT_ORDER', 'tradeID': '371', 'price': 137.876, 'timeInForce': 'GTC', 'triggerCondition': 'DEFAULT', 'reason': 'ON_FILL'}
2020-09-10 14:21:51,439 [    INFO] *****, STORE NOTIF:, Received external transaction TAKE_PROFIT_ORDER with id 372. Skipping transaction.
2020-09-10 14:21:51,439 [    INFO] *****, STORE NOTIF:, {'id': '373', 'time': '1599715278.907071425', 'userID': 12345678, 'accountID': '101-009-12345678-005', 'batchID': '371', 'type': 'STOP_LOSS_ORDER', 'tradeID': '371', 'price': 147.928, 'timeInForce': 'GTC', 'triggerCondition': 'DEFAULT', 'reason': 'ON_FILL'}
2020-09-10 14:21:51,439 [    INFO] *****, STORE NOTIF:, Received external transaction STOP_LOSS_ORDER with id 373. Skipping transaction.
2020-09-10 14:22:00,987 [    INFO] *****, STORE NOTIF:, v20 REST request to https://stream-fxpractice.oanda.com:443/v3/accounts/101-009-12345678-005/transactions/stream has timed out (stream)
2020-09-10 14:22:02,451 [    INFO] *****, STORE NOTIF:, v20 REST request to https://stream-fxpractice.oanda.com:443/v3/accounts/101-009-12345678-005/pricing/stream?instruments=GBP_JPY has timed out (stream)
    :
2020-09-10 14:22:06,300 [    INFO] *****, STORE NOTIF:, Trying to reconnect streaming events (1 of -1)

ORDER STATUS[tradeid:54] (2020-09-10 14:22:03 - )
Main Order: Completed
TP Order: Submitted <-- Why is it "Submitted" and not "Accepted"?
SL Order: Submitted <-- Why is it "Submitted" and not "Accepted"?

T H

By the way, the order (tradid:55) has already been successfully filled. (according to "OANDA Web Trading Platform")
But this orders status also as follows.

Main Order: Accepted <-- Why is it "Accepted" and not "Completed"?
TP Order: Submitted
SL Order: Submitted

(I'm not very good at English, so I don't know what verbs to use.(Filled? Opened?, Executed?) I'm sorry about that...)

could you also provide the code which creates the orders?

I see the issue there, but don't really know what is wrong there.

T H

@dasch That's hard…
But, I can provide only my order method.
This is in next().
This is called after the status of the orders checked.
Nothing in particular. Maybe...

def create_new_order(self, price, exec_type):
    stopprice = price - (self.order_type_sign * self.max_range_price)
    limitprice = price + (self.order_type_sign * self.interval_price)

    self.tradeid_seq +=1
    
    orders = None
    if self.order_type == bt.Order.Buy:
        orders = self.buy_bracket(size=self.order_size, price=price, exectype=exec_type, 
                            tradeid=self.tradeid_seq, 
                            stopprice=stopprice, stopexec=bt.Order.Stop, 
                            limitprice=limitprice, limitexec=bt.Order.Limit)
    elif self.order_type == bt.Order.Sell:
        orders = self.sell_bracket(size=self.order_size, price=price, exectype=exec_type, 
                            tradeid=self.tradeid_seq, 
                            stopprice=stopprice, stopexec=bt.Order.Stop, 
                            limitprice=limitprice, limitexec=bt.Order.Limit)
    
    logger.info("$$$ ORDER $$$ type:{}, size:{}, price:{}, exectype:{}({}), tradeid:{}, stopprice:{}, limitprice:{}".format(
        bt.Order.OrdTypes[self.order_type], self.order_size, price, exec_type, bt.Order.ExecTypes[exec_type], self.tradeid_seq, stopprice, limitprice)
    )
    return orders

the issue is with the tradeid, btoandav20 fails when the order gets replaced. I am not really sure, when i will be able to fix that in btoandav20 but as a workaround, cancel your previous order and create a new order without tradeid.

to explain it more, btoandav20 will not recognize the child order, since they are assigned to the canceled order. the new order will not trigger the child orders.

T H

OMG!! Thank you for your information.
Anyway, I'll try it.