Anyone using Backtrader for live trading
-
Hi All,
Given that backtrader has live trading capability, wander if anyone has/is used/using it for live trading (more so with Interactive Brokers)... -
Mostly paper trading for the moment... As I am still learning the fwk...
-
During development, real orders were issued from backtrader to Interactive Brokers TWS and a maximum of 2 simultaneous data feeds were put in play.
As with anything which you may use to trade money with, you have to be aware of:
-
No matter how much effort is put into keeping backtrader bug free: bugs may be there.
-
Your own code may contain bugs (it doesn't have to throw an exception, it may simply not do what you are expecting it to do)
-
Connectivity issues
- From your internet connection
- From IB
-
TWS will exit every 24 hours. You would have to use one of the 3rd party solutions which are able to restart the platform. backtrader will try to backfill to recover (in most cases the re-start should happen outside of market trading hours, unless we consider the 24x7 nature of Forex)
The best approach would be to set up a test system with a paper trading account (they are associated to your main account to benefit from the same data permissions, but this apparently requires running from the same machine) and test long your scenarios before trying to do it live
-
-
Thanks @backtrader & @remroc,
I have been playing with backtrader & ibtest.py recently and found some below potential issues/enhancements-
CFD's are not supported in backtrader fully, for getting market data / historical data they behave like cash, so need to modify the ibstore.py to include CFD like CASH as given below
''' if contract.m_secType == 'CASH': '''
if contract.m_secType == 'CASH' or contract.m_secType == 'CFD':
@backtrader - could you please consider including above in next release. -
While strategies can be triggered preferably based on real time bars (it could be sourced as rtbar or made up from the market data), while putting orders it is essential to know the bid and ask price as well before putting live order. I couldn't find any way to get both of them in the call to next(), any ideas on
a) how to get the mix of rtbar and market data.
b) within market data should get all (bid, ask and trade (if IB is giving based on security type)), c) how to get all of this in next() and be able to distinguish the fields.
-
-
Do you have a sample of the potential full notation needed for a
CFD
?. For example:- Multipliers?
- Expiration/Strike Dates?
- Long/Short?
Actually none of the above may apply at all. This is a sample from another platform:
IBUS500-SMART-CFD-USD
. This seems a lot more like if aCFD
is treated like a common stock and not as aCASH
product as you mentioned above.
Another example:
IBDE30-CFD-SMART-EUR-BAAVG
Where it is fully unknown what
BAAVG
means and where it is supposed to fit.Bid/Ask Data
The platform delivers
OHLC
bars and bid/ask data is not considered (except forCASH
products which don't offer anything else and theBID
price is followed according to industry practices).What
IB
callsReal Time Bars
is a 5-second snapshot of the market activity and with it comes not bid/ask data.Keeping a record of the
bid/ask
prices would be possible, but it will have to be implemented. -
IB CFD's:
2 types, STOCK CFD - price is that of underlying stock, and INDEX CFD - price is derived by that of underlying index's immediate future contract.Multipliers - not applicable.
Expiration/Strike Dates - not applicable,
Long/Short - both are possible.IB don't give data for stock CFD, we need to source it for underlying stock.
Index CFD's which is relevant following works
IBDE30-CFD-SMART
IBUS500-CFD-SMART -
Trying out on paper trading, found that while it recovers the open positions at start of program, it does not recover the open orders.
-
Because the position has a general meaning for an asset and the orders may have been issued by a different client (manually over
TWS
for example)Your algorithm may be ready to go short if you have a long position, but an open order may be a stop-loss which you have issued as a safety belt for failures in the algorithm (or the platform sustaining it)
It may be a good use case to selectively enable order download.
-
Long/Short
was a badly formulated question in the sense that it tried to ask if, like with Options, you can have a different end symbol if you go for aPUT
or aCALL
.It seems like the
data-source
vstrading-asset
paradigm which can be used in theVisualChart
data feed is applicable to this case too. -
Let me reword it.
Using backtrader for IB paper trading (with simple strategy to buy / sell on particular conditions). This is a failover scenario.-
A limit order is sent to exchange and accepted and got FILLED from this client. For some reason the program crashed, on restart backtrader download the position and I know I got a open position while making decisions - this is fine.
-
A limit order is sent to exchange and accepted (but NOT filled) from this client. For some reason the program crashed, on restart it has no clue of the existing open order at exchange as it does not download the orders (like what it does for positions).
There is an api reqOpenOrder which return open orders for that client, can we please look to add this to TODO list.
-
-
I'm currently testing Backtrader as the platform for our day-trading startup. I have a difficult time to connect to 100+ symbols realtime. The client gets connected and disconnected many times before all the symbols are added. Is there anybody with experience with that many symbols in backtrader?
-
@rastegarr For sure not many people try to use those many symbols regardless of the platform.
One thing to remember is that
TWS
limits the amount of historical requests that can be issued in a certain amount of time.From IB - Historical Data Limitations
Pacing Violations
All of the API technologies support historical data requests. However, requesting the same historical data in a short period of time can cause extra load on the backend and subsequently cause pacing violations. The error code and message that indicates a pacing violation is:
162 - Historical Market Data Service error message: Historical data request pacing violation
The following conditions can cause a pacing violation:
- Making identical historical data requests within 15 seconds;
- Making six or more historical data requests for the same Contract, Exchange and Tick Type within two seconds.
- Also, observe the following limitation when requesting historical data:
Do not make more than 60 historical data requests in any ten-minute period.
If the whatToShow parameter in reqHistoricalData() is set to BID_ASK, then this counts as two requests and we will call BID and ASK historical data separately.Unless you manually disable
backfill_start
(initial backfilling) andbackfill
(after each diconnection) by setting both toFalse
during data feed creation you are continously going over this limit:Do not make more than 60 historical data requests in any ten-minute period.
At least over that one. Some others could kick in. That limit will apply to any platform connecting to
TWS
orIBGateway
.Of course by disabling backfilling, the indicators will take longer to produce usable values.
-
@rastegarr
I am also trying to change code for multiple symbols however got issues in handling strategies next() call. Its call for each bar and couldn't identify for which of data0/1/2... it is called so that rest could be ignored in that call. This is causing multiple signals on same products.
Could you please share something on handling multiple symbols end to end if your have faced similar issue and workaround... -
Check out this link - https://www.backtrader.com/blog/posts/2015-09-03-multidata-strategy/multidata-strategy.html. Maybe it will help.
To issue the order for particular dataN the BUY order code is
self.buy(data=dataN
). -
@skytrading54
next
is not called for individual data feeds.It is called when the next unit of time of any of the data feeds has elapsed.
Scenarios
-
A single data feed:
Straightforward.
next
is called for each bar of the data feed -
B multiple data feeds aligned in time
Straightfoward too. The same principle as above
-
C multiple data feeds which are not aligned in time
One of the data feeds may advance in time whilst the other(s) may remain static. This can happen, for example, if assets from different exchanges are in the system and the exchanges have different trading calendars. An example
SPY
andDAX
. Bank holidays in the USA and Germany are not aligned (some are but most aren't).When the
SPY
is not trading because it's Thanksgiving Day, theDAX
is happily trading. In this casenext
will be called and the following will be true:- The
SPY
data feed will not increase its length and checking the values at[0]
will return the last known values - The
DAX
data feed will increase its length and the values at[0]
will be the current ones - The Strategy will increase its length because one of the data feeds has moved forward (and value, cash and other things may have changed)
You may check this which shows an extreme example in which several trading days were removed from the source:
- The
Scenario C is the one applicable to multiple live data feeds, because some may not deliver data for a given period, whilst others will do.
The smaller the period, the higher the chances that some data feeds will not have data for a given period, whilst others (those with more liquidity) will keep pushing the clock forward.
To understand which data feeds have moved forward use:
len(dataX)
and see if it has increased with regards to the last time (you may also checkdataX.datetime[0]
ordataX.datetime.datetime()
, but this will change in areplay
scenario, with the length remaining constant, hence the recommendation to uselen
) -
-
Initial support for
CFD
products has been added.The default request for any
CFD
security type will be forBID
instead ofTRADES
-
Thanks Backtrader,
I have taken this an ran the newer version but seems its not processing correctly, some obervations
python ibtest_12Jan17.py --port 8099 --clientId 2 --data0 IBUS500-CFD-SMART --timeframe Seconds --compression 5 --broker --rtbar- Historical data request is not seen going through..
***** STORE NOTIF: <error id=16777217, errorCode=420, errorMsg=Invalid Real-time Query:No historical market data for IBUS500/CFD@UKBBO Last 0> - next is not getting called even when real time data is coming through as seen in debug mode.
this is irrespective of --rtbar or not. so far I am not altering anything, will try to run it in trace to see what is happening.
- Historical data request is not seen going through..
-
If the error states:
Invalid Real-time Query
, it seems difficultnext
may be called at all, because the code believes there is no incoming data.That's the reason for the questions earlier in this thread, with regards to for example how to treat the
-BAAVG
string which for exampleSierra Chart
uses. See hereMay it also be that the currency specifier is missing?
-
The issues in processing the data received for CFD's.
Basically the error is its processing the market data returned for CFD's.
CFD's we request only BID and IB provides it in callback tickPrice()
The current implementation of tickprice checks if its CASH then only it process, ignore rest.
tickerId = msg.tickerId
if self.iscash[tickerId]:
........I have made the to include if its CFD's as well.
main change is
tickerId = msg.tickerId
if self.iscash[tickerId] or self.iscfd[tickerId]:
if msg.field == 1: # Bid Price
........
rest of the changes are to create/push/pop self.iscfd dictionary.
All the changes are in ibstore.pyI had forked a copy from dev branch and my changes are checked it in the GitHub
https://github.com/skytrading54/backtradersomehow couldn't create a pull request under
https://github.com/skytrading54/backtraderPlease take a look to merge in the next cycle.
It now works as expected ....
python ibtest_12Jan17.py --port 8099 --clientId 1 --data0 IBHK50-CFD-SMART --timeframe Minutes --compression 1
Data0, 0215, 736342.252778, 2017-01-13T06:04:00.000000, 22953.87, 22954.87, 22952.86, 22953.87, -1.0, 0, 22950.47
Data0, 0216, 736342.253472, 2017-01-13T06:05:00.000000, 22953.87, 22957.87, 22952.86, 22957.87, -1.0, 0, 22952.47
Data0, 0217, 736342.254167, 2017-01-13T06:06:00.000000, 22957.87, 22964.87, 22955.86, 22963.87, -1.0, 0, 22955.67
Data0, 0218, 736342.254861, 2017-01-13T06:07:00.000000, 22963.87, 22963.87, 22960.86, 22960.87, -1.0, 0, 22958.07
***** DATA NOTIF: LIVE
Data0, 0219, 736342.255556, 2017-01-13T06:08:00.000000, 22960.87, 22961.87, 22959.87, 22960.87, 0.0, 0, 22959.47 -
typos ...
- Basically the error is its NOT processing the market data returned for CFD's.
- somehow couldn't create a pull request under
https://github.com/mementum/backtrader