For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

No backfill from IB



  • Hello,

    I'm new to backtrader and trying to use my newly created IB account to run some Strategies with Paper Trading for the moment. When I run the ibtest I cannot get any historical data, the code runs and stays with the output as below. I have an active data subscription to the NASDAQ data and in the TWS I can see it in real-time. This code I was trying to run over the weekend but I guess the backfill should happen anyhow even if the markets are closed?
    I've tried to change the TZ to the US/Eeastern (I'm in France) but it did not change anything.

    Any ideas what I do wrong there? Some setting somewhere in BT or TWS?

    I've also tried to code it myself but the result is similar to the one from ibtest, no backfill data.

    Cheers,
    Jakub

    python ibtest.py --data0 'AAPL-STK-SMART-USD' --port 7497 --timeframe Days
    Server Version: 76
    TWS Time at connection:20180113 23:47:52 CET
    --------------------------------------------------
    Strategy Created
    --------------------------------------------------
    Timezone from ContractDetails: EST5EDT
    Datetime, Open, High, Low, Close, Volume, OpenInterest, SMA
    ***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm connection is OK:hfarm>
    ***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm connection is OK:jfarm>
    ***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm connection is OK:eufarm>
    ***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm connection is OK:usfuture>
    ***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm connection is OK:cashfarm>
    ***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm connection is OK:usfarm.us>
    ***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm connection is OK:usfarm>
    ***** STORE NOTIF: <error id=-1, errorCode=2106, errorMsg=HMDS data farm connection is OK:ilhmds>
    ***** STORE NOTIF: <error id=-1, errorCode=2106, errorMsg=HMDS data farm connection is OK:euhmds>
    ***** STORE NOTIF: <error id=-1, errorCode=2106, errorMsg=HMDS data farm connection is OK:fundfarm>
    ***** STORE NOTIF: <error id=-1, errorCode=2106, errorMsg=HMDS data farm connection is OK:ushmds>
    ***** DATA NOTIF: DELAYED
    


  • I bet your problem is ... the weekend.

    To backfill data you need a reference point, i.e.: real-time data which tells the system up to which it has to download.

    No real-time data (i.e.: weekend), no download.

    Use the --historical flag (which sets the historical parameter of the data feed to True) and things will probably change.



  • Thanks a lot for your answer. I was suspecting something related with the market closure. I will give it a try today to see if things get better. I've tried the option with only historical data before but it makes the system exit just after the last point. In other words it is not live but only for backtesting. This is ok but not what I really want. :)



  • @jakub-wozniak said in No backfill from IB:

    In other words it is not live but only for backtesting

    --historical is clearly historical and not live.

    Your problem is: there is no live data during the weekend to reference the end of the stream. Your paper trading account works with real-time market data and if there is no real-time market data, there is no live nothing.

    During the weekends you can use the demo account from IB which supports only a few tickers, but provides real-time (faux) data and which will work with backfilling. Try the TWTR ticker which used to always work with the demo.



  • Hi,

    Yes, I've tested and you were absolutely right. When the market is open all works as expected for the paper trading account. When it is closed it does no backfill.
    Thanks for the hint about the demo one.

    But I guess this is just something in the implementation of the datafeed in backtrader, right? I would suspect that normally you can ask IB for any historical data at any moment? The real-time data is just needed to establish the from-to range? Am I getting it correctly or there is something else going on behind the scene?
    In any case, I know now what the problem was so I can continue with what I wanted to do.

    Cheers,
    Jakub



  • @jakub-wozniak
    Jakub,

    Can you please post the main code flow you use to connect to IB and trade? I have the same problem with my paper account.

    Thank you.



  • One question: are you using the paper account and the live account at the same time?



  • Yes I am.



  • This may then apply to you: https://ibkr.info/node/1719



  • @paska-houso
    The link you sent prohibits running of live and paper account on different devices, but says nothing about running on same device, both TWS sessions (live + paper) , at the same time, right?
    So, there should not be a problem running live and paper TWS sessions on the same device, at the same time, if I read right.


  • administrators

    The referenced link from above seems to indicate so. But for backtrader there is no distinction as to what is a live and a paper account. Everything is a TCP connection. Data delivery, trading and everything is controlled by TWS.

    Things that could help in diagnosing your problem:

    • Run ibtest.py against your live account. As long as you don't provide the --broker (create an IBBroker) and --trade (engage into actual trading) nothing will be done

    If you don't want to blindly trust not using the options mentioned above, you can also:

    • Set the API to read-only in the TWS settings. Even if backtrader wanted to issue orders, these would be rejected.

    or

    • Remove the trading code from ibtest.py

    You can also show:

    • The command line you used to run ibtest.py
    • The output the command produced