Adding planning tasks



  • Is there any way to add planned tasks to an strategy?
    Imagine I want to run some optimization and changes on the strategies all the days at 23:59. Or for example to buy/sell all the days at an specific datetime. Is there any way to do this?
    If the data feed doesn´t contain data at the 23:59 (or the intraday datetime I need), the strategy never get run at that point, so not sure if this can be done in some way.

    By the way, thanks for all the support and the great platform you´re developing. It´s being very helpful for my purpose.



  • Thinking a little bit more about this functionality; assuming I only have daily bars on the data feed, if I could create those planning tasks and create a MOC (Market on Close) order, for example 10 minutes before the close of the market, this should happen with the close price of the bar, isn't?
    And implement a more realistic proper use of the data, instead of using -coc option (that really I don´t like, as the bar has been consumed and the market is really closed).
    Same could be applied for an MOO (Market on Open) order, for example, sent during the night, with the market already closed and to be executed on the open auction of the market next day.



  • To run script based on schedule you can use several ways -

    1. common windows task scheduler with *.bat file
      just create *.bat file with the following lines and schedule it's run as you wish

    cd C:<Path to the folder with the script>
    python script_name.py --script parameters

    1. windows task scheduler as is
      https://blogs.esri.com/esri/arcgis/2013/07/30/scheduling-a-scrip/

    2. module schedule, looks like the easiest way - https://pypi.python.org/pypi/schedule

    import schedule
    import time
    
    def job():
        print("I'm working...")
    
    schedule.every().day.at("10:30").do(job)
    
    while True:
        schedule.run_pending()
        time.sleep(1)
    
    1. also you can do magic with datetime and time, but after schedule script it will be quite cumbersome.

  • administrators

    At the end of the day and as quickly outlined by @ab_trader you need to have something running in the background. The platform per se runs background threads when talking to external data providers, to manage the events and feed them properly into the platform.

    Options:

    • You run your own threads and have them sleeping until 23:59 kicks in

    • You use alternatives like the ones provided by @ab_trader

    • You try to full integrate your idea in the platform

    The 3rd option could be achieved by means of a ghost data feed which delivers data points, being the datetime the only one of interest to you.

    It is of course always easier said than done, but a brief outline:

    • You could subclass from feeds.DataBase
    • Override the _load method
    • On entering the method and based on the timeframe and compression parameters you calculate the remaining time until you hit the next time boundary.
    • You wait and deliver
    • Everything repeats from the beginning

    Your data feed could be for example ticking each and every minute or could tick every hour or every 24 hours. Just an idea



  • The problem with the first and second option is that it's managed outside the platform and for backtesting purpose will not work (only on real time).
    I'll try to implement it, as you're suggesting, with the lines concept..

    Apart of that, any idea on how to create MOC or MOO orders if possible?


  • administrators

    If MOC stands for Market-On-Close, this is a Close order.

    This order is meant for intraday data, because you issue the order before the market closes.

    If you are testing with daily data and still want to simulate a Close, you should then use broker.set_coc(True).

    MOO?



  • MOO (Market On Open) is default bt order.



  • I agree that the backtesting behavior of backtrader is effectively an MOC. But in a live trading mode, entering an MOC order might be a desirable feature in order to take a Market entry price on close of the next bar. After a bit of research, MOC is just an orderType in the API, so it may be as simple as that.

    https://interactivebrokers.github.io/tws-api/basic_orders.html#gsc.tab=0


  • administrators

    @ab_trader This is inaccurate. The Market order in backtrader takes the 1st incoming tick. In many use cases this is the open price of the next OHLC bar.

    But in replay mode, the tick can be any part of the final OHLC bar (and in live mode too)

    MOO should take the real open price of the next day and not the next incoming tick.


  • administrators

    @RandyT

    The backtesting behavior as explained above is to take the next incoming tick (which in many backtesting cases is the open price of the next bar) You can get the close price by enabling cheat-on-close.

    In live trading mode you cannot decide/guarantee that you will be given the close price of any bar. And MOC orders apply to the closeprice of the trading session (usually after a closing auction)

    With the Interactive Brokers and VisualChart broker interfaces, if you issue a Close order it will be translated to the appropriate broker type (which is MOC for Interactive Brokers and AtClose in VisualChart)


  • administrators

    @backtrader said in Adding planning tasks:

    If MOC stands for Market-On-Close, this is a Close order.

    This order is meant for intraday data, because you issue the order before the market closes.

    If you are testing with daily data and still want to simulate a Close, you should then use broker.set_coc(True).

    MOO?

    MOO can be set by the end user of backtrader by sending kwargs when issuing a buy/sell. See the reference for ib for an example with m_triggerMethod



  • @backtrader My understanding was that we discuss end-of-day backtesting, not real trading. Author said that he has only daily bars.



  • @backtrader Looking at the code now, I see that the use of MOC for Order.Close order type implies a close value during regular trading hours for the traded instrument. An important distinction.

    So if you wanted to close a position at Market on the next tick, you must set OrderType of Order.Market. Do I understand that correctly now?


  • administrators

    @ab_trader replay isn't real trading and one can be replaying daily bars into weekly bars. The usual backtesting use cases will get the open price of the next bar. But not for example when replay is in action, or if a filter has been inserted into the data stream and the bars are for example being chopped in 2 or 3 parts (OHL + C and O + HL + C)


  • administrators

    @RandyT Indeed. A Market order will get the next tick, which as stated above is usually the open price of the next bar (but must not be)

    Under real trading conditions, the market will actually dictate what the real execution price will be, but in liquid markets with a tight spread it will not be that far away from the tick.


  • administrators

    After some consideration, the best option would be to have a background thread which

    • sleeps until the expected time is reached
    • takes ownership of a lock, to avoid race conditions on data manipulation
    • calls a method of the strategy to make whatever changes may be needed
    • release the lock

    The next method:

    • acquires the lock
    • does something
    • releases the lock

    The acquisition/release of the lock is possibly best done inside a with context manager, to avoid having to add release statements at different points of complex if/elif/else logic blocks



  • Ok, so several things:

    • My example was with daily bars, as I think is more close to the reality. At the end, forgetting about the -coc option for a moment, if you have a live order feed sending only daily bars, you could have that planned task at, for example, midday, that send the MOC order and gets executed with the close price of the daily bar.
    • In reality, I have ticks resolution that I intend to use on my strategies, or sometimes other resolutions like minute bars. I´ve tried to create a Close Order during midday, and perfectly gets executed with the close price of the day (as a MOC must work). The curious thing is that if I log the execution on the notify_order method, as this is called with the first price of the next day, the date/time is on the next day (while the price correctly belongs to the close of the day)
    • For the MOO I can send it to IB with kwargs as you´re proposing, but I cannot get the same effect with backtesting, basically because 1) Backtrader doesn´t support natively MOO orders (what about more order types support for the future? :slight_smile: ) or 2) as I don´t have "planning tasks capability" I should know which one is the last bar - 1 on the day, to send a Market Order, before it close, and simulate MOO behaviour...
    • So at the end, I think the planning tasks, could be a great functionality (at least, for my needs it is).

    The question now is:

    1. if I do it with a background thread as proposed, I understand this is a real time capability, but no way to reproduce it on backtesting mode, isn´t?
    2. if implemented as points in lines (kind of time feed), wouldn´t be this more accurate to have same behaviour like in realtime with just 1 code?

  • administrators

    My example was with daily bars, as I think is more close to the reality. At the end, forgetting about the -coc option for a moment, if you have a live order feed sending only daily bars, you could have that planned task at, for example, midday, that send the MOC order and gets executed with the close price of the daily bar.

    There is difference between backtesting with daily bars and having a live feed which delivers daily bars. The latter probably constantly updates the current daily bar and that's why you think that already at noon time it may be possible to send a MOC order.

    To achieve the same effect during backtesting you need one of this:

    • intraday data and replay daily bars
    • Set cheat-on-close and allow using the close price
    • Use filters and break the feed into OHL + C steps (see the Pinkfish Challenge - and the sample in the sources)

    For the MOO I can send it to IB with kwargs as you´re proposing, but I cannot get the same effect with backtesting, basically because 1) Backtrader doesn´t support natively MOO orders (what about more order types support for the future? :slight_smile: ) or 2) as I don´t have "planning tasks capability" I should know which one is the last bar - 1 on the day, to send a Market Order, before it close, and simulate MOO behaviour...

    Yes you can. If you use daily bars, whenever you get a bar during backtesting, you are at the end of the session. Send for both cases (backtesting and live) a Market order and the extra kwargs. The standard backtesting broker will ignore it and the IB broker will overwrite the order type.

    1. if I do it with a background thread as proposed, I understand this is a real time capability, but no way to reproduce it on backtesting mode, isn´t?

    Not at the moment

    1. if implemented as points in lines (kind of time feed), wouldn´t be this more accurate to have same behaviour like in realtime with just 1 code?

    At least it would have the same behavior right now for both scenarios.


Log in to reply
 

Looks like your connection to Backtrader Community was lost, please wait while we try to reconnect.