During the last days I wondered: How do you continue a Backtrader strategy when it got interrupted? Let's say someone pulled the plug from your computer to put the vacuum cleaner in, or (more seriously ;-) ) you need to restart it for system updates or it simply crashed.

As there is at least some money on the line I can imagine one would not like to wake up or come home from work just to see that the computer crashed before a dump in the market and one is still in a position while the Backtrader strategy one developed for that could not run. Or the computer crashed before a run in the market where everyone tells how much of a good trader he is ;-) and one is not in a position as one would have been when the strategy would have had run.

From what I understand there is at least one thing that prevents one to safely automatically restart Backtrader with the strategy: The strategy does not know if one is in a position or not.
The feed would deliver the candle data as usual and most indicators could safely be prefilled to continue the strategy as if it would not have been stopped.
For the Parabolic SAR for example I don't think one can know in advance how many candles need to be prefilled to continue the strategy as if there was no interruption as one doesn't know when the last trend reversed and thus one can't calculate anything about the current SAR. The only solution here would be to prefill pessimistically quite a lot of candles to make sure one catches at least one trend reversal.

Depending on the strategy there can certainly be more internal state that cannot be automatically recovered and I might even overlook some obvious ones.

It is clear that there cannot be some kind of transactions over actual orders. But most of the time the strategy probably doesn't run and I would assume a crash happens at that time or otherwise accept the risk it doesn't. It would be good to at least have some kind of graceful shutdown/restart functionality to automatically recover the important transient state from a crash. I personally see only two options:

1. Whitelist-based store/restore

Save only specific important data at the end of every next() cycle and load it at the start of Backtrader if data can be found (means it run already). Only specific information e.g. if one is in a position or not would be stored and restored.
The disadvantages are:
a) that there is no hook/callback I'm aware of that signals the end of a next() cycle when all notify_order(),... methods have been run.
b) one might want to know if an order has been completed in the mean time. But the list of orders would have been gone after a restart. This is why one would need to store and restore internal Backtrader objects like orders which raises questions like what order properties need to be saved, what references within an order are also needed. This might even practically work with orders but it will quickly become ugly and unmanageble if more Backtrader internal state needs to be saved.

2. Pickling / Unpickling the whole Cerebro object with all its references

This way at least the state will be consistant. But the disadvantage is, that one cannot easily recover the position in the runtime execution stack where the last strategy ended.

Simply calling run() or runstrategies() on the unpickled Cerebro object would reinitialize and thus overwrite some state. Calling _runnext(runstrats) directly would probably miss some (if not all) initializations like timers if they are needed.

To solve that one would need to create something like a continue_strategy() method that expects to be run on an unpickled Cerebro object and does only the necessary initializations. Ideally that would not be too hard as I guess the optimization feature already makes strategies run independently. But unfortunately I don't understand the Backtrader optimization code well enough to utilize the optimization feature for that. I would probably copy & paste some code from run() and runstrategies() in to a continue_strategy() method, throw tests at it and go with that. But without further advise this would still be ugly.

Any advise or hints are much appreciated, even if it covers just some aspects!

Thanks in advance!

Best,

Sandro

@backtrader said in Restarting Backtrader for every candle:

@vladisld said in Restarting Backtrader for every candle:

Just curious - why add the dependency on Google API - just because it fits into the always free plan ?

It's the Google like Amazon Lambda. It is the current trend to have serverless services. Instead of managing a server, a script/function/piece of code is the thing to manage. The rest is managed in the background by the provider (Google, Amazon, ...)

BTW: The problem with Amazon Lambda, Google Functions and so on is, that those code usually needs a lot of dependencies with makes startup time of this code very long. Backtrader is a very very good fit in this aspect as it doesn't has this problem!

During the last days I wondered: How do you continue a Backtrader strategy when it got interrupted? Let's say someone pulled the plug from your computer to put the vacuum cleaner in, or (more seriously ;-) ) you need to restart it for system updates or it simply crashed.

As there is at least some money on the line I can imagine one would not like to wake up or come home from work just to see that the computer crashed before a dump in the market and one is still in a position while the Backtrader strategy one developed for that could not run. Or the computer crashed before a run in the market where everyone tells how much of a good trader he is ;-) and one is not in a position as one would have been when the strategy would have had run.

From what I understand there is at least one thing that prevents one to safely automatically restart Backtrader with the strategy: The strategy does not know if one is in a position or not.
The feed would deliver the candle data as usual and most indicators could safely be prefilled to continue the strategy as if it would not have been stopped.
For the Parabolic SAR for example I don't think one can know in advance how many candles need to be prefilled to continue the strategy as if there was no interruption as one doesn't know when the last trend reversed and thus one can't calculate anything about the current SAR. The only solution here would be to prefill pessimistically quite a lot of candles to make sure one catches at least one trend reversal.

Depending on the strategy there can certainly be more internal state that cannot be automatically recovered and I might even overlook some obvious ones.

It is clear that there cannot be some kind of transactions over actual orders. But most of the time the strategy probably doesn't run and I would assume a crash happens at that time or otherwise accept the risk it doesn't. It would be good to at least have some kind of graceful shutdown/restart functionality to automatically recover the important transient state from a crash. I personally see only two options:

1. Whitelist-based store/restore

Save only specific important data at the end of every next() cycle and load it at the start of Backtrader if data can be found (means it run already). Only specific information e.g. if one is in a position or not would be stored and restored.
The disadvantages are:
a) that there is no hook/callback I'm aware of that signals the end of a next() cycle when all notify_order(),... methods have been run.
b) one might want to know if an order has been completed in the mean time. But the list of orders would have been gone after a restart. This is why one would need to store and restore internal Backtrader objects like orders which raises questions like what order properties need to be saved, what references within an order are also needed. This might even practically work with orders but it will quickly become ugly and unmanageble if more Backtrader internal state needs to be saved.

2. Pickling / Unpickling the whole Cerebro object with all its references

This way at least the state will be consistant. But the disadvantage is, that one cannot easily recover the position in the runtime execution stack where the last strategy ended.

Simply calling run() or runstrategies() on the unpickled Cerebro object would reinitialize and thus overwrite some state. Calling _runnext(runstrats) directly would probably miss some (if not all) initializations like timers if they are needed.

To solve that one would need to create something like a continue_strategy() method that expects to be run on an unpickled Cerebro object and does only the necessary initializations. Ideally that would not be too hard as I guess the optimization feature already makes strategies run independently. But unfortunately I don't understand the Backtrader optimization code well enough to utilize the optimization feature for that. I would probably copy & paste some code from run() and runstrategies() in to a continue_strategy() method, throw tests at it and go with that. But without further advise this would still be ugly.

Any advise or hints are much appreciated, even if it covers just some aspects!

Thanks in advance!

Best,

Sandro

@Sandro-Böhme said in Restarting Backtrader for every candle:

@vladisld said in Restarting Backtrader for every candle:

Just curious - why add the dependency on Google API - just because it fits into the always free plan ?

Yea, partially because it's more cost effective. I really try to avoid monthly costs. Especially if I want to run several strategies and an N1 VM might not be enough it gets cheaper than a full VM.

An other reason is, that I could automate or externally control the strategy creation. Assume you have a strategy with a trading pair and period parameter. You can let a Google Cloud Function listen on a Google Firestore node that contains this parameters as the trade setup. When a new one with the actual trading pair and period/timeframe got created it triggers the Google Cloud Function that listens which can then configure the Google Cloud Scheduler with the strategy that also runs in a separate Function. That strategy Function would start every minute and stop after a next() cycle has ended if a 1 minute period had been configured. The strategy can of course also write to a subnode for the trade setup and report things like current win rate, profit/loss and so on.

This setup is what I'm working on. It is pretty easy to create/manage Firestore nodes using the Google frontend. To keep the things simple for now this is what I would go for at the beginning. But it should also not be too hard to generate a website with this Firestore nodes using a JAMstack.

The actual development and backtesting of the strategy can still be done locally.

I've created proof of work's for the triggers and for running Backtrader in a Google Function. That in general is not a problem. I also worked on a whitelist based persistence of the strategy where I only saved some custom strategy variables and some internal ones from Backtrader. Then I started Backtrader as usual in a Function with a custom start time based on the Cloud Scheduler and used my custom loading to overwrite some strategy variables.
But at some point I was not sure if I really catched all the needed properties to overwrite. This is why started just pickling everything to be sure I've got the whole state from the last run. But now I have no workaround or support to restart Cerebro with that data.

To be fair - it's probably quite complex to support recoverability.

@Sandro-Böhme said in Restarting Backtrader for every candle:

Ok, I thought about a configuration where Backtrader is run for trading as usual but then I wondered how one safely restarts it after it has been stopped. I mean, it probably will stop at one point. But then the strategy doesn't know it's last state e.g. if it's in a position and so on.

I'll create a separate post about that later.

@backtrader said in Restarting Backtrader for every candle:

@Sandro-Böhme said in Restarting Backtrader for every candle:

I'm aware that Backtrader is probably not made for that scenario.

You can remove the "probably". It is not.

@Sandro-Böhme said in Restarting Backtrader for every candle:

Is there a way / workaround to support that scenario with the current version of Backtrader?

No.

@Sandro-Böhme said in Restarting Backtrader for every candle:

Would that "restartability" feature be something that Backtrader could support in the future?

No.

Ok, I thought about a configuration where Backtrader is run for trading as usual but then I wondered how one safely restarts it after it has been stopped. I mean, it probably will stop at one point. But then the strategy doesn't know it's last state e.g. if it's in a position and so on.

@vladisld said in Restarting Backtrader for every candle:

Just curious - why add the dependency on Google API - just because it fits into the always free plan ?

Yea, partially because it's more cost effective. I really try to avoid monthly costs. Especially if I want to run several strategies and an N1 VM might not be enough it gets cheaper than a full VM.

An other reason is, that I could automate or externally control the strategy creation. Assume you have a strategy with a trading pair and period parameter. You can let a Google Cloud Function listen on a Google Firestore node that contains this parameters as the trade setup. When a new one with the actual trading pair and period/timeframe got created it triggers the Google Cloud Function that listens which can then configure the Google Cloud Scheduler with the strategy that also runs in a separate Function. That strategy Function would start every minute and stop after a next() cycle has ended if a 1 minute period had been configured. The strategy can of course also write to a subnode for the trade setup and report things like current win rate, profit/loss and so on.

This setup is what I'm working on. It is pretty easy to create/manage Firestore nodes using the Google frontend. To keep the things simple for now this is what I would go for at the beginning. But it should also not be too hard to generate a website with this Firestore nodes using a JAMstack.

The actual development and backtesting of the strategy can still be done locally.

I've created proof of work's for the triggers and for running Backtrader in a Google Function. That in general is not a problem. I also worked on a whitelist based persistence of the strategy where I only saved some custom strategy variables and some internal ones from Backtrader. Then I started Backtrader as usual in a Function with a custom start time based on the Cloud Scheduler and used my custom loading to overwrite some strategy variables.
But at some point I was not sure if I really catched all the needed properties to overwrite. This is why started just pickling everything to be sure I've got the whole state from the last run. But now I have no workaround or support to restart Cerebro with that data.

Hello,

I would like to run Backtrader in a Google Cloud configuration. More precisely I would like to use the Google Cloud Scheduler to run the Backtrader strategy e.g. every minute in a Google Cloud Function.
But that would mean running Cerebro for the first next() call, pickling the instance before the subsequent next() call and stopping the execution via self.env.runstop(). That will probably take way less than a minute.
At the start of the next minute the Google Cloud Scheduler starts the Google Cloud Function again with the Backtrader runtime. It should then unpickle the Cerebro instance and continue to run it.

I'm aware that Backtrader is probably not made for that scenario.

But that way to run it in the Google Cloud would most probably still be covered by the free plan while still getting a high reliability of the infrastructure (hardware, internet,...) the code runs on.

Pickling and unpickling the state of the Cerebro instance works like a charm (after some changes in my code) but:

1. I don't know when exactly I should pickle the state. It should be at a time when next() has been finished and notify_data(), notify_order() and so on have been called.
2. At the start of the next minute the Cerebro instance will get unpickled but I don't know how to rerun Cerebro with that instance. cerebro.runstrategies(cerebro.strats[0]) would probably be too early.

Is there a way / workaround to support that scenario with the current version of Backtrader?
Would that "restartability" feature be something that Backtrader could support in the future?

Any help is much appreciated!

Thanks,

Sandro

@backtrader said in Matching of stopLimit orders:

@sandro-böhme said in Matching of stopLimit orders:

Somehow from the notify methods I assumed that everything will be handled asynchronously but it doesn't

That's a huge mistake. You may think it's an advantage short-term ... but it isn't. The notify_xxx do simply let you know how things have happened. They imply nothing.

That was indeed a big mistake. Thanks for your support @backtrader!

@backtrader
TL;DR: From my observation it all seems to work now as simple as it can and get exactly the way I wanted it.

Longer Version:
You wrote

the logic is supposed to be located in next

So I thought about a rough idea of extending the broker to have some kind of updateOrder(oldOrder, newArgs) method that cancels the old order, really synchronously waits for for the result and then sends the new stop-limit-sell order. When I looked into the broker I use to see if I can add an order type - surprise, surprise - I recognized that the methods are already synchronous to a big degree. E.g. when the cancel() method returns the order is already canceled on the exchange and this is already reflected in the returning order object. It is the same with the stop-limit-sell order. When the sell() method returns the order is already placed at the exchange.
Now this makes things incredibly simple. I can really just call cancel() and then send my new sell() order with the raised stop-loss price in the next() method directly afterwards because I know that the previous sell order will immediately get canceled before I send the raised stop loss and I actually don't have to wait for any notify_xxx() method to confirm that.
Somehow from the notify methods I assumed that everything will be handled asynchronously but it doesn't. At least not for the Broker implementation I use.
I tested it live on an exchange and it works like a charm!
Of course I still use the notify_xxx methods to get called when the buy or sell orders got matched.

@backtrader said in Matching of stopLimit orders:

@sandro-böhme said in Matching of stopLimit orders:

If I would like to make use of it as a stop price as in (Stop And Reverse) I guess I need to find a way to get the order send and accepted at the beginning of a candle or more precisely during a candle with the current PSAR value

This is NOT possible. Because the PSAR value is only known when the candle is closed.

• In regular mode you get the final PSAR value with the final bar prices

• In replay mode you get the temporary PSAR value with the known bar prices.

In no case can you have the PSAR value at the beginning of the bar. At the beginning of the bar you have the previous PSAR value.

I wondered what would be the use of it if I only have the PSAR stop-loss value of the elapsed bar represented in next() but I figured that of course one can place a market sell order if the market price in next() is below the PSAR one can access in next(). But that might not save the investment very well as the price can have headed south way more than with a stop loss. Especially in Cryptos.

It feels wrong to write that now but my understanding is, that in the current next() call I should get the PSAR value for the subsequent candle.
The Wikipedia page about the Parabolic SAR says

That is, tomorrow's SAR value is built using data available today.

Also if I use the indicator in TradingView (my charting tool of choice) it is set already at the beginning of the candle and doesn't change even though the final high and low values of that candle are obviously not yet established.

At the end of course Backtrader can have used the previous high and low values to generate the current PSAR value at the end of the candle in next(). This is why I logged this values in the indicator and in next() to see if the input of the PSAR is taken from the high and low values available in next(). Every first line comes from the indicator and obviously the every second from the next() call.

2019-02-15T09:27:00, PSAR High: 9.3882 Low: 9.3877
2019-02-15T09:27:00, next() Open: 9.3877, High: 9.3882, Low: 9.3877, Close: 9.3882, PSAR: 9.3505 in next()
...
2019-02-15T09:28:00, PSAR High: 9.3919 Low: 9.3919
2019-02-15T09:28:00, next() Open: 9.3919, High: 9.3919, Low: 9.3919, Close: 9.3919, PSAR: 9.351254 in next()
...
2019-02-15T09:29:00, PSAR High: 9.3839 Low: 9.3839
2019-02-15T09:29:00, next() Open: 9.3839, High: 9.3839, Low: 9.3839, Close: 9.3839, PSAR: 9.35287984 in next()
...
2019-02-15T09:30:00, PSAR High: 9.3893 Low: 9.3893
2019-02-15T09:30:00, next() Open: 9.3893, High: 9.3893, Low: 9.3893, Close: 9.3893, PSAR: 9.3544406464 in next()

I don't know yet why the ohlc values are equal so often (probably an issue specific to my setup) but it seems to me that the PSAR uses the current close values from next(). Which is perfect for me as I can use this value then in next() for the stop-limit-sell order that will have the order placed before the method finished which should mean that it is available quite early in the next candle.

But in the end of course the PSAR indicator or Backtrader platform might do something I don't recognize that leads to next() indeed having the PSAR of the current candle and not the one for the next candle as I expect it.

For me it least it now looks like it works exactly as I want it.

@backtrader said in Matching of stopLimit orders:

@sandro-böhme said in Matching of stopLimit orders:

If you see other problems please tell me as well as I plan to trade real money with that code.

You need to see the problems and think twice about your money.

Thanks for taking the time to respond @Backtrader. That's much appreciated!

Are you expecting people to read and debug your code for you? Problem #1.

No, I don't but I wanted to give the opportunity that people can do that if they want.
I wasn't sure why you was assuming 6.9994 is the Limit price. I worried that you actually had a look in the code and I made a mistake there and mixed something up.

Log things properly, because I can have a quick look at the log and you can today relate the code to the log. When you look at the log in the future you won't know what it relates too. If there are ohlc values and trigger and limit prices, log it all. And log the version of the code you are working with if you ever want to relate the log to the actual code. (you may log a git sha hash or something, but something that lets you relate the log to the actual code)

Thanks for the hint. I agree that it makes sense to at least log the ohlc values and trigger and limit prices.

In any case, setting the Limit price of a sell-StopLimit below the trigger price means you want to give money away for free.

Let me explain why I do that so you can correct me if I'm wrong.

Assume I set the limit order to have the same price as the trigger price to not give away money for free.
As I send a stop limit sell order I expect the market price to drop when the order is triggered and the limit sell order is added to the order book.
But if the price drops very fast not all the other sell orders before mine could have been bought away by buyers before the market price is below my limit sell order. Then my sell order would be too expensive for the buyers and not get filled.
The lower my limit price is compared to the trigger (stop) price the higher the chance that my sell order gets filled.
This is why I would say I give away the money to pay for lowering the risk of the sell order not to get filled.

@sandro-böhme said in Matching of stopLimit orders:

Thanks a lot, that's a very good point!

Problem #2.

It's not a good point. It is simply the point. It doesn't matter if 6.9994 is your trigger or limit price. It is never reached when your order is alive. Your order can be neither triggered nor executed.

Of course you are right again.
I just haven't checked it by myself at that time and wasn't sure if there is maybe (also) a bug in my code which is why I didn't know yet it was the point.

I'll compare it with the other bar and I guess I'll see the difference.
If I don't I'll try to find the matching logic and debug the code to better understand the behavior.

You don't need to understand the matching logic. It is as easy as: it cannot be matched if the order prices are not in the range of the bar.

You are absolutely right. Again... :-)

You need TRADING experience.

Absolutely. I'm working on that as well. To be precise I would say I'm towards the end of the "Conscious Incompetence" stage of the stages of competence. So at least I should have left the Dunning-Kruger effect behind me ;-).
In this thread I already learned quite a bit thanks to your feedback!

When you have TRADING experience, how and when orders are matched will be obvious to you. My suggestion: spend several hours see how things work in a market.

There is no doubt that will happen. Not only regarding trading in general but also when I run my code with real money agains an exchange. I plan to set breakpoints, examine the actual values I get from the exchange and thus always have the chance to stop the code from running. Even though I cannot stop the exchange to execute the order I have send some buy orders already with prices well below the market price to see if they show up on the exchange the way I expected. I also did that with careful debugging and examining the actual values of the objects.

See this other recent thread: Community - Stop Order execution price? Why does the poster believe he has found a flaw in the logic for the execution go a Stop order?

• Simple: the poster has NO trading experience.

Yes I've seen that as I read some threads (and docs of course) before posting. This thread was actually the reason I was specifically talking about a stop-limit order and not only about stop-loss.

@sandro-böhme said in Matching of stopLimit orders:

Basically submitting the sell-StopLimit takes the time of a candle

Problem #3. This is a completely wrong statement. And very dangerous. The candle you are working with contains PAST prices. Here and China. Order submission DOES NOT take the time of a candle. The candle DOES NOT longer exist. You work and can only work with PAST information.

The order is submitted and can ONLY be applied and evaluated with the next set of incoming prices.

Sure I'm sorry that I wasn't more specific to be really clear.
I'll try to describe how I understand that more clearly now so you can correct me if I'm wrong.

The next() method runs after a candle has been closed when all final ohlc data for that candle is available. Judging from the logs the notify_...() methods seem to get called directly after the first next() and before the subsequent next(). One could see this as the time during the candle.
I submit the sell order when I get the "cancel" notification in notify_order(). This is after the previous next() call and before the subsequent next() call. As written above I think this is the time the ohlc data for the subsequent next() candle is created. Technically I cannot really know that it takes the full time of the candle as I could as well get the notify_order() call at the last minute of the 15m candle. So of course you are right that this is a wrong statement.

@sandro-böhme said in Matching of stopLimit orders:

I think the ParabolicSAR assumes that I can get the order accepted at the same candle

Problem #4

The ParabolicSAR assumes nothing. The indicator simply delivers a value. It is a math formula. You are making the assumption. Don't put the the blame on the math. And yes wording is important because it shows how you perceive things.

If I would like to make use of it as a stop price as in (Stop And Reverse) I guess I need to find a way to get the order send and accepted at the beginning of a candle or more precisely during a candle with the current PSAR value. Currently I always have a candle where there is no sell-stop-limit order set as I submit the order during notify_order() which is before next() and get it accepted after next().
But of course you are right. The indicator simply delivers a value as every other indicator.

@sandro-böhme said in Matching of stopLimit orders:

Is there a way I can get the next() and notify_...() methods called multiple times during a candle independent of the time frame e.g. in a 1m timeframe during backtesting and live trading?

Problem #5

Is there any way in which you can get future information? Yes, by entering the vortex which is generated with tachyon particles and a thoron emitter whilst the frequency of the shields is remodulated to match the rotating harmonics of the reverse polarity of the feedback impulse traveling over the tractor beam.

Oh yes, you will read about replaying: Docs - Data Replay. But of course, you will then have to decide when to make the trade within the bar and even with replay ... the prices you see are already PAST prices. What you see is always PAST data.

Thanks for that information that is very helpful! I'll have a look into data-replay.