For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

Preparing my trading architecture for use with backtrader



  • Hi all, I've been building a trading bot on the OANDA API for sometime now, letting it run on practice accounts, modifying it, letting it run for a bit longer. I was completely new to trading prior to this project!

    I would like to backtest this trading strategy now and iteratively improve it that way, but in all my exploring of backtrader (and many other) available Python frameworks' sample code and documentation I haven't found a clear solution for backtesting my approach. I attempted a version using the order_history sample code on backtrader Github but could not get it working..

    I'm thinking I may need to modify my architecture to be more "strategy-like" to match the simple SMA crossover type examples I'm seeing around for backtesting?

    I run my trading architecture with a very simple looking function, like so:

    trade_setup_dict = trader(df_ETF)
    

    ...where df_ETF is the most recent 150 candles, pulled from the OANDA API. After running a great deal of code and analyzing the potential trade, it returns the dictionary trade_setup_dict, which looks like this:

    {
    'TRADE_IT': 1, 
    'Entry Price': 0.77638, 
    'Stop Loss': 0.77666, 
    'Take Profit 1': 0.77618, 
    'Take Profit 2': 0.77618, 
    'Type of Trade': 'LONG', 
    'Pct of Available Cash': 0.05, 
    }
    

    where TRADE_IT is 1 if we want to make a trade at all (the rest should be self explanatory...)

    The examples I've been able to understand from backtrader do not appear to receive detailed trade setups like this very clearly.

    Could anyone advise as to how I might be able to "port" my trading strategy to backtrader?

    Thanks for any of your time @backtrader, I'm grateful for your work and hopeful to learn more!



  • Lets assume that you call trader every bar. Then I think you have two options:

    • write an indicator which will return you required dictionary and then in the strategy next you will issue buy/sell order if TRADE_IT == 1

    • in the strategy next call your trader function and issue buy/sell order if TRADE_IT == 1

    In both cases you will need to rework your trader function so it can process bt datafeed. Or you can read prices separately into your function and into bt, but I think it will be less clear.



  • Thank you for your reply. I'm trying to understand your response:

    @ab_trader said in Preparing my trading architecture for use with backtrader:

    write an indicator which will return you required dictionary and then in the strategy next you will issue buy/sell order if TRADE_IT == 1

    Isn't my function the indicator in this case? Or do I need to do something backtrader specific to make it an 'indicator'?

    In both cases you will need to rework your trader function so it can process bt datafeed.

    What does that mean? I have tons of OHLC data saved offline that I would like to use for backtesting, is that possible?



  • @tw00000

    Isn't my function the indicator in this case? Or do I need to do something backtrader specific to make it an 'indicator'?

    Good question. I didn't see your function, but common sense and your questions tell me that it will need to be adjusted to be used in bt. Here is the link on how bt treats indicators - Docs - Using Indicators and Docs - Indicator Development

    What does that mean? I have tons of OHLC data saved offline that I would like to use for backtesting, is that possible?

    In order to test something using bt you need to import data feed. So you can use this imported data feed in your indicator. Or read external file with indicator and sync it to imported data feed. Which is probably more complex. Here is the link to data feeds - Docs - Data Feeds

    But I believe the best link (based on your questions) you might want to use first time is here - Docs - Quickstart



  • Good question. I didn't see your function, but common sense and your questions tell me that it will need to be adjusted to be used in bt. Here is the link on how bt treats indicators - Docs - Using Indicators and Docs - Indicator Development

    I provided a pretty clear description of the function above, it receives 150 lines of OHLC data and returns a trade setup dictionary that includes whether or not to place a trade.

    But I believe the best link (based on your questions) you might want to use first time is here - Docs - Quickstart

    I've experimented a lot with backtrader and built a pretty in depth implementation using the order-history sample as a starting point, but I was ultimately unsuccessful which is why I came to the community.

    I was hoping for a bit more in-depth answer than 'read the docs', but I appreciate you taking the time to respond in any case.



  • Any hope for a reply from @backtrader :) ?



  • Your's choice. Good luck.



  • @tw00000 ok, I'll try to explain in more details. The things you need to do in your backtesting script if you want to use Backtrader are:

    Read the Quickstart if you have not already. It's crucial to get familiar with the basic terms of Backtrader.

    Write a strategy Python class (inherits from backtrader.Strategy, see example here)

    • the next() function is where you will be calling your trader() function and reacting on the results it will provide (the trade_setup_dict)
    • use the slicing method to get an array of last 150 values of high, low, open, close, volume. You'll get 5 arrays that way so either can combine them to a structure your trader() function understands or modify the function to accept the 5 arrays separately.
    • call your trader() function with the OHLCV data obtained in the previous step in the next() function to get the trade_setup_dict results
    • based on the results, call the buy() or sell() functions to place an order (see the docs). Since you know your entry price, stoploss and take profit, it's best to use the bracker order functionality
    • if you do it this way, you will want to skip the first 150 bars of your data to make sure you can slice the 150 last OHLC values to pass to the trader() function - simply keep checking the len(self.datas[0].close) until it is more than 150

    Next, define a Datafeed that will allow Backtrader to understand the format of your data.

    • I don't know the format of your saved OHLC data so you need to study the Datafeed docs to find out which of the already implemented ones will work for your data format (e.g. if your data is in CSV format, the GenericCSVData type will probably work with just some configuration).

    Then, do the usual Backtrader procedure (all the steps are described in the Quickstart):

    • instantiate Cerebro
    • add your datafeed to Cerebro
    • set the starting cash amount
    • configure Cerebro to use the right Sizer - the PercentSizer is probably the one you want since your trader() function returns the order size in percents (assuming of total available account size)
    • run Cerebro
    • plot if wanted

  • administrators

    The summary from @tomasrollo should be more than enought.

    I would only like to point out that the trader function above seems to play the role of next in that it generates a trade which has also the information an indicator (or indicator-like logic) has generated.

    @tw00000 said in Preparing my trading architecture for use with backtrader:

    'TRADE_IT': 1, 
    

    Pack the generation of TRADE_IT in an Indicator and the price generation logic can remain in next or be even delegated to a Sizer

    Note:

    @tomasrollo said in Preparing my trading architecture for use with backtrader:

    use the slicing method to get an array of last 150 values of high, low, open, close, volume. You'll get 5 arrays that way so either can combine them to a structure your trader() function understands or modify the function to accept the 5 arrays separately.

    150 values will be available for the slice if there are indicators which need that buffering. If the logic doesn't need indicators, the easiest ways to ensure that such buffering is available from the data when next is 1st called are:

    • Create a Simple Moving Average with period 150 as in: bt.ind.SMA(period=150)

    • Create a delayed version of the data with a period of 150, for example: self.data.close(ago=-150) # or self.data.close(-150)



  • Thank you @backtrader and @tomasrollo for your super helpful posts, I think in combination I have a lot to work with here. I'll report back ASAP!



  • So, I got something working! Thanks greatly to @tomasrollo and @backtrader for their help. I've followed Tomas' steps pretty much completely, a test is running currently, we'll see if it works in a bit! I haven't figured out how to use the Sizer functionality and multiple take profits yet but I'm interested, for now just using Target Price 1 as my only take profit with the Bracket functionality.

    One question, as I think the next step will be optimizing backtrader to run more efficiently so I can test more nimbly...

    @backtrader said in Preparing my trading architecture for use with backtrader:

    I would only like to point out that the trader function above seems to play the role of next in that it generates a trade which has also the information an indicator (or indicator-like logic) has generated.

    Right now, I'm not doing anything in __init__ . My strategy is such that most or all of the indicators need to be re-painted each iteration. In that case, it seems like the trade logic ("indicator" that generates the TRADE_IT: 1 and the entry/TP/SL) must be in next() ... is this a correct understanding of the difference between creating an 'indicator' to be used in __init__ and using my own logic in next() ?

    My code is like so:

    def next(self):
            
            if len(self.datas[0].close) > window:
        
    # This function assembles a numpy array from the bt lines behind the scenes for evaluation, then returns a dict with trade instructions# 
                try: 
                    trade_setup = my_strategy.run_the_indicators(
                                        data=[self.datas[0].open.get(size=window),
                                            self.datas[0].high.get(size=window),
                                            self.datas[0].low.get(size=window),
                                            self.datas[0].close.get(size=window),
                                            [bt.utils.date.num2date(date) for date in self.datas[0].datetime.get(size=window)]
                                             ],
                                        instrument_name=instrument_name,
                                        params={'count':window, 'round_value':5}
                                        )
                except Exception as e:
                    trade_setup = {'TRADE_IT': 0,
                     'Entry Price': 0,
                     'Stop Loss': 0,
                     'Target Price 1': 0,
                     'Target Price 2': 0,
                     'Type of Trade': None,
                     'Pct of Available Cash': 0,
                     'TP1 vs TP2 Split': 0,
                    print("Trade logic failed with: ", e)
    
                if trade_setup['TRADE_IT'] == 1:
                    if trade_setup['Type of Trade'] == 'SHORT':
                        self.sell_bracket(limitprice=trade_setup['Target Price 1'], price=trade_setup['Entry Price'], stopprice=trade_setup['Stop Loss'])
    
                    if trade_setup['Type of Trade'] == 'LONG':
                        self.buy_bracket(limitprice=trade_setup['Target Price 1'], price=trade_setup['Entry Price'], stopprice=trade_setup['Stop Loss'])
    

  • administrators

    The logic cannot be elsewhere but next, because it is what will be called on a regular basis.

    The difference between creating and operating with indicators is explained here: Docs - Platform Concepts - Stage 1 and Stage 2

    You may also look at the lifecycle of the Strategy: Docs - Strategy