For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/
Beginner optimizer and analyzer help
-
Hi,
Just got started with backtrader and I am blown away with the things you can do. I am trying to do a very simple strategy to make sure I can get the optimizer and analyzer working as expected but I cannot seem to have any luck with this.
The strategy works 100% without the optimizer and analyzer, but I dont seem to understand how these work together.
here is my code:
from __future__ import (absolute_import, division, print_function, unicode_literals) import datetime # For datetime objects import os.path # To manage paths import sys # To find out the script name (in argv[0]) from strategy import * # Import the backtrader platform import backtrader as bt import backtrader.indicators as btind def optimizer_callbacks(cb): print('optcallback executed') if __name__ == '__main__': cerebro = bt.Cerebro(optreturn=False) cerebro.addanalyzer(bt.analyzers.SharpeRatio, _name='sharpe_ratio') cerebro.optcallback(optimizer_callbacks) cerebro.broker.setcommission(commission=0.025) # 2.5% commission cerebro.optstrategy(SMA_CrossOver, fast=range(3,5), slow=range(19,21) ) cerebro.addsizer(bt.sizers.SizerFix, stake=1) start_portfolio_value = 10000.0 cerebro.broker.setcash(start_portfolio_value) #print('Starting Portfolio Value: %.2f' % cerebro.broker.getvalue()) data = bt.feeds.CCXT(exchange='binance', symbol='BTC/USDT', name="btc_usd_tick", # Do not pass values before this date fromdate=datetime.datetime(2019, 1, 1), # Do not pass values before this date todate=datetime.datetime(2019, 6, 1), timeframe=bt.TimeFrame.Days, compression=1, ) cerebro.adddata(data) optimized_runs = cerebro.run(maxcpus=1) # results final_results_list = [] for run in optimized_runs: for strategy in run: PnL = round(strategy.broker.get_value() - 10000, 2) sharpe = strategy.analyzers.sharpe_ratio.get_analysis() final_results_list.append([strategy.params.fast, strategy.params.slow, PnL, sharpe['sharperatio']]) for item in final_results_list: print(item) sort_by_sharpe = sorted(final_results_list, key=lambda x: x[3], reverse=True) for line in sort_by_sharpe[:5]: print(line)and my very simple strategy:
class SMA_CrossOver(bt.Strategy): params = (('fast', 3), ('slow', 20)) def __init__(self): self.buysig = {} for d in self.getdatanames(): sma_fast = btind.SMA(self.getdatabyname(d), period=self.p.fast) sma_slow = btind.SMA(self.getdatabyname(d), period=self.p.slow) self.buysig[d] = btind.CrossOver(sma_fast, sma_slow) def next(self): for d in self.getdatanames(): if self.getpositionbyname(d).size: if self.buysig[d] < 0: self.sell(data=self.getdatabyname(d)) print('sell!') elif self.buysig[d] > 0: self.buy(data=self.getdatabyname(d)) print('buy!')my output unfortunately gives me the following:
buy! sell! buy! optcallback executed optcallback executed optcallback executed [3, 19, 0.0, None] [3, 20, 0.0, None] [4, 19, 0.0, None] [4, 20, 0.0, None] Traceback (most recent call last): File "...\trading_optimizer.py", line 56, in <module> reverse=True) TypeError: '<' not supported between instances of 'NoneType' and 'NoneType'To me it seems it is only executing the strategy once (as there is no buy! or sell! in my output), and in the analyzer results it doesnt even give me the output to that first run?
Next steps once I have this working :
- compare different assets (e.g. BTC/USDT vs BTC/ETH)
- compare different strategies
to see which gives a better results in my analyzers - is it possible to do this in 1 run?
- I need to work out how to print my optimizer_callbacks with the strategy name and parameters.
Any assistance would be most appreciated.
Thanks again for an amazing library.
-
So I have been working on this and have made some progress, making some changes along the way:
I added TradeAnalyzer as well to the analyzers to see if I could try something different and I have made some progress - I am now able to successfully get the results of the first strategy run:
cerebro.optstrategy(SMA_CrossOver, fast=range(17,20) )optimized_runs = cerebro.run(maxcpus=1) print('runs completed: ' + str(len(optimized_runs))) for optimized_run in optimized_runs: for strategy in optimized_run: print(strategy.params.fast) try: print(strategy.analyzers.trades.get_analysis()) print('---------------------------------------') except: print('not enough data...')but I am still unable to access the other analyzers results - it says there is not enough data
run begin optcallback executed optcallback executed optcallback executed runs completed: 3 17 AutoOrderedDict([('total', AutoOrderedDict([('total', 10), ('open', 1), ('closed', 9)])), ('streak', AutoOrderedDict([('won', AutoOrderedDict([('current', 0), ('longest', 4)])), ('lost', AutoOrderedDict([('current', 2), ('longest', 2)]))])), ('pnl', AutoOrderedDict([('gross', AutoOrderedDict([('total', 639.347), ('average', 71.03855555555555)])), ('net', AutoOrderedDict([('total', 273.2601249999998), ('average', 30.362236111111088)]))])), ('won', AutoOrderedDict([('total', 5), ('pnl', AutoOrderedDict([('total', 511.07400000000007), ('average', 102.21480000000001), ('max', 228.05882499999996)]))])), ('lost', AutoOrderedDict([('total', 4), ('pnl', AutoOrderedDict([('total', -237.81387500000028), ('average', -59.45346875000007), ('max', -204.41035000000005)]))])), ('long', AutoOrderedDict([('total', 9), ('pnl', AutoOrderedDict([('total', 273.2601249999998), ('average', 30.362236111111088), ('won', AutoOrderedDict([('total', 511.07400000000007), ('average', 102.21480000000001), ('max', 228.05882499999996)])), ('lost', AutoOrderedDict([('total', -237.81387500000028), ('average', -59.45346875000007), ('max', -204.41035000000005)]))])), ('won', 5), ('lost', 4)])), ('short', AutoOrderedDict([('total', 0), ('pnl', AutoOrderedDict([('total', 0.0), ('average', 0.0), ('won', AutoOrderedDict([('total', 0.0), ('average', 0.0), ('max', 0.0)])), ('lost', AutoOrderedDict([('total', 0.0), ('average', 0.0), ('max', 0.0)]))])), ('won', 0), ('lost', 0)])), ('len', AutoOrderedDict([('total', 297), ('average', 33.0), ('max', 67), ('min', 13), ('won', AutoOrderedDict([('total', 142), ('average', 28.4), ('max', 48), ('min', 13)])), ('lost', AutoOrderedDict([('total', 155), ('average', 38.75), ('max', 67), ('min', 20)])), ('long', AutoOrderedDict([('total', 297), ('average', 33.0), ('max', 67), ('min', 13), ('won', AutoOrderedDict([('total', 142), ('average', 28.4), ('max', 48), ('min', 13)])), ('lost', AutoOrderedDict([('total', 155), ('average', 38.75), ('max', 67), ('min', 20)]))])), ('short', AutoOrderedDict([('total', 0), ('average', 0.0), ('max', 0), ('min', 2147483647), ('won', AutoOrderedDict([('total', 0), ('average', 0.0), ('max', 0), ('min', 2147483647)])), ('lost', AutoOrderedDict([('total', 0), ('average', 0.0), ('max', 0), ('min', 2147483647)]))]))]))]) --------------------------------------- 18 AutoOrderedDict([('total', AutoOrderedDict([('total', 0)]))]) --------------------------------------- 19 AutoOrderedDict([('total', AutoOrderedDict([('total', 0)]))]) --------------------------------------I know the other values do have results and trades if I set those first, I just dont know what I need to do different to access them?
-
It appears that your second and third trade analyzer have no trades. Is there a reason why there would be no trades with those settings?
-
@run-out said in Beginner optimizer and analyzer help:
It appears that your second and third trade analyzer have no trades. Is there a reason why there would be no trades with those settings?
No they definately should have trades - I have adjusted the parameters (the run above was 17,20) to 18,20 and now the results are:
18 AutoOrderedDict([('total', AutoOrderedDict([('total', 10), ('open', 1), ('closed', 9)])), ('streak', AutoOrderedDict([('won', AutoOrderedDict([('current', 0), ('longest', 4)])), ('lost', AutoOrderedDict([('current', 2), ('longest', 2)]))])), ('pnl', AutoOrderedDict([('gross', AutoOrderedDict([('total', 684.163), ('average', 76.01811111111111)])), ('net', AutoOrderedDict([('total', 323.06872499999986), ('average', 35.89652499999998)]))])), ('won', AutoOrderedDict([('total', 5), ('pnl', AutoOrderedDict([('total', 458.521875), ('average', 91.704375), ('max', 199.78830000000002)]))])), ('lost', AutoOrderedDict([('total', 4), ('pnl', AutoOrderedDict([('total', -135.45315000000014), ('average', -33.863287500000034), ('max', -52.87622500000009)]))])), ('long', AutoOrderedDict([('total', 9), ('pnl', AutoOrderedDict([('total', 323.06872499999986), ('average', 35.89652499999998), ('won', AutoOrderedDict([('total', 458.521875), ('average', 91.704375), ('max', 199.78830000000002)])), ('lost', AutoOrderedDict([('total', -135.45315000000014), ('average', -33.863287500000034), ('max', -52.87622500000009)]))])), ('won', 5), ('lost', 4)])), ('short', AutoOrderedDict([('total', 0), ('pnl', AutoOrderedDict([('total', 0.0), ('average', 0.0), ('won', AutoOrderedDict([('total', 0.0), ('average', 0.0), ('max', 0.0)])), ('lost', AutoOrderedDict([('total', 0.0), ('average', 0.0), ('max', 0.0)]))])), ('won', 0), ('lost', 0)])), ('len', AutoOrderedDict([('total', 259), ('average', 28.77777777777778), ('max', 48), ('min', 14), ('won', AutoOrderedDict([('total', 173), ('average', 34.6), ('max', 48), ('min', 18)])), ('lost', AutoOrderedDict([('total', 86), ('average', 21.5), ('max', 30), ('min', 14)])), ('long', AutoOrderedDict([('total', 259), ('average', 28.77777777777778), ('max', 48), ('min', 14), ('won', AutoOrderedDict([('total', 173), ('average', 34.6), ('max', 48), ('min', 18)])), ('lost', AutoOrderedDict([('total', 86), ('average', 21.5), ('max', 30), ('min', 14)]))])), ('short', AutoOrderedDict([('total', 0), ('average', 0.0), ('max', 0), ('min', 2147483647), ('won', AutoOrderedDict([('total', 0), ('average', 0.0), ('max', 0), ('min', 2147483647)])), ('lost', AutoOrderedDict([('total', 0), ('average', 0.0), ('max', 0), ('min', 2147483647)]))]))]))]) --------------------------------------- 19 AutoOrderedDict([('total', AutoOrderedDict([('total', 0)]))])So it is only getting the results for the first run - eventhough 18 was second in the first range and gave no values but when it was first in the range it did give values.
-
Could you post your full code please?
-
Here is the full code with everything
from __future__ import (absolute_import, division, print_function, unicode_literals) import datetime # For datetime objects import os.path # To manage paths import backtrader as bt import backtrader.indicators as btind class BBands(bt.Strategy): params = (('BBandsperiod', 20),('logging',True)) def log(self, txt, dt=None): ''' Logging function fot this strategy''' if self.params.logging: dt = dt or self.datas[0].datetime.date(0) print('%s, %s' % (dt.isoformat(), txt)) def __init__(self): # Keep a reference to the "close" line in the data[0] dataseries self.dataclose = self.datas[0].close # To keep track of pending orders and buy price/commission self.order = None self.buyprice = None self.buycomm = None self.redline = None self.blueline = None # Add a BBand indicator self.bband = bt.indicators.BBands(self.datas[0], period=self.params.BBandsperiod) def notify_order(self, order): if order.status in [order.Submitted, order.Accepted]: # Buy/Sell order submitted/accepted to/by broker - Nothing to do return # Check if an order has been completed # Attention: broker could reject order if not enougth cash if order.status in [order.Canceled, order.Margin]: if order.isbuy(): self.log( 'BUY FAILED, Cancelled or Margin' ) self.log if order.status in [order.Completed, order.Canceled, order.Margin]: if order.isbuy(): self.log( 'BUY EXECUTED, Price: %.2f, Cost: %.2f, Comm %.2f' % (order.executed.price, order.executed.value, order.executed.comm)) self.buyprice = order.executed.price self.buycomm = order.executed.comm else: # Sell self.log('SELL EXECUTED, Price: %.2f, Cost: %.2f, Comm %.2f' % (order.executed.price, order.executed.value, order.executed.comm)) self.bar_executed = len(self) # Write down: no pending order self.order = None def notify_trade(self, trade): if not trade.isclosed: return self.log('OPERATION PROFIT, GROSS %.2f, NET %.2f' % (trade.pnl, trade.pnlcomm)) def next(self): # Simply log the closing price of the series from the reference #self.log('Close, %.2f' % self.dataclose[0]) # Check if an order is pending ... if yes, we cannot send a 2nd one if self.order: return if self.dataclose < self.bband.lines.bot and not self.position: self.redline = True if self.dataclose > self.bband.lines.top and self.position: self.blueline = True if self.dataclose > self.bband.lines.mid and not self.position and self.redline: # BUY, BUY, BUY!!! (with all possible default parameters) self.log('BUY CREATE, %.2f' % self.dataclose[0]) # Keep track of the created order to avoid a 2nd order self.order = self.buy() if self.dataclose > self.bband.lines.top and not self.position: # BUY, BUY, BUY!!! (with all possible default parameters) self.log('BUY CREATE, %.2f' % self.dataclose[0]) # Keep track of the created order to avoid a 2nd order self.order = self.buy() if self.dataclose < self.bband.lines.mid and self.position and self.blueline: # SELL, SELL, SELL!!! (with all possible default parameters) self.log('SELL CREATE, %.2f' % self.dataclose[0]) self.blueline = False self.redline = False # Keep track of the created order to avoid a 2nd order self.order = self.sell() def optimizer_callbacks(cb): print('optcallback executed') if __name__ == '__main__': print('Starting up Cerebro...') cerebro = bt.Cerebro(optreturn=False) cerebro.addanalyzer(bt.analyzers.SharpeRatio, _name='sharpe_ratio') cerebro.addanalyzer(bt.analyzers.TradeAnalyzer, _name="trades") cerebro.optcallback(optimizer_callbacks) cerebro.broker.setcommission(commission=0.025) # 2.5% commission cerebro.optstrategy(BBands, BBandsperiod=range(13,15), logging=False ) cerebro.addsizer(bt.sizers.SizerFix, stake=0.1) start_portfolio_value = 10000.0 cerebro.broker.setcash(start_portfolio_value) #print('Starting Portfolio Value: %.2f' % cerebro.broker.getvalue()) data = bt.feeds.CCXT(exchange='binance', symbol='BTC/USDT', name="btc_usd_tick", # Do not pass values before this date fromdate=datetime.datetime(2019, 1, 1), # Do not pass values before this date todate=datetime.datetime(2020, 5, 1), timeframe=bt.TimeFrame.Days, compression=1, ) cerebro.adddata(data) print('run begin') optimized_runs = cerebro.run(maxcpus=1) print('runs completed: ' + str(len(optimized_runs))) for optimized_run in optimized_runs: for strategy in optimized_run: print(strategy.params.BBandsperiod) try: print(strategy.analyzers.trades.get_analysis()) print(strategy.analyzers.sharpe_ratio.get_analysis()) print('---------------------------------------') except: print('not enough data...')with my output:
Starting up Cerebro... run begin optcallback executed optcallback executed runs completed: 2 13 AutoOrderedDict([('total', AutoOrderedDict([('total', 11), ('open', 1), ('closed', 10)])), ('streak', AutoOrderedDict([('won', AutoOrderedDict([('current', 0), ('longest', 4)])), ('lost', AutoOrderedDict([('current', 1), ('longest', 1)]))])), ('pnl', AutoOrderedDict([('gross', AutoOrderedDict([('total', 412.5020000000002), ('average', 41.25020000000002)])), ('net', AutoOrderedDict([('total', -2.847749999999877), ('average', -0.2847749999999877)]))])), ('won', AutoOrderedDict([('total', 7), ('pnl', AutoOrderedDict([('total', 524.1731250000003), ('average', 74.88187500000004), ('max', 228.45857500000002)]))])), ('lost', AutoOrderedDict([('total', 3), ('pnl', AutoOrderedDict([('total', -527.0208750000002), ('average', -175.67362500000004), ('max', -267.76225000000005)]))])), ('long', AutoOrderedDict([('total', 10), ('pnl', AutoOrderedDict([('total', -2.847749999999877), ('average', -0.2847749999999877), ('won', AutoOrderedDict([('total', 524.1731250000003), ('average', 74.88187500000004), ('max', 228.45857500000002)])), ('lost', AutoOrderedDict([('total', -527.0208750000002), ('average', -175.67362500000004), ('max', -267.76225000000005)]))])), ('won', 7), ('lost', 3)])), ('short', AutoOrderedDict([('total', 0), ('pnl', AutoOrderedDict([('total', 0.0), ('average', 0.0), ('won', AutoOrderedDict([('total', 0.0), ('average', 0.0), ('max', 0.0)])), ('lost', AutoOrderedDict([('total', 0.0), ('average', 0.0), ('max', 0.0)]))])), ('won', 0), ('lost', 0)])), ('len', AutoOrderedDict([('total', 305), ('average', 30.5), ('max', 55), ('min', 13), ('won', AutoOrderedDict([('total', 173), ('average', 24.714285714285715), ('max', 55), ('min', 13)])), ('lost', AutoOrderedDict([('total', 132), ('average', 44.0), ('max', 50), ('min', 37)])), ('long', AutoOrderedDict([('total', 305), ('average', 30.5), ('max', 55), ('min', 13), ('won', AutoOrderedDict([('total', 173), ('average', 24.714285714285715), ('max', 55), ('min', 13)])), ('lost', AutoOrderedDict([('total', 132), ('average', 44.0), ('max', 50), ('min', 37)]))])), ('short', AutoOrderedDict([('total', 0), ('average', 0.0), ('max', 0), ('min', 2147483647), ('won', AutoOrderedDict([('total', 0), ('average', 0.0), ('max', 0), ('min', 2147483647)])), ('lost', AutoOrderedDict([('total', 0), ('average', 0.0), ('max', 0), ('min', 2147483647)]))]))]))]) OrderedDict([('sharperatio', -0.46681509014083633)]) --------------------------------------- 14 AutoOrderedDict([('total', AutoOrderedDict([('total', 0)]))]) OrderedDict([('sharperatio', None)]) --------------------------------------- -
I ran your code although using local ES mini data since I don't have the CCXT version installed. I did not have a problem and I received two results. This suggest then:
a) CCXT doesn't like you pulling data so fast... try installing the data locally or...
b) your installed version of Backtrader is doing something funny.Perhaps try running your code with some local data and see what happens.
-
@dracount Also... Not sure if this will help. but consider adding in optdatas=True when instatiating cerebro...
cerebro = bt.Cerebro(optreturn=False, optdatas=True)Read here in the docs. This might help if you are having a data loading problem.
-
Thank you - I have managed to get some historical data and test it out and it works as expected.
Is there a way to save the data that is used in bt.feeds.CCXT ?
-
Yes there is a way. You can create an analyzer that will track the ohlcv of the backtest, and then save the output after the test.
Here is the one I sometimes use.
class OHLCV(bt.analyzers.Analyzer): """This analyzer reports the OHLCV of each of datas. Params: - timeframe (default: ``None``) If ``None`` then the timeframe of the 1st data of the system will be used - compression (default: ``None``) Only used for sub-day timeframes to for example work on an hourly timeframe by specifying "TimeFrame.Minutes" and 60 as compression If ``None`` then the compression of the 1st data of the system will be used Methods: - get_analysis Returns a dictionary with returns as values and the datetime points for each return as keys """ def start(self): tf = min(d._timeframe for d in self.datas) self._usedate = tf >= bt.TimeFrame.Days self.rets = {} def next(self): self.rets[self.data.datetime.datetime()] = [ self.datas[0].open[0], self.datas[0].high[0], self.datas[0].low[0], self.datas[0].close[0], self.datas[0].volume[0], ] def get_analysis(self): return self.retsAdd it to cerebro when you are starting cerebro up.
cerebro.addanalyzer(OHLCV, _name="ohlcv")Then when you run cerebro, if:
results = cerebro.run()Then you can get your data from results.
results[0].analyzers.getbyname("ohlcv").get_analysis()This will yield a dictionary with the data. The datetime will be the key of the dictionary, and the ohlcv the values in a list.
