For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/
How is Getvalue() Calculated?
-
Hi there,
How exactly is getvalue() calculated? I understand it is the sum of everything you have invested in market + cash?
This should then mean that getvalue() -getcash() = sum of current value of everything invested in market (stocks).
However, when I add up the value of my stocks held, it does not equal this.
See my example print out below (totalwealth = getvalue(), invested = getvalue() - cash()), my invested value says only $27,000. But when you add up my stocks held, its more like $66,000...
2012-12-28, ORDER CANCELLED - TP: ABC, Trade Ref: 1037 2012-12-28, SELL EXECUTED - Closed Long Pos: SUN, Ref: 1039, Price: 9.97, PNL: 123.47, Purchase Cost: 8379.68, Comm 10.24 2012-12-28, SELL EXECUTED - Closed Long Pos: CTX, Ref: 1040, Price: 19.10, PNL: 520.93, Purchase Cost: 8284.17, Comm 10.61 2012-12-28, BUY EXECUTED - Entered Long Pos: TTS, Ref: 1041, Price: 3.01, Cost: 8298.57, Size: 2757.00, Comm 10.00 2012-12-28, BUY EXECUTED - Entered Long Pos: CHC, Ref: 1042, Price: 3.32, Cost: 8373.04, Size: 2522.00, Comm 10.09 2012-12-28, OPERATION PROFIT: SUN, GROSS 123.47, NET 103.13 2012-12-28, CANCELLING STOP-LOSS: SUN, Ref: 216 2012-12-28, CANCELLING TAKE-PROFIT: SUN, Ref: 216 2012-12-28, OPERATION PROFIT: CTX, GROSS 520.93, NET 500.34 2012-12-28, CANCELLING STOP-LOSS: CTX, Ref: 211 2012-12-28, CANCELLING TAKE-PROFIT: CTX, Ref: 211 2012-12-28, Stock held: TTS, Close: 3.01, score: 0.93, Score Yest: 0.92, Posn: 2757.00, hold days 1 2012-12-28, CREATE SL: TTS, Trigger Price: 2.71 2012-12-28, CREATE TP: TTS, Trigger Price: 9.03 2012-12-28, Stock held: MTS, Close: 3.38, score: 1.00, Score Yest: 0.99, Posn: 2569.00, hold days 6 2012-12-28, Stock held: FLT, Close: 27.03, score: 0.99, Score Yest: 1.00, Posn: 302.00, hold days 27 2012-12-28, CREATE BUY: ABC, Close: 3.11, score: 0.94 2012-12-28, Stock held: MYR, Close: 2.01, score: 1.00, Score Yest: 1.00, Posn: 4221.00, hold days 25 2012-12-28, Stock held: CHC, Close: 3.31, score: 0.95, Score Yest: 0.92, Posn: 2522.00, hold days 1 2012-12-28, CREATE SL: CHC, Trigger Price: 2.98 2012-12-28, CREATE TP: CHC, Trigger Price: 9.93 2012-12-28, Stock held: MQA, Close: 1.68, score: 0.97, Score Yest: 0.97, Posn: 5076.00, hold days 7 2012-12-28, CREATE BUY: SMX, Close: 4.64, score: 0.91 2012-12-28, Stocks Held: 6, Total Wealth: 104603, Invested: 27280, Cash-On-Hand (pre-today's buys): 77323 2012-12-28, Ending Wealth: 104603, Invested: 27280, Cash-On-Hand: 77323,Perhaps I am interpreting the
getvalue()definition wrong? Otherwise, any ideas why these numbers dont equate?Thank you
-
Not much can be really said about an isolated log. You mention that the value of your stocks is more like $66,000 but why is it so is also not shown.
You may be short on some, you may have your own commission scheme objects changing the calculation value, you may have tagged some as a future and only the margin is taken into account ...
-
@backtrader, given your response, I am assuming my assumption of the definition of getvalue() is correct.
Given that I am trying to determine what the value of my invested stocks is. I was hoping to use getvalue() - getcash(), which is what this log line does:
2012-12-28, Ending Wealth: 104603, Invested: 27280, Cash-On-Hand: 77323However when you add up each of the
stock held:lines; i.e.2012-12-28, Stock held: TTS, Close: 3.01, score: 0.93, Score Yest: 0.92, Posn: 2757.00, hold days 1 2012-12-28, Stock held: MTS, Close: 3.38, score: 1.00, Score Yest: 0.99, Posn: 2569.00, hold days 6 2012-12-28, Stock held: FLT, Close: 27.03, score: 0.99, Score Yest: 1.00, Posn: 302.00, hold days 27 2012-12-28, Stock held: MYR, Close: 2.01, score: 1.00, Score Yest: 1.00, Posn: 4221.00, hold days 25 2012-12-28, Stock held: CHC, Close: 3.31, score: 0.95, Score Yest: 0.92, Posn: 2522.00, hold days 1 2012-12-28, Stock held: MQA, Close: 1.68, score: 0.97, Score Yest: 0.97, Posn: 5076.00, hold days 7You get $66,000 (Close*Posn for each line). Hence my question. Why would
getvalue() - getcash()not equal the summation of stocks held? -
That's still an isolated log. What you are printing and from where and what the actual operations are is unclear.
To start with:
Sum(Close * Posn)is nowhere near66000, which is apparently the basis of the problem.Things may be wrong in the platform and contain 1 million bugs, but any kind of statement should at least try to show why it is wrong.
-
Hi @backtrader, appologies for my maths error, must have been a different run... :-)
As requested, here is my full script. Please ignore references to "Score" (i have set to "1" and commented out irrelevant bits).
The issue remains though, at the end of the analysis 14 stocks are held, but the "invested" amount in this case is -$95k...!
My script, log and plot are below.
I look forward to your advise on how to fix this! I would like to use Stop-Loss / Take-Profit functionality, however I will need to sort out this negative stock investment issue for multiple stocks!
Thank you kindly,
CWEfrom __future__ import (absolute_import, division, print_function, unicode_literals) import argparse import pandas as pd import numpy as np import datetime from scipy.stats import norm import math import backtrader as bt import backtrader.indicators as btind import backtrader.feeds as btfeeds import glob import ntpath def parse_args(): parser = argparse.ArgumentParser(description='MultiData Strategy') parser.add_argument('--Custom_Alg', default=False, # True OR False... NOT 'True' OR 'False' help='True = Use custom alg') parser.add_argument('--SLTP_On', default=True, # True OR False... NOT 'True' OR 'False' help='True = Use Stop-Loss & Take-Profit Orders, False = do NOT use SL & TP orders') parser.add_argument('--stoploss', action='store', default=0.10, type=float, help=('sell a long position if loss exceeds')) parser.add_argument('--takeprofit', action='store', default=2.00, type=float, help=('Exit a long position if profit exceeds')) parser.add_argument('--data0', '-d0', default=r'T:\PD_Stock_Data\DBLOAD', help='Directory of CSV data source') parser.add_argument('--fromdate', '-f', default='2012-01-01', help='Starting date in YYYY-MM-DD format') parser.add_argument('--todate', '-t', default='2013-12-31', help='Ending date in YYYY-MM-DD format') parser.add_argument('--limitpct', action='store', default=0.005, type=float, help=('For buying at LIMIT, this will only purchase if the price is less than (1-limitpct)*Closing price')) parser.add_argument('--validdays', action='store', default=30, type=int, help=('The number of days which a buy order remains valid')) parser.add_argument('--sellscore', action='store', default=-0.91, type=float, help=('Max score for a sell')) parser.add_argument('--marketindex', default='XJO', help='XAO = All Ords, XJO = ASX200') parser.add_argument('--startingcash', default=100000, type=int, help='Starting Cash') parser.add_argument('--minholddays', default=3, type=int, help='Dont exit a market position until have held stock for at least this many days (excl. Stop-Loss and TP). May assist stopping exiting/cancelling orders when they are still being accepted by broker (i.e. day after entering mkt).') parser.add_argument('--pctperstock', action='store', #0.083 = 1/12... i.e. a portfolio of up to 12 stocks default=0.083, type=float, #i.e. 10% portfolio value in each stock help=('Pct of portfolio starting cash to invest in each stock purchase')) parser.add_argument('--maxpctperstock', action='store', default=0.20, type=float, help=('Max pct portfolio to invest in any porticular stock')) parser.add_argument('--mintrade', default=1000, type=float, help='Smallest dollar value to invest in a stock (if cash level below amount required for pctperstock)') parser.add_argument('--tradefee', default=10.0, type=float, help='CMC Markets Fee per stock trade (BUY OR SELL)') parser.add_argument('--alg_buyscore', #only used if Custom_Alg ==True action='store', # 0.91884558 default=0.91, type=float, help=('Min score for a buy')) return parser.parse_args() #Excel sheet with ASX200 index constituents (used for chosing stocks to analyse in Backtrader) def LoadIndicies(Excel_Path, Excel_Sheet): # Load ASX200 Excel File ASX200 = pd.read_excel(Excel_Path, sheetname=Excel_Sheet) Index_Constituents = ASX200.to_dict(orient='list') for key, value in Index_Constituents.items(): Index_Constituents[key] = [x for x in value if str(x) != 'nan'] # drop any "blank" (NaN) tickers from the index constituents table IndexDates = sorted(Index_Constituents.keys()) IndexDates.append(datetime.datetime.now().strftime("%Y-%m-%d")) # ordered list of the Index constituent Dates, with todays date at the end return Index_Constituents, IndexDates def LoadStockData(CSV_path=None): args = parse_args() if CSV_path is None: raise RuntimeError("no stock folder directory specifed.") allFiles = glob.glob(CSV_path + "/*.csv") Stocks = {} # Create a DICTIONARY object to store the entire contents of all dataframes, allows for easy reference to / looping through dataframes by a string of their name, i.e. : 'CSL' for file_ in allFiles: name = ntpath.basename(file_[:-4]) # Set DF name = basename (not path) of the CSV. [:-4] gets rid of the '.CSV' extention. Stocks[name] = pd.read_csv(file_, index_col='Date', parse_dates=True, header=0) return Stocks class StockLoader(btfeeds.PandasData): args = parse_args() params = ( ('openinterest', None), # None= column not present ('TOTAL_SCORE', -1)) # -1 = autodetect position or case-wise equal name if args.Custom_Alg == True: lines = ('TOTAL_SCORE',) if args.Custom_Alg == True: datafields = btfeeds.PandasData.datafields + (['TOTAL_SCORE']) else: datafields = btfeeds.PandasData.datafields class st(bt.Strategy): args = parse_args() params = ( #NB: self.p = self.params ('printlog', True), ) def log(self, txt, dt=None, doprint=False): if self.p.printlog or doprint: dt = dt or self.datas[0].datetime.date(0) print('%s - %s' % (dt.isoformat(), txt)) def __init__(self): self.order = {} #Order: market entry self.order_out = {} #Order: market exit self.order_sl = {} #Stop-Loss self.order_tp = {} #Take-Profit self.bar_executed = {} self.sma_short = {} self.sma_long = {} for i, d in enumerate(d for d in self.datas): self.order[d._name] = None self.order_out[d._name] = None self.order_sl[d._name] = None self.order_tp[d._name] = None self.bar_executed[d._name] = None self.sma_short[d._name] = bt.indicators.SimpleMovingAverage(d, period=42) #np.round(pd.rolling_mean(d.Close, window=42),2) self.sma_long[d._name] = bt.indicators.SimpleMovingAverage(d, period=252) #np.round(pd.rolling_mean(d.Close, window=252),2) def start(self): pass def notify_trade(self, trade): #NB: "print(Trade.__dict__)" if trade.isclosed: #Market Position exited self.log('OPERATION PROFIT: %s, Gross: %2f, Net: %2f' %(trade.data._name, trade.pnl, trade.pnlcomm)) def notify_order(self, order): #used to check order info ord = self.order[order.data._name] ord_out = self.order_out[order.data._name] ord_sl = self.order_sl[order.data._name] ord_tp = self.order_tp[order.data._name] if order.status in [order.Submitted, order.Accepted]: return #do nothing elif order.status in [order.Margin, order.Rejected, order.Completed, order.Cancelled]: if order.isbuy(): buysell = 'BUY' elif order.issell(): buysell = 'SELL' if ord and order == ord: type = 'Enter' self.bar_executed[order.data._name] = len(order.data) #length of dataframe when enter market (used to calc days held) elif ord_out and order == ord_out: type = 'Exit' elif ord_sl and order == ord_sl: type = 'Stop-Loss' elif ord_tp and order == ord_tp: type = 'Take-Profit' self.log('%s %s: %s, Type: %s, Ref: %s, Price: %.2f, Cost: %.2f, Size: %.2f, Comm %.2f' %(buysell, order.Status[order.status], order.data._name, type, order.ref, order.executed.price, order.executed.value, order.executed.size, order.executed.comm)) if not order.alive():# indicate no order is pending, allows new orders if ord and order == ord: self.order[order.data._name] = None elif ord_out and order == ord_out: self.order_out[order.data._name] = None elif ord_sl and order == ord_sl: self.order_sl[order.data._name] = None elif ord_tp and order == ord_tp: self.order_tp[order.data._name] = None def prenext(self): #overrides PRENEXT() so that the "NEXT()" calculations runs regardless of when each stock data date range starts. self.next() def next(self): today = self.getdatabyname(args.marketindex).datetime.date(0) weekday = today.isoweekday() #Monday = 1, Sunday = 7 if weekday in range(1,8): # analyse on all weekdays (MONDAY to SUNDAY) num_long = 0 #number long stocks #IdealLongPortf = pd.DataFrame(columns=('Stock', 'Score','Close','Current Position', 'Ideal Position', 'Pos Delta Value', 'Go NoGo')) #ideal stock positions at end of each next() iteration for i, d in enumerate(d for d in self.datas if len(d) and d._name != args.marketindex): # Loop through Universe of Stocks. "If Len(d)" is used to check that all datafeeds have delivered values. as if using minute data, some may have had many minutes, 500, and another may not have 1 record yet (if its still on daily) position = self.broker.getposition(d) positiondol = float(self.broker.getposition(d).size*d.close[0]) cash = self.broker.getcash() #total available cash if position.size == 0 \ and self.order[d._name] is None \ and self.order_out[d._name] is None \ and self.order_sl[d._name] is None \ and self.order_tp[d._name] is None \ and d.close[0] > 0 \ and self.sma_short[d._name][0] > self.sma_long[d._name][0]: #and d.lines.TOTAL_SCORE[0] >= args.alg_buyscore: #IdealLongPortf.append([d._name, d.lines.TOTAL_SCORE[0], d.close[0], position.size, np.NaN, np.NaN,np.NaN]) buylimit = d.close[0]*(1-args.limitpct) if args.SLTP_On == True: stop_loss = d.close[0]*(1.0 - args.stoploss) take_profit = d.close[0]*(1.0 + args.takeprofit) o1 = self.buy(data = d, exectype=bt.Order.Limit, price=buylimit, valid=today + datetime.timedelta(days=args.validdays), transmit=False) self.log('CREATE BUY: %s, Close: %2f, Buy @: %2f, Oref: %i, Score: %2f' %(d._name, d.close[0], buylimit, o1.ref, 1)) #d.lines.TOTAL_SCORE[0])) o2 = self.sell(data = d, size = o1.size, # could be an issue with re-balancing!!! exectype=bt.Order.Stop, price=stop_loss, parent=o1, transmit=False) self.log('CREATE Stop-Loss: %s, Sell Stop @: %2f, Oref: %i' %(d._name, stop_loss, o2.ref)) o3 = self.sell(data = d, size = o1.size, exectype=bt.Order.Limit, price=take_profit, parent=o1, transmit=True) self.log('CREATE Take-Profit: %s, Sell Limit @: %2f, Oref: %i' %(d._name, take_profit, o3.ref)) self.order[d._name] = o1 self.order_sl[d._name] = o2 self.order_tp[d._name] = o3 else: o1 = self.buy(data = d, exectype=bt.Order.Limit, price=buylimit, valid=today + datetime.timedelta(days=args.validdays)) self.log('CREATE BUY: %s, Close: %2f, Buy @: %2f, Oref: %i, Score: %2f' %(d._name, d.close[0], buylimit, o1.ref, 1)) #d.lines.TOTAL_SCORE[0])) self.order[d._name] = o1 elif position.size > 0: # Currently LONG daysheld = len(d) - self.bar_executed[d._name] + 1 self.log('Stock Held: %s, Close: %2f, Posn: %i, Posn($): %2f, Hold Days: %i, Score: %2f, Score Yest: %2f' %(d._name, d.close[0], position.size, positiondol, daysheld, 1, #d.lines.TOTAL_SCORE[0], 1)) #d.lines.TOTAL_SCORE[-1])) num_long +=1 if self.order[d._name] is None \ and self.order_out[d._name] is None \ and daysheld >= args.minholddays \ and self.sma_short[d._name][0] < self.sma_long[d._name][0]: #and d.lines.TOTAL_SCORE[0] < args.alg_buyscore: self.log('CLOSING LONG POSITION: %s, Close: %2f, Score: %2f' %(d._name, d.close[0], 1)) #d.lines.TOTAL_SCORE[0])) # cancel SL/TP 1st to avoid possibility of duplicate exit orders if self.order_sl[d._name]: self.log('CANCELLING SL & TP for: %s, SL ref: %i, TP ref: %i' %(d._name,self.order_sl[d._name].ref, self.order_tp[d._name].ref)) self.broker.cancel(self.order_sl[d._name]) #this automatically cancels the TP too #if self.order_tp[d._name]: # self.log('CANCELLING TP for: %s, ref: %s' %(d._name,self.order_tp[d._name].ref)) # self.broker.cancel(self.order_tp[d._name]) o4 = self.close(data=d) self.order_out[d._name] = o4 totalwealth = self.broker.getvalue() cash = self.broker.getcash() invested = totalwealth - cash self.log("Stocks Held: %s, Total Wealth: %i, Invested: %i, Cash-On-Hand: %i" %(str(num_long), totalwealth, invested, cash)) def stop(self): pass class PortfolioSizer(bt.Sizer): def _getsizing(self, comminfo, cash, data, isbuy): args = parse_args() position = self.broker.getposition(data) price = data.close[0] investment = args.startingcash * args.pctperstock if cash < investment: investment = max(cash,args.mintrade) # i.e. never invest less than the "mintrade" $value qty = math.floor(investment/price) # This method returns the desired size for the buy/sell operation if isbuy: # if buying if position.size < 0: # if currently short, buy the amount which are short to close out trade. return -position.size elif position.size > 0: return 0 # dont buy if already hold else: return qty # num. stocks to LONG if not isbuy: # if selling.. if position.size < 0: return 0 # dont sell if already SHORT elif position.size > 0: return position.size # currently Long... sell what hold else: return qty # num. stocks to SHORT def RunStrategy(): args = parse_args() cerebro = bt.Cerebro() cerebro.addstrategy(st) # strats = cerebro.optstrategy(st,maperiod=range(10, 31)) #date range to backtest tradingdates = Stocks[args.marketindex].loc[ (Stocks[args.marketindex].index>=datetime.datetime.strptime(args.fromdate, "%Y-%m-%d")) & (Stocks[args.marketindex].index<datetime.datetime.strptime(args.todate, "%Y-%m-%d")) ] #Load 200 stocks into Backtrader (specified in the Index_constituents list) for ticker in Index_Constituents[IndexDates[3]]: datarange = Stocks[ticker].loc[ (Stocks[ticker].index>=datetime.datetime.strptime(args.fromdate, "%Y-%m-%d")) & (Stocks[ticker].index<datetime.datetime.strptime(args.todate, "%Y-%m-%d")) ] #REINDEX to make sure the stock has the exact same trading days as the MARKET INDEX. Reindex ffill doesn't fill GAPS. Therefore also apply FILLNA datarange.reindex(tradingdates.index, method='ffill').fillna(method='ffill',inplace=True) data = StockLoader(dataname=datarange) data.plotinfo.plot=False cerebro.adddata(data, name=ticker) data = btfeeds.PandasData(dataname=tradingdates, openinterest=None) #load market index (for date referencing) cerebro.adddata(data, name=args.marketindex) #cerebro.addanalyzer(CurrentBuysAnalyzer, ) cerebro.addanalyzer(bt.analyzers.TradeAnalyzer, ) #length of holds etc cerebro.addanalyzer(bt.analyzers.TimeReturn, timeframe=bt.TimeFrame.Years) cerebro.addanalyzer(bt.analyzers.SharpeRatio, timeframe=bt.TimeFrame.Years, riskfreerate=0.03, annualize=True) cerebro.addanalyzer(bt.analyzers.SQN) cerebro.addanalyzer(bt.analyzers.DrawDown, ) cerebro.broker.setcash(args.startingcash) # set starting cash cerebro.addsizer(PortfolioSizer) commission = float(args.tradefee/(args.pctperstock*args.startingcash)) print("The Commission rate is: %0.5f" % (commission)) cerebro.broker.setcommission(commission=commission) cerebro.run(runonce=False, writer=True) cerebro.plot(volume=False, stdstats=False) ''' zdown=True: Rotation of the date labes on the x axis stdstats=False: Disable the standard plotted observers numfigs=1: Plot on one chart ''' if __name__ == '__main__': # if python is running this script (module) as the main program, then __name__ == __main__, and this block of code will run. # However, if another script (module) is IMPORTING this script (module), this block of code WILL NOT RUN, but the above functions can be called. args = parse_args() t1 = datetime.datetime.now() print('Processing Commenced: {}'.format(str(t1))) #Load stocks from local drive for analysis Stocks = LoadStockData(args.data0) t2 = datetime.datetime.now() # Dictionary of Index Constituents (and their stock dataframes) Index_Constituents, IndexDates = LoadIndicies( Excel_Path='T:\Google Drive\Capriole\CAPRIOLEPROCESSOR\TickerUpdater.xlsm', Excel_Sheet='ASX200_Const_Updated') print('ASX200 constituent list date: {}'.format(IndexDates[3])) if args.Custom_Alg == True: initiate() t3 = datetime.datetime.now() RunStrategy() t4 = datetime.datetime.now() #TIMER print('Run-time - TOTAL: {0}'.format(datetime.datetime.now() - t1)) print('Run-time - Load Data: {0}'.format(t2 - t1)) if t3: print('Run-time - Algorithm: {0}'.format(t3 - t2)) print('Run-time - Strategy Back-test: {0}'.format(t4 - t3)) else: print('Run-time - Strategy Back-test: {0}'.format(t4 - t2))And last few days of log extract (wont let me copy all):
2013-12-20 - Stocks Held: 14, Total Wealth: 159974, Invested: -92641, Cash-On-Hand: 252615 2013-12-23 - Stock Held: WBC, Close: 31.744440, Posn: 319, Posn($): 10126.476360, Hold Days: 250, Score: 1.000000, Score Yest: 1.000000 2013-12-23 - Stock Held: ANZ, Close: 31.950001, Posn: 332, Posn($): 10607.400332, Hold Days: 250, Score: 1.000000, Score Yest: 1.000000 2013-12-23 - Stock Held: WES, Close: 43.412518, Posn: 225, Posn($): 9767.816550, Hold Days: 250, Score: 1.000000, Score Yest: 1.000000 2013-12-23 - Stock Held: WOW, Close: 33.610001, Posn: 281, Posn($): 9444.410281, Hold Days: 250, Score: 1.000000, Score Yest: 1.000000 2013-12-23 - Stock Held: WPL, Close: 38.900002, Posn: 246, Posn($): 9569.400492, Hold Days: 250, Score: 1.000000, Score Yest: 1.000000 2013-12-23 - Stock Held: CSL, Close: 68.300003, Posn: 152, Posn($): 10381.600456, Hold Days: 250, Score: 1.000000, Score Yest: 1.000000 2013-12-23 - Stock Held: DL_FOX, Close: 37.700001, Posn: 363, Posn($): 13685.100363, Hold Days: 250, Score: 1.000000, Score Yest: 1.000000 2013-12-23 - Stock Held: BXB, Close: 9.110000, Posn: 139, Posn($): 1266.290000, Hold Days: 247, Score: 1.000000, Score Yest: 1.000000 2013-12-23 - Stock Held: GMG, Close: 4.790000, Posn: 273, Posn($): 1307.670000, Hold Days: 246, Score: 1.000000, Score Yest: 1.000000 2013-12-23 - Stock Held: DL_AIO, Close: 5.156790, Posn: 1919, Posn($): 9895.880010, Hold Days: 237, Score: 1.000000, Score Yest: 1.000000 2013-12-23 - Stock Held: CGF, Close: 6.050000, Posn: 1959, Posn($): 11851.950000, Hold Days: 226, Score: 1.000000, Score Yest: 1.000000 2013-12-23 - Stock Held: AGO, Close: 1.185000, Posn: 6803, Posn($): 8061.555000, Hold Days: 10, Score: 1.000000, Score Yest: 1.000000 2013-12-23 - Stock Held: CDD, Close: 3.827047, Posn: 2136, Posn($): 8174.572392, Hold Days: 42, Score: 1.000000, Score Yest: 1.000000 2013-12-23 - Stock Held: AOG, Close: 1.950000, Posn: 4761, Posn($): 9283.950000, Hold Days: 40, Score: 1.000000, Score Yest: 1.000000 2013-12-23 - Stocks Held: 14, Total Wealth: 159800, Invested: -92815, Cash-On-Hand: 252615 2013-12-24 - Stock Held: WBC, Close: 31.963230, Posn: 319, Posn($): 10196.270370, Hold Days: 251, Score: 1.000000, Score Yest: 1.000000 2013-12-24 - Stock Held: ANZ, Close: 32.130001, Posn: 332, Posn($): 10667.160332, Hold Days: 251, Score: 1.000000, Score Yest: 1.000000 2013-12-24 - Stock Held: WES, Close: 43.671818, Posn: 225, Posn($): 9826.159050, Hold Days: 251, Score: 1.000000, Score Yest: 1.000000 2013-12-24 - Stock Held: WOW, Close: 33.700001, Posn: 281, Posn($): 9469.700281, Hold Days: 251, Score: 1.000000, Score Yest: 1.000000 2013-12-24 - Stock Held: WPL, Close: 39.099998, Posn: 246, Posn($): 9618.599508, Hold Days: 251, Score: 1.000000, Score Yest: 1.000000 2013-12-24 - Stock Held: CSL, Close: 68.830002, Posn: 152, Posn($): 10462.160304, Hold Days: 251, Score: 1.000000, Score Yest: 1.000000 2013-12-24 - Stock Held: DL_FOX, Close: 37.980000, Posn: 363, Posn($): 13786.740000, Hold Days: 251, Score: 1.000000, Score Yest: 1.000000 2013-12-24 - Stock Held: BXB, Close: 9.210000, Posn: 139, Posn($): 1280.190000, Hold Days: 248, Score: 1.000000, Score Yest: 1.000000 2013-12-24 - Stock Held: GMG, Close: 4.790000, Posn: 273, Posn($): 1307.670000, Hold Days: 247, Score: 1.000000, Score Yest: 1.000000 2013-12-24 - Stock Held: DL_AIO, Close: 5.165837, Posn: 1919, Posn($): 9913.241203, Hold Days: 238, Score: 1.000000, Score Yest: 1.000000 2013-12-24 - Stock Held: CGF, Close: 6.130000, Posn: 1959, Posn($): 12008.670000, Hold Days: 227, Score: 1.000000, Score Yest: 1.000000 2013-12-24 - Stock Held: AGO, Close: 1.180000, Posn: 6803, Posn($): 8027.540000, Hold Days: 11, Score: 1.000000, Score Yest: 1.000000 2013-12-24 - Stock Held: CDD, Close: 3.872812, Posn: 2136, Posn($): 8272.326432, Hold Days: 43, Score: 1.000000, Score Yest: 1.000000 2013-12-24 - Stock Held: AOG, Close: 1.950000, Posn: 4761, Posn($): 9283.950000, Hold Days: 41, Score: 1.000000, Score Yest: 1.000000 2013-12-24 - Stocks Held: 14, Total Wealth: 157471, Invested: -95144, Cash-On-Hand: 252615 2013-12-27 - Stock Held: WBC, Close: 31.913506, Posn: 319, Posn($): 10180.408414, Hold Days: 252, Score: 1.000000, Score Yest: 1.000000 2013-12-27 - Stock Held: ANZ, Close: 32.160000, Posn: 332, Posn($): 10677.120000, Hold Days: 252, Score: 1.000000, Score Yest: 1.000000 2013-12-27 - Stock Held: WES, Close: 43.921139, Posn: 225, Posn($): 9882.256275, Hold Days: 252, Score: 1.000000, Score Yest: 1.000000 2013-12-27 - Stock Held: WOW, Close: 33.709999, Posn: 281, Posn($): 9472.509719, Hold Days: 252, Score: 1.000000, Score Yest: 1.000000 2013-12-27 - Stock Held: WPL, Close: 39.099998, Posn: 246, Posn($): 9618.599508, Hold Days: 252, Score: 1.000000, Score Yest: 1.000000 2013-12-27 - Stock Held: CSL, Close: 68.400002, Posn: 152, Posn($): 10396.800304, Hold Days: 252, Score: 1.000000, Score Yest: 1.000000 2013-12-27 - Stock Held: DL_FOX, Close: 38.590000, Posn: 363, Posn($): 14008.170000, Hold Days: 252, Score: 1.000000, Score Yest: 1.000000 2013-12-27 - Stock Held: BXB, Close: 9.180000, Posn: 139, Posn($): 1276.020000, Hold Days: 249, Score: 1.000000, Score Yest: 1.000000 2013-12-27 - Stock Held: GMG, Close: 4.750000, Posn: 273, Posn($): 1296.750000, Hold Days: 248, Score: 1.000000, Score Yest: 1.000000 2013-12-27 - Stock Held: DL_AIO, Close: 5.138696, Posn: 1919, Posn($): 9861.157624, Hold Days: 239, Score: 1.000000, Score Yest: 1.000000 2013-12-27 - Stock Held: CGF, Close: 6.160000, Posn: 1959, Posn($): 12067.440000, Hold Days: 228, Score: 1.000000, Score Yest: 1.000000 2013-12-27 - Stock Held: AGO, Close: 1.185000, Posn: 6803, Posn($): 8061.555000, Hold Days: 12, Score: 1.000000, Score Yest: 1.000000 2013-12-27 - Stock Held: CDD, Close: 3.981502, Posn: 2136, Posn($): 8504.488272, Hold Days: 44, Score: 1.000000, Score Yest: 1.000000 2013-12-27 - Stock Held: AOG, Close: 1.990000, Posn: 4761, Posn($): 9474.390000, Hold Days: 42, Score: 1.000000, Score Yest: 1.000000 2013-12-27 - Stocks Held: 14, Total Wealth: 158338, Invested: -94277, Cash-On-Hand: 252615 2013-12-30 - Stock Held: WBC, Close: 32.142239, Posn: 319, Posn($): 10253.374241, Hold Days: 253, Score: 1.000000, Score Yest: 1.000000 2013-12-30 - Stock Held: ANZ, Close: 32.209999, Posn: 332, Posn($): 10693.719668, Hold Days: 253, Score: 1.000000, Score Yest: 1.000000 2013-12-30 - Stock Held: WES, Close: 43.931114, Posn: 225, Posn($): 9884.500650, Hold Days: 253, Score: 1.000000, Score Yest: 1.000000 2013-12-30 - Stock Held: WOW, Close: 33.849998, Posn: 281, Posn($): 9511.849438, Hold Days: 253, Score: 1.000000, Score Yest: 1.000000 2013-12-30 - Stock Held: WPL, Close: 39.169998, Posn: 246, Posn($): 9635.819508, Hold Days: 253, Score: 1.000000, Score Yest: 1.000000 2013-12-30 - Stock Held: CSL, Close: 68.900002, Posn: 152, Posn($): 10472.800304, Hold Days: 253, Score: 1.000000, Score Yest: 1.000000 2013-12-30 - Stock Held: DL_FOX, Close: 38.639999, Posn: 363, Posn($): 14026.319637, Hold Days: 253, Score: 1.000000, Score Yest: 1.000000 2013-12-30 - Stock Held: BXB, Close: 9.180000, Posn: 139, Posn($): 1276.020000, Hold Days: 250, Score: 1.000000, Score Yest: 1.000000 2013-12-30 - Stock Held: GMG, Close: 4.790000, Posn: 273, Posn($): 1307.670000, Hold Days: 249, Score: 1.000000, Score Yest: 1.000000 2013-12-30 - Stock Held: DL_AIO, Close: 5.183931, Posn: 1919, Posn($): 9947.963589, Hold Days: 240, Score: 1.000000, Score Yest: 1.000000 2013-12-30 - Stock Held: CGF, Close: 6.170000, Posn: 1959, Posn($): 12087.030000, Hold Days: 229, Score: 1.000000, Score Yest: 1.000000 2013-12-30 - Stock Held: AGO, Close: 1.195000, Posn: 6803, Posn($): 8129.585000, Hold Days: 13, Score: 1.000000, Score Yest: 1.000000 2013-12-30 - Stock Held: CDD, Close: 4.010105, Posn: 2136, Posn($): 8565.584280, Hold Days: 45, Score: 1.000000, Score Yest: 1.000000 2013-12-30 - Stock Held: AOG, Close: 2.130000, Posn: 4761, Posn($): 10140.930000, Hold Days: 43, Score: 1.000000, Score Yest: 1.000000 2013-12-30 - Stocks Held: 14, Total Wealth: 157279, Invested: -95336, Cash-On-Hand: 252615 =============================================================================== Cerebro: ----------------------------------------------------------------------------- - Datas: +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ -
@backtrader, heres the plot
-
The chart seems to indicate that you are entering short positions.
The code only prints position sizes and holding length for long operations (
position.size > 0), which makes it impossible to (using the log) prove or disprove that theory.No traces of operations are present in the log, but it seems reasonable to assume that many
selloperations are started which contribute to an increase inshortpositions.And that's the key issue:
- Each short position decreases the
value - Each short position increases the
cash
- Each short position decreases the
-
Thanks @backtrader, good feedback but I am unsure how this could be occurring? I have probably made a mistake somewhere, but my logic is based on there being no short positions whatsoever. In my next statement I only enter long positions. And my Sizer() is (hopefully) also supporting that.
Is my order dictionaries logic and SL/TP creation/cancelling logic sound? Can you please verify that for me?
In addition, the analyser log says no short positions are entered, heres the log print out following the
datasinfo:- Strategies: +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ - st: ************************************************************************* - Params: - printlog: True ************************************************************************* - Indicators: ....................................................................... - SimpleMovingAverage: - Lines: sma ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Params: - period: 252 ************************************************************************* - Observers: ....................................................................... - Broker: - Lines: cash, value - Params: None ....................................................................... - BuySell: - Lines: buy, sell ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Params: - barplot: True - bardist: 0.015 ....................................................................... - DataTrades: - Lines: BHP, CBA, WBC, ANZ, NAB, TLS, WES, WOW, RIO, WPL, CSL, WFD, DL_FOX, NCM, FMG, AMP, QBE, SUN, ORG, MQG, BXB, STO, CCL, AMC, IAG, OSH, ORI, TCL, AGL, AZJ, SGP, CWN, GMG, GPT, SYD, RMD, CIM, WOR, RHC, ASX, DL_NVN, LLC, IPL, SHL, CTX, MGR, CPU, DXS, DL_AIO, FBU, APA, COH, TTS, JHX, ILU, ALQ, AST, WHC, BEN, QAN, BLD, DL_TOL, SPK, VCX, TWE, MTS, SGR, GNC, FLT, SVW, RRL, DL_CPA, SEK, DUE, BOQ, PTM, AWC, MND, TAH, SKI, OZL, TPM, DL_PNA, HVN, ANN, PRY, ABC, SUL, DL_ALZ, SGM, FGX, BSL, CGF, BPT, MIN, IOF, UGL, CAR, NVT, DOW, ALL, SWM, AGO, IFL, DL_AUT, QUB, NUF, HGG, MSB, DL_ENV, PPT, DLX, SFR, MYR, DL_DJS, CQR, ARI, CWY, EVN, DL_GFF, FXJ, KAR, LYC, BWP, SDL, DL_WTF, IGO, RSG, DL_AQA, FXL, IRE, VAH, MMS, MML, JBH, BKN, CSR, DL_BRS, CDD, ABP, CHC, MGX, IVC, PRU, PDN, BLY, ASL, SAI, SCP, WSA, SIP, BRG, SXY, OGC, CDU, MQA, DL_DML, SLR, MRM, DL_IIN, BDR, SXL, SRX, SBM, GWA, KCN, TEN, DL_MTU, DL_DLS, EWC, AWE, BRU, DL_SKE, GUD, FWD, AAD, NST, DL_PBG, NWH, CAB, AQG, ACR, DL_CPL, DL_MBN, DCG, BBG, TRS, EHL, GBG, AOG, DL_SGT, IMD, TRY, MDL, FDM, SMX, BRL, SAR, DL_GRY, XJO ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Params: - usenames: True ************************************************************************* - Analyzers: ....................................................................... - Value: - Begin: 100000 - End: 157279.72151316737 ....................................................................... - tradeanalyzer: - Params: None ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Analysis: """"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" - total: - total: 93 - open: 52 - closed: 41 """"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" - streak: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - won: - current: 0 - longest: 2 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - lost: - current: 6 - longest: 29 """"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" - pnl: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - gross: - total: 21406.054992015008 - average: 522.0989022442685 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - net: - total: 20736.80284161358 - average: 505.7756790637458 """"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" - won: - total: 4 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - pnl: - total: 43579.02415400791 - average: 10894.756038501977 - max: 43189.75324506506 """"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" - lost: - total: 37 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - pnl: - total: -22842.22131239433 - average: -617.3573327674143 - max: -989.3569277108446 """"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" - long: - total: 41 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - pnl: - total: 20736.80284161358 - average: 505.7756790637458 ############################################################### - won: - total: 43579.02415400791 - average: 10894.756038501977 - max: 43189.75324506506 ############################################################### - lost: - total: -22842.22131239433 - average: -617.3573327674143 - max: -989.3569277108446 - won: 4 - lost: 37 """"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" - short: - total: 0 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - pnl: - total: 0.0 - average: 0.0 ############################################################### - won: - total: 0.0 - average: 0.0 - max: 0.0 ############################################################### - lost: - total: 0.0 - average: 0.0 - max: 0.0 - won: 0 - lost: 0 """"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" - len: - total: 2205 - average: 53.78048780487805 - max: 218 - min: 1 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - won: - total: 449 - average: 112.25 - max: 186 - min: 3 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - lost: - total: 1756 - average: 47.45945945945946 - max: 218 - min: 1 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - long: - total: 2205 - average: 53.78048780487805 - max: 218 - min: 1 ############################################################### - won: - total: 449 - average: 112.25 - max: 186 - min: 3 ############################################################### - lost: - total: 1756 - average: 47.45945945945946 - max: 218 - min: 1 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - short: - total: 0 - average: 0.0 - max: 0 - min: 9223372036854775807 ############################################################### - won: - total: 0 - average: 0.0 - max: 0 - min: 9223372036854775807 ############################################################### - lost: - total: 0 - average: 0.0 - max: 0 - min: 9223372036854775807 ....................................................................... - timereturn: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Params: - timeframe: 8 - compression: None - _doprenext: True - data: None - firstopen: True ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Analysis: - 2012-12-31: 0.0002235522925362332 - 2013-12-31: 0.5724456912924965 ....................................................................... - sharperatio: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Params: - timeframe: 8 - compression: 1 - riskfreerate: 0.03 - factor: None - convertrate: True - annualize: True - stddev_sample: False - daysfactor: None - legacyannual: False ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Analysis: - sharperatio: 0.895926963750468 ....................................................................... - sqn: - Params: None ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Analysis: - sqn: 0.47926917418139037 - trades: 41 ....................................................................... - drawdown: - Params: None ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Analysis: - len: 128 - drawdown: 9.523888291773911 - moneydown: 16555.911499416543 """"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" - max: - len: 128 - drawdown: 15.175583721262242 - moneydown: 26380.57202521086 Run-time - TOTAL: 0:03:05.675929 Run-time - Load Data: 0:00:23.584653 Process finished with exit code 1Thank you
PS: I searched the entire log for -ve positions ("posn: -" or "posn($): -") and there are no results, which would also suggest there is no shorting? -
The best approach would be to print the actual position of all assets without restricting it to
position.size > 0Given that the cash keeps on growing and growing it would seem like if the
shorttrades were never closed. If the assumption is right, they would never show up in the analyzer, because they are still alive. -
my wrong suggestion
-
Thanks @backtrader, i will try debuging with negative positions. But why are they occuring??
It must be because of the stop loss and take profit. I dont have this issue when i turn that functionality off. Can you tell me how I have implemented the stop loss and take profit functionality incorrectly? Am I cancelling orders OK? Are my new market positions managing them OK?@ab_trader not sure on your comment there :-)
-
Another long shot is that the code apparently fill values with
NaNwithfillna. If the prices areNaNthat could have effect in the different calculations of indicators and in the broker.To actually try to verify, the first thing would be to go down from 200 data feeds to
1and then2. -
-
Thanks @backtrader,
fillnapropogates the last non-NaN value forward to fill any holes (NaN values) in data that may exist. Its a common technique in investment analysis.As mentioned, the code above works fine when I set
SLTP_On=Falseand don't use stop-loss take profit functionality, so I am inclined to think I am either most likely executing this functionality wrong, or it doesnt work properly...?Can you please advise if I am implementing this functionality correct, with the dictionarys, order cancelling and creation? I.e. the
nextstatement, theinitstep andnotify_orderstepSome extracts of interest - INIT:
self.order = {} #Order: market entry self.order_out = {} #Order: market exit self.order_sl = {} #Stop-Loss self.order_tp = {} #Take-Profit self.bar_executed = {} self.sma_short = {} self.sma_long = {} for i, d in enumerate(d for d in self.datas): self.order[d._name] = None self.order_out[d._name] = None self.order_sl[d._name] = None self.order_tp[d._name] = None self.bar_executed[d._name] = NoneNOTIFY_ORDER:
if not order.alive():# indicate no order is pending, allows new orders if ord and order == ord: self.order[order.data._name] = None elif ord_out and order == ord_out: self.order_out[order.data._name] = None elif ord_sl and order == ord_sl: self.order_sl[order.data._name] = None elif ord_tp and order == ord_tp: self.order_tp[order.data._name] = NoneNEXT:
# cancel SL/TP 1st to avoid possibility of duplicate exit orders if self.order_sl[d._name]: self.log('CANCELLING SL & TP for: %s, SL ref: %i, TP ref: %i' %(d._name,self.order_sl[d._name].ref, self.order_tp[d._name].ref)) self.broker.cancel(self.order_sl[d._name]) #this automatically cancels the TP too #if self.order_tp[d._name]: # self.log('CANCELLING TP for: %s, ref: %s' %(d._name,self.order_tp[d._name].ref)) # self.broker.cancel(self.order_tp[d._name]) o4 = self.close(data=d) self.order_out[d._name] = o4 -
The code has obviously been read above and hints offered. Sensible steps to find the problems could be
- Reduce the number of assets in play from
200to1or2. Even better if done with reference data feeds which need no manipulation. - If testing with
200tickers is wished, check for duplicate tickers - Output the position of all assets and not only that of the assets for which
position.size > 0 - Stop execution as soon as a position has gone short
- Match the executed orders to the generated orders which should show which order is the one producing the likely culprit (being the suspicion that for several assets the code is generating short positions which increase the cash level)
- Use a simple sizer or even a fixed stake to start with
The reference implementation offered in the blog post above may be used as a starting base if wished.
- Reduce the number of assets in play from
-
Thank you @backtrader, I can confirm the issue is resolved and likely was a result of my version issues.
No short positions are entered, and sum of stocks held = getvalue() - getcash() as expected!! :-)
Cheers!
-
@cwse Hi CWE,
Can you please let me know what was changed compared to the code in your original post?
Thanks and Cheers,
Tamás -
@holicst I upgraded my backtrader version, and used the logic provided in the above example @backtrader posted
-
@cwse all right, thanks!
-
Hi there,
After spending a fair amount of time to make sense the relationship between
order.executed.valuevs (self.broker.getvalue()-self.broker.getcash()) , I just want to point out thatorder.executed.valueis the value of the current order position using the opening price of the NEXT period while theself.broker.getvalue()use the close price of the CURRENT period. Hence, you may find a small difference if the close price of the current period is different from the open price of the next period.
