For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

Shorting values are negative



  • Hi, it is me again. I really appreciate everyone's commitment to this community!

    While I was making a long&short strategy with coding helps from people here, I confronted another problem.

    My strategy is the following :

    For shorting strategy (Which is the code that I wrote below), within the universe, I short 2 stocks (maximum positions that I preset).

    If the price reaches a profit-realisation or loss-cut level then I cover the stocks then enter the market again with shorting 2 different positions within the universe.

    This portfolio will be rebalanced every Q ends regardless of the current stocks' prices.

    This is my code:

    #Short 
    class Short(bt.Strategy):
        
        params = (
        ('losscut', 0.15),  # Different params for losscut & profit realisation & maximum stock holdings 
        ('profit', 0.15),
        ('max_stock', 2), 
        ) 
        
    
            
        def notify_order(self, order):
    
            if order.status in [order.Submitted, order.Accepted]:
                self.order = order
                return
    
            if order.status in [order.Completed]:
                if order.isbuy(): #Trying to track entry price for each stock position we entered
                    self.entry_order.update({order.data._name : order.executed.price})
    
                    print('Date : {}, Current Order is Buy : {}, Price : {}, Value : {}, Commission : {}'.format(
                        self.datetime.date(), order.data._name,order.executed.price, order.executed.value, order.executed.comm))
                    
                else:
                    print('Date : {}, Current Order is Sell : {}, Price : {}, Value : {}, Commission : {}'.format(
                        self.datetime.date(), order.data._name,order.executed.price, order.executed.value, order.executed.comm))
    
            # Sentinel to None: new orders allowed
            self.order = None  
        
        def __init__(self):
            # __init__ = creates object defined by the Class. INITIALISES the object - sets all attributes
            # To keep track of pending orders, 현재 보유 중인 종목, 투자되었던 종목, Entry Price
            self.entry = {}
            self.entry_order = {}
            self.Short_list = []
            self.Shorted_list = []
            self.rebalanced = []
            self.order = None
            
        def next(self):
            
            for i, d in enumerate(self.datas):
                dt, dn = self.datetime.date(), d._name
                pos = self.getposition(d).size
                offset = BMonthEnd()
    
                if pos == 0:
                    if dn not in self.Shorted_list:
                        if len(self.Short_list) == self.params.max_stock:
                            return
                        
                        else:
                            self.order_target_percent(data=d, target = -(1/float(self.params.max_stock)))
                            self.Shorted_list.append(dn)
                            self.Short_list.append(dn)
                            self.entry.update({dn : d.close[0]})
                            
                else:
                    if dt.month in [3,6,9,12]:
                        #Last day of current month
                        if dt.day == offset.rollforward(dt).day:
                            self.order_target_percent(data=d, target = 0)
                            self.rebalanced.append(dn)
                            self.Short_list.remove(dn)
                            return
                        #Last day of previous month
                        #print(offset.rollback(dt))
                        
                    if d.close[0] > self.entry[dn]*float(1 + self.params.losscut):
                        self.order_target_percent(data=d, target = 0.0)
                        self.Short_list.remove(dn)
                        
                    elif d.close[0] < self.entry[dn]*float(1 - self.params.profit):               
                        self.order_target_percent(data=d, target = 0.0)
                        self.Short_list.remove(dn)
                        
    
                        
    # Trade list similar to Amibroker output
    class open_list(bt.Analyzer):
    
        def get_analysis(self):
    
            return self.trades
    
    
        def __init__(self):
    
            self.trades = []
            self.cumprofit = 0.0
    
    
        def notify_trade(self, trade):
    
            if trade.isclosed:
    
                dir = 'short'
                if trade.history[0].event.size > 0: dir = 'Short'
    
                pricein = trade.history[len(trade.history)-1].status.price
    
                datein = bt.num2date(trade.history[0].status.dt)-timedelta(days=1)
                
    
                if trade.data._timeframe >= bt.TimeFrame.Days:
                    datein = datein.date()
    
                pnl = trade.history[len(trade.history)-1].status.pnlcomm
                barlen = trade.history[len(trade.history)-1].status.barlen
                pbar = pnl / barlen
                self.cumprofit += pnl
    
                size = value = 0.0
                for record in trade.history:
                    if abs(size) < abs(record.status.size):
                        size = record.status.size
                        value = record.status.value
    
                highest_in_trade = max(trade.data.high.get(ago=0, size=barlen+1))
                lowest_in_trade = min(trade.data.low.get(ago=0, size=barlen+1))
                hp = 100 * (highest_in_trade - pricein) / pricein
                lp = 100 * (lowest_in_trade - pricein) / pricein
                if dir == 'long':
                    mfe = hp
                    mae = lp
                if dir == 'short':
                    mfe = -lp
                    mae = -hp
    
                self.trades.append({'ref': trade.ref, 'ticker': trade.data._name, 'dir': dir,
                     'datein': datein, 'pricein': pricein, 'pnl': pnl,
                     'size': size, 'value': value, 'cumpnl': self.cumprofit,
                     'nbars': barlen, 'pnl/bar': round(pbar, 2),
                     'mfe%': round(mfe, 2), 'mae%': round(mae, 2)})
                
    
    # Trade list similar to Amibroker output
    class trade_list(bt.Analyzer):
    
        def get_analysis(self):
    
            return self.trades
    
    
        def __init__(self):
    
            self.trades = []
            self.cumprofit = 0.0
    
    
        def notify_trade(self, trade):
    
            if trade.isclosed:
                offset = BMonthEnd()
                dir = 'short'
                if trade.history[0].event.size > 0: dir = 'Short'
    
                pricein = trade.history[len(trade.history)-1].status.price
                priceout = trade.history[len(trade.history)-1].event.price
                
                datein = bt.num2date(trade.history[0].status.dt)-timedelta(days=1)
                
                dateout = bt.num2date(trade.history[len(trade.history)-1].status.dt)-timedelta(days=1)
                if dateout.weekday() == 5:      #if it's Saturday
                    dateout = dateout - timedelta(days = 1) #then make it Friday
                elif dateout.weekday() == 6:      #if it's Sunday
                    dateout = dateout - timedelta(days = 2); #then make it Friday
        
                if dateout.strftime("%Y-%m-%d") == offset.rollforward(bt.num2date(trade.history[len(trade.history)-1].status.dt)-timedelta(days=1)).strftime("%Y-%m-%d"):
                    status = 'Rebalanced'
                else:
                    if pricein > priceout:
                        status = 'Loss-Cut'
                    else:
                        status = 'Profit-realisation'
    
                if trade.data._timeframe >= bt.TimeFrame.Days:
                    datein = datein.date()
                    dateout = dateout.date()
    
                pcntchange = 100 * priceout / pricein - 100
                pnl = trade.history[len(trade.history)-1].status.pnlcomm
                barlen = trade.history[len(trade.history)-1].status.barlen
                pbar = pnl / barlen
                self.cumprofit += pnl
    
                size = value = 0.0
                for record in trade.history:
                    if abs(size) < abs(record.status.size):
                        size = record.status.size
                        value = record.status.value
    
                highest_in_trade = max(trade.data.high.get(ago=0, size=barlen+1))
                lowest_in_trade = min(trade.data.low.get(ago=0, size=barlen+1))
                hp = 100 * (highest_in_trade - pricein) / pricein
                lp = 100 * (lowest_in_trade - pricein) / pricein
                if dir == 'Short':
                    mfe = hp
                    mae = lp
                if dir == 'short':
                    mfe = -lp
                    mae = -hp
    
                self.trades.append({'ref': trade.ref, 'ticker': trade.data._name, 'dir': dir, 'Status' : status,
                     'datein': datein, 'pricein': pricein, 'dateout': dateout, 'priceout': priceout,
                     'chng%': round(pcntchange, 2), 'pnl': pnl,
                     'size': size, 'value': value, 'cumpnl': self.cumprofit,
                     'nbars': barlen, 'pnl/bar': round(pbar, 2),
                     'mfe%': round(mfe, 2), 'mae%': round(mae, 2)})   
             
            
    class EIKON_HLOC(bt.feeds.GenericCSVData):
        params = (
            ('nullvalue' , float('NaN')),
            ('dtformat', '%Y-%m-%d'),
            ('datetime', 0),
            ('time',-1),
            ('open', 1),
            ('high', 2),
            ('low', 3),
            ('close', 4),
            ('volume', 5),
            ('openinterest', -1),
        )
        
    
    #Variable for our starting cash
    startcash = 1000000000
    
    #Create an instance of cerebro
    cerebro = bt.Cerebro(cheat_on_open=False)
    
    #Add our strategy
    cerebro.addstrategy(Short)
    
    datalist = []
    for i in bascket:
        datalist.append(('{}.csv'.format(i), i))
    
    for i in range(len(datalist)):
        data = EIKON_HLOC(dataname = datalist[i][0])
        cerebro.adddata(data, name=datalist[i][1])
        
    # Set our desired cash start
    cerebro.broker.setcash(startcash)
    cerebro.broker.set_coc(True)
    cerebro.broker.setcommission(commission=0.005)
    
    # add analyzers
    cerebro.addanalyzer(trade_list, _name='trade_list')
    cerebro.addanalyzer(open_list, _name='open_list')
    # run backtest
    strats = cerebro.run(tradehistory=True)
    # get analyzers data
    trade_list = strats[0].analyzers.trade_list.get_analysis()
    open_list = strats[0].analyzers.open_list.get_analysis()
    print (tabulate(open_list, headers="keys"))
    print (tabulate(trade_list, headers="keys"))
    #Finally plot the end results
    cerebro.plot()
    

    And the result is following:

    Date : 2015-01-05, Current Order is Sell : 000060, Price : 12600.0, Value : -499993200.0, Commission : 2499966.0
    Date : 2015-01-05, Current Order is Sell : 000250, Price : 10050.0, Value : -499997550.0, Commission : 2499987.75
    Date : 2015-03-16, Current Order is Buy : 000250, Price : 11650.0, Value : -499997550.0, Commission : 2897995.75
    Date : 2015-03-16, Current Order is Sell : 005380, Price : 172000.0, Value : -470592000.0, Commission : 2352960.0
    Date : 2015-04-01, Current Order is Buy : 000060, Price : 11400.0, Value : -499993200.0, Commission : 2261874.0
    Date : 2015-04-02, Current Order is Sell : 005930, Price : 28460.0, Value : -488686660.0, Commission : 2443433.3000000003
    Date : 2015-06-03, Current Order is Buy : 005380, Price : 138500.0, Value : -470592000.0, Commission : 1894680.0
    Date : 2015-06-03, Current Order is Sell : 006730, Price : 18784.0, Value : -542951520.0, Commission : 2714757.6
    Date : 2015-07-01, Current Order is Buy : 005930, Price : 25360.0, Value : -488686660.0, Commission : 2177282.8000000003
    Date : 2015-07-02, Current Order is Sell : 034310, Price : 21850.0, Value : -546075200.0, Commission : 2730376.0
    Date : 2015-07-23, Current Order is Buy : 034310, Price : 25300.0, Value : -546075200.0, Commission : 3161488.0
    Date : 2015-07-23, Current Order is Sell : 034730, Price : 302500.0, Value : -477647500.0, Commission : 2388237.5
    Date : 2015-08-21, Current Order is Buy : 034730, Price : 256000.0, Value : -477647500.0, Commission : 2021120.0000000002
    Date : 2015-08-21, Current Order is Sell : 078340, Price : 111900.0, Value : -574158900.0, Commission : 2870794.5
    Date : 2015-08-25, Current Order is Buy : 006730, Price : 15713.0, Value : -542951520.0, Commission : 2270921.325
    Date : 2015-08-25, Current Order is Sell : 098460, Price : 32200.0, Value : -607742800.0, Commission : 3038714.0
    Date : 2015-09-01, Current Order is Buy : 098460, Price : 38000.0, Value : -607742800.0, Commission : 3586060.0
    Date : 2015-10-01, Current Order is Buy : 078340, Price : 116300.0, Value : -574158900.0, Commission : 2983676.5
    

    The question is... OK, so I take short positions at the beginning and cover them when stocks meet profit-realisation or loss-cut or even rebalancing conditions. Then there must be + values coming from shorting and - values from buying (if the values are calculated based on cash flows).

    But why do I get negative values for both sell & buy ? I was expecting a positive value from cash inflows of short selling but the figure here shows a negative value. Also, as you see below, even though prices are different, I still get the same absolute values from sell & buy. I don't know what I am missing here :(

    Date : 2015-01-05, Current Order is Sell : 000250, Price : 10050.0, Value : -499997550.0, Commission : 2499987.75
    Date : 2015-03-16, Current Order is Buy : 000250, Price : 11650.0, Value : -499997550.0, Commission : 2897995.75
    


    Also, as you can see the order transaction date from the result, sometimes the sell & buy order takes place within the same day but sometimes it doesn't.

    Could you guys help me why I am having these issues?

    Thank you!!!


  • administrators

    @crespecular said in Shorting values are negative:

    I was expecting a positive value from cash inflows of short selling

    There is cash inflow, you simply have to check the amount of cash in your account and not the details of the order, which simply says that instead of decreasing your account by x is decreasing your account by -x (i.e.: is adding cash to your account)

    A simple plot will show you how your cash increases when you engage into short selling.



  • Thank you for your quick response!

    Btw, when you say cash inflow, do you mean pnl? or is there any code that I can retrieve current cash balance + cash generate(or loss) from the transaction?

    :)


  • administrators

    @crespecular said in Shorting values are negative:

    Btw, when you say cash inflow, do you mean pnl?

    No, I mean cash.

    @crespecular said in Shorting values are negative:

    or is there any code that I can retrieve current cash balance

    broker.get_cash()? Docs - Broker

    @crespecular said in Shorting values are negative:

    cash generate(or loss) from the transaction?

    Use for example Trade instances. Docs - Trade



  • Thanks a lot!