Hi there,

I am new to backtrader, it seems a powerful platform!

I have a pre-written trading algorithm which modifies the pandas dataframes for many stocks, adding additional columns to the standard OHLCV dataframe. Many new columns are added for the time series to calculate different characteristics.

What is the best way to bring these dataframes, with my columns, into Back Trader? Can I just use the standard "data= bt.feeds.PandasData(..."?

How can I then call these columns in "Def next()" so that I can say things like "if Column X > Column Y Then BUY BUY BUY!!" ...i.e. self.order = self.buy().

Appreciate your help! Hopefully I don't need to re-write my code and can just pull the dataframes into Backtrader!

Thank you!
cwse

Does backtrader use arithmetic or log returns?

If both, when where and why?

Thanks :-)

Hi there,

How exactly is getvalue() calculated? I understand it is the sum of everything you have invested in market + cash?

This should then mean that getvalue() -getcash() = sum of current value of everything invested in market (stocks).

However, when I add up the value of my stocks held, it does not equal this.

See my example print out below (totalwealth = getvalue(), invested = getvalue() - cash()), my invested value says only $27,000. But when you add up my stocks held, its more like $66,000...

2012-12-28, ORDER CANCELLED - TP: ABC, Trade Ref: 1037
2012-12-28, SELL EXECUTED - Closed Long Pos: SUN, Ref: 1039, Price: 9.97, PNL: 123.47, Purchase Cost: 8379.68, Comm 10.24
2012-12-28, SELL EXECUTED - Closed Long Pos: CTX, Ref: 1040, Price: 19.10, PNL: 520.93, Purchase Cost: 8284.17, Comm 10.61
2012-12-28, BUY EXECUTED - Entered Long Pos: TTS, Ref: 1041, Price: 3.01, Cost: 8298.57, Size: 2757.00, Comm 10.00
2012-12-28, BUY EXECUTED - Entered Long Pos: CHC, Ref: 1042, Price: 3.32, Cost: 8373.04, Size: 2522.00, Comm 10.09
2012-12-28, OPERATION PROFIT: SUN, GROSS 123.47, NET 103.13
2012-12-28, CANCELLING STOP-LOSS: SUN, Ref: 216
2012-12-28, CANCELLING TAKE-PROFIT: SUN, Ref: 216
2012-12-28, OPERATION PROFIT: CTX, GROSS 520.93, NET 500.34
2012-12-28, CANCELLING STOP-LOSS: CTX, Ref: 211
2012-12-28, CANCELLING TAKE-PROFIT: CTX, Ref: 211
2012-12-28, Stock held: TTS, Close: 3.01, score: 0.93, Score Yest: 0.92, Posn: 2757.00, hold days 1
2012-12-28, CREATE SL: TTS, Trigger Price: 2.71
2012-12-28, CREATE TP: TTS, Trigger Price: 9.03
2012-12-28, Stock held: MTS, Close: 3.38, score: 1.00, Score Yest: 0.99, Posn: 2569.00, hold days 6
2012-12-28, Stock held: FLT, Close: 27.03, score: 0.99, Score Yest: 1.00, Posn: 302.00, hold days 27
2012-12-28, CREATE BUY: ABC, Close: 3.11, score: 0.94
2012-12-28, Stock held: MYR, Close: 2.01, score: 1.00, Score Yest: 1.00, Posn: 4221.00, hold days 25
2012-12-28, Stock held: CHC, Close: 3.31, score: 0.95, Score Yest: 0.92, Posn: 2522.00, hold days 1
2012-12-28, CREATE SL: CHC, Trigger Price: 2.98
2012-12-28, CREATE TP: CHC, Trigger Price: 9.93
2012-12-28, Stock held: MQA, Close: 1.68, score: 0.97, Score Yest: 0.97, Posn: 5076.00, hold days 7
2012-12-28, CREATE BUY: SMX, Close: 4.64, score: 0.91
2012-12-28, Stocks Held: 6, Total Wealth: 104603, Invested: 27280, Cash-On-Hand (pre-today's buys): 77323
2012-12-28, Ending Wealth: 104603, Invested: 27280, Cash-On-Hand: 77323,

Perhaps I am interpreting the getvalue() definition wrong? Otherwise, any ideas why these numbers dont equate?

Thank you

Hi,

Is it possible to get Days Held and Unrealised return in Next for stocks which have non-zero positions?

Thanks!

Is it possible to self.close() when you have an open Stop order, without running the risk of the Stop and Close executing at the same time (and resulting in an unwanted net short position)?

It seems you have to first cancel the Stop, then wait until the next "next" iteration to "Close" the position. But working with daily data, this means up to 3 days before you can get out of the market.

I have tried self.cancel followed by self.Close in the one Next() iteration, but if the Stop has not yet been Accepted by the Broker (i.e. was recently created), it appears this "self.cancel" step doesn't work (and you have to wait another next iteration).

Would love anyone's thoughts on how to manage stops without waiting multiple "next" iterations!

Thanks @backtrader, sounds good. Any tips how I could edit the following commission scheme to do just that?

class CommInfo_Custom(bt.CommInfoBase):
    params = (
        ('commtype', bt.CommInfoBase.COMM_PERC), #commission to be understood as %
        # Custom broker commission fee params:
        ('min_fee', 0),  # The broker uses a %commission fee, with the "min_fee" as the minimum fee per trade. 
    )

    def _getcommission(self, size, price, pseudoexec):
        commvalue = abs(size) * price * self.p.commission         # %based commission value
        commvalue = max(self.p.min_fee,commvalue)                 # choose the greater of $min_fee and percentage calculated
        return commvalue

Much appreciated

Hello again!

I am trying to get the interest expense in Next() block, but the problem is when an position is closed, the bbroker self.d_credit[data] is reset to zero, so I cannot get the interest applicable to the last day the stock was held. Is it possible to get interest each day in next()?

Also: I notice the credit interest is included in order.executed.comm; is it possible to get the interest separately out of this?

Thanks!!

Is it possible to apply "Cheat on Open" and Cheat on close" for specific trades (i.e. for select order placements), as opposed to applying it across the board in cerebro: cerebro = bt.Cerebro(cheat_on_open=True)?

Thank you @backtrader,

When you say it is OK to replace it, would something like the following (psuedo code) be acceptable?

def next(self):
    if market_scenario_one:
        self.broker.addcommissioninfo(comminfo_one)
    else:
        self.broker.addcommissioninfo(comminfo_two)

...be OK, assuming comminfo_one and comminfo_two are defined bt.CommInfoBase objects?

Cheers,
cwse

Hello all!

I am wondering if it is possible to change the commission during "next()" iterations?

I have cases where if certain conditions are met, I want to go "leveraged long with margin" (eg. 2x leverage)... but if these conditions are not met, I do not want any leverage, just normal buy and sell trades (without any margin requirements).

I tried using "self.broker.addcommissioninfo" at the start of each "next()" and adding a margin comminfo where appropriate, but I do not believe this is the correct way to do it.

Would love to know if this is possible at all? I am using my own custom comminfo if that matters (code below).

class CommInfo_Custom(bt.CommInfoBase):
    params = (
        ('commtype', bt.CommInfoBase.COMM_PERC),
        ('min_fee', 0),  
    )

    def _getcommission(self, size, price, pseudoexec):
        commvalue = abs(size) * price * self.p.commission         # %based commission value
        commvalue = max(self.p.min_fee,commvalue)                 # choose the greater of $min_fee and percentage calculated
        return commvalue

Desired commission info to "use" when in normal conditions:

comminfo = CommInfo_Custom(commission=v.comm_pct
                                           ,min_fee=0.0
                                           ,interest=v.short_holding_fee
                                           ,interest_long=False
                                           ,percabs=True
                                           ,automargin=False
                                           ,leverage=1.0
                                           ,mult=1.0)

Desired commission info to "use" when in "leverage" conditions:

        comminfo_leveraged_long = CommInfo_Custom(commission=v.comm_pct
                                   ,min_fee=0.0
                                   ,interest=v.short_holding_fee
                                   ,interest_long=True
                                   ,percabs=True
                                   ,automargin=0.5 #must hold 0.5*price to hold a margin position
                                   ,leverage=2.0 #Amount of leverage for the asset with regards to the needed cash
                                   ,mult=2.0) #multiplier applied to the asset for value/profit

Thanks so much!!

Hi @backtrader

I am sorry this is turning to be a difficult to answer, it must be very complex process to add a line to an indicator in Backtrader. I am passing many lines into my strategy at present and do not need help with extending data feeds, this process works just fine thank you.

Apologies if you think I am twisting things, I just didn't want to copy my over 2000 lines of code as I think it is distracting from the question at hand. I don't think it is a good use of your or my time to turn this into a full code review. I can assure you I am passing over 20 lines via pandas dataframes with ~30columns into the custom data loader with no issues, using my analysiscols to specify the list of column names to pass as lines. These lines are used by my Backtrader strategy in many places.

Given it appears to not be a straightforward process to pass a line to an indicator, and I am not aware of any documentation on how to specify a line for use in an indicator, I will just calculate the SMA outside of BT (in pandas) and pass it into backtrader as another line.

I am a big fan of backtrader software and use it daily, so this workaround will do just fine.

Thank you!

Hi @backtrader

Here is my data loader, I use many custom "analysis_cols"

class CustomBTStockLoader(btfeeds.PandasData):
    #  None : column not present
    #  -1 : autodetect position or case-wise equal name
    #  >= 0 : numeric index to the colum in the pandas dataframe
    #  string : column name (as index) in the pandas dataframe


    params = (
                 ('openinterest', None),
                 ('open', 'Open'),
                 ('high', 'High'),
                 ('low', 'Low'),
                 ('close','Close'),
                 ('volume','Volume')
             ) \
             + tuple((e,-1) for e in v.analysiscols)   #<--- this loads my custom data columns.

    if v.Custom_Alg == True:
        lines = tuple(v.analysiscols) #('Opportunity',)
        datafields = btfeeds.PandasData.datafields + v.analysiscols
    else:
        datafields = btfeeds.PandasData.datafields

I feel my question is pretty straight forward though and we are going on a bit of a tangent.

How do you specify the data which an indicator uses?

I believe it should just be a matter of editing this single line of code (following) where the indicator is created, to somehow specify the "line" of interest. Could you please tell me how?

self.sma_mth[d] = bt.indicators.SimpleMovingAverage(d, period=21)

Hi @backtrader

I am not trying to prove anything, I am simply asking for help. How do I set a indicator (eg bt.indicators.simpleMovingAverage) to use a specific line?

Note on the above error it specifically says OHLC

This section is at the start of my Strategy code in the init:

    def __init__(self):
        self.o = {}  # orders per data (main, stop, limit, manual-close)
        self.sma_mth = {}
        self.sma50 = {}
        self.sma20 = {}
        self.sma10 = {}
        self.sma5 = {}
        self.sma3 = {}
        self.sma2 = {}
        #self.bollinger = {}
        for i, d in enumerate(d for d in self.datas):
            self.sma_mth[d] = bt.indicators.SimpleMovingAverage(d, period=21) #https://cryptocurrencyhaus.com/crypto-trading-how-to-use-simple-moving-averages/
            self.sma50[d] = bt.indicators.SimpleMovingAverage(d, period=50)
            self.sma20[d] = bt.indicators.SimpleMovingAverage(d, period=20) #https://cryptocurrencyhaus.com/crypto-trading-how-to-use-simple-moving-averages/
            self.sma10[d] = bt.indicators.SimpleMovingAverage(d, period=10)
            self.sma5[d] = bt.indicators.SimpleMovingAverage(d, period=5)
            self.sma3[d] = bt.indicators.SimpleMovingAverage(d, period=3)
            self.sma2[d] = bt.indicators.SimpleMovingAverage(d, period=2)

My data has been fed to backtrader and is hence referd to as self.datas.

Any tips helpers or pointers in the right direction, on how to use custom lines here would be much appreciated!

Thank you!!