So in the file lib/python3.6/site-packages/backtrader/brokers/, I copied bbroker.py to mybbroker.py which is the same as the original except for the function _try_exec_limit which I have replaced by:

    def _try_exec_limit(self, order, popen, phigh, plow, plimit):
        if order.isbuy():
            if 1==0:
                #plimit >= popen:
                # open smaller/equal than requested - buy cheaper
                pmax = min(phigh, plimit)
                p = self._slip_up(pmax, popen, doslip=self.p.slip_open,
                                  lim=True)
                self._execute(order, ago=0, price=p)
            elif plimit >= plow:
                # day low below req price ... match limit price
                self._execute(order, ago=0, price=plimit)

        else:  # Sell
            if 1==0:
                #plimit <= popen:
                # open greater/equal than requested - sell more expensive
                pmin = max(plow, plimit)
                p = self._slip_down(plimit, popen, doslip=self.p.slip_open,
                                    lim=True)
                self._execute(order, ago=0, price=p)
            elif plimit <= phigh:
                # day high above req price ... match limit price
                self._execute(order, ago=0, price=plimit)

Now my questions is how can I tell BT to use mybbroker.py instead of bbroker.py? (I understand I can simply over-write bbroker.py but I'm sure there is a more elegant way to switch these things. I just don't know how)

Thanks in advance,

@BT: thanks for the quick answer. Just to clarify, when I wrote

Say for cases when the OHLC bars are constructed from intraday data?

I meant from say when using 1Min OHLC bars (so the bars are not daily frequencies).

When you write:

Yes. Write a broker and plug it in.

can you kindly give me the name of the function to change, so I can find it in the source and do that?

Thanks in advance,

P.S. I understand the behavior I'am trying to mute is this one:

        if order.isbuy():
            if plimit >= popen:
                # open smaller/equal than requested - buy cheaper
                pmax = min(phigh, plimit)
                p = self._slip_up(pmax, popen, doslip=self.p.slip_open,
                                  lim=True)
self._execute(order, ago=0, price=p)

and

            if plimit <= popen:
                # open greater/equal than requested - sell more expensive
                pmin = max(plow, plimit)
                p = self._slip_down(plimit, popen, doslip=self.p.slip_open,
                                    lim=True)
self._execute(order, ago=0, price=p)

Could you kindly flesh out how to go about overloading the broker class so as to mute these conditions? I could not find a documentation about this,

Consider the following lines in the BT doc

*Price Matching:

backtrader tries to provide most realistic execution price for Limit orders.

Using the 4 price spots (Open/High/Low/Close) it can be partially inferred if the requested price can be improved.

For Buy Orders

        Case 1:

        If the open price of the bar is below the limit price the order executes immediately with the open price. The order has been swept during the opening phase of the session*

Is there anyway to switch this behavior off? Say for cases when the OHLC bars are constructed from intraday data?

Basically to have the open price be treated just as the other ones (for a limit buy, if open price < smaller than limit price, execute the limit at the limit price).

Alternatively, would it be possible to have BT ignore the open price altogether? I mean is it safe to feed BT an open-less file or do I risk running into undefined behavior?

I was wondering if there was a way to get getposition().size, or the same information, either in an observer or somehow out of cerebro.run() e.g. I want to get a vector the same length as the data with at each time point the number of position held by the strategy at that point in time.

One solution is to do something like this in the Strategy:

    def next(self):
        contract         = self.datas[0]
        current_position = self.getposition(contract).size
        self.datas[0].position_size[0] = current_position

where position_size is an 0 valued column I added to the data.

which can then be recovered from as:

result = cerebro.run()
result[0].datas[0].position_size.get(size = dataframe.shape[0])

where dataframe is the original data fed to cerebro.

@backtrader As usual, thanks for your answer.

I might be wrong on this, but it seems to me that backtrader (the library) is more flexible. It seems to me that a simple block like this in the backtrader.Strategy class could help toughening up the broker:

def next(self):
        contract    = self.datas[0]
        current_position = self.getposition(contract).size
         
        min_nav    = self.broker.getvalue() -\
                                     ((self.datas[0].close[0] - self.datas[0].high[0])\
                                       if current_position < 0 else\
                                     (self.datas[0].close[0] - self.datas[0].low[0])) *\
                                     self.datas[0].multiplier[0] * current_position
        margin_req = abs(current_position) * self.datas[0].MARGIN[0]
        
        if min_nav <= margin_req:
            if self.order:
                self.cancel(self.order)                
            if current_position < 0:
                self.order = self.buy(exectype = backtrader.Order.Limit,
                                                    data     = contract, 
                                                    price    = self.datas[0].high[0],
                                                  size     = abs(current_position))
            else:
                self.order = self.sell(exectype = backtrader.Order.Limit,
                                                   data     = contract, 
                                                   price    = self.datas[0].low[0],
                                                  size     = current_position)
    else:   
        #normal trading logic

Is this foolish?

Hi all,

bt.observers.Broker is returning negative values. Here is the relevant code:

        cerebro   = backtrader.Cerebro()
        cerebro.addobserver(backtrader.observers.Broker)
        # Add a strategy
        cerebro.addstrategy(StrategyBaseTrading)
        data      = PandasDataStrat1(dataname     = dataframe, 
                                     datetime     = -1, 
                                     open         = -1, 
                                     high         = -1, 
                                     low          = -1, 
                                     close        = -1,
                                     volume       = -1,
                                     openinterest = None)
        cerebro.adddata(data)
        cerebro.broker\
               .setcash(10000.0)
        cerebro.broker\
               .setcommission(commission = 2.0 
                              margin     = 500.0, 
                              mult       = 1000.0)
                            
        return cerebro.run()

then, I find that values of:

results[0].observers[0].value.get(size = dataframe.shape[0])

are negative (at the end of the series)

My questions are:

• what does this mean? Do I read this correctly to mean that if I liquidate my positions at that point, I owe my broker money?
• is there a way to force the broker to act tougher? (liquidate the position if the value is < margin_reqs_per_contract * num_contract).

Lemme know if I can help by providing more code/details the full example (I did not want to cluter the question as I suspect my misunderstanding of the docs is the likely culprit).

@backtrader: Perfect. Thanks for the pointer, it indeed addresses my problem. Just to be sure, could you confirm that the backtrader-preferred way to add an observer as a new column to the original dataframe (that was fed to backtrader) is:

dataframe['observer'] = result[0].observers[2].executed_size.get(size = dataframe.shape[0])

thank you

@backtrader Thanks for the pointer.. But I must be doing something wrong because the output I get don't seem to match the one from the csv file.

result[0].observers[2].lines.executed_size

In [44]: len(list(result[0].observers[2].lines.executed_size))
Out[44]: 3364

(this is the correct length: that of the data).

but I must be doing something wrong because these are all nan values:

In [39]: pandas.DataFrame(list(result[0].observers[2].lines.executed_size)).dropna()
Out[39]:
Empty DataFrame
Columns: [0]
Index: []

whereas when I export the result to a csv:

        cerebro.addwriter(backtrader.WriterFile, 
                          csv = True,
                          out = outfile)

I can clearly see the executed_size column is not all empty.

full code

class maCross(backtrader.Strategy):
    '''
    For an official backtrader blog on this topic please take a look at:

    https://www.backtrader.com/blog/posts/2017-04-09-multi-example/multi-example.html

    oneplot = Force all datas to plot on the same master.
    '''
    params = (
    ('sma1', 80),
    ('sma2', 200),
    ('oneplot', True),
    ('list_plus', [])
    )

    def __init__(self):
        '''
        Create an dictionary of indicators so that we can dynamically add the
        indicators to the strategy using a loop. This mean the strategy will
        work with any number of data feeds. 
        '''
        self.inds = dict()#           p0 = pandas.Series(self.data.close.get(size=5))
#           print(p0.head(5).transpose())
        for i, d in enumerate(self.datas):
            self.inds[d] = dict()
            self.inds[d]['sma1'] = backtrader.indicators.SMA(d.close, 
                                                             period=self.params.sma1)
            self.inds[d]['sma2'] = backtrader.indicators.SMA(d.close, 
                                                             period=self.params.sma2)
            if d._name in self.params.list_plus:
                self.inds[d]['cross'] = backtrader.indicators.CrossOver(self.inds[d]['sma1'],
                                                                        self.inds[d]['sma2'])
            else:
                self.inds[d]['cross'] = backtrader.indicators.CrossOver(self.inds[d]['sma2'],
                                                                        self.inds[d]['sma1']) 
          
            if i > 0: #Check we are not on the first loop of data feed:
                if self.p.oneplot == True:
                    d.plotinfo.plotmaster = self.datas[0]

    def next(self):
        for i, d in enumerate(self.datas):
            dt, dn = self.datetime.date(), d._name
            pos = self.getposition(d).size

            if not pos:  # no market / no orders
                if self.inds[d]['cross'][0] == 1:
                    self.buy(data=d, size=1)
                elif self.inds[d]['cross'][0] == -1:
                    self.sell(data=d, size=1)
            else:
                if self.inds[d]['cross'][0] == 1:
                    self.close(data=d)
                    self.buy(data=d, size=1)
                elif self.inds[d]['cross'][0] == -1:
                    self.close(data=d)
                    self.sell(data=d, size=1)

class SimpleObserver(backtrader.observer.Observer):
    lines = ('executed_size', )

    def next(self):

        for order in self._owner._orderspending:

            if order.data is not self.data:
                continue

            if order.status in [backtrader.Order.Completed]:
                self.lines.executed_size[0]  = order.executed.size

then

        dataframe = self.get_data(file_name)
        margin    = 10000
        # Create a cerebro entity
        cerebro   = backtrader.Cerebro()
        cerebro.addobserver(SimpleObserver)
        # Add a strategy
        cerebro.addstrategy(maCross)
        data      = PandasDataStrat1(dataname     = dataframe, 
                                     datetime     = -1, 
                                     open         = -1, 
                                     high         = -1, 
                                     low          = -1, 
                                     close        = -1,
                                     volume       = -1,
                                     openinterest = None)
        cerebro.adddata(data)
        start_cash = 100000.0
        cerebro.broker\
               .setcash(start_cash)
        cerebro.broker\
               .setcommission(commission = 2.0, 
                              margin     = margin, 
                              mult       = 10.0)
        result = cerebro.run()

@backtrader : sorry, my question was poorly formulated.

What I meant is that after I run the code:

        cerebro   = backtrader.Cerebro()
        cerebro.addobserver(PositionObserver)
        # Add a strategy
        cerebro.addstrategy(StrategyBaseTrading)

        data = PandasDataStrat1(dataname     = dataframe, 
                                datetime     = -1, 
                                open         = -1, 
                                high         = -1, 
                                low          = -1, 
                                close        = -1,
                                volume       = -1,
                                openinterest = None)
        cerebro.adddata(data)
        cerebro.broker\
               .setcash(100000.0)
        cerebro.broker\
               .setcommission(commission = 2.0, 
                              margin     = 10000.0, 
                              mult       = 10.0)

is there a way to extract the content of lines.executed_position created by the Observer out of the cerebero object?

basically, right now, I do:

        cerebro.addwriter(backtrader.WriterFile, 
                          csv = True, 
                          out = 'results.csv')
        cerebro.run()

        def find_skipped_rows(file_name, string):
            sentinel = []
            counter  = 0
            with open(file_name) as fh:
                for line in fh:
                    counter += 1
                    if line.startswith(string):
                        sentinel.append(counter)
            return [i for i in range(counter) if i < sentinel[0] or i >= sentinel[1] - 1]

        skip_rows = find_skipped_rows(outfile, '===')
        executed_position  = pandas.read_csv(outfile, skiprows = skip_rows).executed_position

so basically I'm looking for a way to get the content of executed_position from the Cerebro object directly (without the write to csv, read from csv step)

thanks

I would like to keep the value of the observer in a list (rather than write it out to a csv and read the csv latter). Can this be done?

Here is my custom observer:

class PositionObserver(backtrader.observer.Observer):
    lines = ('position', 'price', )

    plotinfo = dict(plot=True, subplot=True, plotlinelabels=True)

    plotlines = dict(
        executed_position = dict(marker='*', markersize=8.0, color='lime', fillstyle='full'),
        executed_price    = dict(marker='*', markersize=8.0, color='lime', fillstyle='full')
    )

    def next(self):

        for order in self._owner._orderspending:

            if order.data is not self.data:
                continue

            self.lines.executed_position[0] = order.executed.size
            self.lines.executed_price[0]    = order.executed.price