Hi, Backtraders!

I want to test an intraday strategy that involves close positions at the close of each day. Interactive brokers have a type of order to achieve this: Market-on-Close (MOC) Orders. Reading the documentation of backtrader it seems that it doesn't have this feature in backtest mode.

From a point o view, it will not consider appropriate to ask a question while rising a solution because it could induce bias but I will make an exception looking for feedback of my approach in case no other possible solution arrives:

To achieve this kind of behavior I thought to modify my data feed to include two additional bars after the last bar of the day (16:00:00). One will be at 16:00:01 to create a market order and the other one will be at 16:00:02 to execute the order. Both will have in the OHLC data the close price of the 16:00:00 bar.

I'm not simply close my positions at 15:49:00 (supposing minute bars) in order to be executed at 16:00 because I'm not using time bars. So, I have uncertainty about at what time it will be the previous bar to the close of the day.

Could you think in another way to incorporate (or simulate) on-close orders in backtesting with backtrader? Or could you share an inconvenience that you see in my approach?

Cheers!

@backtrader Thanks for your response. I apologize for not making a clear question. But the link you provide solve my doubts about how backtrader executes orders. Very interesting mechanism.

Greetings.

Hi, Backtraders.

I have a doubt in regards to how backtrader operates bracket orders (and perhaps limit and stop orders too). For instance, in a long position, the upper bracket will check if the low price of the current bar is greater than or equal to the value and then close the position at close price for that bar. Is this the way backtrader works?

Greetings.

Agree with you @backtrader... If it ain't broke, don't fix it

When I open the TWS I'm able to choose what sub account I want to operate. The problem is that I can not find a way to connect Backtrader to a specific one.

Does Backtrader have this feature? If it doesn't, any clue how it could be added?

Hi, @backtrader and Bactraders!

I tried the second option but it didn't go as expected. I read the documentation and review every post about the backfill_from and tried several things from there without success.

The issue is that the timestamp I get from the data when the next() is called jump from 12:00 to 5:00 of the other day instead of continue to 12:01 and so on until 16:00 and then 9:30 of the next day. Here an extract of my prints:

2018-10-01T11:56:00
2018-10-01T11:57:00
2018-10-01T11:58:00
2018-10-01T11:59:00
2018-10-01T12:00:00
2018-10-02T05:31:00
2018-10-02T05:32:00
2018-10-02T05:33:00
2018-10-02T05:34:00
2018-10-02T05:35:00
2018-10-02T05:36:00
2018-10-02T05:37:00

I'm very sure that my local CSVs have data only between 9:30 and 16:00 hrs.

Let me detail about my code. This is how I'm connecting to IB:

cerebro = bt.Cerebro()
store = bt.stores.IBStore(host='127.0.0.1',port=4002, clientId=36, reconnect=-1)

My data is in minute bars and I'm reading it as follow:

data0 = btfeed.GenericCSVData(
        dataname=file_path,

        fromdate=datetime.datetime(2018, 10, 1, 9, 30, 0),
        todate=datetime.datetime(2018, 12, 12, 9, 30, 0),

        timeframe=bt.TimeFrame.Minutes,
        compression=1,

        dtformat='%Y%m%d',
        tmformat='%H:%M:%S',

        datetime=0,
        time=1,
        open=3,
        high=4,
        low=5,
        close=6,
        volume=8,
        openinterest=-1,
)

Then, adding the data:

contract = 'TWTR-STK-SMART-USD'
data = store.getdata(dataname=contract, rtbar=True,
                             timeframe=bt.TimeFrame.Ticks, historical=False,
                             #todate=datetime.datetime.now(),
                             fromdate=datetime.datetime(2018, 10, 1, 9, 30, 0),
                             tz=pytz.timezone('US/Eastern'),
                             backfill_from=data0,
                             #sessionstart=datetime.time(9, 30, 0),
                             #sessionend=datetime.time(16, 0, 0),
                             )

cerebro.resampledata(data, timeframe=bt.TimeFrame.Minutes, compression=1)

Finally, I run cerebro as:

cerebro.broker = store.getbroker()

cerebro.addstrategy(TestStrategy)

cerebro.run(exactbars=1, runonce=False)

I have tried several combinations of parameters both, in store.getdata() and in btfeed.GenericCSVData() but none of them work. So, any help will be appreciated.

Thanks in advance.

@backtrader Thank you very much for your useful answer. Greetings!

It seems to be a bug in pyfolio.py:

        pss = self.rets['positions']
        ps = [[k] + v[-2:] for k, v in iteritems(pss)]
        cols = ps.pop(0)  # headers are in the first entry
        positions = DF.from_records(ps, index=cols[0], columns=cols)
        positions.index = pandas.to_datetime(positions.index)
        positions.index = positions.index.tz_localize('UTC')

Line 130 is ps = [[k] + v[-2:] for k, v in iteritems(pss)] but it should be ps = [[k] + v[:] for k, v in iteritems(pss)]. With that change now I get the right positions in my strategy:

If the bug could be confirmed, it would be great to get fixed in a future release.

Happy trading!

Hi, Backtraders!

I facing an issue retrieving the positions from a backtesting in Backtrader. I'm using Pyfolio analyzer to try to achieve this but it doesn't work properly.

This is what I got in positions for my strategy:

I tried the following function of Pyfolio but it doesn't work either:

The code I am using in backtrader to run the strategy is this one:

    cerebro = bt.Cerebro()

    # Add pyfolio analyzer
    cerebro.addanalyzer(bt.analyzers.PyFolio, _name='pyfolio')

    # Data feed --------------------------------
    for equity in equity_list:
        data = get_data(
            hp.cd_dospuntos(os.getcwd(), 2) + '/input_gral/backtest_data/' + equity + '.csv',
            from_date,
            to_date
        )
        cerebro.adddata(data, name=equity)

    # Set our desired cash start
    cerebro.broker.setcash(100000.0)

    # Comissions
    cerebro.broker.setcommission(commission=0.001)

    # Configuramos el slipage a 0.5%
    cerebro.broker.set_slippage_perc(perc=0.005,
                                     slip_open=True, slip_limit=True,
                                     slip_match=True, slip_out=False)

    # When working with large percentages it would be good to disable checksubmit in the broker. The rationale being
    # that if the current total portfolio is large, the broker might reject some orders if no cash is left.
    # You could order the orders in such a way that this doesn't happen, but disabling checksubmitwould be easier.
    cerebro.broker.set_checksubmit(checksubmit=False)

    # Strategy
    cerebro.addstrategy(
        TestStrategy,
        buy_and_hold=buy_and_hold,
        sellAll_date=sellAll_date,
    )

    start = time.time()
    print('Starting Portfolio Value: %.2f' % cerebro.broker.getvalue())

    # Execute
    results = cerebro.run(runonce=False)

    print('Final Portfolio Value: %.2f' % cerebro.broker.getvalue())
    end = time.time()

    # Obtiene los resultados para el posterior análisis con pyfolio
    strat = results[0]
    pyfoliozer = strat.analyzers.getbyname('pyfolio')
    returns, positions, transactions, gross_lev = pyfoliozer.get_pf_items()

    # Grabamos los resultados a un archivo pickle
    filepath = 'out/BTesting_results.pkl'
    if buy_and_hold:
        filepath = 'out/BTesting_results_buy_and_hold.pkl'
    file = open(filepath, 'wb')
    pikle_dict = {
        'returns': returns,
        'positions': positions,
        'transactions': transactions,
        'gross_lev': gross_lev,
    }
    pickle.dump(pikle_dict, file)

Any help is welcome. Thanks in advance.

@backtrader Thank you very much for your response and suggestions. I appreciate it.