Backtrader Community
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

    multi asset, time-frame complications (backfilling with IB?)

    General Code/Help
    2
    3
    698
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • H
      hbf last edited by hbf

      I hope this can help others that wish to implement a multi time-frame and asset strategy.

      I am having problems with the strategy actually running for what seems to be higher timeframe/compression combinations.

      When I run the code below with arguments --compression0 2 --timeframe1 minute --compression1 5 the strategy works as intended and I am receiving the signal output during the backfill process.

      Server Version: 76
TWS Time at connection:20190104 14:37:53 AEST
***** EUR.USD-CASH-IDEALPRO-60m DATA NOTIF: DELAYED
***** GBP.USD-CASH-IDEALPRO-60m DATA NOTIF: DELAYED
***** GBP.AUD-CASH-IDEALPRO-60m DATA NOTIF: DELAYED
2019-01-03T10:00:00 EUR.USD-CASH-IDEALPRO-1d signal: none
2019-01-03T10:00:00 GBP.USD-CASH-IDEALPRO-1d signal: none
2019-01-03T10:00:00 GBP.AUD-CASH-IDEALPRO-1d signal: none
2019-01-03T10:02:00 EUR.USD-CASH-IDEALPRO-1d signal: buy
2019-01-03T10:02:00 GBP.USD-CASH-IDEALPRO-1d signal: buy
2019-01-03T10:02:00 GBP.AUD-CASH-IDEALPRO-1d signal: sell
...
2019-01-03T22:36:00 EUR.USD-CASH-IDEALPRO-1d signal: buy
2019-01-03T22:36:00 GBP.USD-CASH-IDEALPRO-1d signal: buy
2019-01-03T22:36:00 GBP.AUD-CASH-IDEALPRO-1d signal: sell
***** EUR.USD-CASH-IDEALPRO-60m DATA NOTIF: LIVE
***** GBP.USD-CASH-IDEALPRO-60m DATA NOTIF: LIVE
***** GBP.AUD-CASH-IDEALPRO-60m DATA NOTIF: LIVE
2019-01-03T22:38:00 EUR.USD-CASH-IDEALPRO-1d signal: buy
2019-01-03T22:38:00 EUR.USD-CASH-IDEALPRO-60m signal: buy order id: 1
2019-01-03T22:38:00 GBP.USD-CASH-IDEALPRO-1d signal: buy
2019-01-03T22:38:00 GBP.USD-CASH-IDEALPRO-60m signal: buy order id: 2
2019-01-03T22:38:00 GBP.AUD-CASH-IDEALPRO-1d signal: sell
2019-01-03T22:38:00 GBP.AUD-CASH-IDEALPRO-60m signal: sell order id: 3

      However, when using the same code with arguments --compression0 60 --timeframe1 daily --compression1 1, backtrader connects to the broker but fails to produce any of the signals. The output below is all that is received and the strategy fails to execute (I have kept it running for multiple hours with no additional output). Running with debug shows that messages are being received from the broker, but nothing happens.

      Server Version: 76
TWS Time at connection:20190104 14:37:15 AEST
***** EUR.USD-CASH-IDEALPRO-60m DATA NOTIF: DELAYED
***** GBP.USD-CASH-IDEALPRO-60m DATA NOTIF: DELAYED
***** GBP.AUD-CASH-IDEALPRO-60m DATA NOTIF: DELAYED
***** EUR.USD-CASH-IDEALPRO-60m DATA NOTIF: LIVE
***** GBP.USD-CASH-IDEALPRO-60m DATA NOTIF: LIVE
***** GBP.AUD-CASH-IDEALPRO-60m DATA NOTIF: LIVE

      The above example works when using 1hr/1day historical data saved locally. I feel like this has to do with IB but my understanding is that 1 year of data is within the historical download limits (needed for 200 day moving average). Potentially there is a problem with using multiple assets?

      I feel a bit stuck as I have been working on this for some time to no avail. I would appreciate if anyone has had any luck with the problems I am facing.

      Thank you for your time!

      from __future__ import (absolute_import, division, print_function,
                        unicode_literals)

import argparse
import datetime
import math

import pandas as pd
import backtrader as bt

class ParcelSizer(bt.Sizer):

    params = dict(stake=10000)

    def _getsizing(self, comminfo, cash, data, isbuy):
        sz = math.floor(self.p.stake / data.close[0])
        return int(sz)

class CFDSt(bt.Strategy):
    params = dict(
        smaperiod=200,
        rsiperiod=5,
        atrperiod=10,
    )

    def notify_order(self, order):
        if order.status == order.Submitted:
            return

        dt, dn = self.data.datetime.datetime(0).isoformat(), order.data._name
        print('{} {} Order {} Status {}'.format(
            dt, dn, order.ref, order.getstatusname())
        )

        if order.status == order.Completed:
            return

        whichord = ['main', 'stop', 'limit', 'close']
        if not order.alive():  # not alive - nullify
            dorders = self.o[order.data]
            idx = dorders.index(order)
            dorders[idx] = None
            print('{} {} Order no longer alive {} Ref'.format(dt, dn,
                                                              whichord[idx]))

            if all(x is None for x in dorders):
                dorders[:] = []  # empty list - New orders allowed

    def notify_trade(self, trade):
        dt, dn = self.data.datetime.datetime(0).isoformat(), trade.data._name
        if trade.isclosed:
            print('{} {} Gross Profit {} Net Profit {}' \
            .format(dt, dn, round(trade.pnl,2), round(trade.pnlcomm,2)))


    def __init__(self):
        self.o = dict()  # orders per data (main, stop, limit, manual-close)
        self.lendata = dict()
        self.inds = dict()

        for i, d in enumerate(self.datas):
            self.inds[d._name] = dict()
            self.inds[d._name]['sma'] = bt.indicators.SMA(d,
                                            period=self.p.smaperiod)
            self.inds[d._name]['signal'] = 'none'

    data_live = False

    def notify_data(self, data, status, *args, **kwargs):
        print('*' * 5, data._name ,'DATA NOTIF:', data._getstatusname(status),
              *args)
        if status == data.LIVE:
            self.data_live = True

    def nextstart(self):
        for i, d in enumerate(self.datas):
            self.lendata[d] = len(d)

    def next(self):
        for i, d in enumerate(self.datas):

            dt, dn = self.data.datetime.datetime(0).isoformat(), d._name
            if (i + 1) % 2 != 0:
                # every odd data feed (shorter timeframe first)
                pos = self.getposition(d).size # get position
                dnht = self.datas[i+1]._name # higher timeframe dataname

                print('{} {} signal: {}'.format(dt, dnht,
                    self.inds[dnht]['signal'])) # print the signal from the higher timeframe

                if self.inds[dnht]['signal'] == 'buy' and self.data_live:
                    self.o[d] = [self.buy(data=d)]
                    print('{} {} signal: {} order id: {}'.format(dt, dn,
                        self.inds[dnht]['signal'], self.o[d][0].ref))
                    self.inds[dnht]['signal'] = 'none' # reset signal

                elif self.inds[dnht]['signal'] == 'sell' and self.data_live:
                    self.o[d] = [self.sell(data=d)]
                    print('{} {} signal: {} order id: {}'.format(dt, dn,
                        self.inds[dnht]['signal'], self.o[d][0].ref))
                    self.inds[dnht]['signal'] = 'none'

                else:
                    pass

            else:
                # every even data feed (higher timeframe second)
                pos = self.getposition(self.datas[i-1]).size
                # position on smaller time frame
                dnlt = self.datas[i-1]._name # lower timeframe dataname

                if self.lendata[d] < len(d):
                    # new higher timeframe bar
                    self.lendata[d] += 1

                    if not pos and not self.o.get(self.datas[i-1], None):
                        # no orders/position
                        if d.close[0] < self.inds[dn]['sma'][0]:
                            self.inds[dn]['signal'] = 'sell'

                        elif d.close[0] > self.inds[dn]['sma'][0]:
                            self.inds[dn]['signal'] = 'buy'

                        else:
                            self.inds[dn]['signal'] = 'none'

                    else:
                        pass

def runstrat(args=None):

    args = parse_args(args)
    cerebro = bt.Cerebro()

    # Store
    store = bt.stores.IBStore(port=7497, host='127.0.0.1', _debug=args.debug,
                              notifyall=args.notifyall, reconnect=-1,
                              timeout=60)

    securities = ['EUR.USD-CASH-IDEALPRO','GBP.USD-CASH-IDEALPRO',
                  'GBP.AUD-CASH-IDEALPRO']

    tframes = dict(seconds=bt.TimeFrame.Seconds,
                   minute=bt.TimeFrame.Minutes, daily=bt.TimeFrame.Days,
                   weekly=bt.TimeFrame.Weeks, monthly=bt.TimeFrame.Months)

    for security, contract in zip(securities,contracts):
        data = store.getdata(dataname=security, tradename=contract,
                             timeframe=tframes[args.timeframe],
                             compression=int(args.compression))

        cerebro.resampledata(data, timeframe=tframes[args.timeframe0],
                             compression=int(args.compression0),
                             name=str(security+args.name0))

        cerebro.resampledata(data, timeframe=tframes[args.timeframe1],
                             compression=int(args.compression1),
                             name=str(security+args.name1))

    # Broker
    cerebro.broker = store.getbroker()

    # Strategy
    cerebro.addstrategy(CFDSt)

    # Sizer
    cerebro.addsizer(ParcelSizer)

    # Execute
    cerebro.run()

def parse_args(pargs=None):
    parser = argparse.ArgumentParser(
        formatter_class=argparse.ArgumentDefaultsHelpFormatter,
        description=(
            'Multiple Values'
        )
    )

    parser.add_argument('--cerebro', required=False, default='',
                        metavar='kwargs', help='kwargs in key=value format')

    parser.add_argument('--debug', required=False,
                        default=False, action='store',
                        help='Debug parameter')

    parser.add_argument('--notifyall', required=False,
                        default=False, action='store',
                        help='If True, any error message sent by IB is relayed')

    parser.add_argument('--broker', required=False, default='',
                        metavar='kwargs', help='kwargs in key=value format')

    parser.add_argument('--sizer', required=False, default='',
                        metavar='kwargs', help='kwargs in key=value format')

    parser.add_argument('--strat', required=False, default='',
                        metavar='kwargs', help='kwargs in key=value format')

    parser.add_argument('--timeframe', required=False,
                        default='minute', action='store',
                        help='datafeed timeframe parameter')

    parser.add_argument('--compression', required=False,
                        default=5, action='store',
                        help='datafeed compression parameter')

    parser.add_argument('--timeframe0', required=False,
                        default='minute', action='store',
                        help='datafeed0 timeframe parameter (resampled)')

    parser.add_argument('--compression0', required=False,
                        default=60, action='store',
                        help='datafeed0 compression parameter (resampled)')

    parser.add_argument('--name0', required=False,
                        default='-60m', action='store',
                        help='datafeed0 name parameter (resampled)')

    parser.add_argument('--timeframe1', required=False,
                        default='daily', action='store',
                        help='datafeed1 timeframe parameter (resampled)')

    parser.add_argument('--compression1', required=False,
                        default=1, action='store',
                        help='datafeed1 compression parameter (resampled)')

    parser.add_argument('--name1', required=False,
                        default='-1d', action='store',
                        help='datafeed1 name parameter (resampled)')

    parser.add_argument('--asx', required=False,
                        default='20', action='store',
                        help='asx group parameter')

    return parser.parse_args(pargs)


if __name__ == '__main__':
    runstrat()
      1 Reply Last reply Reply Quote 0
      • B
        backtrader administrators last edited by backtrader

        Get independent data feeds (even if for the same underlying asset) for each resampling and give it a try. The timeframes are too far apart and (without calculating it) it can't probably not get enough historical data in 60-minutes format to fulfill the 1-day needs.

        Do it in general, even if the timeframes are not so different, given the constraints IB puts in place for data downloading.

        1 Reply Last reply Reply Quote 1
        • H
          hbf last edited by

          Thank you very much! This appears to have solved the issue.

          1 Reply Last reply Reply Quote 0
          • 1 / 1
          • First post
            Last post
          Copyright © 2016, 2017, 2018, 2019, 2020, 2021 NodeBB Forums | Contributors