Navigation

    Backtrader Community

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

    IB delayed vs live timeframes (interday trading)

    General Code/Help
    2
    3
    914
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • S
      seano last edited by

      Hi Backtrader (and community),

      I'm automating my existing manual system. I've recently hit something I'm struggling to troubleshoot so hope you can help. Intent is to use backtrader for live trading strategies that take a pre-screened short-list of LSE stocks and waits for break-outs (hitting new highs) before buying with a fixed/trailing stop. I can get the multiple datafeeds and trade triggers working ok, but have issues with the timeframes when implementing live system. Need to use daily data, looking at new highs of most recent close for the break-outs - only really need to run the script daily as most positions will be bought at market and held for more than a day (potentially a week or more).

      I used ibtest.py as original template. When I set timeframe = days and compression = 1 in datafeed the historical 'delayed' data is daily but it switches to tick data when switching to 'live' feed from IB. If I use resample then I'm getting only a small set of the daily data (missing days) so suspect it's actually resampling the daily data.

      Some specific questions:

      • Am I missing something obvious here in terms of parameters/approach? Or is this intentional logic seeing as daily close data wouldn't ever be available until market close anyway...
      • Should I just ignore the fact that live will call next() for every tick and just build functionality to only look at first tick on calling (if triggering script daily) or perhaps using timers?

      Below is main snippet I'm using to test:

      def runstrategy():
          
          # Hardcode the args
          args = dict(
              host='127.0.0.1',
              port=7497, #7496 for live,
              clientId=None,
              data0='VOD-STK-LSE', #'TWTR', 'EUR.USD-CASH-IDEALPRO', 'RWA-STK-LSE'
              data1=None,
              resample=False, #False,
              timeframe='Days', #bt.TimeFrame.Names[0], #Try 'Minutes' or 'Days'
              compression=1, #5,
          )
          
          # Create a cerebro
          cerebro = bt.Cerebro()
      
          #Use store method
          ibstore = bt.stores.IBStore(host=args['host'], port=args['port'], clientId=args['clientId'])
          broker = ibstore.getbroker()
          cerebro.setbroker(broker)
      
          #Set the timeframe
          timeframe = bt.TimeFrame.TFrame(args['timeframe'])
      
          if args['resample']:
              datatf = bt.TimeFrame.Ticks
              datacomp = 1
          else:
              datatf = timeframe
              datacomp = args['compression']
      
          #IBDataFactory = ibstore.getdata if args['usestore'] else bt.feeds.IBData
          IBDataFactory = ibstore.getdata
      
          #Create the data using datatf and datacomp
          data0 = IBDataFactory(dataname=args['data0'], timeframe=datatf, compression=datacomp)
      
          #Feed the strategy with resampled data if requested
          if args['resample']:
              cerebro.resampledata(data0, timeframe=timeframe, compression=args['compression'])
          else:
              cerebro.adddata(data0)
      
          # Add the strategy
          cerebro.addstrategy(TestStrategy)
      
          cerebro.run()
      

      Sample output:

      Data0, 0248, 737092.0, 2019-02-01T23:59:59.999989, 138.28, 139.42, 136.8, 138.84, 54488196.0, 0.0, 136.928
      Data0, 0249, 737095.0, 2019-02-04T23:59:59.999989, 139.2, 139.76, 136.7, 138.1, 52807693.0, 0.0, 137.46
      Data0, 0250, 737096.0, 2019-02-05T23:59:59.999989, 137.84, 141.54, 137.82, 141.06, 55387568.0, 0.0, 138.656
      Data0, 0251, 737097.0, 2019-02-06T23:59:59.999989, 142.2, 143.62, 142.0, 142.4, 55193955.0, 0.0, 139.796
      Data0, 0252, 737098.0, 2019-02-07T23:59:59.999989, 141.5, 141.76, 138.84, 139.38, 60064548.0, 0.0, 139.956
      Data0, 0253, 737098.0, 2019-02-08T00:00:00.000000, 139.0, 139.46, 138.52, 138.96, 5210654.0, 0.0, 139.98
      ***** DATA NOTIF: LIVE
      Data0, 0254, 737098.396565, 2019-02-08T09:31:03.195999, 138.96, 138.96, 138.96, 138.96, 8127.0, 0.0, 140.152
      Data0, 0255, 737098.397154, 2019-02-08T09:31:54.083995, 138.96, 138.96, 138.96, 138.96, 600.0, 0.0, 139.732
      Data0, 0256, 737098.397257, 2019-02-08T09:32:03.009997, 138.92, 138.92, 138.92, 138.92, 24490.0, 0.0, 139.036
      Data0, 0257, 737098.397405, 2019-02-08T09:32:15.790003, 138.9, 138.9, 138.9, 138.9, 4412.0, 0.0, 138.94
      Data0, 0258, 737098.397521, 2019-02-08T09:32:25.818997, 138.88, 138.88, 138.88, 138.88, 3223.0, 0.0, 138.924
      

      Loving the package and support! Appreciate any guidance. Many thanks.

      B 1 Reply Last reply Reply Quote 0
      • B
        backtrader administrators @seano last edited by

        @seano said in IB delayed vs live timeframes (interday trading):

        I used ibtest.py as original template. When I set timeframe = days and compression = 1 in datafeed the historical 'delayed' data is daily but it switches to tick data when switching to 'live' feed from IB

        There is no switch in backtrader. IB doesn't provide full bars in real-time, it does only provide ticks (or 5-seconds real-time bars). Those ticks need to be resampled to days. The execution snippets for ibtest.py show it.

        @seano said in IB delayed vs live timeframes (interday trading):

        • Am I missing something obvious here in terms of parameters/approach? Or is this intentional logic seeing as daily close data wouldn't ever be available until market close anyway...

        The daily close is obviously not available until the market closes, so you may want to reformulate your question.

        @seano said in IB delayed vs live timeframes (interday trading):

        • Should I just ignore the fact that live will call next() for every tick and just build functionality to only look at first tick on calling (if triggering script daily) or perhaps using timers?

        Resampling should do it, but you say you have missing bars, but we haven't seen that above.

        @seano said in IB delayed vs live timeframes (interday trading):

        Sample output:

        What was the input? (Because you have different execution paths)

        1 Reply Last reply Reply Quote 0
        • S
          seano last edited by

          @backtrader said in IB delayed vs live timeframes (interday trading):

          Thanks for quick response. To rephrase my question, I was asking whether there was intentional logic to 'switch' from daily to tick (or 5-seconds real-time bars) when going from delayed to live data... but as explained, there is no such switch so problem must be elsewhere.

          Just ran the script for TWTR using resampling and you will see that the delayed data below has missing dates:

          Data0, 0108, 737069.208333, 2019-01-10T00:00:00.000000, 29.9, 32.4, 29.76, 32.02, 470227.0, 0.0, 29.28
          Data0, 0109, 737072.208333, 2019-01-13T00:00:00.000000, 32.02, 33.5, 32.02, 32.77, 462363.0, 0.0, 30.448
          Data0, 0110, 737076.208333, 2019-01-17T00:00:00.000000, 32.75, 33.35, 32.12, 32.49, 313474.0, 0.0, 31.232
          Data0, 0111, 737079.208333, 2019-01-20T00:00:00.000000, 32.32, 33.89, 32.24, 33.2, 255231.0, 0.0, 32.116
          Data0, 0112, 737084.208333, 2019-01-25T00:00:00.000000, 33.01, 33.35, 30.72, 31.58, 455923.0, 0.0, 32.412
          Data0, 0113, 737089.208333, 2019-01-30T00:00:00.000000, 31.66, 33.67, 31.46, 31.91, 602043.0, 0.0, 32.39
          Data0, 0114, 737092.208333, 2019-02-02T00:00:00.000000, 32.0, 34.09, 31.42, 33.2, 526106.0, 0.0, 32.476
          Data0, 0115, 737097.208333, 2019-02-07T00:00:00.000000, 33.3, 35.29, 33.24, 35.05, 621249.0, 0.0, 32.988
          ***** DATA NOTIF: LIVE
          

          Full script below:

          from __future__ import (absolute_import, division, print_function,
                                  unicode_literals)
          
          import argparse
          import datetime
          
          # The above could be sent to an independent module
          import backtrader as bt
          from backtrader.utils import flushfile  # win32 quick stdout flushing
          
          
          class TestStrategy(bt.Strategy):
              params = dict(
                  smaperiod=5,
                  trade=True,
                  stake=10,
                  exectype=bt.Order.Market,
                  stopafter=20,
                  valid=None,
                  cancel=0,
                  donotsell=False,
                  stoptrail=False,
                  stoptraillimit=False,
                  trailamount=None,
                  trailpercent=None,
                  limitoffset=None,
                  oca=False,
                  bracket=False,
                  stop_loss=0.01, # NEW: price is 5% less than the entry point (Redundant if use trail I presume)
                  trail=False, # NEW: NOTE IT IS SIMILAR TO stoptrail; set trail=False if want fixed stop specified in stop_loss; OR set to a numeric value (e.g. 0.10) will tell the strategy to use a StopTrail
              )
          
          
              def __init__(self):
                  # To control operation entries
                  self.orderid = list()
                  self.order = None
          
                  self.counttostop = 0
                  self.datastatus = 0
          
                  # NEW: Keep a reference to the "close" line in the data[0] dataseries
                  self.dataclose = self.data0.close
                  
                  # NEW: Add a Highest Close indicator
                  # TODO: Note that doing this with period=100 means trades don't start until after 100 periods
                  # Should look at a way to simply keep a running record of highest high (in all history) instead
                  self.the_highest_close = bt.ind.Highest(self.data0.close, period=5, subplot=False)
                  
                  # Create SMA on 2nd data
                  self.sma = bt.indicators.MovAv.SMA(self.data, period=self.p.smaperiod)
          
                  print('--------------------------------------------------')
                  print('Strategy Created')
                  print('--------------------------------------------------')
          
              def notify_data(self, data, status, *args, **kwargs):
                  print('*' * 5, 'DATA NOTIF:', data._getstatusname(status), *args)
                  if status == data.LIVE:
                      self.counttostop = self.p.stopafter
                      self.datastatus = 1
          
              def notify_store(self, msg, *args, **kwargs):
                  print('*' * 5, 'STORE NOTIF:', msg)
          
              def notify_order(self, order):
                  if order.status in [order.Completed, order.Cancelled, order.Rejected]:
                      self.order = None
          
                  print('-' * 50, 'ORDER BEGIN', datetime.datetime.now())
                  print(order)
                  print('-' * 50, 'ORDER END')
          
              def notify_trade(self, trade):
                  print('-' * 50, 'TRADE BEGIN', datetime.datetime.now())
                  print(trade)
                  print('-' * 50, 'TRADE END')
          
              def prenext(self):
                  self.next(frompre=True)
          
              def next(self, frompre=False):
                  txt = list()
                  txt.append('Data0')
                  txt.append('%04d' % len(self.data0))
                  dtfmt = '%Y-%m-%dT%H:%M:%S.%f'
                  txt.append('{}'.format(self.data.datetime[0]))
                  txt.append('%s' % self.data.datetime.datetime(0).strftime(dtfmt))
                  txt.append('{}'.format(self.data.open[0]))
                  txt.append('{}'.format(self.data.high[0]))
                  txt.append('{}'.format(self.data.low[0]))
                  txt.append('{}'.format(self.data.close[0]))
                  txt.append('{}'.format(self.data.volume[0]))
                  txt.append('{}'.format(self.data.openinterest[0]))
                  txt.append('{}'.format(self.sma[0]))
                  #txt.append('{}'.format(self.the_highest_close[0])) #NEW: to see if it should trigger
                  #txt.append('{}'.format(self.dataclose[0])) #NEW: to see if it should trigger
                  print(', '.join(txt))
          
                  if len(self.datas) > 1 and len(self.data1):
                      txt = list()
                      txt.append('Data1')
                      txt.append('%04d' % len(self.data1))
                      dtfmt = '%Y-%m-%dT%H:%M:%S.%f'
                      txt.append('{}'.format(self.data1.datetime[0]))
                      txt.append('%s' % self.data1.datetime.datetime(0).strftime(dtfmt))
                      txt.append('{}'.format(self.data1.open[0]))
                      txt.append('{}'.format(self.data1.high[0]))
                      txt.append('{}'.format(self.data1.low[0]))
                      txt.append('{}'.format(self.data1.close[0]))
                      txt.append('{}'.format(self.data1.volume[0]))
                      txt.append('{}'.format(self.data1.openinterest[0]))
                      txt.append('{}'.format(float('NaN')))
                      print(', '.join(txt))
          
                  if self.counttostop:  # stop after x live lines
                      self.counttostop -= 1
                      if not self.counttostop:
                          self.env.runstop()
                          return
          
                  if not self.p.trade:
                      return
          
                  if self.datastatus and not self.position: # and len(self.orderid) < 1: #NEW Removed this
                      exectype = self.p.exectype if not self.p.oca else bt.Order.Limit
                      close = self.data0.close[0]
                      price = round(close * 0.90, 2)
                      
                      # NEW: Buy rule => Not yet ... we MIGHT BUY if ... 
                      if self.dataclose[0] > self.the_highest_close[-1]:
                              # A BREAKOUT !!!
          
                              # BUY, BUY, BUY!!! (with default parameters)
                              # self.log('BUY CREATE, %.2f' % self.dataclose[0])
          
                              # Keep track of the created order to avoid a 2nd order
                              # self.order = self.buy()
                              
                              self.order = self.buy(size=self.p.stake,
                                            exectype=exectype,
                                            price=price,
                                            valid=self.p.valid,
                                            transmit=not self.p.bracket)
                              
                              # Set a stop loss at the time of buy (need to enable cheat on close in __init__() first)
                              if not self.params.trail:
                                  stop_price = round(self.dataclose[0] * (1.0 - self.params.stop_loss), 2)
                                  self.sell(size=self.p.stake, exectype=bt.Order.Stop, price=stop_price)
                              else:
                                  # Originally used trailamount... but I changed to trailpercent
                                  # https://www.backtrader.com/docu/order-creation-execution/bracket/bracket.html?highlight=stoptrail
                                  self.sell(size=self.p.stake, exectype=bt.Order.StopTrail,
                                            trailpercent=self.p.trail)
                      
                      
                      self.orderid.append(self.order)
                      
                      # TODO: Look into if I should be using these instead of my stop logic above
                      if self.p.bracket:
                          # low side
                          self.sell(size=self.p.stake,
                                    exectype=bt.Order.Stop,
                                    price=round(price * 0.90, 2),
                                    valid=self.p.valid,
                                    transmit=False,
                                    parent=self.order)
          
                          # high side
                          self.sell(size=self.p.stake,
                                    exectype=bt.Order.Limit,
                                    price=round(close * 1.10, 2),
                                    valid=self.p.valid,
                                    transmit=True,
                                    parent=self.order)
          
                      elif self.p.oca:
                          self.buy(size=self.p.stake,
                                   exectype=bt.Order.Limit,
                                   price=round(self.data0.close[0] * 0.80, 2),
                                   oco=self.order)
          
                      elif self.p.stoptrail:
                          self.sell(size=self.p.stake,
                                    exectype=bt.Order.StopTrail,
                                    # price=round(self.data0.close[0] * 0.90, 2),
                                    valid=self.p.valid,
                                    trailamount=self.p.trailamount,
                                    trailpercent=self.p.trailpercent)
          
                      elif self.p.stoptraillimit:
                          p = round(self.data0.close[0] - self.p.trailamount, 2)
                          # p = self.data0.close[0]
                          self.sell(size=self.p.stake,
                                    exectype=bt.Order.StopTrailLimit,
                                    price=p,
                                    plimit=p + self.p.limitoffset,
                                    valid=self.p.valid,
                                    trailamount=self.p.trailamount,
                                    trailpercent=self.p.trailpercent)
          
                  # TAKE THIS OUT AND JUST HAVE THE STOP LOSS
                  #elif self.position.size > 0 and not self.p.donotsell:
                  #    if self.order is None:
                  #        self.order = self.sell(size=self.p.stake // 2,
                  #                               exectype=bt.Order.Market,
                  #                               price=self.data0.close[0])
          
                  elif self.order is not None and self.p.cancel:
                      if self.datastatus > self.p.cancel:
                          self.cancel(self.order)
          
                  if self.datastatus:
                      self.datastatus += 1
          
              def start(self):
                  if self.data0.contractdetails is not None:
                      print('Timezone from ContractDetails: {}'.format(
                            self.data0.contractdetails.m_timeZoneId))
          
                  header = ['Datetime', 'Open', 'High', 'Low', 'Close', 'Volume',
                            'OpenInterest', 'SMA']
                  print(', '.join(header))
          
                  self.done = False
          
          
          def runstrategy():
              
              # Hardcode the args
              args = dict(
                  host='127.0.0.1',
                  port=7497, #7496 for live,
                  clientId=None,
                  data0='TWTR', #'TWTR', #None, #'EUR.USD-CASH-IDEALPRO'RWA-STK-LSE'
                  data1=None,
                  resample=True, #False,
                  timeframe='Days', #bt.TimeFrame.Names[0], #Try 'Minutes' or 'Days'
                  compression=1, #5,
              )
              
              # Create a cerebro
              cerebro = bt.Cerebro()
          
              #Use store method
              ibstore = bt.stores.IBStore(host=args['host'], port=args['port'], clientId=args['clientId'])
              broker = ibstore.getbroker()
              cerebro.setbroker(broker)
          
              #Set the timeframe
              timeframe = bt.TimeFrame.TFrame(args['timeframe'])
          
              if args['resample']:
                  datatf = bt.TimeFrame.Ticks
                  datacomp = 1
              else:
                  datatf = timeframe
                  datacomp = args['compression']
          
              #IBDataFactory = ibstore.getdata if args['usestore'] else bt.feeds.IBData
              IBDataFactory = ibstore.getdata
          
              #Create the data using datatf and datacomp
              data0 = IBDataFactory(dataname=args['data0'], timeframe=datatf, compression=datacomp)
          
              #Feed the strategy with resampled data if requested
              if args['resample']:
                  cerebro.resampledata(data0, timeframe=timeframe, compression=args['compression'])
              else:
                  cerebro.adddata(data0)
          
              # Add the strategy
              cerebro.addstrategy(TestStrategy)
          
              cerebro.run()
          
          
          if __name__ == '__main__':
              runstrategy()
          
          1 Reply Last reply Reply Quote 0
          • 1 / 1
          • First post
            Last post
          Copyright © 2016, 2017, 2018, 2019, 2020, 2021 NodeBB Forums | Contributors