Backtrader Community

    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    1. Home
    2. ard9000
    3. Posts
    For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/
    A
    • Profile
    • Following 0
    • Followers 1
    • Topics 4
    • Posts 8
    • Best 0
    • Controversial 0
    • Groups 0

    Posts made by ard9000

    • Adding data after cerebro.run()

      Hello Backtrader Community,

      I have a strategy that I create manual entries with using a charting application and GUI. I sling entries to BackTrader via ZeroMQ which runs in a separate thread and creates the entries. Once the position is live, the BackTrader script manages the position the stop loss. It works really well using the timer function which is called every second to check or new orders received via ZeroMQ.

      I'm running on IQFeed 1 minute data and using our FIX interface we developed for our firm with BackTrader.

      The only downside is I have to add the universe of 400 symbols to my script and then break it up into 4 smaller scripts, 100 on each symbols, so that each next() event is proficient. I run the 4 scripts in parallel concurrently and it works really well, just I'm using a lot of CPU and bandwidth.

      What I'm looking for is a way to add symbols on an as-needed basis so that I can create orders on symbols that have not been added via cerebro.adddata() when the script starts. This way, I can run a single smaller script and it just adds symbols as it receives entry orders for them. At the end of session, it clears the list and starts fresh the following session. The price/volume bars are not a concern since I have a separate method that pulls in price bars outside of self.datas as a workaround.

      For example, if I add data for SPY and QQQ when the script starts and then I receive an order for EWZ, which was not added using .adddata() when the script started, it will of course throw errors for self.getpositionbyname('EWZ) if I try to create an order object on this instance.

      Can .adddata() be called during .run() ?

      Any other suggestions?

      posted in General Code/Help
      A
      ard9000
    • Multiple exit stop orders at different prices

      I have a strategy that I'm running live with our FIX implementation and noticed there is a difference in the way the stops are executed live and with the default BT broker.

      An example:

      1. Buy 1000 shares of AAPL @ 200
      2. Place a stop loss for 1000 shares @ 198
      3. Buy another 1000 shares of AAPL @ 202
      4. Place a stop loss order for the second 1000 shares @ 201

      For a long position, the stop order looks like this in our code:

      # live trading
      stop_order = self.sell(data_min, price=stop_price, exectype=bt.Order.Stop, size=size)
      
      # backtesting 
      stop_order = self.close(data_min, price=stop_price, exectype=bt.Order.Stop, size=size)
      

      In this case, size = 1000. For backtesting, if I use self.sell or self.buy, the default BT broker doesn't take it as a stop loss order, it takes it as a stop entry and executes the order immediately. It works fine with self.close instead

      In live trading, this works as expected. If I look at live orders on the FIX side, there will be a stop order for 1000 @ 198 and a second order for 1000 @ 201.

      In backtesting, if the #4 stop is hit, it sells all 2000 shares and ignores the size= parameter. Is this how this works in the default BT broker or is there an additional parameter required.

      posted in General Code/Help
      A
      ard9000
    • RE: More IB bugs in live trading

      The test script that demonstrates this with IQFeed with the long IB symbols. If I'm doing something incorrectly here, I'd appreciate what is wrong.

      
      #!/usr/bin/env python
      """
      IQFeed live data
      
      IB Brokerage 
      
      """
      
      import pandas as pd
      
      import argparse
      import datetime
      import os
      import sys
      
      from pytz import timezone
      from pandas_market_calendars import get_calendar
      from pandas import Series, DataFrame, bdate_range
      
      import backtrader as bt
      
      class Strategy(bt.Strategy):
          """Trading strategy class."""
      
          params = (
              ('stocks', ('AAPL-STK-SMART-USD','TWTR-STK-SMART-USD','CAT-STK-SMART-USD',)),
              ('timezone', timezone('US/Eastern')),
              ('rsize', 1000),
              ('stdperiod', 30),
              ('debug', True),
              ('info', True),
          )
      
          def __init__(self):
              bt.Strategy.__init__(self)
      
              # create IB contract objects
      
              ibstore = bt.stores.ibstore.IBStore() # to be able to call makecontract API
              self.contracts = [ibstore.makecontract(*stock.split('-')) for stock in self.p.stocks]
              for i, stock in enumerate(self.p.stocks):
                  self.datas[i].tradecontract = self.contracts[i]
              
              # initialize parameters
      
              self.position_sizes = { 'AAPL-STK-SMART-USD': 1200,
                                      'TWTR-STK-SMART-USD': 0,
                                      'CAT-STK-SMART-USD': 300}
                                      
              
              self.stop_orders = {}
              self.df_min = {}
              self.daily = {}
              self.orders = []
              self.entry_price = {}
                    
              # set early close days
              calendar = get_calendar('NYSE')
              now = datetime.datetime.now()
              schedule = calendar.schedule(start_date=now - datetime.timedelta(weeks=1),
                                           end_date=now + datetime.timedelta(weeks=200))
              self.early_closes = calendar.early_closes(schedule).index
      
              self.p.debug = True
      
          def debug(self, msg):
              if self.p.debug:
                  dtime = self.datas[0].datetime
                  print('debug: %s %s, %s' % (dtime.date(0, tz=self.p.timezone),
                                              dtime.time(0, tz=self.p.timezone), msg))
      
          def info(self, msg):
              if self.p.info:
                  dtime = self.datas[0].datetime
                  print('info: %s %s, %s' % (dtime.date(0, tz=self.p.timezone),
                                             dtime.time(0, tz=self.p.timezone), msg))
      
          def notify_order(self, order):
              if order.status == order.Completed:
                  # calculate cost basis
                  stock = order.p.data._name
                  value = order.executed.value + order.executed.comm
                  entry_price = abs(value / order.executed.size)
                  self.info('%s: %s order executed: price: %.2f, value: %.2f, commm %.2f, '
                            'cost basis: %.2f, stock price: %s' %
                            (stock, order.ref, order.executed.price, order.executed.value,
                             order.executed.comm, entry_price, order.p.data[0]))
                  self.orders.remove(order)
                  
                 
      
          def notify_trade(self, trade):
              if trade.isclosed:
      
                  self.info('Trade value: %.2f, P/L %.2f, P/L comm: %.2f' %
                            (trade.value, trade.pnl, trade.pnlcomm ))
                 
          def eod(self):
              """
              Calculate time of eod actions.
              """
              if self.data0.datetime.date() in self.early_closes:
                  return datetime.time(12, 30)
              else:
                  return datetime.time(15, 54)
      
      
          def next(self):
              """Main processing is happening here. This method is called for every bar."""
          
             # check to make sure all data has been accumulated before calling next
      
              ctime = self.data0.datetime.time(0, tz=self.p.timezone)
              cdate = self.data0.datetime.date(0, tz=self.p.timezone)
              cdt = self.data0.datetime.datetime(0, tz=self.p.timezone)
              tz = self.p.timezone
      
              # skip bars from history
              if datetime.datetime.now(tz=tz) - tz.localize(self.data0.datetime.datetime()) >= datetime.timedelta(minutes=1):
                  return
      
              # check to make sure all data has been accumulated before calling next
              
              if len(set(data.datetime.datetime() for data in self.datas )) > 1:                                                                             
                  # skip processing if all minute bars not arrived yet                                                                                                                     
                  return
      
              # don't process bars after 16:00 or before 9:30
              
              for data in self.datas:
                  if data.datetime.time(0, tz=self.p.timezone) < datetime.time(9,30) or self.data.datetime.time(0, tz=self.p.timezone) > datetime.time(16,0,0):
                      return
      
                          
              for i, stock in enumerate(self.p.stocks):
                  
                 # get position data - first method using getdatabyname(stock)
                   
                  data_min = self.datas[i]
                  position = self.getposition(data_min)
      
                 # if there is no position and it's not set to zero, create a short order for it
      
                  if not position.size and self.position_sizes[stock] != 0:
                     order = self.sell(data_min, size=self.position_sizes[stock])
                     self.orders.append(order)
                     self.stop_orders[stock] = ''
      
                 # second position method
      
                  data_min2 = self.getdatabyname(stock.split("-")[0])
                  position2 = self.getpositionbyname(stock.split("-")[0])
      
                  position_size2 = position2.size
                  entry_price2 = position2.price
      
                  self.info(" \n getpositionbyname method %s position size: %s entry price: %s" %(stock, position_size2, entry_price2))
      
                 # demonstrate stop orders not being set
      
                  if position.size != 0 and position.price:
      
                      self.entry_price[stock] = position.price
      
                      self.info("%s position size: %s entry price: %s current close: %s" %(stock, position.size, position.price, data_min.close[0]))
      
                      if not self.stop_orders.get(stock) and self.entry_price[stock]:
                          stop_price = self.entry_price[stock] + .3
                          stop_order = self.close(data_min, price=stop_price, exectype=bt.Order.Stop)
                          self.stop_orders[stock] = stop_order
                          self.orders.append(stop_order)
                          self.info(" %s: stop order sent at %.2f" %(stock,stop_price))
      
                      
      
      def parse_args():
          parser = argparse.ArgumentParser(
                       formatter_class=argparse.ArgumentDefaultsHelpFormatter,
                       description='Live strategy run on Interactive Brokers')
      
          parser.add_argument('--host', default='127.0.0.1',
                              required=False, action='store',
                              help='Host for the Interactive Brokers TWS Connection')
      
          parser.add_argument('--qcheck', default=0.5, type=float,
                              required=False, action='store',
                              help=('Timeout for periodic '
                                    'notification/resampling/replaying check'))
      
          parser.add_argument('--port', default=7497, type=int,
                              required=False, action='store',
                              help='Port for the Interactive Brokers TWS Connection')
      
          parser.add_argument('--clientId', default=None, type=int,
                              required=False, action='store',
                              help='Client Id to connect to TWS (default: random)')
      
          parser.add_argument('--reconnect', default=3, type=int,
                              required=False, action='store',
                              help='Number of recconnection attempts to TWS')
      
          parser.add_argument('--timeout', default=180, type=float,
                              required=False, action='store',
                              help='Timeout between reconnection attempts to TWS')
      
          parser.add_argument('--debug',
                              required=False, action='store_true',
                              help='Display all info received form IB')
      
          return parser.parse_args()
                      
      def main(argv):
          """Entry point."""
          args = parse_args()
      
          # Create a cerebro entity
          cerebro = bt.Cerebro(maxcpus=1, oldsync=False)
      
          # Set broker
          broker = bt.brokers.IBBroker(host=args.host, port=args.port,
                                       clientId=args.clientId, reconnect=args.reconnect,
                                       timeout=args.timeout, _debug=args.debug)
          cerebro.setbroker(broker)
      
          # Create store
          store = bt.stores.IQFeedStore(reconnect=args.reconnect,
                                        timeout=args.timeout,
                                        debug=args.debug)
          
          # Add data feeds
          
          for stock in Strategy.params.stocks:
              ticker = stock.split('-')[0]
      
              hist_start_date = bdate_range(end=datetime.datetime.now(), periods=Strategy.params.stdperiod + 2)[0].to_pydatetime()
              data = bt.feeds.IQFeed(dataname=ticker, name=ticker,
                                     tz=Strategy.params.timezone,
                                     timeframe=bt.TimeFrame.Minutes,
                                     sessionstart=datetime.time(hour=9, minute=30),
                                     sessionend=datetime.time(hour=16, minute=00, second=59),
                                     fromdate=hist_start_date)
              cerebro.adddata(data)
      
          # Print out the starting conditions
          print 'Starting Portfolio Value: %.2f' % cerebro.broker.getvalue()
      
          # Add a strategy
          cerebro.addstrategy(Strategy)
      
          # Run over everything
          cerebro.run()
      
      if __name__ == '__main__':
          sys.exit(main(sys.argv))
      
      posted in General Code/Help
      A
      ard9000
    • RE: More IB bugs in live trading

      I'm sorry for the 'more bugs' headline, I agree it was harsh and not productive to resolving the issue.

      I am a licensed professional trader in the US, I'm a partner at a prop firm.I really love the platform and am championing bringing it into our firm as our python platform. We've been using it for the past 18 months to develop algos with and are now implementing it for live trading. We're using IB to test with as we build a FIX broker interface to work with Backtrader and our servers.

      My code does indeed look like that. We had Ed Bartosh integrate IQFeed into Backtrader for us. Ed also integrated CCXT into the platform for crypto trading, both of which we are giving to the Backtrader community for (hopefully) future inclusion in the platform. Both are available from his git branch. We've been debugging the IQFeed implementation for months in live trading, I believe it's production worthy.

      So we don't have to deal with using the IB symbol in our code 'AAPL-STK-SMART-USD' I run the code like this:

      
       params = (
              ('stocks', ('AAPL','TWTR','VIX.XO',)),
              ('tradable', ('AAPL','TWTR',)),
              ('timezone', timezone('US/Eastern')),)
      
      for i, stock in enumerate(self.p.stocks):
                  if stock in self.p.tradable:
                      self.contracts.append(ibstore.makecontract(stock,'STK','SMART','USD'))
                      self.datas[i].tradecontract = self.contracts[i]
      

      In main, I setup the symbols this way:

      # Set broker
          broker = bt.brokers.IBBroker(host=args.host, port=args.port,
                                       clientId=args.clientId, reconnect=args.reconnect,
                                       timeout=args.timeout, _debug=args.debug)
          cerebro.setbroker(broker)
      
          # Create store
          store = bt.stores.IQFeedStore(reconnect=args.reconnect,
                                        timeout=args.timeout,
                                        debug=args.debug)
          
          # Add data feeds
          
          for stock in Strategy.params.stocks:
              ticker = stock.split('-')[0]
      
              hist_start_date = bdate_range(end=datetime.datetime.now(), periods=Strategy.params.stdperiod + 2)[0].to_pydatetime()
              data = bt.feeds.IQFeed(dataname=ticker, name=ticker,
                                     tz=Strategy.params.timezone,
                                     timeframe=bt.TimeFrame.Minutes,
                                     sessionstart=datetime.time(hour=9, minute=30),
                                     sessionend=datetime.time(hour=16, minute=00, second=59),
                                     fromdate=hist_start_date)
              cerebro.adddata(data)
      

      Doing it this way, you can see we have the ticker = stock.split('-')[0] left over from when we were using the long IB symbols in self.p.stocks.

      We've made a test script to verify the bugs in the position.size and position.price when multiple positions are open in the same script. The other problem is the stop loss orders, they are intermittent.

       stop_order = self.close(data_min, price=stop_price, exectype=bt.Order.Stop)
       self.stop_orders[stock] = stop_order
       self.orders.append(stop_order)
      

      If the script is run 10 times, sometimes the stop orders will appear in IB TWS, other times they won't. There seems to be no reproducible pattern as to why the sometimes do and don't appear. All other orders work fine.

      Since this platform is free (and I really like it) community driven and to encourage growth of the platform, we're going to fix these now and give them to the community. Even though we will never trade live with IB. If you can give Ed some help when he needs it to expedite fixing these, it would be very helpful for everybody.

      posted in General Code/Help
      A
      ard9000
    • RE: Stop orders on interactive brokers

      @ard9000

      Bump.

      My hopes to this being resolved are fading quick. It doesn't appear, based on this and my other post about the getposition() bugs in live IB trading, that anyone is using BT for live equities trading.

      posted in General Code/Help
      A
      ard9000
    • More IB bugs in live trading

      I have a strategy that I'm running with my IB paper trading account. If I'm running three symbols, says AAPL, JNJ, CSCO and the strategy status is:

      AAPL long 1200 shares
      JNJ flat no position
      CSCO long 800 shares

      When live trading with IB, if my code looks like this:

      data = self.datas[i]
      position = getposition(data)
      
      entry_price = position.price
      position_size = position.size
      

      Let's say I'm pulling data for AAPL this way. For AAPL, it returns:

      position.size = 2000 and
      position.entry_price = the correct entry price

      The entry price works correctly, but the position.size returns the aggregate of all open positions. However the result gets stranger when using a different method. If I use:

      data = getdatabyname('AAPL')
      position = getposition(data)
      
      entry_price = position.price
      position_size = position.size
      

      The result of entry_price is an average across all symbols and position_size is still the aggregate of all open positions. The same thing happens if I use:

      data = self.datas[i]
      position = getpositionbyname('AAPL')
      
      entry_price = position.price
      position_size = position.size
      

      These all work fine in backtesting. I verified that [i] is the correct position in datas[i]. Is there something I'm missing here?

      posted in General Code/Help
      A
      ard9000
    • RE: Stop orders on interactive brokers

      The result is the same if I use:

      stop_order = self.close(data_min, price=stop_price, exectype=bt.Order.Stop)
      
      stop_order = self.buy(data_min, price=stop_price, exectype=bt.Order.Stop)
      
      stop_order = self.sell(data_min, price=stop_price, exectype=bt.Order.Stop)
      
      
      posted in General Code/Help
      A
      ard9000
    • Stop orders on interactive brokers

      I have a strategy that I'm implementing using the IB broker to interface with for my test account. Everything works great except stop orders. In my backtest strategy I have:

      stop_order = self.close(data_min, price=stop_price, exectype=bt.Order.Stop)
      self.orders.append(stop_order)
      

      Where data_min is the minute data feed I'm using. When stop orders are placed this way using exectype=bt.Order.Stop, there is no stop loss order created. Either with IB or within the strategy itself, if the price moves through the stop price, it never exits.

      All of the other entries and exits with BT work great with IB.

      Is there a specific exectype for IB market stop loss orders?

      posted in General Code/Help
      A
      ard9000
    • 1 / 1