Backtrader Community

    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    1. Home
    2. blonc
    For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/
    B
    • Profile
    • Following 0
    • Followers 2
    • Topics 16
    • Posts 44
    • Best 5
    • Controversial 1
    • Groups 0

    blonc

    @blonc

    7
    Reputation
    1036
    Profile views
    44
    Posts
    2
    Followers
    0
    Following
    Joined Last Online

    blonc Unfollow Follow

    Best posts made by blonc

    • marketstore NEW data feed complete code & example

      to read more about marketstore. MarketStore is a database server optimized for financial timeseries data. I am integrating MS because it as well allows for real-time streaming via a websocket of the bars from the db. IMO this will allow for consistency between realtime and historical testing. Data is stored in tick compression and then using On-disk Aggregate updates the downsample data compressed into any timeframe you want. Easiest way to get this to work is to build MS from source in a docker, its already integrated with GDAX and works out the box.

      The next step for me is to integrate polygon.io which is REALTIME trade by trade SIP data for 200$ a month, latency is under 10ms too if you are colocated in or around ny4.

      will update with the polygon connector to MS once it is complete and will post the live datafeed code as well once done. Will update this code too once all the rest is complete.

      thanks,

      datafeed:

              import backtrader as bt
              from ..utils import date2num
              import backtrader.feed as feed
              import datetime
              import pytz
              import pymarketstore as pymkts
      
      
              class MarketStore(feed.DataBase):
      
                  params = (
                      ('dataname', None),
                      ('fromdate', datetime.date(1990, 1, 1)),
                      ('todate', datetime.date(2050,1,1)),
                      ('name', ''),
                      ('compression', 1),
                      ('timeframe', bt.TimeFrame.Days),
                      ('host', '127.0.0.1'),
                      ('port', '5993'),
                      ('symbol', None),
                      ('query_timeframe', None),
      
                  )
      
                  def start(self):
                      super(MarketStore, self).start()
      
                      self.ndb = pymkts.Client('http://{host}:{port}/rpc'.format(
                          host=self.p.host,
                          port=self.p.port
                      ))
      
                      qstr = pymkts.Params(self.p.symbol,
                                           self.p.query_timeframe,
                                           'OHLCV',
                                           start=self.p.fromdate.isoformat(),
                                           end=self.p.todate.isoformat())
      
      
                      dbars = list(self.ndb.query(qstr).first().array)
      
                      self.biter = iter(dbars)
      
      
                  def _load(self):
                      try:
                          bar = next(self.biter)
                      except StopIteration:
                          return False
      
      
                      self.l.datetime[0] = date2num(datetime.datetime.fromtimestamp(bar[0],pytz.utc))
                      self.l.open[0] = bar[1]
                      self.l.high[0] = bar[2]
                      self.l.low[0] = bar[3]
                      self.l.close[0] = bar[4]
                      self.l.volume[0] = bar[5]
      
                      return True
      

      simple test strategy:

          from __future__ import (absolute_import, division, print_function,
                                  unicode_literals)
          import pytz
          import datetime
          import backtrader as bt
          
          class Master(bt.Strategy):
          
              def __init__(self):
                  self.dataopen = self.datas[0].open
                  self.datahigh = self.datas[0].high
                  self.datalow = self.datas[0].low
                  self.dataclose = self.datas[0].close
          
              def next(self):
                  # self.log('Line')
                  print(self.datas[0].datetime.date(0), self.datas[0].datetime.time(0),
                        self.dataopen[0],
                        self.datahigh[0],
                        self.datalow[0],
                        self.dataclose[0],
                        self.datas[0]._name
                        )
          
          def runstrat():
              cerebro = bt.Cerebro()
          
              cerebro.broker.setcash(550000.0)
          
              data123 = bt.feeds.MarketStore(
                  symbol='ETH',
                  name='ETH',
                  query_timeframe='1Hour',
                  timeframe=bt.TimeFrame.Minutes,
                  fromdate=datetime.date(2018,6,3),
                  todate=datetime.date(2018, 6,3),
                  # sessionstart=datetime.time(7),
                  # sessionend=datetime.time(10),
                  tz=pytz.timezone('US/Eastern'),
          
              )
          
              cerebro.adddata(data123)
          
              cerebro.addstrategy(Master)
          
              cerebro.run()
          
              print('finished')
          
          if __name__ == '__main__':
              runstrat()
      
      posted in General Code/Help
      B
      blonc
    • passing data to strategy via an *args

      I have a pairs strategy, that I want to re-use with multiple pairs. When I add the strategy as thus:

      `cerebro.addstrategy(TestStrategy,d0=NTRS,d1=PNC)`
      

      and then for my main :

      `def __init__(self,d0,d1):`
      

      with in that I try to pull the data from that

         ` self.data_0_close = self.d0.close
          self.data_1_close = self.d1.close`
      

      however when I run the strategy I get the following error:

      'Lines_LineSeries_LineIterator_DataAccessor_Strateg' object has no attribute 'd0'

      posted in General Code/Help
      B
      blonc
    • RE: passing data to strategy via an *args

      @nooby_mcnoob
      thanks you, that makes more sense.

      for anyone in the future: if you are using multiple data sets and want to define the data that goes specifically to that strategy. You need to define the data when you import it.

      data_feed_X = bt.feeds.GenericCSVData()

      when you add a strategy you need to pass the data you want to that strategy IE

      cerebro.addstrategy(TestStrategy,d0=data_feed_X,d1=data_feed_Y)

      then with in the strategy you need to add params that pull the data into the strategy, IE:

      params = ( ('d0', None ), ('d1', None), )

      now you are set you can call:

      self.params.d0.close[0]

      and it will work just like calling data0 or datas0 or so on.

      posted in General Code/Help
      B
      blonc
    • RE: Strategy Selection - Passing dynamic strategy name

      ** solved **

      passing a variable to the strategy params() I can then pull that that variable in my print line. The variables sit under strat.params. . or looking something like the following:

      results = cerebro.run()
      
      strats = [x[0] for x in results]  # flatten the result
      
      for i, strat in enumerate(strats):
          rets = strat.analyzers.returns.get_analysis()
          print('Strat {} Name {}:\n  - analyzer: {}\n'.format(
              i, strat.params.example_param, rets))
      
      posted in General Code/Help
      B
      blonc
    • RE: How to: change currency in datafeed

      @jacob gonna be a lot easier to just write a simple csv data feed handler and a lot cleaner. most the code is there to do it, just change as needed.. import the two files, do the conversion and pass it along. this will be totally reusable too for any two files you need.

      posted in General Code/Help
      B
      blonc
    • calling data in int() by name / dataname

      data by name.

      referencing data by _name

      when adding data I do have the attribute name in this example well say name = SWK with in the bt.feeds.GenericCSVData()

      Now with in def __init__(self): I am trying to call the data by name and can not seem to get it... IE
      self.data_SWK.close does not work... self.datas_SWK.close does not work either.

      referencing source data_NAME

      line 109 ` - self.dataX_name -> self.datas[X].name . But this is for an analyzer, maybe its not possible any other place.

      Maybe I am miss interpreting the documents.

      posted in General Code/Help
      B
      blonc

    Latest posts made by blonc

    • RE: How to: change currency in datafeed

      if you are looking for something quick and dirty this bit of code can do literally everything you want. just import the CSV files, do the math in a np.array and then pass it to _load as a iter . you can add params at the top to pass from the strategy side and there you go.

      class MyCryptoCSVData(bt.CSVDataBase):
          params = (
         ('filepath1', None),
         ('filepath2', None),
         ('anything you want', None),
        )
      def start(self):
          # read in the CSV values, do your math and put the list into a iter()
          pass
      
      def stop(self):
          # close your read files      
      
      def _loadline(self, linetokens):
          i = itertools.count(0)
      
          dttxt = linetokens[next(i)]
          # Format is YYYY-MM-DD
          y = int(dttxt[0:4])
          m = int(dttxt[5:7])
          d = int(dttxt[8:10])
      
          dt = datetime.datetime(y, m, d)
          dtnum = date2num(dt)
      
          self.lines.datetime[0] = dtnum
          self.lines.open[0] = float(linetokens[next(i)])
          self.lines.high[0] = float(linetokens[next(i)])
          self.lines.low[0] = float(linetokens[next(i)])
          self.lines.close[0] = float(linetokens[next(i)])
          self.lines.volume[0] = float(linetokens[next(i)])
          self.lines.openinterest[0] = float(linetokens[next(i)])
      
          return True
      
      posted in General Code/Help
      B
      blonc
    • RE: How to: change currency in datafeed

      @jacob gonna be a lot easier to just write a simple csv data feed handler and a lot cleaner. most the code is there to do it, just change as needed.. import the two files, do the conversion and pass it along. this will be totally reusable too for any two files you need.

      posted in General Code/Help
      B
      blonc
    • RE: How to: change currency in datafeed

      @Jacob To better understand how a data feed works maybe read this first CSV Data Feed Development , then move onto this Extending a Datafeed , and everything you want to do should be possible.

      posted in General Code/Help
      B
      blonc
    • RE: How to: change currency in datafeed

      @Jacob add another set of data that is daily data for forex for the conversion needed. then do arithmetic with the two datasets self.datas[0].close vs self.datas[1].close to give you the needed outputs.

      posted in General Code/Help
      B
      blonc
    • marketstore NEW data feed complete code & example

      to read more about marketstore. MarketStore is a database server optimized for financial timeseries data. I am integrating MS because it as well allows for real-time streaming via a websocket of the bars from the db. IMO this will allow for consistency between realtime and historical testing. Data is stored in tick compression and then using On-disk Aggregate updates the downsample data compressed into any timeframe you want. Easiest way to get this to work is to build MS from source in a docker, its already integrated with GDAX and works out the box.

      The next step for me is to integrate polygon.io which is REALTIME trade by trade SIP data for 200$ a month, latency is under 10ms too if you are colocated in or around ny4.

      will update with the polygon connector to MS once it is complete and will post the live datafeed code as well once done. Will update this code too once all the rest is complete.

      thanks,

      datafeed:

              import backtrader as bt
              from ..utils import date2num
              import backtrader.feed as feed
              import datetime
              import pytz
              import pymarketstore as pymkts
      
      
              class MarketStore(feed.DataBase):
      
                  params = (
                      ('dataname', None),
                      ('fromdate', datetime.date(1990, 1, 1)),
                      ('todate', datetime.date(2050,1,1)),
                      ('name', ''),
                      ('compression', 1),
                      ('timeframe', bt.TimeFrame.Days),
                      ('host', '127.0.0.1'),
                      ('port', '5993'),
                      ('symbol', None),
                      ('query_timeframe', None),
      
                  )
      
                  def start(self):
                      super(MarketStore, self).start()
      
                      self.ndb = pymkts.Client('http://{host}:{port}/rpc'.format(
                          host=self.p.host,
                          port=self.p.port
                      ))
      
                      qstr = pymkts.Params(self.p.symbol,
                                           self.p.query_timeframe,
                                           'OHLCV',
                                           start=self.p.fromdate.isoformat(),
                                           end=self.p.todate.isoformat())
      
      
                      dbars = list(self.ndb.query(qstr).first().array)
      
                      self.biter = iter(dbars)
      
      
                  def _load(self):
                      try:
                          bar = next(self.biter)
                      except StopIteration:
                          return False
      
      
                      self.l.datetime[0] = date2num(datetime.datetime.fromtimestamp(bar[0],pytz.utc))
                      self.l.open[0] = bar[1]
                      self.l.high[0] = bar[2]
                      self.l.low[0] = bar[3]
                      self.l.close[0] = bar[4]
                      self.l.volume[0] = bar[5]
      
                      return True
      

      simple test strategy:

          from __future__ import (absolute_import, division, print_function,
                                  unicode_literals)
          import pytz
          import datetime
          import backtrader as bt
          
          class Master(bt.Strategy):
          
              def __init__(self):
                  self.dataopen = self.datas[0].open
                  self.datahigh = self.datas[0].high
                  self.datalow = self.datas[0].low
                  self.dataclose = self.datas[0].close
          
              def next(self):
                  # self.log('Line')
                  print(self.datas[0].datetime.date(0), self.datas[0].datetime.time(0),
                        self.dataopen[0],
                        self.datahigh[0],
                        self.datalow[0],
                        self.dataclose[0],
                        self.datas[0]._name
                        )
          
          def runstrat():
              cerebro = bt.Cerebro()
          
              cerebro.broker.setcash(550000.0)
          
              data123 = bt.feeds.MarketStore(
                  symbol='ETH',
                  name='ETH',
                  query_timeframe='1Hour',
                  timeframe=bt.TimeFrame.Minutes,
                  fromdate=datetime.date(2018,6,3),
                  todate=datetime.date(2018, 6,3),
                  # sessionstart=datetime.time(7),
                  # sessionend=datetime.time(10),
                  tz=pytz.timezone('US/Eastern'),
          
              )
          
              cerebro.adddata(data123)
          
              cerebro.addstrategy(Master)
          
              cerebro.run()
          
              print('finished')
          
          if __name__ == '__main__':
              runstrat()
      
      posted in General Code/Help
      B
      blonc
    • RE: backtrader giving me identical datetime prints on every line

      SOLVED

      the data feed was fine, it was with in the next() have to split from datetime to date and separate time

          class Master(bt.Strategy):
          
              def __init__(self):
                  self.dataclose = self.datas[0].close
          
              def next(self):
                  # self.log('Line')
                  print(self.datas[0].datetime.date(0), self.datas[0].datetime.time(0))
          
          def runstrat():
              cerebro = bt.Cerebro()
              cerebro.broker.setcash(550000.0)
          
              data123 = bt.feeds.MarketStore(
                  symbol='BTC',
                  query_timeframe='1Min',
                  timeframe=bt.TimeFrame.Minutes,
          
              )
          
              cerebro.adddata(data123)
          
              cerebro.addstrategy(Master)
          
              cerebro.run()
          
              print('finished')
          
          if __name__ == '__main__':
              runstrat()
      

      output

          2018-06-03 12:00:00 7685.13
          2018-06-03 12:01:00 7685.13
          2018-06-03 12:02:00 7685.13
          2018-06-03 12:03:00 7685.13
          2018-06-03 12:04:00 7685.13
          2018-06-03 12:05:00 7685.12
          2018-06-03 12:06:00 7685.12
          2018-06-03 12:07:00 7681.01
          2018-06-03 12:08:00 7681.01
          2018-06-03 12:09:00 7681.01
          finished
      
      posted in General Code/Help
      B
      blonc
    • backtrader giving me identical datetime prints on every line

      I have checked all the logical places for this, but still backtrader things I am passing daily bars. the strategy code is as followed:

          class Master(bt.Strategy):
          
              def __init__(self):
                  self.dataclose = self.datas[0].close
                  self.dt = self.datas[0].datetime.datetime(0)
              def next(self):
                  print(self.dt, self.dataclose[0])
          
          
          def runstrat():
              cerebro = bt.Cerebro()
              cerebro.broker.setcash(550000.0)
          
              data123 = bt.feeds.MarketStore(
                  symbol='ETH',
                  query_timeframe='1Min',
                  timeframe=bt.TimeFrame.Minutes,
              )
          
              cerebro.adddata(data123)
          
              cerebro.addstrategy(Master)
          
              cerebro.run()
          
              print('finished')
          
          if __name__ == '__main__':
              runstrat()
      

      the output is as followed. The first 10 lines comes from the datafeed directly, I have a line right after it loads the datetime to print(datetime.datetime.fromtimestamp(bar[0]).isoformat()) . So I know both time and date are being passed correctly after it gets passed through the date2num function :

          ## from datafeed directly
          2018-06-03T11:13:00
          2018-06-03T11:14:00
          2018-06-03T11:15:00
          2018-06-03T11:16:00
          2018-06-03T11:17:00
          2018-06-03T11:18:00
          2018-06-03T11:19:00
          2018-06-03T11:20:00
          2018-06-03T11:21:00
          2018-06-03T11:22:00
          ## from BT strategy printing as though it is daily bars
          2018-06-03 11:22:00 615.41
          2018-06-03 11:22:00 614.88
          2018-06-03 11:22:00 614.51
          2018-06-03 11:22:00 614.71
          2018-06-03 11:22:00 615.01
          2018-06-03 11:22:00 616.1
          2018-06-03 11:22:00 616.54
          2018-06-03 11:22:00 616.74
          2018-06-03 11:22:00 616.5
          2018-06-03 11:22:00 616.11
          finished
      
      posted in General Code/Help datetime date2num print
      B
      blonc
    • RE: marketstore * new data feed integration

      SOLVED

      the fromdate with in the datafeed initialize was a integer and needed to be a datetime object . IE datetime.date(2018, 6, 1)

      posted in General Code/Help
      B
      blonc
    • RE: marketstore * new data feed integration

      an example of it passing through function fine but with in backtrader still the error of AttributeError: 'str' object has no attribute 'toordinal'

          import math
          import datetime
          from backtrader import date2num
          
          bars = [(1527991500, 7635.41, 7635.41, 7635.41, 7635.41, 0.13048232),
           (1527991560, 7635.4, 7635.41, 7635.4, 7635.41, 0.3689),
           (1527991620, 7635.41, 7635.41, 7635.41, 7635.41, 3.5814399599999995),
           (1527991680, 7635.41, 7635.41, 7635.41, 7635.41, 1.41122419),]
          
          biter = iter(bars)
          bar = next(biter)
          
          date2num(datetime.datetime.utcfromtimestamp(int(bar[0])))
      
      posted in General Code/Help
      B
      blonc
    • RE: marketstore * new data feed integration

      the issue seems to be with the date2num function . I have converted my array of epoch back into datetime objects and it seems to pass fine through the function. However, i still get the same error in backtrader. I updated the code as followed :

       self.l.datetime[0] = date2num(datetime.datetime.utcfromtimestamp(int(bar[0])))
      
      posted in General Code/Help
      B
      blonc