Navigation

    Backtrader Community

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    1. Home
    2. sfkiwi
    3. Posts
    For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/
    S
    • Profile
    • Following 1
    • Followers 0
    • Topics 15
    • Posts 31
    • Best 2
    • Groups 0

    Posts made by sfkiwi

    • RE: How to save indicator config to use in different strategy

      Thanks, in the end that is pretty much what I did.
      There's likely a much more elegant way to do this than how I have done it though

      # define_indicators.py
      
      def save_indicators():
      
        indicators = {
      
          'SMA': {'ind': bt.ind.SMA, 'data': 0, 'args': {'period':10}},
          'trend': {'ind': bt.ind.SMA, 'data': 3, 'args': {'period':10}},
          'rsi_1M': { 'ind': bt.ind.RSI, 'data': 0, 'args': None },
          'rsi_5M': { 'ind': bt.ind.RSI, 'data': 1, 'args': None},
          'rsi_30M': { 'ind': bt.ind.RSI, 'data': 2, 'args': None },
          'rsi_1H': { 'ind': bt.ind.RSI, 'data': 3, 'args': None },
          'rsi_4H': {'ind': bt.ind.RSI, 'data': 4, 'args': None},
          'sto_1M': {'ind': StochasticRSI, 'data': 'rsi_1M', 'args': None},
          .....
        }
      
        DATA_DIRNAME.mkdir(parents=True, exist_ok=True)
        with open(INDICATORS_FILENAME, 'wb') as f:
          pickle.dump(indicators, f, pickle.HIGHEST_PROTOCOL)
      
      
      def build_indicators(datas, indicators):
      
          ind = dict()
          for k, v in indicators.items():
      
              kwargs = {}
              if v['args']:
                  kwargs = v['args']
      
              if type(v['data']) is str:
                  data = [ind[v['data']]]
              elif type(v['data']) is list:
                  data = list()
                  for d in v['data']:
                      if type(d) is tuple:
                          data.append(getattr(ind[d[0]], d[1]))
                      else:
                          data.append(ind[d])
              else:
                  data = [datas[v['data']]]
              
              ind[k] = v['ind'](*data, plotname=k, **kwargs)
          
          return ind
      
      if __name__ == '__main__':
        save_indicators()
      
      # strategy.py
      import backtrader as bt
      from define_indicators import build_indicators
      
      class St(bt.Strategy):
      
        def __init__(self):
      
              f = open(INDICATORS_FILENAME, 'rb')
              indicators = pickle.load(f)
      
              self.indicators = build_indicators(self.datas, indicators)
      posted in Indicators/Strategies/Analyzers
      S
      sfkiwi
    • How to save indicator config to use in different strategy

      I am trying to figure out a way to save the configuration of all of my indicators in one strategy and then load the same configuration into a different strategy (there are more than 50 indicators and the indicators may change from time to time so it needs to be done programmatically).

      For example, say in strategy1 I have an SMA and an RSI, I want to be able to load a file from disk that will tell strategy2 to create an SMA and an RSI with the same config (including plotname) as in Strategy1. Strategy 1 and Strategy 2 have different logic and both use different feeds so the rest of the cerebro instance is quite different between Strategy1 and Strategy2.

      I had tried pickling but keep getting cannot serialize '_io.TextIOWrapper' object exceptions, which is also discussed in this thread. However pickling probably wouldn't work in any case because each indicator holds references to the cerebro context and those get pickled as well.

      Is there a way to save a config, along with the indicator plotname etc that could then be used to instantiate an indentical copy in another strategy?

      posted in Indicators/Strategies/Analyzers
      S
      sfkiwi
    • RE: How to get underlying timeframe when using replaydata

      Nevermind, I had taken the idea of using the self._timeframe from the ccxt feed when instead I should have been using self.p.timeframe which doesn't get modified by the replaydata. Works perfectly now.

      posted in General Discussion
      S
      sfkiwi
    • How to get underlying timeframe when using replaydata

      I have created a custom feed for coinbase data using their websocket interface and am trying to get the replaydata to work. The feed supports both historic and realtime data so if I want to replay ticks on minute bars for example I would do the following

      # Start with 255 mins of historic data before going to LIVE
      hist_start_date = datetime.utcnow() - timedelta(seconds=60*255)
      
      # Get a tick data feed as the underlying data
      data = CoinbaseProFeed(symbol="BTC-USD",
                             name="btc_usd_tick",
                             fromdate=hist_start_date,
                             timeframe=bt.TimeFrame.Ticks)
      
      # replay the tick feed as minute bars
      cerebro.replaydata(data,
                         timeframe=bt.TimeFrame.Minutes,
                         compression=1)
      

      However, the feed only sees the timeframe of Minutes (self._timeframe) which is what the strategy would expect, but the feed needs to know that the underlying data is ticks because that's what cerebro is expecting in order to do the minute bar construction over the top.

      I see that there is a self.replaying flag to check that the data is being replayed but I don't see anywhere to fetch the underlying timeframe from.

      posted in General Discussion
      S
      sfkiwi
    • RE: marketstore NEW data feed complete code & example

      @blonc said in marketstore NEW data feed complete code & example:

      I am integrating MS because it as well allows for real-time streaming via a websocket of the bars from the db. IMO this will allow for consistency between realtime and historical testing. Data is stored in tick compression and then using On-disk Aggregate updates the downsample data compressed into any timeframe you want. Easiest way to get this to work is to build MS from source in a docker, its already integrated with GDAX and works out the box.

      It looks like you were originally trying to have MarketStore feed both historical and live data into backtrader. I have the MarketStore running with the GDAX feeder running, however the feeder is only using the historical data API to fetch the data even if the timeframe is 1Min, which means that its always a little behind real-time. And likewise the code above also uses the polling API to retrieve the data from the MarketStore.

      Did you ever get it working with websocket data feeding in and a websocket connection from backtrader to the MarketStore?

      posted in General Code/Help
      S
      sfkiwi
    • RE: marketstore NEW data feed complete code & example

      @blonc said in marketstore NEW data feed complete code & example:

      The next step for me is to integrate polygon.io which is REALTIME trade by trade SIP data for 200$ a month, latency is under 10ms too if you are colocated in or around ny4.

      What additional latency are you getting on the GDAX connection (compared to a direct GDAX API connection) when streaming via the MarketStore websocket connector. e.g. By the time it comes down from the GDAX API, writes to disk in MarketStore, then gets re-streamed out, how far behind the real-time market data are you?

      Do you think you could run this with real-time L1 or L2 data feed?

      posted in General Code/Help
      S
      sfkiwi
    • RE: Anyone use backtrader to do live trading on Bitcoin exchange?

      I noticed something strange with the ccxt fetchohlc() in that sometimes it returns candles up to the last closed candle and sometimes it returns the current live (not yet closed candle).

      I used the follow code to get the last 20 candles which uses the same method that the ccxt backtrader uses to get live candle data.

      df = pd.DataFrame(gdax.fetchOhlcv('BTC/USD', timeframe='1m', limit=20))
      df['Date'] = pd.to_datetime(df[0], unit='ms')
      df.set_index('Date', inplace=True)
      df.drop(0, axis=1, inplace=True)
      df.columns = ['open', 'high','low','close','volume']
      df['highlow'] = df['high'] - df['low']; df
      

      This outputs a dataframe:

                                 
      Date	                open    high    low     close   volume        highlow
      2018-08-21 09:15:00	6452.99	6453.00	6452.99	6452.99	0.382287	0.01
      2018-08-21 09:16:00	6453.00	6453.00	6452.99	6453.00	0.231581	0.01
      2018-08-21 09:17:00	6453.00	6453.00	6453.00	6453.00	0.033058	0.00
      2018-08-21 09:18:00	6453.00	6453.00	6452.99	6453.00	0.244911	0.01
      2018-08-21 09:19:00	6453.00	6453.00	6452.99	6452.99	2.142046	0.01
      2018-08-21 09:20:00	6453.00	6453.00	6438.91	6441.62	10.994071	14.09
      2018-08-21 09:21:00	6441.62	6445.98	6441.62	6445.98	1.862376	4.36
      2018-08-21 09:22:00	6445.97	6445.97	6431.64	6437.10	12.625005	14.33
      2018-08-21 09:23:00	6437.10	6445.67	6437.10	6443.40	1.902326	8.57
      2018-08-21 09:24:00	6443.39	6443.40	6443.39	6443.39	0.531487	0.01
      2018-08-21 09:25:00	6443.40	6443.40	6435.62	6435.62	1.435376	7.78
      2018-08-21 09:26:00	6436.35	6437.32	6433.71	6433.71	1.582333	3.61
      2018-08-21 09:27:00	6435.23	6435.23	6435.22	6435.22	0.054494	0.01
      2018-08-21 09:28:00	6435.22	6436.00	6435.22	6436.00	0.099945	0.78
      2018-08-21 09:29:00	6435.99	6435.99	6433.71	6433.71	1.374433	2.28
      2018-08-21 09:30:00	6433.71	6433.71	6431.00	6432.77	1.015340	2.71
      2018-08-21 09:31:00	6433.62	6433.62	6432.76	6433.00	0.172707	0.86
      2018-08-21 09:32:00	6432.99	6433.00	6432.99	6432.99	0.555686	0.01
      2018-08-21 09:33:00	6432.99	6432.99	6427.75	6431.62	3.599003	5.24
      2018-08-21 09:34:00	6431.63	6431.63	6431.63	6431.63	0.059508	0.00
      

      When I ran this 4 times consecutively within less than 1 second I got the following results (only last 3 lines shown)

      2018-08-21 09:32:00	6432.99	6433.00	6432.99	6432.99	0.555686	0.01
      2018-08-21 09:33:00	6432.99	6432.99	6427.75	6431.62	3.599003	5.24
      2018-08-21 09:34:00	6431.63	6431.63	6431.63	6431.63	0.059508	0.00
      
      2018-08-21 09:31:00	6433.62	6433.62	6432.76	6433.00	0.172707	0.86
      2018-08-21 09:32:00	6432.99	6433.00	6432.99	6432.99	0.555686	0.01
      2018-08-21 09:33:00	6432.99	6432.99	6427.75	6431.62	3.599003	5.24
      
      2018-08-21 09:32:00	6432.99	6433.00	6432.99	6432.99	0.555686	0.01
      2018-08-21 09:33:00	6432.99	6432.99	6427.75	6431.62	3.599003	5.24
      2018-08-21 09:34:00	6431.63	6431.63	6431.63	6431.63	0.062508	0.00
      
      2018-08-21 09:32:00	6432.99	6433.00	6432.99	6432.99	0.555686	0.01
      2018-08-21 09:33:00	6432.99	6432.99	6427.75	6431.62	3.599003	5.24
      2018-08-21 09:34:00	6431.63	6431.63	6431.63	6431.63	0.059508	0.00
      

      As you can see, sometimes it returns the 9:34 candle (which at the time had not closed) and sometimes it returns the 9:33 candle (which has closed) as the most recent.

      When I ran this a few minutes later I get

      2018-08-21 09:32:00	6432.99	6433.00	6432.99	6432.99	0.555686	0.01
      2018-08-21 09:33:00	6432.99	6432.99	6427.75	6431.62	3.599003	5.24
      2018-08-21 09:34:00	6431.63	6433.00	6431.62	6433.00	4.551569	1.38
      2018-08-21 09:35:00	6433.38	6434.68	6433.38	6434.68	12.601649	1.30
      2018-08-21 09:36:00	6436.36	6444.00	6436.36	6443.99	3.252023	7.64
      

      As you can see the 9:34 candle closed at 6433.00 not at 6431.63.

      This means that when you are getting 1m live data from backtrader you may actually be getting a live candle instead of the actual data because backtrader/ccxt will record the time of the latest candle and only give candles with timestamps later than the previous.

      In fact this actually looks like what I'm getting

      ***** DATA NOTIF: LIVE
      ***** NEXT: 2018-08-21 09:04:00 btc_usd_min 6459.99 6460.0 6459.99 6460.0 0.02234232 Minute 3
      ***** NEXT: 2018-08-21 09:05:00 btc_usd_min 6455.06 6455.06 6455.06 6455.06 0.05325761 Minute 4
      ***** NEXT: 2018-08-21 09:06:00 btc_usd_min 6455.06 6460.92 6455.06 6460.92 0.75941057 Minute 5
      ***** NEXT: 2018-08-21 09:07:00 btc_usd_min 6460.95 6460.95 6460.95 6460.95 0.00362068 Minute 6
      ***** NEXT: 2018-08-21 09:08:00 btc_usd_min 6457.84 6457.84 6457.84 6457.84 0.03702458 Minute 7
      ***** NEXT: 2018-08-21 09:09:00 btc_usd_min 6452.82 6452.82 6452.82 6452.82 1.21548925 Minute 8
      ***** NEXT: 2018-08-21 09:10:00 btc_usd_min 6452.81 6452.81 6452.81 6452.81 0.0226 Minute 9
      

      Minute 5 appears to be a full candle whereas Minute 7 (for example) appears to be a partially completed live candle

      posted in General Discussion
      S
      sfkiwi
    • RE: Anyone use backtrader to do live trading on Bitcoin exchange?

      I've been using backtrader for sometime but I'm new to the ccxt live data. Thank you @Ed-Bartosh for all the work you've done on this. amazing!

      When using live data I'm noticing quite a significant delay of the data by about 30 seconds or so. Is this expected behavior? My hope is to run this as close to real-time as possible.

      Even though the data status shows LIVE, the first data is actually about 15mins behind real-time but starts to catch up after around 30 seconds, although it never actually fully catches up and ends up always being about 10-15 seconds behind the real-time data. I'm using TradingView and pro.coinbase.com as my benchmark as I'm using gdax as the feed.

      ***** DATA NOTIF: LIVE
      ***** NEXT: 2018-08-21 08:15:33.492003 btc_usd_tick 6475.01 6475.01 6475.01 6475.01 0.01 Tick 1
      ***** NEXT: 2018-08-21 08:15:33.532004 btc_usd_tick 6475.91 6475.91 6475.91 6475.91 0.57164401 Tick 2
      ***** NEXT: 2018-08-21 08:15:46.381995 btc_usd_tick 6475.91 6475.91 6475.91 6475.91 0.0016879 Tick 3
      ***** NEXT: 2018-08-21 08:15:53.060997 btc_usd_tick 6475.9 6475.9 6475.9 6475.9 0.0034 Tick 4
      ***** NEXT: 2018-08-21 08:16:04.493001 btc_usd_tick 6475.91 6475.91 6475.91 6475.91 0.0028 Tick 5
      ***** NEXT: 2018-08-21 08:16:16.126000 btc_usd_tick 6475.9 6475.9 6475.9 6475.9 0.0097 Tick 6
      ***** NEXT: 2018-08-21 08:16:28.048999 btc_usd_tick 6475.9 6475.9 6475.9 6475.9 0.1667 Tick 7
      ***** NEXT: 2018-08-21 08:16:45.003001 btc_usd_tick 6475.9 6475.9 6475.9 6475.9 1.0 Tick 8
      ***** NEXT: 2018-08-21 08:16:45.043002 btc_usd_tick 6474.6 6474.6 6474.6 6474.6 0.001 Tick 9
      ***** NEXT: 2018-08-21 08:16:49.953004 btc_usd_tick 6471.01 6471.01 6471.01 6471.01 0.00345867 Tick 10
      ***** NEXT: 2018-08-21 08:16:50.023995 btc_usd_tick 6471.01 6471.01 6471.01 6471.01 0.045419 Tick 11
      ***** NEXT: 2018-08-21 08:17:05.876002 btc_usd_tick 6471.01 6471.01 6471.01 6471.01 0.15756512 Tick 12
      ***** NEXT: 2018-08-21 08:17:10.691003 btc_usd_tick 6471.01 6471.01 6471.01 6471.01 0.0079974 Tick 13
      ***** NEXT: 2018-08-21 08:17:17.021002 btc_usd_tick 6471.01 6471.01 6471.01 6471.01 0.00315375 Tick 14
      ***** NEXT: 2018-08-21 08:17:21.066002 btc_usd_tick 6468.35 6468.35 6468.35 6468.35 0.00062 Tick 15
      ***** NEXT: 2018-08-21 08:17:21.703003 btc_usd_tick 6467.06 6467.06 6467.06 6467.06 0.01 Tick 16
      ***** NEXT: 2018-08-21 08:17:49.377999 btc_usd_tick 6465.19 6465.19 6465.19 6465.19 0.03187985 Tick 17
      ***** NEXT: 2018-08-21 08:18:01.357998 btc_usd_tick 6465.19 6465.19 6465.19 6465.19 0.0515 Tick 18
      ***** NEXT: 2018-08-21 08:18:04.042001 btc_usd_tick 6465.2 6465.2 6465.2 6465.2 0.00384762 Tick 19
      ***** NEXT: 2018-08-21 08:18:11.765002 btc_usd_tick 6465.19 6465.19 6465.19 6465.19 0.00383559 Tick 20
      ***** NEXT: 2018-08-21 08:18:11.801997 btc_usd_tick 6465.0 6465.0 6465.0 6465.0 4.99793959 Tick 21
      
      class TestStrategy(bt.Strategy):
          def next(self):
              for data in self.datas:
                  print('*' * 5, 'NEXT:', bt.num2date(data.datetime[0]), data._name, data.open[0], data.high[0],
                        data.low[0], data.close[0], data.volume[0],
                        bt.TimeFrame.getname(data._timeframe), len(data))
      
          def notify_order(self, order):
              print('*' * 5, "NOTIFY ORDER", order)
              
          def notify_data(self, data, status, *args, **kwargs):
              print('*' * 5, 'DATA NOTIF:', data._getstatusname(status), *args)
      
      def runstrategy(argv):
          # Create a cerebro
          cerebro = bt.Cerebro()
      
          # Create broker
          broker_config = {'urls': {'api': 'https://api.pro.coinbase.com'},
                           'apiKey': '{}'.format(gdax.apiKey),
                           'secret': '{}'.format(gdax.secret),
                           'password': '{}'.format(gdax.password)
                          }
      
          # Create data feeds
          data_ticks = bt.feeds.CCXT(exchange='gdax', symbol='BTC/USD', name="btc_usd_tick",
                                     timeframe=bt.TimeFrame.Ticks, compression=1)
          cerebro.adddata(data_ticks)
      
          # Add the strategy
          cerebro.addstrategy(TestStrategy)
      
          # Run the strategy
          cerebro.run()
      
      posted in General Discussion
      S
      sfkiwi
    • Re: PyFolio analyzer multiple data sources not working

      Re: PyFolio analyzer multiple data sources not working.

      I see the above topic was posted back in October 2017 but I'm wondering if this was resolved somewhere else. I'm also finding the exact same thing when using multiple data sources with the pyfolio analyzer. The source code line mentioned in the original post refers to

      ps = [[k] + v[-2:] for k, v in iteritems(pss)]
      

      where v contains the positions for each of the data feeds. [-2:] means that its only using the last two entries (the last symbol and the cash positions). If the line instead is changed to

      ps = [[k] + v for k, v in iteritems(pss)]
      

      Then positions for all datafeeds are saved and pyfolio gives the correct position and full tear sheets. Otherwise pyfolio thinks there was only one symbol in the portfolio.

      A workaround of course is to add your own PositionValues analyzer

      # existing pyfolio analyzer
      cerebro.addanalyzer(bt.analyzers.PyFolio, _name='pyfolio',
                              timeframe=bt.TimeFrame.Days)
      pyfoliozer = strat.analyzers.getbyname('pyfolio',)
      returns, positions, transactions, gross_lev = pyfoliozer.get_pf_items() #<--positions only holds data for one symbol here
      
      # Add an extra PositionsValue analyzer to capture positions for all symbols
      from backtrader.utils.py3 import iteritems
      cerebro.addanalyzer(bt.analyzers.PositionsValue, headers=True, cash=True, _name='mypositions')
      p = strat.analyzers.getbyname('mypositions').get_analysis()
      mypositions = [[k] + v for k, v in iteritems(p)] #<-- Now positions will hold data for all symbols
      cols = mypositions.pop(0)  # headers are in the first entry
      mypositions = pd.DataFrame.from_records(mypositions, index=cols[0], columns=cols)
      
      pf.create_full_tear_sheet(
          returns, # from pyfolio analyzer
          positions=mypositions, #<-- from additional PositionsValue analyzer 
          transactions=transactions, # from pyfolio analyzer
          gross_lev=gross_lev, # from pyfolio analyzer
          round_trips=True)
      
      posted in Indicators/Strategies/Analyzers
      S
      sfkiwi
    • preventing labels for all data feeds showing when plotting

      I have a few hundred data feeds that Im running a portfolio strategy over. I would like to plot just the observer results and perhaps one of the data feeds as a reference but I haven't figured out how to turn off the data labels showing for the trades observer.

      I am setting plotinfo.plot = False for all the data feeds except the first one which seems to work when I use just 6 of the data feeds but when I add all the data feeds I get the plot show below.

      With all feeds
      0_1532551515241_Screen Shot 2018-07-25 at 10.41.48 PM.png

      With just 6 feeds
      0_1532552121784_Screen Shot 2018-07-25 at 10.54.32 PM.png

      #run
      cerebro = bt.Cerebro()
      cerebro.addstrategy(St)
      cerebro.broker.setcash(20000.0)
      cerebro.addobserver(bt.observers.TimeReturn, timeframe=bt.TimeFrame.NoTimeFrame)
      cerebro.addobserver(bt.observers.Benchmark, data=benchmark, timeframe=bt.TimeFrame.NoTimeFrame)
      cerebro.adddata(benchmark, name='SPY')
      for feed in feeds:
          cerebro.adddata(feed[1], name=feed[0])
      results = cerebro.run()
      cerebro.plot(iplot=False, style='candle')
      
      #strategy
      for d in self.datas:
          d.plotinfo.plot = False
      self.data1.plotinfo.plot = True
      
      posted in General Code/Help
      S
      sfkiwi
    • RE: Trying to run from latest source code

      @backtrader What I was trying to do was have my project import the library from the source directly rather than from the installed package. I'm fairly new to python so if there is a standard way to achieve this then please enlighten me.

      For now I have just added my local source code path to the front of sys.path which works and doesn't break things.

      posted in General Code/Help
      S
      sfkiwi
    • Trying to run from latest source code

      I am trying to run backtrader from the latest source code but am getting an error when I execute cerebro.run()

      AttributeError: 'NoneType' object has no attribute '_next_stid'
      

      Update & Solution: I resolved this issue, so posting the solution here. I have left the original question below for context. The issue was occurring because in my Jupyter Notebook I was importing from libs.backtrader to make sure that I was importing the source code and not the installed pip install version. The source code, however uses import backtrader as bt throughout which was pulling in the install version. There is a function called findowner which attempts to search the call stack for the owner of the strategy and because of the different imports the stack frames were different and couldn't find the cerebro instance and hence findowner returned None which triggered the error.
      Solution: I modified my Jupyter Notebook so that it too had import backtrader as bt but before the import statement I added sys.path = ['libs/backtrader'] + sys.path which prepended my local directory to the sys.path which caused both my Jupyter Notebook and the source code to find the backtrader module in the local directory before searching the pip install paths.


      Original Question:

      I have been successfully running backtrader from a pip install inside a conda env and am running inside a Jupyter Notebook environment. Everything has been working fine using the installed version of backtrader.

      I have made a couple of minor changes to the source code so I have cloned the backtrader repo to a local folder and have simlinked the folder inside my Jupyter Notebook folder.

      # common root folder is 'code'
      cd code/
      
      # clone backtrader into code/backtrader
      git clone https://github.com/backtrader/backtrader.git
      
      # code/project is where the jupyter notebook is
      # change directory to libs/ directory inside project directory
      cd project/libs/
      
      #inside libs/ directory create a symlink to backtrader folder
      ln -s ~/.../code/backtrader backtrader
      
      # /code/backtrader - backtrader project folder
      # /code/project - my jupyter notebook folder
      # /code/project/libs/backtrader - symlinked backtrader folder
      # /code/project/data/equities - folder containing dataset
      

      Inside my jupyter notebook I am importing the project as

      from libs.backtrader import backtrader as bt
      from libs.backtrader.backtrader import indicators as btind
      from libs.backtrader.backtrader import feeds as btfeeds
      

      I am using one of the example strategies and sample data sets

      class TestStrategy(bt.Strategy):
      
          def log(self, txt, dt=None):
              ''' Logging function fot this strategy'''
              dt = dt or self.datas[0].datetime.date(0)
              print('%s, %s' % (dt.isoformat(), txt))
      
          def __init__(self):
              # Keep a reference to the "close" line in the data[0] dataseries
              self.dataclose = self.datas[0].close
      
          def next(self):
              # Simply log the closing price of the series from the reference
              self.log('Close, %.2f' % self.dataclose[0])
      
              if self.dataclose[0] < self.dataclose[-1]:
                  # current close less than previous close
      
                  if self.dataclose[-1] < self.dataclose[-2]:
                      # previous close less than the previous close
      
                      # BUY, BUY, BUY!!! (with all possible default parameters)
                      self.log('BUY CREATE, %.2f' % self.dataclose[0])
                      self.buy()
      

      Then I run the strategy as

      cerebro = bt.Cerebro()
      
      # Add the strategy
      cerebro.addstrategy(TestStrategy)
      
      # path to data
      datapath = 'data/equities/orcl-1995-2014.txt'
      
      # Create a Data Feed
      data = bt.feeds.YahooFinanceCSVData(
          dataname=datapath,
          # Do not pass values before this date
          fromdate=datetime.datetime(2000, 1, 1),
          # Do not pass values before this date
          todate=datetime.datetime(2000, 12, 31),
          # Do not pass values after this date
          reverse=False)
      
      # Add the Data Feed to Cerebro
      cerebro.adddata(data)
      
      # Set up broker
      cerebro.broker.setcash(100000.0)
      
      # Run 
      cerebro.run()
      

      Running this produces the following error. Running the exact same code using the pip installed version of backtrader does not cause the error

      ---------------------------------------------------------------------------
      AttributeError                            Traceback (most recent call last)
      <ipython-input-40-166ad5063c49> in <module>()
      ----> 1 cerebro.run(runonce=False)
      
      ~/Dropbox/Work/Code/Trading/strategies/libs/backtrader/backtrader/cerebro.py in run(self, **kwargs)
         1125             # let's skip process "spawning"
         1126             for iterstrat in iterstrats:
      -> 1127                 runstrat = self.runstrategies(iterstrat)
         1128                 self.runstrats.append(runstrat)
         1129                 if self._dooptimize:
      
      ~/Dropbox/Work/Code/Trading/strategies/libs/backtrader/backtrader/cerebro.py in runstrategies(self, iterstrat, predata)
         1215             sargs = self.datas + list(sargs)
         1216             try:
      -> 1217                 strat = stratcls(*sargs, **skwargs)
         1218             except bt.errors.StrategySkipError:
         1219                 continue  # do not add strategy to the mix
      
      ~/Dropbox/Work/Code/Trading/strategies/libs/backtrader/backtrader/metabase.py in __call__(cls, *args, **kwargs)
           84     def __call__(cls, *args, **kwargs):
           85         cls, args, kwargs = cls.doprenew(*args, **kwargs)
      ---> 86         _obj, args, kwargs = cls.donew(*args, **kwargs)
           87         _obj, args, kwargs = cls.dopreinit(_obj, *args, **kwargs)
           88         _obj, args, kwargs = cls.doinit(_obj, *args, **kwargs)
      
      ~/Dropbox/Work/Code/Trading/strategies/libs/backtrader/backtrader/strategy.py in donew(cls, *args, **kwargs)
           70         # Find the owner and store it
           71         _obj.env = _obj.cerebro = cerebro = findowner(_obj, bt.Cerebro)
      ---> 72         _obj._id = cerebro._next_stid()
           73 
           74         return _obj, args, kwargs
      
      AttributeError: 'NoneType' object has no attribute '_next_stid'
      

      It seems like findowner is not finding the cerebro instance. I am wondering if this is because of the way strategy.py it is importing backtrader and instead of using the local code, is importing the pip installed version

      # strategy.py 
         32   from .utils.py3 import (filter, keys, integer_types, iteritems, itervalues,
         33       map, MAXINT, string_types, with_metaclass)
       
      >> 34   import backtrader as bt
         35   from .lineiterator import LineIterator, StrategyBase
      
      posted in General Code/Help
      S
      sfkiwi
    • RE: DivideZeroError on some indicators

      @backtrader

      Ok but, unlikely as it is, the price stayed stationary for more than 20mins so shouldn't the indicators still be protected from a divide by zero error?

      I am adding this protection to my fork of the latest code in the repo although having some difficulty getting the latest code in the repo to run without error. Once I get it to work I can submit a PR unless there is a reason you don't want this protection on the indicators?

      posted in Indicators/Strategies/Analyzers
      S
      sfkiwi
    • RE: DivideZeroError on some indicators

      @backtrader

      Here is a log of the close price of the data just before the error occurs. You can see that there is a period where the price doesn't change for 19 consecutive minutes.

      Only the RSI and Stochastic Indicators implement SafeDiv and these indicators don't seem to cause the issue to occur. Its other indicators like BollingerBands that trigger this error when enabled.

      2018-05-10 03:25:00, running 9286.70
      2018-05-10 03:26:00, running 9291.65
      2018-05-10 03:27:00, running 9291.65
      2018-05-10 03:28:00, running 9291.70
      2018-05-10 03:29:00, running 9291.70
      2018-05-10 03:30:00, running 9291.70
      2018-05-10 03:31:00, running 9291.70
      2018-05-10 03:32:00, running 9291.70
      2018-05-10 03:33:00, running 9291.70
      2018-05-10 03:34:00, running 9291.70
      2018-05-10 03:35:00, running 9291.70
      2018-05-10 03:36:00, running 9291.70
      2018-05-10 03:37:00, running 9291.70
      2018-05-10 03:38:00, running 9291.70
      2018-05-10 03:39:00, running 9291.70
      2018-05-10 03:40:00, running 9291.70
      2018-05-10 03:41:00, running 9291.70
      2018-05-10 03:42:00, running 9291.70
      2018-05-10 03:43:00, running 9291.70
      2018-05-10 03:44:00, running 9291.70
      2018-05-10 03:45:00, running 9291.70
      2018-05-10 03:46:00, running 9291.70
      

      Edit:
      One of the indicators that is causing a problem is the BollingerBandsPct. Looking into the code for the indicator it seems that if the close price doesn't change for more than the period of the indicator (default period=20) then the line.top and line.bottom will both be equal to the close price as there is no stddev for a series of values that are all the same (stddev=0).

      If both line.top and line.bottom are equal to each other then in the Percent calculation the denominator will be top - bottom = 0.

      class BollingerBands(Indicator):
          '''
          Defined by John Bollinger in the 80s. It measures volatility by defining
          upper and lower bands at distance x standard deviations
      
          Formula:
            - midband = SimpleMovingAverage(close, period)
            - topband = midband + devfactor * StandardDeviation(data, period)
            - botband = midband - devfactor * StandardDeviation(data, period)
      
          See:
            - http://en.wikipedia.org/wiki/Bollinger_Bands
          '''
          alias = ('BBands',)
      
          lines = ('mid', 'top', 'bot',)
          params = (('period', 20), ('devfactor', 2.0), ('movav', MovAv.Simple),)
      
          plotinfo = dict(subplot=False)
          plotlines = dict(
              mid=dict(ls='--'),
              top=dict(_samecolor=True),
              bot=dict(_samecolor=True),
          )
      
          def _plotlabel(self):
              plabels = [self.p.period, self.p.devfactor]
              plabels += [self.p.movav] * self.p.notdefault('movav')
              return plabels
      
          def __init__(self):
              self.lines.mid = ma = self.p.movav(self.data, period=self.p.period)
      
              #########################################
              # if the last 20 closing prices are all the same then StdDev will evaluate 
              # to zero
              #########################################
              stddev = self.p.devfactor * StdDev(self.data, ma, period=self.p.period,
                                                 movav=self.p.movav)
      
              #########################################
              # both self.lines.top and self.lines.bottom will be equal to the ma
              # which is now equal to the close price
              #########################################
              self.lines.top = ma + stddev
              self.lines.bot = ma - stddev
      
              super(BollingerBands, self).__init__()
      
      
      class BollingerBandsPct(BollingerBands):
          '''
          Extends the Bollinger Bands with a Percentage line
          '''
          lines = ('pctb',)
          plotlines = dict(pctb=dict(_name='%B'))  # display the line as %B on chart
      
          def __init__(self):
              super(BollingerBandsPct, self).__init__()
      
              #########################################
              # The denominator here will evaluate to zero as self.l.top
              # and self.l.bot are the same.
              #########################################
              self.l.pctb = (self.data - self.l.bot) / (self.l.top - self.l.bot)
      

      Edit2:
      I duplicated the BollingerBandsPct in my own code and replaced the percent calculation with bt.DivByZero and it resolved the problem for this indicator. Unfortunately there are many others similar to this.

      Is there a particular reason why only the Stochastic and the Rsi indicators use the safeDiv function and not the other indicators?

      class MyBollingerBandsPct(btind.BollingerBands):
          '''
          Extends the Bollinger Bands with a Percentage line
          '''
          lines = ('pctb',)
          plotlines = dict(pctb=dict(_name='%B'))  # display the line as %B on chart
      
          def __init__(self):
              super(MyBollingerBandsPct, self).__init__()
              self.l.pctb = bt.DivByZero((self.data - self.l.bot), (self.l.top - self.l.bot), zero=0)
      
      posted in Indicators/Strategies/Analyzers
      S
      sfkiwi
    • RE: DivideZeroError on some indicators

      @backtrader

      Here is the associated code. I'm not sure exactly where the error is being triggered. It looks to me to be in the next() function of the strategy which triggers an operation on an indicator, however the indicators that seem to be causing the problem are not accessed in the Strategy.

      # Inside Strategy
      
      def __init__(self):
      
        # These indicators don't cause any problem
        self.trend = btind.SMA(self.data3, period=10); self.trend.csv =self.p.l
        self.trendslope = self.trend - self.trend(-1); self.trendslope.csv = self.p.l
      
        self.r1 = btind.RSI(self.data0, plotname='rsi_1M'); self.r1.csv = self.p.l
        self.r5 = btind.RSI(self.data1, plotname='rsi_5M'); self.r5.csv = self.p.l
        self.r30 = btind.RSI(self.data2, plotname='rsi_30M'); self.r30.csv = self.p.l
        self.r60 = btind.RSI(self.data3, plotname='rsi_1H'); self.r60.csv = self.p.l
        self.r240 = btind.RSI(self.data4, plotname='rsi_4H'); self.r240.csv = self.p.l
              
        self.s1 = StochasticRSI(self.r1, plotname='sto_1M'); self.s1.csv = self.p.l
        self.s5 = StochasticRSI(self.r5, plotname='sto_5M'); self.s5.csv = self.p.l
        self.s30 = StochasticRSI(self.r30, plotname='sto_30M'); self.s30.csv = self.p.l
        self.s60 = StochasticRSI(self.r60, plotname='sto_1H'); self.s60.csv = self.p.l
        self.s240 = StochasticRSI(self.r240, plotname='sto_4H'); self.s240.csv = self.p.l
              
        btind.ADX(plot=True).csv = self.p.l
        btind.ADXR(plot=False).csv = self.p.l
        btind.ATR(plot=True).csv = self.p.l
        btind.AbsPriceOsc(plot=False).csv = self.p.l
        btind.AccDeOsc(plot=True).csv = self.p.l
      
        #####
        # Enabling any of the following indicators causes the DivideZeroError, but 
        # there are other indicators not shown here that also cause the problem
        btind.AdaptiveMovingAverageOsc(plot=False).csv = self.p.l
        btind.AdaptiveMovingAverage(plot=False).csv = self.p.l
        btind.BBands(plot=False).csv = self.p.l
        ######
      
      def next(self):
        if self.orefs:
             return
      
        if not self.position:
          
          # Long Entry
          if self.s240.D[0] < 20 and self.trendslope > 0:
      
            if ((self.s60.D[0] > self.s60.K[0]) and 
              (self.s30.D[0] > self.s30.K[0]) and 
              (self.s5.D[0] > self.s5.K[0])):
      
              if (self.s1.K[0] < 20) and (self.s1.D[0] > self.s1.K[0]):
      
                os = self.buy_bracket(
                  exectype=bt.Order.Market, 
                  stopprice=self.data0.close[0] * (1 - self.p.sl), 
                  limitprice=self.data0.close[0] * (1 + self.p.tp))
      
                self.orefs = [o.ref for o in os]
      
      def getwriterheaders(self):
        self.indobscsv = [self]
                  
        indobs = itertools.chain(
          self.getindicators_lines(), self.getobservers())
          self.indobscsv.extend(filter(lambda x: x.csv, indobs))
                  
        headers = list()
                  
        # prepare the indicators/observers data headers
        for iocsv in self.indobscsv:
          name = iocsv.plotinfo.plotname or iocsv.__class__.__name__
          linealiases = iocsv.getlinealiases()
                  
          def joinnames(alias):
            if alias:
              join = name + '_' + alias
              return join
            else:
              return alias
                                      
          aliases = tuple(map(joinnames, linealiases))
          headers.extend(aliases)
                  
          return headers
                      
      def getwritervalues(self):
        values = list()
                  
        for iocsv in self.indobscsv:
          name = iocsv.plotinfo.plotname or iocsv.__class__.__name__
          lio = len(iocsv)
          if lio:
            values.extend(map(lambda l: l[0], iocsv.lines.itersize()))
          else:
            values.extend([''] * iocsv.lines.size())
                  
          return values
      
      
      # Running the Strategy
      cerebro = bt.Cerebro()
      cerebro.addstrategy(St, tp=0.005, sl=0.007, printlog=True, l=True)
      cerebro.addwriter(bt.WriterFile, csv=True, csv_counter=False, out='log.csv')
      
      # Data is a Pandas DataFrame of 1min OHLCV
      data = bt.feeds.PandasData(dataname=candles_test, openinterest=None, timeframe=bt.TimeFrame.Minutes)
      cerebro.adddata(data)
      cerebro.resampledata(data, timeframe=bt.TimeFrame.Minutes, compression=5, boundoff=1)
      cerebro.resampledata(data, timeframe=bt.TimeFrame.Minutes, compression=30, boundoff=1)
      cerebro.resampledata(data, timeframe=bt.TimeFrame.Minutes, compression=60, boundoff=1)
      cerebro.resampledata(data, timeframe=bt.TimeFrame.Minutes, compression=240, boundoff=1)
      cerebro.broker.setcash(20000.0)
      cerebro.addsizer(bt.sizers.PercentSizer, percents=98)
      cerebro.broker.setcommission(commission=0.00125)
      cerebro.run(runonce=False)
      
      posted in Indicators/Strategies/Analyzers
      S
      sfkiwi
    • DivideZeroError on some indicators

      First off, there is a related post on DivideZeroError here but it doesn't seem to answer the issue that I am experiencing.

      I have quite a large number of indicators in my strategy but some of them keep throwing the same DivideZeroError. As far as I can tell I'm using the indicators correctly, for example BollingerBands are fairly widely used.

      BollingerBands as far as I can tell doesn't have any division in it but every time I enable it I get this same error.

      ~/miniconda3/envs/pythontrading/lib/python3.5/site-packages/backtrader/linebuffer.py in next(self)
          743     def next(self):
          744         if self.bline:
      --> 745             self[0] = self.operation(self.a[0], self.b[0])
          746         elif not self.r:
          747             if not self.btime:
      
      ZeroDivisionError: float division by zero
      

      I have tried putting a try/Except block around both the init and the next functions and neither catch the error so it seems like its happening somewhere else but only gets triggered when I enable these specific indicators.

      I am running a minute timescale datafeed which is then resampled to 5M, 30M, 60M 4H.

      posted in Indicators/Strategies/Analyzers
      S
      sfkiwi
    • RE: Formatting of output csv from bt.Writer

      In order to read in as a dataframe what I really want is a row of headers that contain the labels for each data point and then each subsequent row represents the value at a point in time.

      I looked into subclassing bt.Writer but the Header and each line of values comes pre-populated from the Strategy and DataSeries classes and the way that it is formatted doesn't provide anyway of knowing where each indicator starts and ends.

      What I ended up doing instead was to override the getWriterHeaders and getWriterValues function in the Strategy. This allowed me to format the header and the values in the way I wanted.

      This works for the indicators and any data coming from the Strategy, however it doesn't solve the problem for the data series. I can't see any easy way to override these functions for the data series.

      posted in General Code/Help
      S
      sfkiwi
    • Formatting of output csv from bt.Writer

      I am trying to collect some data points during backtest that I can process using Pandas. I am using bt.Writer to output my data feeds and indicators to a csv but the format is not very friendly to importing using pd.read_csv due to multiple duplicated column headers and extra lines that are not part of the csv.

      Is there a way to custom format the output of the csv from bt.Writer?

      posted in General Code/Help
      S
      sfkiwi
    • LinePlotterIndicator doesn't write to csv

      I am using the bt.Writer to write to a csv and already have my data feeds and indicators writing successfully to the file.

      I would now like to create some custom parameters, lets say for example (2 * close(-1) / volume), and have those output to the csv. I tried to create a LinePlotterIndicator in my strategy in the init function.

      pricevol = 2 * self.data.close(-1) / self.data.volume
      pv = bt.LinePlotterIndicator(pricevol, name='pricevol')
      pv.csv = True
      

      As you can see in the plot below, I am seeing the result of LinePlotterIndicator in red, but I'm also getting 10 other values that seem to be related to the LinesCouplers. I think the LinesCouplers are from using the PivotPoint indicator although I don't seem to be able to get rid of them even when I set plot=False on the PP indicator.

      0_1531901240217_Screen Shot 2018-07-18 at 10.03.58 AM.png

      In the csv file, I just see see 11 empty columns for pricevol

      0_1531901296404_Screen Shot 2018-07-18 at 10.07.39 AM.png

      How can I get the pricevol to output to the csv and just as one column instead of 11.

      posted in General Code/Help
      S
      sfkiwi
    • Create scrollable plot / keyboard shortcut to save plot to .png

      I have a number of data feeds and indicators that I would like to plot but they need more vertical resolution than my display is capable of. Is there any way to make the plot scrollable vertically so that all the subplots can be displayed without squashing them all in.

      I have changed the default plot size so that it has a large vertical height which creates an acceptable plot, however its not possible to view as the bottom is off the display and the plot window can't be resized or moved.

      It is possible to save the plots to a .png which would also be acceptable, however the save button is also off the bottom of the screen. Is there a keyboard shortcut to save the plot to .png?

      posted in General Code/Help
      S
      sfkiwi
    • 1
    • 2
    • 1 / 2