Navigation

    Backtrader Community

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

    create a CSV of the result of optstrategy

    Indicators/Strategies/Analyzers
    7
    17
    8992
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • C
      cortex last edited by backtrader

      hello,
      i'm trying to print the result of optstrategy for this i to use something like this:

      import csv
      b = open('test.csv', 'a')
      a = csv.writer(b,delimiter=',')
      list = ['test', 'self.broker.startingcash', 'self.broker.getvalue()']
      a.writerows([list])
      b.close()
      

      that i try to insert in the def stop section. it doesn't work.
      anyone has a clue ?
      here the code i run:

      from __future__ import (absolute_import, division, print_function,
                              unicode_literals)
      
      import datetime  # For datetime objects
      import os.path  # To manage paths
      import sys  # To find out the script name (in argv[0])
      import backtrader.feeds as btfeeds
      import numpy as np
      import csv
      
      # Import the backtrader platform
      import backtrader as bt
      
      
      # Create a Stratey
      class TestStrategy(bt.Strategy):
          params = (
              ('maperiod', 15),
              ('printlog', False),
          )
      
          def log(self, txt, dt=None, doprint=False):
              ''' Logging function fot this strategy'''
              if self.params.printlog or doprint:
                  dt = dt or self.datas[0].datetime.date(0)
                  print('%s, %s' % (dt.isoformat(), txt))
      
          def __init__(self):
              # Keep a reference to the "close" line in the data[0] dataseries
              self.dataclose = self.datas[0].close
      
              # To keep track of pending orders and buy price/commission
              self.order = None
              self.buyprice = None
              self.buycomm = None
      
              # Add a MovingAverageSimple indicator
              self.sma = bt.indicators.SimpleMovingAverage(
                  self.datas[0], period=self.params.maperiod)
      
          def notify_order(self, order):
              if order.status in [order.Submitted, order.Accepted]:
                  # Buy/Sell order submitted/accepted to/by broker - Nothing to do
                  return
      
              # Check if an order has been completed
              # Attention: broker could reject order if not enougth cash
              if order.status in [order.Completed, order.Canceled, order.Margin]:
                  if order.isbuy():
                      self.log(
                          'BUY EXECUTED, Price: %.2f, Cost: %.2f, Comm %.2f' %
                          (order.executed.price,
                           order.executed.value,
                           order.executed.comm))
      
                      self.buyprice = order.executed.price
                      self.buycomm = order.executed.comm
                  else:  # Sell
                      self.log('SELL EXECUTED, Price: %.2f, Cost: %.2f, Comm %.2f' %
                               (order.executed.price,
                                order.executed.value,
                                order.executed.comm))
      
                  self.bar_executed = len(self)
      
              # Write down: no pending order
              self.order = None
      
          def notify_trade(self, trade):
              if not trade.isclosed:
                  return
      
              self.log('OPERATION PROFIT, GROSS %.2f, NET %.2f' %
                       (trade.pnl, trade.pnlcomm))
      
          def next(self):
              # Simply log the closing price of the series from the reference
              self.log('Close, %.2f' % self.dataclose[0])
      
              # Check if an order is pending ... if yes, we cannot send a 2nd one
              if self.order:
                  return
      
              # Check if we are in the market
              if not self.position:
      
                  # Not yet ... we MIGHT BUY if ...
                  if self.dataclose[0] > self.sma[0]:
      
                      # BUY, BUY, BUY!!! (with all possible default parameters)
                      self.log('BUY CREATE, %.2f' % self.dataclose[0])
      
                      # Keep track of the created order to avoid a 2nd order
                      self.order = self.buy()
      
              else:
      
                  if self.dataclose[0] < self.sma[0]:
                      # SELL, SELL, SELL!!! (with all possible default parameters)
                      self.log('SELL CREATE, %.2f' % self.dataclose[0])
      
                      # Keep track of the created order to avoid a 2nd order
                      self.order = self.sell()
      
          def stop(self):
              self.log('(MA Period %2d) Ending Value %.2f' %
                       (self.params.maperiod, self.broker.getvalue()), doprint=True)
              list = [self.params.maperiod, self.broker.startingcash, self.broker.getvalue()]
              b = open('test.csv', 'a')
              a = csv.writer(b)
              a.writerows([list])
              b.close()
      
      
      if __name__ == '__main__':
          # Create a cerebro entity
          cerebro = bt.Cerebro()
      
          # Add a strategy
          strats = cerebro.optstrategy(
              TestStrategy,
              maperiod=range(10, 31))
      
          # Datas are in a subfolder of the samples. Need to find where the script is
          # because it could have been called from anywhere
          modpath = os.path.dirname(os.path.abspath(sys.argv[0]))
          datapath = os.path.join(modpath, './GLDGDX/GLD20151130to20170124.csv')
      
          fromdate = datetime.datetime(2016, 10, 02)  # datetime.datetime.strptime(args.fromdate, '%Y-%m-%d')
          todate = datetime.datetime(2016, 11, 20)  # datetime.datetime.strptime(args.todate, '%Y-%m-%d')
      
          # Create the 2nd data
      
          data = btfeeds.GenericCSVData(
              dataname=datapath,
      
              fromdate=fromdate,
              todate=todate,
      
              nullvalue=0.0,
      
              dtformat=('%Y-%m-%d %H:%M:%S'),
              datetime=1,
              high=3,
              low=4,
              open=2,
              close=5,
              volume=6,
              openinterest=-1,
          )
          # Add the Data Feed to Cerebro
          cerebro.adddata(data)
      
          # Set our desired cash start
          cerebro.broker.setcash(10000000.0)
      
          # Add a FixedSize sizer according to the stake
          cerebro.addsizer(bt.sizers.FixedSize, stake=10)
      
          # Set the commission
          cerebro.broker.setcommission(commission=0.0)
      
          # Run over everything
          cerebro.run()
      
      1 Reply Last reply Reply Quote 0
      • Brad Lloyd
        Brad Lloyd last edited by

        try this:
        cerebro.addwriter(bt.WriterFile, csv = True, out='your_strategy_results')

        C 1 Reply Last reply Reply Quote 0
        • A
          ab_trader last edited by

          Maybe you need to try a.writerow([list]).

          What error was returned?

          • If my answer helped, hit reputation up arrow at lower right corner of the post.
          • Python Debugging With Pdb
          • New to python and bt - check this out
          C 1 Reply Last reply Reply Quote 0
          • C
            cortex @ab_trader last edited by backtrader

            @ab_trader
            it was working in fact with an s or without and s.
            I didn't pay close attention enough.

            here the result with the originale code:

            14,10000000.0,9999912.600000035
            
            16,10000000.0,9999920.800000025
            
            12,10000000.0,9999908.300000027
            
            10,10000000.0,9999897.00000004
            
            17,10000000.0,9999929.200000012
            
            15,10000000.0,9999921.400000028
            
            13,10000000.0,9999907.600000022
            
            11,10000000.0,9999905.400000053
            
            18,10000000.0,9999936.400000017
            
            20,10000000.0,9999941.199999996
            
            22,10000000.0,9999932.200000025
            
            24,10000000.0,9999937.000000019
            
            21,10000000.0,9999940.200000029
            
            19,10000000.0,9999935.100000003
            
            23,10000000.0,9999937.500000019
            
            25,10000000.0,9999938.100000018
            
            30,10000000.0,9999946.000000006
            
            28,10000000.0,9999943.700000001
            
            26,10000000.0,9999942.499999996
            
            29,10000000.0,9999954.699999997
            
            27,10000000.0,9999943.599999996
            .
            

            I have space between lines I don't how to avoid to create them

            A 1 Reply Last reply Reply Quote 0
            • C
              cortex @Brad Lloyd last edited by backtrader

              @Brad-Lloyd
              hello Brad , I want to be able to control the format of the exit.
              when I used the line you proposed :

              Traceback (most recent call last):
                File "C:/Users/mteddy/PycharmProjects/datacollection/Datahackathon/letsoptimise.py", line 168, in <module>
                  cerebro.run()
                File "C:\Users\mteddy\Anaconda2\lib\site-packages\backtrader\cerebro.py", line 801, in run
                  self.runstrats = list(pool.map(self, iterstrats))
                File "C:\Users\mteddy\Anaconda2\lib\multiprocessing\pool.py", line 251, in map
                  return self.map_async(func, iterable, chunksize).get()
                File "C:\Users\mteddy\Anaconda2\lib\multiprocessing\pool.py", line 567, in get
                  raise self._value
              ValueError: I/O operation on closed file
              
              1 Reply Last reply Reply Quote 0
              • A
                ab_trader @cortex last edited by

                @cortex check this out -
                http://stackoverflow.com/questions/3348460/csv-file-written-with-python-has-blank-lines-between-each-row

                • If my answer helped, hit reputation up arrow at lower right corner of the post.
                • Python Debugging With Pdb
                • New to python and bt - check this out
                1 Reply Last reply Reply Quote 0
                • B
                  backtrader administrators last edited by

                  This is really an unexpected use case, because the writer works by collecting data in real time and there is single writer instance running in the master process and not an instance per worker. Hence the closed file error notification, because the process forking has invalidated the existing file descriptor in the master.

                  It should be clear that if one tried to write to the same file from multiple processes, something unexpected would be happening.

                  The writer (with csv=True will also print all the values the data feeds have produced and that wouldn't help with multiple processes which do actually process the same data feeds)

                  Cerebro returns (a cut-down version unless optreturn=False) the strategies which have been executed. The best way to print things out is to rely on the values collected by the analyzers attached to the strategies. Of course, collecting value and cash in the stop method is a valid approach too.

                  Let's assume the case in which a single strategy is being optimized

                  
                  class MyStrategy(bt.Strategy):
                      ...
                      def stop(self):
                          self.thevalue = self.broker.get_value()
                          self.thecash = self.broker.get_cash()
                  
                  cerebro.optstrategy(MyStrategy, myparam1=range(val1, valn), ...)
                  
                  strats = cerebro.run()  # list of list
                  
                  # list comprehensions separated for clarity
                  # Get the element 0 (there was only 1 strategy in each run) of each optimization
                  st0 = [s[0] for s in strats]
                  
                  # Get the period, value and cash in text format comma separated in a list from each strategy
                  periodcashvalues = [','.join([str(s.p.maperiod), str(s.thecash), str(s.thevalue)]) for s in st0]
                  
                  # print out the generated list separating with newlines
                  print('\n'.join(cashvalues))
                  
                  T 1 Reply Last reply Reply Quote 1
                  • T
                    Taewoo Kim @backtrader last edited by Taewoo Kim

                    @backtrader said in create a CSV of the result of optstrategy:

                    The writer (with csv=True will also print all the values the data feeds have produced and that wouldn't help with multiple processes which do actually process the same data feeds)

                    Cerebro returns (a cut-down version unless optreturn=False) the strategies which have been executed. The best way to print things out is to rely on the values collected by the analyzers attached to the strategies. Of course, collecting value and cash in the stop method is a valid approach too.

                    How would I get the strategy to also print out the analyzers ? I am using optstrategy as well as all the standard analyzers:

                    cerebro.addanalyzer(btanalyzers.SharpeRatio, _name='mySharpeRatio', timeframe=bt.TimeFrame.Days, compression=1, riskfreerate=0.0)
                    cerebro.addanalyzer(btanalyzers.DrawDown, _name='myDrawDown')
                    cerebro.addanalyzer(btanalyzers.AnnualReturn, _name='myAnnualReturn')
                    #cerebro.addanalyzer(btanalyzers.GrossLeverage, _name='myGrossLeverage')
                    cerebro.addanalyzer(btanalyzers.TimeDrawDown, _name='myTimeDrawDown')
                    

                    I saw your blog post on strategy selection, but i have a different use case - 1 strategy, multiple parameters.

                    I changed the code from

                    cerebro.addstrategy(MyStrat,
                        timeframe_0_tf = "1Min",
                        timeframe_1_tf = "1H",
                        timeframe_0_num_bars_in_sr = 3,
                        timeframe_1_num_bars_in_sr = 3,
                        csv_file="data.csv",
                        sr_indicator = False,
                        min_spread = 0.0010,
                        proximity_to_sr = 0.2
                        )
                    

                    to

                    cerebro.optstrategy(MyStrat,
                        timeframe_0_tf = "1Min",
                        timeframe_1_tf = "1H",
                        timeframe_0_num_bars_in_sr = 3, 
                        timeframe_1_num_bars_in_sr = 3,
                        csv_file="data.csv",
                        sr_indicator = False,
                        min_spread = list(np.arange(0.0010,0.0020,0.0005)), 
                        proximity_to_sr = 0.2
                        )
                    

                    I.e. one line min_spread went from float to list

                    Nothing else changed. I run the code, and i get this error

                    Traceback (most recent call last):
                      File "C:\Python36\lib\threading.py", line 916, in _bootstrap_inner
                        self.run()
                      File "C:\Python36\lib\threading.py", line 864, in run
                        self._target(*self._args, **self._kwargs)
                      File "C:\Python36\lib\multiprocessing\pool.py", line 429, in _handle_results
                        task = get()
                      File "C:\Python36\lib\multiprocessing\connection.py", line 251, in recv
                        return _ForkingPickler.loads(buf.getbuffer())
                    AttributeError: Can't get attribute 'Lines_LineSeries_LineIterator_DataAccessor_ObserverBase_Observer_DataTrades_95681bf7d78147dca4d7f72d5ed03808' on <module 'backtrader.lineseries' from 'C:\\Python36\\lib\\site-packages\\backtrader\\lineseries.py'>
                    
                    B 1 Reply Last reply Reply Quote 0
                    • B
                      backtrader administrators @Taewoo Kim last edited by

                      @Taewoo-Kim said in create a CSV of the result of optstrategy:

                      How would I get the strategy to also print out the analyzers ?

                      Either with writer=True to cerebro or adding your own writer as pointed out by @Brad-Lloyd

                      cerebro.addwriter(bt.WriterFile, csv = True, out='your_strategy_results')

                      But in a multiprocess scenario this isn't sensible because the output will get interleaved and be most probably unusable.

                      That's why the blog post mentioned by you uses a standard print after the strategies have run.

                      Nothing else changed

                      Compared to the post, there are several changes, including the use of multiple data feeds. This obviously shows that the DataTrades observer has a problem with being pickled

                      1 Reply Last reply Reply Quote 0
                      • B
                        backtrader administrators last edited by

                        In any case and to avoid createing the DataTrades observer (which may be considered pointelss, since plotting in an optimization makes no sense) you may run with cerebro.run(stdstats=False)

                        1 Reply Last reply Reply Quote 0
                        • B
                          backtrader administrators last edited by

                          And upon inspection of the case it is not easily solvable. The DataTrades observer (which is used when multiple data feeds are in place) generates its lines automatically when the number of data feeds is known (that is: when the execution of each strategy starts).

                          Because of this, the attempt to place the dynamically generated classes at module level to enable pickling is not successful.

                          1 Reply Last reply Reply Quote 0
                          • T
                            Taewoo Kim last edited by

                            is it possible to access the analyzers from within the strategy?

                            I have in main:

                            cerebro.addanalyzer(btanalyzers.DrawDown, _name='myDrawDown')
                            

                            If i do (in strategy)

                            def stop(self):
                                print(self.analyzers.mySharpeRatio.get_analysis())
                            

                            it would just print a blank dict

                            However, if I call it after cerebro runs via:

                            thestrats = cerebro.run()
                            
                            print(thestrats[0].analyzers.mySharpeRatio.get_analysis())
                            

                            it seems to work.

                            Work around?

                            B 1 Reply Last reply Reply Quote 0
                            • B
                              backtrader administrators @Taewoo Kim last edited by

                              @Taewoo-Kim said in create a CSV of the result of optstrategy:

                              is it possible to access the analyzers from within the strategy?

                              The SharpeRatio is calculated after the strategy has stopped. It's not meant as a rolling analyzer (which calculates during the development of the strategy) That's the only reason for the difference.

                              1 Reply Last reply Reply Quote 0
                              • G
                                Gtrade last edited by Gtrade

                                Hello @backtrader, I have the same problem with the SQN analyzer. Is there a work around ?
                                My idea was to create a Strategy subclass that would automatically log the KPIs to a file..
                                Thanks for your great support !

                                1 Reply Last reply Reply Quote 0
                                • B
                                  backtrader administrators last edited by

                                  You could subclass Cerebro and its run method to achieve the same effect.

                                  1 Reply Last reply Reply Quote 0
                                  • S
                                    scottz1 last edited by

                                    I run an optimization and print the values of the parameters in my strategy 'stop' function.
                                    I want to include max drawdown in that report, using the DrawDown analyzer.
                                    As Taewoo mentions, calling the analyzer in the 'stop' function would be ideal. Is it viable for the framework to run the final calculations in the analyzers just before calling strategy.stop?
                                    For the alternative, I don't know what code I would need to include in a subclassed Cerebro.run. Assuming I did that, then my cerebro derived class would need to call the strategy to get all it's parameter values -- somehow after the strategy runs but before the strategy object dies.
                                    What is the best solution path?

                                    1 Reply Last reply Reply Quote 0
                                    • S
                                      scottz1 last edited by

                                      I just checked, and strategy.stop() does obtain drawdown analyzer results. Nevermind.

                                      drawdown = self.analyzers.drawdown.get_analysis()
                                      print(drawdown.max.drawdown)
                                      
                                      1 Reply Last reply Reply Quote 1
                                      • 1 / 1
                                      • First post
                                        Last post
                                      Copyright © 2016, 2017, 2018, 2019, 2020, 2021 NodeBB Forums | Contributors