For code/output blocks: Use ``` (aka backtick or grave accent) in a single line before and after the block. See: http://commonmark.org/help/

How to add event information into strategy



  • Hi everyone,

    I'm a total noob to BackTrader but first let me say, this platform is awesome! Thanks to everyone for making this available!

    I do have a question, hopefully an easy one... I want to add event data to a backtest (and a live strategy for that matter) but I'm unsure how.

    Let's start with backtesting....

    I'd think like to add this data feed so that when next gets called the value shows up just like any other data (like Close or whatever).

    First, is that how experienced backtraders would expect this new data to appear?

    Second, this is what I did (and where I'm stuck).

    I made a csv for the events that just appends an "Event" Column. I'm starting simple, with one event:

    Date,Open,High,Low,Close,Adj Close,Volume,Event
    2014-01-03 07:30:00,,,,,,,4
    

    Then I made a new feed...

    class GenericCSV_Event(GenericCSVData):
    
        # Add a 'event' line to the inherited ones from the base class
        lines = ('event',)
    
        # openinterest in GenericCSVData has index 7 ... add 1
        # add the parameter to the parameters inherited from the base class
        params = (('Event', 8),)
    

    I created the feed and added it alongside a stock feed:

        cerebro.adddata(data)
        cerebro.adddata(event_data)
    

    And, im stuck. I can't around a few errors like:

    AttributeError: 'AutoInfoClass_LineRoot_LineMultiple_LineSeries_Dat' object has no attribute 'event'

    I'm kinda confused about the capitalization of the "Event" in my csv file (which I copied from the orcl-2014.txt file) and the use of event in the GenericCSV_Event class.

    Anyway, here is the whole example... perhaps someone can point me in the right direction?

    Thanks!

    from __future__ import (absolute_import, division, print_function,
                            unicode_literals)
    
    import datetime  # For datetime objects
    import os.path  # To manage paths
    import sys  # To find out the script name (in argv[0])
    from backtrader.feeds import GenericCSVData
    
    # Import the backtrader platform
    import backtrader as bt
    
    class GenericCSV_Event(GenericCSVData):
    
        # Add a 'event' line to the inherited ones from the base class
        lines = ('event',)
    
        # openinterest in GenericCSVData has index 7 ... add 1
        # add the parameter to the parameters inherited from the base class
        params = (('event', 8),)
    
    
    # Create a Stratey
    class TestStrategy(bt.Strategy):
    
        def log(self, txt, dt=None):
            ''' Logging function for this strategy'''
            dt = dt or self.datas[0].datetime.date(0)
            event = self.datas[0].event[0]
            print('%s, %s' % (dt.isoformat(), txt))
            print("Event: {0}".format(event))
    
        def __init__(self):
            # Keep a reference to the "close" line in the data[0] dataseries
            self.dataclose = self.datas[0].close
    
        def next(self):
            # Simply log the closing price of the series from the reference
            self.log('Close, %.2f' % self.dataclose[0])
    
    
    if __name__ == '__main__':
        # Create a cerebro entity
        cerebro = bt.Cerebro()
    
        # Add a strategy
        cerebro.addstrategy(TestStrategy)
    
        # Datas are in a subfolder of the samples. Need to find where the script is
        # because it could have been called from anywhere
        modpath = os.path.dirname(os.path.abspath(sys.argv[0]))
        datapath = os.path.join(modpath, '../backtrader/datas/orcl-2014.txt')
    
        # Create a Data Feed
        data = bt.feeds.YahooFinanceCSVData(
            dataname=datapath,
            # Do not pass values before this date
            fromdate=datetime.datetime(2014, 1, 1),
            # Do not pass values before this date
            todate=datetime.datetime(2014, 1, 5),
            # Do not pass values after this date
            reverse=False)
    
        # now add the event data:
        modpath = os.path.dirname(os.path.abspath(sys.argv[0]))
        datapath = os.path.join(modpath, '../backtrader/datas/events.csv')
        # Create a Data Feed
        event_data = GenericCSV_Event(
            dataname=datapath,
            # Do not pass values before this date
            fromdate=datetime.datetime(2014, 1, 1),
            # Do not pass values before this date
            todate=datetime.datetime(2014, 1, 5),
            # Do not pass values after this date
            reverse=False)
    
    
        # Add the Data Feed to Cerebro
        cerebro.adddata(data)
        cerebro.adddata(event_data)
    
        # Set our desired cash start
        cerebro.broker.setcash(100000.0)
    
        # Print out the starting conditions
        print('Starting Portfolio Value: %.2f' % cerebro.broker.getvalue())
    
        # Run over everything
        cerebro.run()
    
        # Print out the final result
        print('Final Portfolio Value: %.2f' % cerebro.broker.getvalue())
    
    

    Also, I loved the line: "Blistering Barnacles!!! The system made money ... something must be wrong"



  • OK, so this morning, I tried again... I got it! Posting here for others.

    BUT, I do have a question... I'm getting events for dates that don't appear in my event list. It's like the event data is being backfilled (ie: the last event data is being passed to the current bar). Is there a way to prevent that? I've tried backfill=False but that doesn't seem to work (possibly not a property of the CSV feed).

    Here's the example... this event file has two events in the same week. But instead of just showing up in the bar of those two days, there's an event for every bar.

    Thanks!

    CSV File:

    TIMESTAMP,EVENT
    2014-01-07,4
    2014-01-09,24
    

    Output: (how do I prevent the events from showing up on the 3 days of the week without events?)

    Starting Portfolio Value: 100000.00
    2014-01-07, Close, 37.85
    2014-01-07, Event, 4
    2014-01-08, Close, 37.72
    2014-01-08, Event, 4
    2014-01-09, Close, 37.65
    2014-01-09, Event, 24
    2014-01-10, Close, 38.11
    2014-01-10, Event, 24
    Final Portfolio Value: 100000.00
    

    Algo:

    from __future__ import (absolute_import, division, print_function,
                            unicode_literals)
    
    import datetime  # For datetime objects
    import os.path  # To manage paths
    import sys  # To find out the script name (in argv[0])
    from backtrader.feeds import GenericCSVData
    
    # Import the backtrader platform
    import backtrader as bt
    
    
    class EventCSV(GenericCSVData):
        linesoverride = True  # discard usual OHLC structure
        # datetime must be present and last
        lines = ('event', 'datetime')
        #lines = ('datetime', 'event')
        # datetime (always 1st) and then the desired order for
        params = (
            ('dtformat', '%Y-%m-%d'),
            ('datetime', 0), # inherited from parent class
            ('event', 1),  # default field pos 1
        )
    
    # Create a Stratey
    class TestStrategy(bt.Strategy):
    
        def log(self, txt, dt=None):
            ''' Logging function for this strategy'''
            dt = dt or self.datas[0].datetime.date(0)
            print('%s, %s' % (dt.isoformat(), txt))
    
    
        def __init__(self):
            # Keep a reference to the "close" line in the data[0] dataseries
            self.dataclose = self.datas[0].close
            self.event_data = self.datas[1]
    
        def next(self):
            # Simply log the closing price of the series from the reference
            self.log('Close, %.2f' % self.dataclose[0])
            self.log('Event, %d' % self.event_data[0])
    
    
    
    if __name__ == '__main__':
        # Create a cerebro entity
        cerebro = bt.Cerebro()
    
        # Add a strategy
        cerebro.addstrategy(TestStrategy)
    
        # Datas are in a subfolder of the samples. Need to find where the script is
        # because it could have been called from anywhere
        modpath = os.path.dirname(os.path.abspath(sys.argv[0]))
        datapath = os.path.join(modpath, '../backtrader/datas/orcl-2014.txt')
    
        # Create a Data Feed
        data = bt.feeds.YahooFinanceCSVData(
            dataname=datapath,
            # Do not pass values before this date
            fromdate=datetime.datetime(2014, 1, 6),
            # Do not pass values after this date
            todate=datetime.datetime(2014, 1, 11),
            reverse=False)
    
        # now add the event data:
        modpath = os.path.dirname(os.path.abspath(sys.argv[0]))
        datapath = os.path.join(modpath, '../backtrader/datas/events_02.csv')
        # Create a Data Feed
        event_data = EventCSV(
            dataname=datapath,
            # Do not pass values before this date
            fromdate=datetime.datetime(2014, 1, 6),
            # Do not pass values after this date
            todate=datetime.datetime(2014, 1, 11),
            backfill=False,
            reverse=False)
    
    
        # Add the Data Feed to Cerebro
        cerebro.adddata(data)
        cerebro.adddata(event_data, name='event_data')
    
        # Set our desired cash start
        cerebro.broker.setcash(100000.0)
    
        # Print out the starting conditions
        print('Starting Portfolio Value: %.2f' % cerebro.broker.getvalue())
    
        # Run over everything
        cerebro.run()
    
        # Print out the final result
        print('Final Portfolio Value: %.2f' % cerebro.broker.getvalue())
    
    

    Thanks!



  • Hi,

    Another way of doing this could be to create an indicator events, load the csv in a panda df during the init and at each bar check whether or not the date is in the df. So the indicator would return true if the date is an event and false if not.



  • Yes!!! This is (potentially) great! It might even solve my problem of using CSVs for live trading. I'll attempt to create an indicator that reads the csv into a df in next() . This way it always grabs the latest file on disk.

    Stay tuned...

    Thanks!


  • administrators

    @brettelliot said in How to add event information into strategy:

    this event file has two events in the same week. But instead of just showing up in the bar of those two days, there's an event for every bar.

    This is because you have implemented your event as a data feed. And the platform presents you with the latest available data point. It's not actually showing twice (or three times). You are reading it twice without noticing that the date to which the data point refers has not changed.

    The same holds true if you for example have the SP500 and the DAX. The trading calendars are not equal and you may get a new data point for the DAX, whilst the SP500 may present you the data point from the previous day (because the markets in the USA are closed for the 4th of July)

    If you implement it as an Indicator as suggested by @Laurent-Michelizza (which has already been discussed several times) you can decide in the next method of the indicator if the data point of your event has to be presented or not (for example by checking if the date for the data on which the indicator is applied is the same as the one for the event)



  • Thank you @backtrader and @Laurent-Michelizza ! I've gotten this working with an indicator. Code posted below for posterity:

    from __future__ import (absolute_import, division, print_function,
                            unicode_literals)
    
    import datetime  # For datetime objects
    import os.path  # To manage paths
    import sys  # To find out the script name (in argv[0])
    import pandas as pd
    import numpy as np
    
    # Import the backtrader platform
    import backtrader as bt
    
    class EventIndicator(bt.Indicator):
        lines = ('event_line',)
    
        def next(self):
    
            df = pd.read_csv("../backtrader/datas/events_02.csv",
                                        parse_dates=True,
                                        index_col=0)
    
            dt = self.datas[0].datetime.date(0)
            try:
                event = df.loc[dt, 'EVENT']
                #print("{0}, Event: {1}".format(dt, event))
                self.lines.event_line[0] = event
            except KeyError:
                pass
    
    
    
    # Create a Stratey
    class TestStrategy(bt.Strategy):
    
        def log(self, txt, dt=None):
            ''' Logging function for this strategy'''
            dt = dt or self.datas[0].datetime.date(0)
            print("{}: {}".format(dt.isoformat(), txt))
    
    
        def __init__(self):
            # Keep a reference to the event indicator
            self.event_indicator = EventIndicator(self.data)
    
        def next(self):
            # Simply log the whenever there was an event from the event indicator
            event = self.event_indicator[0]
            if not np.isnan(event):
                self.log("Event: {}".format(event))
            if event >= 10:
                self.buy()
    
    
    if __name__ == '__main__':
        # Create a cerebro entity
        cerebro = bt.Cerebro()
    
        # Add a strategy
        cerebro.addstrategy(TestStrategy)
    
        # Datas are in a subfolder of the samples. Need to find where the script is
        # because it could have been called from anywhere
        modpath = os.path.dirname(os.path.abspath(sys.argv[0]))
        datapath = os.path.join(modpath, "../backtrader/datas/orcl-2014.txt")
    
        # Create a Data Feed
        data = bt.feeds.YahooFinanceCSVData(
            dataname=datapath,
            # Do not pass values before this date
            fromdate=datetime.datetime(2014, 1, 6),
            # Do not pass values after this date
            todate=datetime.datetime(2014, 1, 11),
            reverse=False)
    
        # Add the Data Feed to Cerebro
        cerebro.adddata(data)
    
        # Set our desired cash start
        cerebro.broker.setcash(100000.0)
    
        # Print out the starting conditions
        print('Starting Portfolio Value: %.2f' % cerebro.broker.getvalue())
    
        # Run over everything
        cerebro.run()
    
        # Print out the final result
        print('Final Portfolio Value: %.2f' % cerebro.broker.getvalue())
    
    


  • I did approximately the same to consider stock splits.


Log in to reply
 

Looks like your connection to Backtrader Community was lost, please wait while we try to reconnect.