Getting historical volume/close/open/etc data in a strategy

Zubin Bharucha

Dear bt community,

First of all, let me commend you on this amazing software! What you have created here is truly amazing!

I am quite new to bt and while I've been able to do everything I've wanted so far, the one thing that escapes me is the ability to retrieve historical data in a strategy. What I would like to do is to make a buy/sell decision based on the average traded volume over the last x days (this is obviously not a real-world scenario, just an example). I looked through the documentation and I see that it's possible to get individual values, e.g., something like self.datas[0].volume[-10]. However, what I want to do is to get something like self.datas[0].volume[-1 : -10 : -1], i.e., to get the last 10 traded volumes in one shot. If I do this, however, I get a TypeError.

Here's a concrete example (hacked from an example in the quickstart documentation):

import datetime  # For datetime objects
import numpy

# Import the backtrader platform
import backtrader as bt

# Create a Stratey
class TestStrategy(bt.Strategy):

    def __init__(self):
        # Keep a reference to the "close" line in the data[0] dataseries
        self.datavolume = self.datas[0].volume

    def next(self):

        # historical volume based decisions <- THIS DOES NOT WORK!!
        if self.datavolume[0] / numpy.mean(self.datavolume[-1 : -10 : -1]) >= 1.25:
            # BUY, BUY, BUY!!! (with all possible default parameters)
            self.buy()

        # single-value volume based decisions <- THIS WORKS
        if self.datavolume[0] > self.datavolume[-1]:
            # current close less than previous close

            if self.datavolume[-1] > self.datavolume[-2]:

                # BUY, BUY, BUY!!! (with all possible default parameters)
                self.buy()

if __name__ == '__main__':
    # Create a cerebro entity
    cerebro = bt.Cerebro()

    # Add a strategy
    cerebro.addstrategy(TestStrategy)

    # Datas are in a subfolder of the samples. Need to find where the script is
    # because it could have been called from anywhere
    datapath = 'goog.csv'

    # Create a Data Feed
    data = bt.feeds.YahooFinanceCSVData(
        dataname=datapath,
        # Do not pass values before this date
        fromdate=datetime.datetime(2016, 1, 1),
        # Do not pass values before this date
        todate=datetime.datetime(2016, 12, 31),
        # Do not pass values after this date
        reverse=True)

    # Add the Data Feed to Cerebro
    cerebro.adddata(data)

    # Set our desired cash start
    cerebro.broker.setcash(100000.0)

    # Print out the starting conditions
    print('Starting Portfolio Value: %.2f' % cerebro.broker.getvalue())

    # Run over everything
    cerebro.run()

    # Print out the final result
    print('Final Portfolio Value: %.2f' % cerebro.broker.getvalue())

Please see the part where I say "<- THIS DOES NOT WORK!!" :). The exact error is:

<path>python bt_example.py
Starting Portfolio Value: 100000.00
Traceback (most recent call last):
  File "bt_example.py", line 64, in <module>
    cerebro.run()
  File "C:\Python27\lib\site-packages\backtrader\cerebro.py", line 810, in run
    runstrat = self.runstrategies(iterstrat)
  File "C:\Python27\lib\site-packages\backtrader\cerebro.py", line 929, in runstrategies
    self._runonce(runstrats)
  File "C:\Python27\lib\site-packages\backtrader\cerebro.py", line 1302, in _runonce
    strat._oncepost(dt0)
  File "C:\Python27\lib\site-packages\backtrader\strategy.py", line 269, in _oncepost
    self.nextstart()  # only called for the 1st value
  File "C:\Python27\lib\site-packages\backtrader\lineiterator.py", line 324, in nextstart
    self.next()
  File "bt_example.py", line 18, in next
    if self.datavolume[0] / numpy.mean(self.datavolume[-1 : -10 : -1]) >= 1.25:
  File "C:\Python27\lib\site-packages\backtrader\linebuffer.py", line 163, in __getitem__
    return self.array[self.idx + ago]
TypeError: unsupported operand type(s) for +: 'int' and 'slice'

Does anyone know how I can achieve what I want without resorting to for loops, etc.?

Many thanks in advance!

backtrader

@Zubin-Bharucha said in Getting historical volume/close/open/etc data in a strategy:

if self.datavolume[0] / numpy.mean(self.datavolume[-1 : -10 : -1]) >= 1.25:

Slicing is not part of the functionality of lines. You may try: self.data.volume.get(size=x, ago=n)` where

ago says how many bars backwards to move before collecting data (0 is current, -1 is the last period, -2 the one before ...). This will be the last point in the array

In any case you can also create a moving average on volume and have a direct operation as a signal

@Zubin-Bharucha said in Getting historical volume/close/open/etc data in a strategy:

Create a Strategy

class TestStrategy(bt.Strategy):

    def __init__(self):
        # Keep a reference to the "close" line in the data[0] dataseries
        self.mysignal = (self.data.volume / bt.ind.Average(self.data.volume, period=X)) >= 1.25

    def next(self):

        # historical volume based decisions <- THIS DOES NOT WORK!!
        if self.mysignal:  # equivalent to if self.signal[0]:
            # BUY, BUY, BUY!!! (with all possible default parameters)

Zubin Bharucha

Hi again @backtrader,

Many thanks for your quick answer! I'm afraid I'll require a follow-up :(.

First things first, where is the get() API documented? I couldn't find it.

I tried your first solution using the get() API and this works:

import datetime  # For datetime objects
import numpy
# Import the backtrader platform
import backtrader as bt


# Create a Stratey
class TestStrategy(bt.Strategy):

    def __init__(self):
        # Keep a reference to the "close" line in the data[0] dataseries
        self.datavolume = self.datas[0].volume
        
    def next(self):

        # historical volume based decisions <- WORKS!!
        if self.datavolume[0] / numpy.mean(self.datavolume.get(size=10, ago=-1)) >= 1.25:
            print('buy')
            self.buy()

if __name__ == '__main__':
    # Create a cerebro entity
    cerebro = bt.Cerebro()

    # Add a strategy
    cerebro.addstrategy(TestStrategy)

    # Datas are in a subfolder of the samples. Need to find where the script is
    # because it could have been called from anywhere
    datapath = 'goog.csv'

    # Create a Data Feed
    data = bt.feeds.YahooFinanceCSVData(
        dataname=datapath,
        # Do not pass values before this date
        fromdate=datetime.datetime(2016, 1, 1),
        # Do not pass values before this date
        todate=datetime.datetime(2016, 12, 31),
        # Do not pass values after this date
        reverse=True)

    # Add the Data Feed to Cerebro
    cerebro.adddata(data)

    # Set our desired cash start
    cerebro.broker.setcash(100000.0)

    # Print out the starting conditions
    print('Starting Portfolio Value: %.2f' % cerebro.broker.getvalue())

    # Run over everything
    cerebro.run()

    # Print out the final result
    print('Final Portfolio Value: %.2f' % cerebro.broker.getvalue())

Here above, I hope I'm using the size and ago parameters properly. Based on your description, I'm supposing that size indicates my "lookback" period and ago indicates from what point prior to the current data point the lookback shall begin.

Unfortunately, I couldn't get your second solution to work (maybe I've missed something):

import datetime  # For datetime objects
# Import the backtrader platform
import backtrader as bt


# Create a Stratey
class TestStrategy(bt.Strategy):

    def __init__(self):
        # Keep a reference to the "close" line in the data[0] dataseries
        self.datavolume = self.datas[0].volume
        self.mysignal = (self.data.volume / bt.ind.Average(self.data.volume, period=10)) >= 1.25 #<- DOES NOT WORK!!

    def next(self):

        # signal-based decision
        if self.mysignal:
            print('buy')
            self.buy()

if __name__ == '__main__':
    # Create a cerebro entity
    cerebro = bt.Cerebro()

    # Add a strategy
    cerebro.addstrategy(TestStrategy)

    # Datas are in a subfolder of the samples. Need to find where the script is
    # because it could have been called from anywhere
    datapath = 'goog.csv'

    # Create a Data Feed
    data = bt.feeds.YahooFinanceCSVData(
        dataname=datapath,
        # Do not pass values before this date
        fromdate=datetime.datetime(2016, 1, 1),
        # Do not pass values before this date
        todate=datetime.datetime(2016, 12, 31),
        # Do not pass values after this date
        reverse=True)

    # Add the Data Feed to Cerebro
    cerebro.adddata(data)

    # Set our desired cash start
    cerebro.broker.setcash(100000.0)

    # Print out the starting conditions
    print('Starting Portfolio Value: %.2f' % cerebro.broker.getvalue())

    # Run over everything
    cerebro.run()

    # Print out the final result
    print('Final Portfolio Value: %.2f' % cerebro.broker.getvalue())

The line where I add the signal fails with the error:

<path>python bt_example.py
Starting Portfolio Value: 100000.00
Traceback (most recent call last):
  File "bt_example.py", line 74, in <module>
    cerebro.run()
  File "C:\Python27\lib\site-packages\backtrader\cerebro.py", line 810, in run
    runstrat = self.runstrategies(iterstrat)
  File "C:\Python27\lib\site-packages\backtrader\cerebro.py", line 877, in runstrategies
    strat = stratcls(*sargs, **skwargs)
  File "C:\Python27\lib\site-packages\backtrader\metabase.py", line 87, in __call__
    _obj, args, kwargs = cls.doinit(_obj, *args, **kwargs)
  File "C:\Python27\lib\site-packages\backtrader\metabase.py", line 77, in doinit
    _obj.__init__(*args, **kwargs)
  File "bt_example.py", line 13, in __init__
    self.mysignal = (self.data.volume / bt.ind.Average(self.data.volume, period=10)) >= 1.25
TypeError: unsupported operand type(s) for /: 'LineBuffer' and 'Average'

Also, even if this were to work, how would we calculate the average for all volume values starting from the previous day and going X days back? As far as I understand, with this code snippet, the signal is generated when the current volume divided by the average of the last X volume values exceeds or equals 1.25.

One more question: is there a simple way to filter out volume values which are 0?

Finally, I may be wrong, but there's perhaps a typo in the indicator documentation (scroll down to the documentation for "Average"):
Shouldn't it be:
Formula:
av = sum(data(period)) / period

backtrader

@Zubin-Bharucha said in Getting historical volume/close/open/etc data in a strategy:

Unfortunately, I couldn't get your second solution to work (maybe I've missed something):

Please update to the latest version of backtrader 1.9.42.116. This adds support for 2 internal operations if division is not imported from the future. See here: Community - Log Returns

@Zubin-Bharucha said in Getting historical volume/close/open/etc data in a strategy:

Also, even if this were to work, how would we calculate the average for all volume values starting from the previous day and going X days back? As far as I understand, with this code snippet, the signal is generated when the current volume divided by the average of the last X volume values exceeds or equals 1.25.

The syntax during init generates a lazily evaluated object which will at each time divide the current volumen between the average of the last X periods (10 in your case) and then compares it to 1.25 and produces a 0 or 1 (False or True)

@Zubin-Bharucha said in Getting historical volume/close/open/etc data in a strategy:

One more question: is there a simple way to filter out volume values which are 0?

Filter how? Remove? Replace? You can create an indicator on the volume and set the value you wish (either the volume if non zero or another if zero)

You can also add a filter: Docs - Filters

Zubin Bharucha

Once again, thank you so much, @backtrader, for your quick and helpful answer. I think I have everything I need for now.