Help importing tick data...

Oliver Bradley

I am trying to import forex tick data into backtrader. It has the following shape:

20181202 170009143,1.135000,1.135500,0
20181202 170042643,1.134960,1.135460,0
20181202 170116393,1.134970,1.135470,0
20181202 170137393,1.134850,1.135350,0

The format is YYYYMMDD HHMMSSfff,Bid,Ask,Volume

Volume is always zero as forex doesn't report trade volume. Here is my code (copied from documentation):

from __future__ import (absolute_import, division, print_function,
                        unicode_literals)

import argparse

import backtrader as bt
import backtrader.feeds as btfeeds


def runstrat():
    args = parse_args()

    # Create a cerebro entity
    cerebro = bt.Cerebro(stdstats=False)

    # Add a strategy
    cerebro.addstrategy(bt.Strategy)

    # Load the Data
    datapath = args.dataname or './DAT_ASCII_EURUSD_T_201812.csv'

    data = btfeeds.GenericCSVData(
        dataname=datapath,
        dtformat='%Y%m%d %H%M%S%f',
        timeframe=bt.TimeFrame.Ticks,
    )

    # Handy dictionary for the argument timeframe conversion
    tframes = dict(
        ticks=bt.TimeFrame.Ticks,
        microseconds=bt.TimeFrame.MicroSeconds,
        seconds=bt.TimeFrame.Seconds,
        minutes=bt.TimeFrame.Minutes,
        daily=bt.TimeFrame.Days,
        weekly=bt.TimeFrame.Weeks,
        monthly=bt.TimeFrame.Months)

    # Resample the data
    data = cerebro.resampledata(data,
                                timeframe=tframes[args.timeframe],
                                compression=args.compression)

    # add a writer
    cerebro.addwriter(bt.WriterFile, csv=True)

    # Run over everything
    cerebro.run()

    # Plot the result
    cerebro.plot(style='bar')


def parse_args():
    parser = argparse.ArgumentParser(
        description='Resampling script down to tick data')

    parser.add_argument('--dataname', default='', required=False,
                        help='File Data to Load')

    parser.add_argument('--timeframe', default='ticks', required=False,
                        choices=['ticks', 'microseconds', 'seconds',
                                 'minutes', 'daily', 'weekly', 'monthly'],
                        help='Timeframe to resample to')

    parser.add_argument('--compression', default=1, required=False, type=int,
                        help=('Compress n bars into 1'))

    return parser.parse_args()


if __name__ == '__main__':
    runstrat()

My goal is to successfully import this data but I'm getting "IndexError: list index out of range" and "backtrader/feeds/csvgeneric.py", line 148, in _loadline csvfield = linetokens[csvidx]"

I also would like to understand how to create tick volume when compressing this data into a timeframe.

Any help is appreciated (I'm a python noob)

backtrader

https://www.backtrader.com/docu/dataautoref/#genericcsvdata

You have to indicate the position in which column each field is and which fields are not present (with -1)

In your loading example not a single column is specified and the code tries to find the values in the default column ordering (datetime, OHLCV)

In any case, your data is NOT tick data. Your data is bid/ask, which isn't the same. You have to choose in which fields you want to load the data.