Hi,
I am sorry, I feel other already had this trouble, but I couldn't exactly find any solution.
1/Context
I intend to study/implement strategies to be run with crypto-currency.
I could spot the "Backtest-rookies" blog that gives some tutorials with CCXT, panda & backtrader: awsome! :)
I used the script they propose to extract data from a crypto exchange using CCXT and saving it in a CSV file:
https://backtest-rookies.com/2018/03/08/download-cryptocurrency-data-with-ccxt/
(for backtesting 1st, this is what I need)
In the script, panda.DataFrame is used to store OHLCV data, and to print it in a CSV file
# Get data
data = exchange.fetch_ohlcv(args.symbol, args.timeframe)
header = ['Timestamp', 'Open', 'High', 'Low', 'Close', 'Volume']
df = pd.DataFrame(data, columns=header).set_index('Timestamp')
# Save it
symbol_out = args.symbol.replace("/","")
filename = '{}-{}-{}.csv'.format(args.exchange, symbol_out,args.timeframe)
df.to_csv(filename)
2/ Settings
To import back data in backtrader, I used the script given in backtradder help:
https://www.backtrader.com/docu/pandas-datafeed/pandas-datafeed.html?highlight=panda
After encountering an error with the initial script that was already mentionned by another user in the forum, I tweaked the initial script with the following line:
https://community.backtrader.com/topic/676/bug-using-pandas-hdf
So the modified script:
from __future__ import (absolute_import, division, print_function,
unicode_literals)
import argparse
import backtrader as bt
import backtrader.feeds as btfeeds
import pandas as pd
def runstrat():
args = parse_args()
# Create a cerebro entity
cerebro = bt.Cerebro(stdstats=False)
# Add a strategy
cerebro.addstrategy(bt.Strategy)
# Get a pandas dataframe
datapath = ('../data_crypto/binance-BTCUSDT-1h_light.csv')
# Simulate the header row isn't there if noheaders requested
skiprows = 1 if args.noheaders else 0
header = None if args.noheaders else 0
dataframe = pd.read_csv(datapath,
skiprows=skiprows,
header=header,
parse_dates=True,
index_col=0)
dataframe.index = pd.to_datetime(dataframe.index)
if not args.noprint:
print('--------------------------------------------------')
print(dataframe)
print('--------------------------------------------------')
# Pass it to the backtrader datafeed and add it to the cerebro
data = bt.feeds.PandasData(dataname=dataframe)
cerebro.adddata(data)
# Run over everything
cerebro.run()
# Plot the result
cerebro.plot(style='bar')
def parse_args():
parser = argparse.ArgumentParser(
description='Pandas test script')
parser.add_argument('--noheaders', action='store_true', default=False,
required=False,
help='Do not use header rows')
parser.add_argument('--noprint', action='store_true', default=False,
help='Print the dataframe')
return parser.parse_args()
if __name__ == '__main__':
runstrat()
Added line is:
dataframe.index = pd.to_datetime(dataframe.index)
3/ Trouble
When displayed by backtrader in the log, I can see that date & time is lost. It prints date starting from 1970-01-01 00:25:54
Well, in 1970, I am quite sure Bitcoin was not existing yet... :)
I can further add that time data in saved CSV file reads for instance:
Timestamp,Open,High,Low,Close,Volume
1554897600000,5197.01,5245.0,5195.33,5234.42,1635.59772
1554901200000,5234.42,5246.39,5216.17,5231.85,1342.13614
1554904800000,5231.85,5237.33,5220.0,5231.0,899.434882
1554908400000,5231.01,5246.0,5212.04,5245.06,1175.720961
Here is what it looks like in the log after reading by backtrader:
Open High Low Close Volume
Timestamp
1970-01-01 00:25:54.897600 5197.01 5245.00 5195.33 5234.42 1635.597720
1970-01-01 00:25:54.901200 5234.42 5246.39 5216.17 5231.85 1342.136140
1970-01-01 00:25:54.904800 5231.85 5237.33 5220.00 5231.00 899.434882
1970-01-01 00:25:54.908400 5231.01 5246.00 5212.04 5245.06 1175.720961
I understand I am losing the date/time information when reading back the CSV data.
Please, does anyone know what would be the trick I am missing (probably a whole lot...)
I thank you in advance for your help.
Have a good day.
Best regards,
Pierre
