Generic CSV data feed, indicator problems on plot

s1000rr

Hi,

New user and am running into problems with plotting issues when running a data feed from generic csv. I will list the problems below:

1: Not reading first ~11 lines of input file. I used a small sample of the data file 2006-day-001.txt from github

Data file:
2006-01-02,3578.73,3605.95,3578.73,3604.33,0,0
2006-01-03,3604.08,3638.42,3601.84,3614.34,0,0
2006-01-04,3615.23,3652.46,3615.23,3652.46,0,0
2006-01-05,3652.19,3661.65,3643.17,3650.24,0,0
2006-01-06,3650.54,3666.99,3647.66,3666.99,0,0
2006-01-09,3667.10,3685.99,3667.10,3671.78,0,0
2006-01-10,3671.23,3671.23,3638.77,3644.94,0,0
2006-01-11,3645.73,3674.31,3645.73,3668.61,0,0
2006-01-12,3667.16,3676.00,3656.99,3670.20,0,0
2006-01-13,3670.27,3670.27,3618.06,3629.25,0,0
2006-01-16,3628.73,3649.10,3621.03,3644.41,0,0
2006-01-17,3639.57,3639.57,3606.54,3610.07,0,0
2006-01-18,3609.34,3609.34,3550.16,3570.17,0,0
2006-01-19,3572.19,3597.34,3572.19,3593.22,0,0
2006-01-20,3593.16,3612.37,3550.80,3550.80,0,0
2006-01-23,3550.24,3550.24,3515.07,3544.31,0,0
2006-01-24,3544.78,3553.16,3526.37,3532.68,0,0
2006-01-25,3532.72,3578.00,3532.72,3578.00,0,0
2006-01-26,3578.92,3641.42,3577.98,3641.42,0,0
2006-01-27,3643.35,3685.48,3643.35,3685.48,0,0
2006-01-30,3684.38,3685.65,3664.45,3677.52,0,0
2006-01-31,3676.71,3707.63,3671.67,3691.41,0,0
2006-02-01,3686.16,3728.80,3674.89,3728.25,0,0
2006-02-02,3728.92,3745.14,3677.05,3677.05,0,0
2006-02-03,3677.05,3696.00,3652.76,3678.48,0,0
2006-02-06,3678.87,3704.17,3672.53,3682.32,0,0
2006-02-07,3682.97,3698.63,3656.20,3680.80,0,0
2006-02-08,3680.05,3680.05,3637.93,3671.37,0,0
2006-02-09,3672.34,3726.81,3672.34,3726.81,0,0
2006-02-10,3725.18,3735.14,3692.63,3695.63,0,0
2006-02-13,3696.09,3727.46,3684.83,3727.46,0,0
2006-02-14,3728.16,3744.66,3707.25,3734.48,0,0
2006-02-15,3733.97,3749.36,3720.41,3729.79,0,0
2006-02-16,3730.82,3756.47,3730.82,3756.47,0,0
2006-02-17,3757.34,3777.16,3749.94,3767.70,0,0
2006-02-20,3767.11,3769.16,3749.88,3766.74,0,0
2006-02-21,3767.21,3800.78,3767.21,3779.51,0,0
2006-02-22,3778.02,3818.48,3771.06,3818.48,0,0
2006-02-23,3819.56,3831.16,3796.21,3813.29,0,0
2006-02-24,3812.76,3826.00,3805.55,3826.00,0,0
2006-02-27,3828.99,3840.56,3819.65,3840.56,0,0
2006-02-28,3840.31,3840.31,3769.25,3774.51,0,0
2006-03-01,3775.23,3806.34,3772.49,3806.03,0,0
2006-03-02,3807.30,3820.55,3745.46,3763.73,0,0
2006-03-03,3763.95,3774.03,3715.35,3733.95,0,0
2006-03-06,3737.58,3766.47,3737.58,3754.07,0,0
2006-03-07,3751.30,3751.30,3719.92,3745.20,0,0
2006-03-08,3745.10,3757.16,3702.04,3727.96,0,0
2006-03-09,3736.61,3765.56,3736.61,3757.59,0,0
2006-03-10,3754.13,3798.46,3741.51,3798.46,0,0
2006-03-13,3801.03,3827.45,3801.03,3824.97,0,0
2006-03-14,3823.18,3833.48,3808.96,3833.48,0,0
2006-03-15,3834.11,3853.33,3834.11,3842.16,0,0
2006-03-16,3844.15,3847.88,3822.56,3839.71,0,0
2006-03-17,3840.20,3874.64,3820.50,3832.43,0,0
2006-03-20,3833.25,3863.95,3833.11,3842.03,0,0
2006-03-21,3842.49,3848.17,3811.02,3848.17,0,0
2006-03-22,3840.27,3872.62,3827.40,3868.48,0,0

Printout of data in def next(self)
2006-01-17T23:59:59.999989   3639.57   3639.57   3606.54   3610.07   0.0
2006-01-18T23:59:59.999989   3609.34   3609.34   3550.16   3570.17   0.0
2006-01-19T23:59:59.999989   3572.19   3597.34   3572.19   3593.22   0.0
2006-01-20T23:59:59.999989   3593.16   3612.37   3550.8   3550.8   0.0
2006-01-23T23:59:59.999989   3550.24   3550.24   3515.07   3544.31   0.0
2006-01-24T23:59:59.999989   3544.78   3553.16   3526.37   3532.68   0.0
2006-01-25T23:59:59.999989   3532.72   3578.0   3532.72   3578.0   0.0
2006-01-26T23:59:59.999989   3578.92   3641.42   3577.98   3641.42   0.0
2006-01-27T23:59:59.999989   3643.35   3685.48   3643.35   3685.48   0.0
2006-01-30T23:59:59.999989   3684.38   3685.65   3664.45   3677.52   0.0
2006-01-31T23:59:59.999989   3676.71   3707.63   3671.67   3691.41   0.0
2006-02-01T23:59:59.999989   3686.16   3728.8   3674.89   3728.25   0.0
2006-02-02T23:59:59.999989   3728.92   3745.14   3677.05   3677.05   0.0
2006-02-03T23:59:59.999989   3677.05   3696.0   3652.76   3678.48   0.0
2006-02-06T23:59:59.999989   3678.87   3704.17   3672.53   3682.32   0.0
2006-02-07T23:59:59.999989   3682.97   3698.63   3656.2   3680.8   0.0
2006-02-08T23:59:59.999989   3680.05   3680.05   3637.93   3671.37   0.0
2006-02-09T23:59:59.999989   3672.34   3726.81   3672.34   3726.81   0.0
2006-02-10T23:59:59.999989   3725.18   3735.14   3692.63   3695.63   0.0
2006-02-13T23:59:59.999989   3696.09   3727.46   3684.83   3727.46   0.0
2006-02-14T23:59:59.999989   3728.16   3744.66   3707.25   3734.48   0.0
2006-02-15T23:59:59.999989   3733.97   3749.36   3720.41   3729.79   0.0
2006-02-16T23:59:59.999989   3730.82   3756.47   3730.82   3756.47   0.0
2006-02-17T23:59:59.999989   3757.34   3777.16   3749.94   3767.7   0.0
2006-02-20T23:59:59.999989   3767.11   3769.16   3749.88   3766.74   0.0
2006-02-21T23:59:59.999989   3767.21   3800.78   3767.21   3779.51   0.0
2006-02-22T23:59:59.999989   3778.02   3818.48   3771.06   3818.48   0.0
2006-02-23T23:59:59.999989   3819.56   3831.16   3796.21   3813.29   0.0
2006-02-24T23:59:59.999989   3812.76   3826.0   3805.55   3826.0   0.0
2006-02-27T23:59:59.999989   3828.99   3840.56   3819.65   3840.56   0.0
2006-02-28T23:59:59.999989   3840.31   3840.31   3769.25   3774.51   0.0
2006-03-01T23:59:59.999989   3775.23   3806.34   3772.49   3806.03   0.0
2006-03-02T23:59:59.999989   3807.3   3820.55   3745.46   3763.73   0.0
2006-03-03T23:59:59.999989   3763.95   3774.03   3715.35   3733.95   0.0
2006-03-06T23:59:59.999989   3737.58   3766.47   3737.58   3754.07   0.0
2006-03-07T23:59:59.999989   3751.3   3751.3   3719.92   3745.2   0.0
2006-03-08T23:59:59.999989   3745.1   3757.16   3702.04   3727.96   0.0
2006-03-09T23:59:59.999989   3736.61   3765.56   3736.61   3757.59   0.0
2006-03-10T23:59:59.999989   3754.13   3798.46   3741.51   3798.46   0.0
2006-03-13T23:59:59.999989   3801.03   3827.45   3801.03   3824.97   0.0
2006-03-14T23:59:59.999989   3823.18   3833.48   3808.96   3833.48   0.0
2006-03-15T23:59:59.999989   3834.11   3853.33   3834.11   3842.16   0.0
2006-03-16T23:59:59.999989   3844.15   3847.88   3822.56   3839.71   0.0
2006-03-17T23:59:59.999989   3840.2   3874.64   3820.5   3832.43   0.0
2006-03-20T23:59:59.999989   3833.25   3863.95   3833.11   3842.03   0.0
2006-03-21T23:59:59.999989   3842.49   3848.17   3811.02   3848.17   0.0
2006-03-22T23:59:59.999989   3840.27   3872.62   3827.4   3868.48   0.0

2: Buy and Sell indicators are shifted above/below the stock price

Attached below is the plot output, please note how the green and red indicators are shifted of the stock price. This vertical shift gets much worse if the stock price has greater swings over a similar period of time and the indicators will be completely off the screen in some instances.

3: Inconsistent time separation

Using the same plot from above we can see the varying distance that separates the vertical date lines.

Code:

This is sample code from the main page with the addition of the printout and generic csv input.

class SmaCross(bt.SignalStrategy):
    params = (('pfast', 5), ('pslow', 10),)
    def __init__(self):
        sma1, sma2 = bt.ind.SMA(period=self.p.pfast), bt.ind.SMA(period=self.p.pslow)
        self.signal_add(bt.SIGNAL_LONG, bt.ind.CrossOver(sma1, sma2))

    def next(self):
    	print(self.data.datetime.datetime(0).isoformat(), " " , 
    		  self.data.open[0], " ", 
    		  self.data.high[0], " ", 
    		  self.data.low[0], " ", 
    		  self.data.close[0], " ", 
    		  self.data.volume[0])

cerebro = bt.Cerebro()

data2 = btfeed.GenericCSVData(
    dataname='2006-day-001-smallsample.csv',
    dtformat=('%Y-%m-%d'),
    datetime=0,
    open=1,
    high=2,
    low=3,
    close=4,
    volume=5,
    openinterest=6,
    time=-1,
    timeframe=bt.TimeFrame.Days,
    compression=1,
)

cerebro.adddata(data2)
cerebro.addstrategy(SmaCross)
cerebro.run()
cerebro.plot()

Any ideas what I'm doing wrong with the Generic CSV data feed?

Thanks

backtrader

@s1000rr said in Generic CSV data feed, indicator problems on plot:

1: Not reading first ~11 lines of input file. I used a small sample of the data file 2006-day-001.txt from github

Minimum period. The lines are all being read. Read:

@s1000rr said in Generic CSV data feed, indicator problems on plot:

2: Buy and Sell indicators are shifted above/below the stock price

Read the documentation for the BuySell observer. It is an observer which simply indicates when and what happened. Detailed values are given to you in order execution.

Docs - Observer Reference

@s1000rr said in Generic CSV data feed, indicator problems on plot:

3: Inconsistent time separation
Using the same plot from above we can see the varying distance that separates the vertical date lines.

You probably want to elaborate on what you mean here.

In any case if you are expecting a charting platform, backtrader isn't. The charts are meant only as a quick visual aid for inspection and are not the core of the platform. It's just an add-on built on top of matplotlib

s1000rr

Hi,

1: ahhh got it, thanks for the explanation.

2: I was just curious why the actual observers were shifted of the plot visually, understand that it's normal.

3: This again was just a visual thing, I have since noticed that on larger data sets it selects a more even distribution of vertical lines to indicate the dates.

Like you said, the charts are just visual aids and I was just curious to see if there were settings that could control the above questions.

Thanks for the help.