To @everyone reading this thread.

The above solution does work, however, there is a better way.

By using the above method by adding two seperate data feeds, resampling them via the timeframe parameter, and adding them to cerebro.adddata() (as shown by the code below), create some errors occur with the calculation of technical indicators (like moving average, RSI, stochastic oscillators, etc.) aswell as the line iterator when executing each tick entry in the data source.

The previous used code:

      datac = bt.feeds.GenericCSVData(dataname = datapath, 
                                    fromdate = datetime.datetime(2021, 1, 4), 
                                    todate = datetime.datetime(2021, 4, 1),
                                    dtformat=('%Y-%m-%d'),
                                    tmformat=('%H:%M:%S:%f'),
                                    timeframe=bt.TimeFrame.MicroSecondss,
                                    compression=1,
                                    date=0,
                                    time=1,
                                    open=2, #acts as bid price
                                    high=3,
                                    low=2,
                                    close=3, #acts as ask price
                                    volume=-1,
                                    openinterest=-1
                                    )
    
       datad = bt.feeds.GenericCSVData(dataname = datapath, 
                                        fromdate = datetime.datetime(2021, 1, 4), 
                                        todate = datetime.datetime(2021, 4, 1),
                                        dtformat=('%Y-%m-%d'),
                                        tmformat=('%H:%M:%S:%f'),
                                        timeframe=bt.TimeFrame.Days,
                                        compression=1,
                                        date=0,
                                        time=1,
                                        open=2,
                                        high=3,
                                        low=-2,
                                        close=3,
                                        volume=-1,
                                        openinterest=-1
                                        )

       cerebro.adddata(datac, name='data')
       cerebro.adddata(datad, name='data1')

In order to fix this if you want to run your strategy every Tick and compute technical indicators on Bar data (daily bar data, for this instance), this can be rectified by adopting cerebro.replaydata(). See below the alternative method to handeling the Tick data source.

datac = bt.feeds.GenericCSVData(dataname = datapath, 
                                    fromdate = datetime.datetime(2021, 1, 4), 
                                    todate = datetime.datetime(2021, 2, 8),
                                    dtformat=('%Y%m%d'),
                                    tmformat=('%H:%M:%S:%f'),
                                    timeframe=bt.TimeFrame.MicroSeconds, 
                                    compression=1,
                                    date=0,
                                    time=1,
                                    open=2,
                                    high=3,
                                    low=2,
                                    close=3,
                                    volume=4,
                                    openinterest=-1
                                    )

    cerebro.replaydata(datac, timeframe=bt.TimeFrame.MicroSeconds, compression=1, name='data')
    cerebro.replaydata(datac, timeframe=bt.TimeFrame.Days, compression=1, name='data1')

You can, therefore, still call your Tick data by data.open[0] and your daily bar data by data1.open[0]

@matt-wilson Now touching on your final post.

@matt-wilson said in Help Understanding Replay on Intraday Data/Compression:

Running this code, my 15 second close and my 5 minute close are identical, the 5 minute close == the 15 close on every bar. I would imagine the 5 minute close should stay as the last fully formed 5 minute bar's close price, until the 15 second dataset reaches the next % 5 minute spot, yes? Or is this a normal output? Output is below:

I will take the hit for this, sorry.
Due to a time interval's close price being dynamic (meaning that it updates continuously until the timeframe is closed) the 15-second and 5-minute intervals close[0] will always be the same.
To explain this better... at any given point in time (lets say at 10 seconds into your execution, the current rulling price will be assigned to the close price of both your 15-second and 5-minute intervals)./
But note that the open price will indicate what we are looking for as these values are static, not dynamic.

So please replace your line with the following top check whether the replaydatya works properly.

print(self.datas[0].datetime.datetime(0).isoformat(),"15-second open:", self.data0.open[0], "5-minute open:", self.data1.open[0])

Chat Thursday mate.

@batman said in Use Tick Data source so that strategy trades per tick but calculates indicators on daily bar resolution:

Or the use of strategy with more than one symbol (with more than one datafeed and replay at same time).

Thanks @Batman

In reply to your questions:

• The strategy I am currently using does not include volume. I will post on this forum as soon as I make use of the volume in a strategy (but it calculates and plots volume correctly).
• I have not had any issues with the plotting. I used the following to plot both data (tick and daily bars) and see all trades correctly

cerebro.replaydata(datac, timeframe=bt.TimeFrame.MicroSeconds, compression=1, name='data')
datac.plotinfo.plotmaster = datac
cerebro.replaydata(datac, timeframe=bt.TimeFrame.Days, compression=1, name='data1')

• I have not yet used the strategy on multiple symbols but will post here when I do.

@matt-wilson Cool man.

Yeah it took me a while to resample/replay my data correctly, but now that it is clear, you can cruise through your backtesting

@emr
Thanks for your interest and your example.
So I am currently adding my own custom broker (MT4) for live trading. Will report back as soon as I succesfully intergrate it to backtrader.

Regarding the bid and ask that you are referring to.
Due to the generic parameters and format of the bt.feeds.GenericCSVData(params), the only prices you can call are open, high, low, and close.
As the data feed doesn’t contain bid and ask parameters, you have to manually assign the bid and ask values of your CSV to open (bid price) and close (ask price) --- given that your CSV actually contains bid and ask prices (e.g. my bid prices were in column 2 of my CSV, therefore I assigned open = 2 in the bt.feed and my ask prices were in column 3 of my CSV, therefore I assigned close = 3 in the bt.feed).
This will sort out your tick data source for backtesting (added by using cerebro.replaydata(data, timeframe=bt.TimeFrame.MicroSeconds, compression=1, name=‘data’)). You will then call your Tick prices as follows in your strategy:

bid = self.data.open[0]
ask = self.data.close[0]

Next you would want to have daily bar data, for example, to calculate technical indicators.
This is then achieved by adding the data using cerebro.replaydata(data, timeframe=bt.TimeFrame.Days, compression=1, name=‘data1’).
The cerebro.replaydata() actually takes all prices (all bid and ask prices) within a daily span and calculates open (first price of that day), high (highest ask), low (lowest bid) and close (final price of that day) for you.
You will then call your Daily Bar prices as follows in your strategy:

open = self.data1.open[0]
high = self.data1.high[0]
low = self.data1.low[0]
close = self.data1.close[0]

I tested this and validated it to actual daily bar data.

@kevkev I would suggest to play around and read exactbars = True vs exactbars = False

Also encountering long runs

@samueltg92 Final shot.

Replace the following in your bt.feed.

fromdate = datetime.datetime(2004, 4, 9),
todate = datetime.datetime(2022,4,4),

@kevkev Hi,

Yes, although your tick data is bid, ask and volume, the cerebro.replaydata actaully calculates OHLCV for you in a new data source. Unfortunately backtrader cannot calculate a "warm start", meaning that it runs from the first data entry. It HAS TO first calculate the technical indicator after 12 30-minute periods before running your strategy (meaning it has a "cold start").

Regarding an additional data source of 30-minute intervals, you can simply add the data via a new bt.feeds. Just make sure that you now actually assign names to the various data to ensure you are tracking and calling them correctly.
Just ensure that their time periods overlap correctly.

@li-mike Not much I can respond to in this post. Maybe explain it a bit better and ill give it a shot

@matt-wilson Trying to replicate something on my side, but struggling.

Have you tried it with only 2 chromosomes before attempting the whole lot?

Hi @Michael-Reilly

Just a few things I would change in your code. Try them out and reply if you get the same error still.

1. I would include an input data source for your technical indicators, for instance:

sma = bt.ind.SMA(self.data, period=50)

2. You define your price as self.data, but self.data is an object, not a value. You need to specify whether you want a open, high, low or close price. If you want, for example, the close price of each bar, you should define it as (This will give you the most recent close price):

price = self.data.close[0]

3. Also, please show how your input data looks from YahooFinance. I might have some advise there for you in your bt.feeds method

@samueltg92 said in ValueError: day is out of range for month:

                           dtformat = '%d.%m.%Y' '%H:%M:%S'

Your dt format is incorrect. You have to define the entire format within one set of ' '. I suggest you replace your dtformat line with:

dtformat = '%d.%m.%Y %H:%M:%S'

Just so that you are aware, this format has to be identical to that in your input data. So for your case, it is 'day.month.year space hour:minute:second'

@bryanpyhk

Where to you initialise Cerebro? Add your entire code here then we can help

@don

Hi.

Lets say you have a moving average names ma (note that this has to be defines in your init() as self.movingaverage = bt.ind.SMA(self.data, period=10) and called in your next() as ma = self.movingaverage[0], which is the currently moving average value).

If your data is already in bar format, you can call the close of the bar by self.data.close[0].

Therefore, in your next() you can include a counter that increases every time self.data.close[0] > ma or self.data.close[0] < ma.

A naive example can look like this:

def __init__(self):
     self.movingaverage = bt.ind.SMA(self.data1, period=10)

def next(self):
     ma = self.movingaverage[0]
     counter_above = 0
     counter_below = 0
     
     if self.data.close[0] > ma:
          counter_above += 1

     elif self.data.close[0] < ma:
          counter_below += 1

     else:
          pass

Now each time is changes (bars which were above go to be below), you can reset the counters.

@pierre-cilliers-0

CORRECTION!!!

change timeframe=bt.TimeFrame.Seconds to timeframe=bt.TimeFrame.Minutes