Then it is quite a surprise.
lets say only 1 min is missing out of 5 needed (most likely cause no trade was actually made in that time, which happens very frequently)

I don't understand why a resample won't happen with 4 out of the 5 minute?
for me, it make more sense to work with the missing values (first price available is open, last for close, min-max for high and low, and sum for volume) instead of disregarding them completely

for the case when we are missing the whole 5 min, I agree 'no data no bar'

I would have expected that if i have at least one bar available I can resample to higher time frame

say we have the following data:
min |

the smaller the timeframe the higher the chance of gaps in the data

My data is of 1-min timeframe and I'm resampling it to 5-min.
but sometimes the 1-min data have gaps (let's say up to 10 min gaps)

what is backtrader expected behavior for resampling if gap is <5-min?
and what if gap>5-min?

@backtrader said in Clarification Regarding Filters:

No idea what you mean. Filters are applied to data. Strategies have nothing to do with filters.

Maybe I should explain a bit what I'm doing and what I'm trying to accomplish.

I'm experimenting with a deep learning library that use backtrader architecture as it backend server. The library take care of all the data feed part so the user entry level to the code is via 'strategy'.

So I end up having one data feed given by the library, but I'm trying to work with a few variation of the data by introducing different filters.

for example, let's say I'm trying to work with the original data and also a resampled version. normally in backtrader I would have used cerebro.adddata() for the first data and cerebro.resampledata(...) for the second.

In my case I can modify only the strategy. so in the init I've tried making a clone() to the data and add a filter to it but It seem I lose the sync (probably because no one is advancing the pointer on next). If I just add the filter I lose the original data.

is there a backtrader solution I can apply? adding the filtered data in a way it would still sync? or dynamically add and remove filters, so I can get one data result with filter and one without? maybe some workaround with filters and indicators?

I am working on a ML project and I use the strategy class for most of the preprocessing stuff.

I have 2 clarification question that I couldn't find answer to:

1. The examples I found so far used filters on the cerebro level (like resample or replay), but can I safely use filters on data in the strategy level?

2. When applying a filter (let's say Resampler), does the filter get evealuated once at the start (to avoid repeatedly applying the same filter to the data)? and the last bar get evaluated on every next (to adjust the boundary of the resampled data to the latest known value)?

@андрей-музыкин This is absolutely amazing!!! I spent a whole week just reviewing the work you did... and I feel like I'm just scratching the surface.

mind blowing!!! thanks a lot for this contribution.

as this is very technical stuff, is there a place maybe to ask questions or exchange ideas?

love this project :)

I'm building a strategy using various indicators and also using crossover between indicators to understand points of interest.

problem is that not all indicators will crossover at the same time but can still be relevant when wanting to make a decision

I wonder what will be the best design approach to achieve a natural sync between indicators

@backtrader Thanks for your detailed reply.

As pointed out above: if the store you work with makes direct requests to the network as a result of an action initiated by the Strategy (hence ... blocking the main thread), the design is wrong.

My focus is crypto currency and I work with the community driven development that tries to bind between Backtrader and CCXT project (which play the broker roll).
but as you mentioned broker and store doesn't include threading to keep all the data well updated, instead it use cache or direct network calls.

which made me wonder how to run strategies in that setup.

clearly for it to work as intended in Backtrader more work on both store and broker is needed.

Ok, so just to see if I understand the concept, at the most simple level.
we expect from the store to take control over updating some of the data independently of what is going on in backtrader using a thread for each thing we wanna keep updated. so for example, as the store get instantiated we will need to create a thread that keep account details up to date at all time. so when we call it from the strategy we get the latest values from the cache.

You probably have a background in Javascript and like many in that camp, you believe that async is the new black.

sorry to disappoint but no background in Javascript or async for that matter :)
just trying to connect the dots with the information I read.

Thanks @backtrader for pointing this out. I didn't know python had a problem with multithreading before.

After reading quite a bit on the subject, I now understand python works best as a single thread. so now I also need to refine my question.

A big part of the bottle neck of having a lot of strategies comes down to I/O requests from the broker, which can consume a lot of time.

And the go to solution seems to be using asynchronous instead of multithreading.

but would it work/make sense to run strategies async?
may it require also changing the store?

let's say for example i have 100 strategies and some of them are calculated quickly and some take couple of minutes.
all i'm asking is it possible to run them all together at the same time or would backtrader can only process them sequentially?

I'm working on a design which involve working with many strategies. some are small and simple and other more complex. each strategy output its decision and on the end the 'master' strategy is making the final decision, which can operate on the market by placing orders or cancel them....

Is there any limitation/reason not to run all/some strategies in parallel?