@андрей-музыкин This is absolutely amazing!!! I spent a whole week just reviewing the work you did... and I feel like I'm just scratching the surface.

mind blowing!!! thanks a lot for this contribution.

as this is very technical stuff, is there a place maybe to ask questions or exchange ideas?

love this project :)

I'm building a strategy using various indicators and also using crossover between indicators to understand points of interest.

problem is that not all indicators will crossover at the same time but can still be relevant when wanting to make a decision

I wonder what will be the best design approach to achieve a natural sync between indicators

@backtrader Thanks for your detailed reply.

As pointed out above: if the store you work with makes direct requests to the network as a result of an action initiated by the Strategy (hence ... blocking the main thread), the design is wrong.

My focus is crypto currency and I work with the community driven development that tries to bind between Backtrader and CCXT project (which play the broker roll).
but as you mentioned broker and store doesn't include threading to keep all the data well updated, instead it use cache or direct network calls.

which made me wonder how to run strategies in that setup.

clearly for it to work as intended in Backtrader more work on both store and broker is needed.

Ok, so just to see if I understand the concept, at the most simple level.
we expect from the store to take control over updating some of the data independently of what is going on in backtrader using a thread for each thing we wanna keep updated. so for example, as the store get instantiated we will need to create a thread that keep account details up to date at all time. so when we call it from the strategy we get the latest values from the cache.

You probably have a background in Javascript and like many in that camp, you believe that async is the new black.

sorry to disappoint but no background in Javascript or async for that matter :)
just trying to connect the dots with the information I read.

Thanks @backtrader for pointing this out. I didn't know python had a problem with multithreading before.

After reading quite a bit on the subject, I now understand python works best as a single thread. so now I also need to refine my question.

A big part of the bottle neck of having a lot of strategies comes down to I/O requests from the broker, which can consume a lot of time.

And the go to solution seems to be using asynchronous instead of multithreading.

but would it work/make sense to run strategies async?
may it require also changing the store?

let's say for example i have 100 strategies and some of them are calculated quickly and some take couple of minutes.
all i'm asking is it possible to run them all together at the same time or would backtrader can only process them sequentially?

I'm working on a design which involve working with many strategies. some are small and simple and other more complex. each strategy output its decision and on the end the 'master' strategy is making the final decision, which can operate on the market by placing orders or cancel them....

Is there any limitation/reason not to run all/some strategies in parallel?

@ed-bartosh

Currently backfill_from is not supported. I'll be happy to see this implemented. I hope it's not that hard to implement considering working example from ibdata.py.

will add it to my backlog developments :)

can you maybe open the option for 'issues' in your repo. I have a couple more ideas for developments and maybe other people also would like to participate. ccxt topic is very active and using this one super long thread is messy

@ed-bartosh ccxt had some unique properties that made it possible to implement the broker straight from BrokerBase and give access to more then 1 exchange simultaneously in a way that feel very integrated and easy to use. remember that ccxt project is all about unifying the variety of exchanges under one api.
and a key part of my development was that the feed had some exchange info inside it.

but to make a more generic broker means to really create a broker that hold a collection of brokers inside (that are not really unified) and switching between them like you proposed before.

The biggest problem this way, is that there are still methods that will get called by Backtrader in the background on the 'active' broker without the intention of the user.

A better solution that may work more like what i wrote is if every feed will hold some broker information in a well known way and then you don't need to guess on what broker.

@Ed-Bartosh I have a special use case.
I'm trying to setup a live feed for Bitstamp and using indicators requires backfilling as well.

The problem I'm facing is that bitstamp doesn't offer ohlcv data as part of it's API's. (i'm assuming ohlcv is required)

side note:
Didn't find any reference in the code to the use of 'backfill_start'?

I have the historical data in a .csv file. is there a way maybe to stitch the two together?

found this in ibdata.py:

  ``backfill_from`` (default: ``None``)

   An additional data source can be passed to do an initial layer of

   backfilling. Once the data source is depleted and if requested,

   backfilling from IB will take place. This is ideally meant to backfill

   from already stored sources like a file on disk, but not limited to.

I have created a project on github based on @Ed-Bartosh work.
it support working simultaneously on multiple ccxt exchanges.

CCXTMultiBroker

key features:

• after creating the multibroker each exchange need to be added manually using 'add_exchange'
• most methods have a 'exchange' param to refer to the wanted exchange. some methods use the exchange info buried in the 'data' (ccxt feed object)
• cash and value defaults are aggregations of all exchanges.
• each exchange have special param 'currency_conversion_rate' that can be updated to convert amounts to one single currency