Those projects and the blog post are describing the behavior of Limit Order Books. Of course when you have real-time you can monitor each and every change of those and see how things are executed (and even estimate how things are going to be executed)
Not bearing any relationship to timeframes (minutes, hours, ...), because you see every change (ticks) is the problem.
Let's imagine the scenario:
You pass 1-minute data to backtrader
A set of indicators gives you the signal to enter the market long (buy)
You fire the order in backtrader
And this is what happens:
backtrader gets the next 1-minute bar
And it tries to see, depending on the execution type, if the order can be matched against the range of prices from that bar
With, for example, Limit orders:
The matching price can be at many points in the bar
Now the question:
Are you going to provide a Limit Order Book for each an every possible price in the 1-minute bar? (Luckily it's not an infinite number, but depending on the range it can be a huge amount of entries in the Limit Order Book)
From a practical point of view the cost of the transaction and simulating the ask-bid spread effect can be achieved by including slippage (which is already supported by the broker)
@etcetc said in Oanda live data - open, close, high, low are the same:
Every printed line is still hourly, even though I would expect these lines to print out 60 times for each "hour".
60 minutes is 1 hour and that's the backfilling before the data goes into live.
In any case your strategy will not be called 60 times every hour
It will be called as many times as ticks are received by the system. Depending on how active the market is ... that could well be some thousand times ... until the hour is complete and the next bar starts.
There are no restrictions as to what you can test with backtrader. And the conditions and requirements of (almost?) any financial instrument can be simulated.
Asset margin requirements
Shorting with interest
In most cases choosing a predefined commission scheme and configuring it is enough:
Docs - Commissions: Stocks vs Futures
Docs - Extending Commissions
Docs - User Defined Commissions
Docs - Commissions: Credit
What backtrader for sure isn't is a data provider. You say
The thing I noted though is that most libraries/platform that make use of Python are focused on equities and US equities in particular
You may be talking about things like Quantopian (which now offers not only US equities but Futures) in which the online platform automatically provides you with data for US Equities.
In backtrader you need to provide the data yourself. There are many ways os instantiating a data source, including online sources, or you can provide yours. See
Docs - Data Feeds Reference
Docs - CSV Data Feed Development
Docs - Binary Datafeed Development
Docs - Extending a Datafeed
Docs - Pandas DataFeed Example
Quick fix, though in no way tested:
in stores/ibstore.py, lines 725, 751 and 836, add 'CMDTY', etc. if contract.m_secType in ['CASH', 'CFD', 'CMDTY']: ..
when defining the data feed, use full kwargs, etc.
data = ibstore.getdata(
dataname = 'XAUUSD',
sectype = 'CMDTY',
exchange = "SMART",
currency = "USD",
The `blaze´ data feed takes data from the iterator. When queried ... an iterator doesn't have an obligation to immediately return. It can block until it decides to return data.
Rather than thinking about "pushing data", you consider that you block until you want to return the data, which is effectively the same thing.
You simply then need to implement a source which implements the iterator protocol (__iter__ or the __len__ / __getitem__ combination). The collections Abstract Base Classes provide most of the machinery you may ever need by simply providing a single method.
Your other option is to do something like Oanda.
When _load is called you return
True if you have filled the buffers
False if the data source is exhausted
None (periodically) to let other elements, like the resampler, decide if a resampled bar has to be returned because the real time clock has gone over a boundary.
There is no obligation to return None if you are only backtesting or you have no plans to mix multiple datas, but you will cooperate best with the other elements if you do.
The periodicity is controlled in this case with a queue.Queue which raises an Exception after a period qget if no message from the actual data source (probably running in a background thread) has arrived.
You also override the method live to return True. Although this only plays a role in a real connection to real live feeds And since your goal is to step, you probably don't need this.
There was a pull request some time ago which added a calculation for aligning unequal (lengthwise) streams when plotting with multiple figures. The calculation was not 100% accurate, but it should be now with this commit.
Quickfix is an implementation of the FIX protocol. The problem with this being that FIX is a protocol but integration with each and every FIX provider requires specific tailoring (and a lot more work than integrating an HTTP stream)
Interactive Brokers offers FIX ... but not to you ... (actually the traffic running between TWS and the server is tagged as FIX by network analyzers)
A FIX implementation would require a funded development project. Your best bet is contacting a professional FIX integrator.