backtrader is not a data provider. The data feeds connect to providers. A provider can be:
A CSV file
A Pandas DataFrame
An online service over Direct Sockets (IB), HTTP (Oanda), COM (VisualChart)
If you have data from those sources which is from futures and with 1-minute resolution, it will be handled.
One use case:
You can implement a logic in your strategy which continuously checks the time of the day to see if only 5 minutes remain until the end of the session. This is the only moment in the session in which your fund executes the daily operations.
If this precise minute is missing (nothing was negotiated), your time comparison will fail. You can enhance the logic to not only check for == but also for >=, but you then need a bool to understand if you have already acted.
The timer will be called only once and 5 minutes before the session end without a constant logic check in next and with no bool involved, because it will only be called once.
The strategy contains the datetime information and things can be done manually. In general is a way to express things in a different way which may be more elegant.
Some people want to do portfolio rotation on the 15th of each month, but that day may be a weekend or a bank holiday. The timer will be called with the next possible day, allowing the rotation to always happen on the 15th or the 1st trading day thereafter (which matches the reality of life)
I only just got around to looking at this again. If anyone else, has the same requirement, the code I developed is as follows:
This commission scheme uses a fixed commission and stamp duty for share
purchases. Share sales are subject only to the fixed commission.
The scheme is intended for trading UK equities on the main market.
params = (
def _getcommission(self, size, price, pseudoexec):
If size is greater than 0, this indicates a long / buying of shares.
If size is less than 0, it idicates a short / selling of shares.
if size > 0:
return self.p.commission + (size * price * self.p.stamp_duty)
elif size < 0:
return 0 #just in case for some reason the size is 0.
@gminorcoles I am just a user of the bt, not an owner. Sometimes I share pieces of the code. @backtrader is an owner.
if you want/can offer any enhancements, then just create pull request with that enhancement. @backtrader will check it and incorporate if it will be suitable from his point of view.
The SharpeRatio analyzer doesn't do rolling calculations, it does the calculation when everything is over.
You can subclass it, override __init__ and create the sub analyzer TimeReturn analyzer with your _doprenext of choice.
You are being fooled by two factors:
Yourself (it happens during the development of backtrader all the time and even more often than that)
1. Why Yourself?
prenext is where you can do dirty things, because it happens before the next phase is in effect. Data feeds and indicators may not be delivering values (or meaningful ones) because the corresponding minimum periods to deliver have not yet been met.
That means that accessing values should be done carefully and with several safeguarding measures in place. The most usual measure is to check if the object which is going to be checked has a positive length. As in:
if len(self.data1): # is not zero
print('it is then guaranteed that it has something like:', self.data.close)
Because the data feeds have different starting points they will start delivering at different times.
2. Why Data Preloading?
Because no safeguards are in place and the default (to speed up things) is to preload the data before doing anything, the python arithmetic for array indexing is taking you somewhere ... which happens to be the last value of the array.
The lifecycle of a Strategy is documented: Docs - Strategy
Those projects and the blog post are describing the behavior of Limit Order Books. Of course when you have real-time you can monitor each and every change of those and see how things are executed (and even estimate how things are going to be executed)
Not bearing any relationship to timeframes (minutes, hours, ...), because you see every change (ticks) is the problem.
Let's imagine the scenario:
You pass 1-minute data to backtrader
A set of indicators gives you the signal to enter the market long (buy)
You fire the order in backtrader
And this is what happens:
backtrader gets the next 1-minute bar
And it tries to see, depending on the execution type, if the order can be matched against the range of prices from that bar
With, for example, Limit orders:
The matching price can be at many points in the bar
Now the question:
Are you going to provide a Limit Order Book for each an every possible price in the 1-minute bar? (Luckily it's not an infinite number, but depending on the range it can be a huge amount of entries in the Limit Order Book)
From a practical point of view the cost of the transaction and simulating the ask-bid spread effect can be achieved by including slippage (which is already supported by the broker)
@etcetc said in Oanda live data - open, close, high, low are the same:
Every printed line is still hourly, even though I would expect these lines to print out 60 times for each "hour".
60 minutes is 1 hour and that's the backfilling before the data goes into live.
In any case your strategy will not be called 60 times every hour
It will be called as many times as ticks are received by the system. Depending on how active the market is ... that could well be some thousand times ... until the hour is complete and the next bar starts.
There are no restrictions as to what you can test with backtrader. And the conditions and requirements of (almost?) any financial instrument can be simulated.
Asset margin requirements
Shorting with interest
In most cases choosing a predefined commission scheme and configuring it is enough:
Docs - Commissions: Stocks vs Futures
Docs - Extending Commissions
Docs - User Defined Commissions
Docs - Commissions: Credit
What backtrader for sure isn't is a data provider. You say
The thing I noted though is that most libraries/platform that make use of Python are focused on equities and US equities in particular
You may be talking about things like Quantopian (which now offers not only US equities but Futures) in which the online platform automatically provides you with data for US Equities.
In backtrader you need to provide the data yourself. There are many ways os instantiating a data source, including online sources, or you can provide yours. See
Docs - Data Feeds Reference
Docs - CSV Data Feed Development
Docs - Binary Datafeed Development
Docs - Extending a Datafeed
Docs - Pandas DataFeed Example
Quick fix, though in no way tested:
in stores/ibstore.py, lines 725, 751 and 836, add 'CMDTY', etc. if contract.m_secType in ['CASH', 'CFD', 'CMDTY']: ..
when defining the data feed, use full kwargs, etc.
data = ibstore.getdata(
dataname = 'XAUUSD',
sectype = 'CMDTY',
exchange = "SMART",
currency = "USD",
The `blaze´ data feed takes data from the iterator. When queried ... an iterator doesn't have an obligation to immediately return. It can block until it decides to return data.
Rather than thinking about "pushing data", you consider that you block until you want to return the data, which is effectively the same thing.
You simply then need to implement a source which implements the iterator protocol (__iter__ or the __len__ / __getitem__ combination). The collections Abstract Base Classes provide most of the machinery you may ever need by simply providing a single method.
Your other option is to do something like Oanda.
When _load is called you return
True if you have filled the buffers
False if the data source is exhausted
None (periodically) to let other elements, like the resampler, decide if a resampled bar has to be returned because the real time clock has gone over a boundary.
There is no obligation to return None if you are only backtesting or you have no plans to mix multiple datas, but you will cooperate best with the other elements if you do.
The periodicity is controlled in this case with a queue.Queue which raises an Exception after a period qget if no message from the actual data source (probably running in a background thread) has arrived.
You also override the method live to return True. Although this only plays a role in a real connection to real live feeds And since your goal is to step, you probably don't need this.