If you can write it down to disk, anything will be compatible.
If you are looking for live minute data with backfilling, the only other such source which is implemented is *[VisualChart][https://www.visualchart.com]. This has the drawback that it only runs under Windows and may be out of your area of influence.
Communication over COM has also small issues and data download may have to be restarted (COM is not one of the strongest aspects in Python)
The problem is:
How do you resample the code?
See the usual values:
open it is only taken 1 at the beginning of a resample cycle and never changed again until delivery
high and low are constanly updated if exceeded until delivery (max and min)
close is always updated until delivery
volume starts at 0 and is incremented with the incoming data
The platform cannot know what to do with an unknown value and therefore nothing is done. The given line doesn't exist for the resampler.
Implement a filter which is plugged into the resampled instance, looks into the original data feed and resamples code with your desired policy.
See: Docs - Filters
Hint: the resampled data instance contains a data attribute which points to the original non-resampled data instance.
Sorry just saw that you replied to this.
Actually I was more considering it in the live environment if/when the strategy graduates from back testing . It would be useful to be able to visually see what the strategy is doing for monitoring purposes and peace of mind of those who might use the strategy but have not developed it.
I understand we can already see buying and selling using the brokers own platform. However you can't see any custom indicators, support lines or other things we can plot on the backtested charts that might have caused the strategy to enter or exit a trade.
Glad it worked.
timeframe=1 seems anyhow odd because that is the actual value for bt.TimeFrame.Ticks and the data seems to be made up of 1-minute bars.
The platform will not complain (especially if no resampling is done), but you can see that later in a plot.
The suggestion would be to use: timeframe=bt.TimeFrame.Minutes (symbolic names seem better)
There doesn't seem to be a fully supported Python API but there are some projects:
With regards to implementation the lates broker to be added was Oanda and should serve as a guidance. How it is foreseen (but not set in stone):
Development of a Store which is a singleton (see the Oanda store -> Source)
This is the entity which actually talks to the API and creates threads and manages events if needed be
Development of a Data Feed (as stated above, see the Oanda data feed -> Source)
Development of a Broker (as stated above, see the Oanda broker -> Source)
backtrader takes a dual approach to the problem. This is controlled with the runonce (boolean) parameter to either a instantiation of Cerebro or to cerebro.run like in
The default is runonce=True
cerebro = Cerebro(runonce=True) # or False
cerebro = Cerebro()
cerebro.run(runonce=True) # or False
This could be called a pseudo-vectorized of half-vectorized approach. Built-in operations feature a once method which calculate things in batch mode in a tight inner loop.
Data feeds are fully pre-loaded
Indicators (and sub-indicators thereof) are pre-calculated in batch-mode
Then, the Strategy instance(s) are run step-by-step
Being the goal to offer an increase in speed, but still allow for fine grained logic in the next method of the strategy
Rough calculations indicate that it is somehow between 20-30% than runonce=False
Drawback: Because indicators are precalculated (and therefore the buffers are preallocated), the data synchronization mechanism cannot pause the actual movement of a data feed when synchronizing the timestamps for the strategy, keeping the buffers to the final same length. This has no actual impact for backtesting but because matplotlib expects all things to have the same x length for plotting, it may not be possible to create a plot of the backtesting.
Drawback 2: The implementation of this mode prevented that some indicators can be fully defined in recursive terms with a single formula. A choice had to be made between having this or having the recursive formulas.
Nice Thing: If a user implements a custom Indicator and only provides a next method (intented for step-by-step, see below), the code automatically detects it and will still pre-calculate the indicator using the next method instead of the missing once method. The calculation loop will not be so tight as it could be, but users don't have to worry about implementing once
This is a 100% step-by-step mode. Also named next because only the next method of the different indicators, strategies et al., play a role.
Everything is calculated one step at a time. The reason being the addition of data feeds which would be providing the data points one step at a time (not necessarily live feeds, it could have been reading out of a socket from a database connection).
If cerebro is run with preload=False (disable the preloading of data feeds) it will switch to this mode.
@Harel-Rozental said in Smart optimizations and Backtrader:
I managed to get my optimizations to run as quick as optstrategy by subclassing Cerebro and pulling out some stuff out of run() and addstrategy() into different functions (one to initialize data loading, and another which gets re-executed from outside the class for optimization).
I also manage multiprocessing from the outside.
Do you mind to share some code? Or kind of a layout of what and where was implemented in more details?
Would be something like this
params = (
datafields = [
'datetime', 'open', 'high', 'low', 'close', 'volume', 'openinterest'
self._df = self.p.dataname
# Set the standard datafields - except for datetime
for datafield in self.datafields[1:]:
# get the column index
colidx = getattr(self.params, datafield)
if colidx < 0:
# column not present -- skip
l = getattr(self.lines, datafield)
l.array = self._df.iloc[:, colidx]
field0 = self.datafields
dts = pd.to_datetime(self.index)
getattr(self.l, field0).array = dts.apply(date2num)
Where datetime is directly taken from the index. The default column offset for the other fields is probably 1-off because of it, but it can luckily be configured
You can directly upload files into the thread. Not much input would be needed.
An alternative is to enclose the data (which is apparently csv) as a code block (with no python specifyer) and it will allow simple copy & paste.