Thanks! Works
cerebro.plot(volume=False)
But in page https://www.backtrader.com/docu/plotting/plotting.html
I found:
# Wether to plot volume or not. Note: if the data in question has no
# volume values, volume plotting will be skipped even if this is True
self.volume = True
A little bit strange, thought changing to Flase will be automatically.
@backtrader said in IndexError in __getitem__ linebuffer (may be need to fix):
that you check the len of every data feed/indicator before attempting to access its values.
Great, thanks, I understood.
@backtrader said in IndexError in __getitem__ linebuffer (may be need to fix):
you need to fix your code and the input you provide to the platform.
May be I wrong, but it seems impossible to fix the input to the platform, because it will be ineffectively every time checking up all .csv files (feeds) for availability of fromdate before added it to cerebro. I have more than 800 files of history, they all may be used in backtesting and they all updating periodically.
As a result one of possible aprroach for me will be to edit the file in framework backtrader\analyzers\positions.py for checking the len of the data:
def next(self):
pvals = [self.strategy.broker.get_value([d]) for d in self.datas if len(d)]
Hello.
One of my datafeed ended up at 2008-05-30:
2008-05-20,10.09,10.1,9.58,9.61,,1292600.0
2008-05-21,9.68,9.69,9.32,9.35,,709700.0
2008-05-22,9.35,9.67,9.28,9.47,,1738500.0
2008-05-23,9.45,9.45,9.23,9.29,,1455300.0
2008-05-27,9.34,9.57,9.3,9.46,,1822400.0
2008-05-28,9.46,9.57,9.22,9.38,,3348100.0
2008-05-29,9.21,9.67,9.2,9.55,,5303600.0
2008-05-30,9.6,9.6,9.33,9.33,,6776700.0
I added feed data to cerebro with parameter fromdate '2008-06-01':
data = bte.feeds.PortfolioFeed(
dataname=os.path.join(path_name, file_name + '.csv'),
name=file_name,
plot=False,
fromdate=p.fromdate)
cerebro.adddata(data)
After running got an error:
Traceback (most recent call last):
File "C:/Users/-/-/Codes Python/Algo/Baskets/Baskets/_main.py", line 306, in <module>
strats = cerebro.run()
File "C:\Users\-\envs\backtrader\lib\site-packages\backtrader\cerebro.py", line 1127, in run
sdf
runstrat = self.runstrategies(iterstrat)
File "C:\Users\-\-\Codes Python\_Frame\bt_external\cerebro_multi.py", line 146, in runstrategies
self._runonce(runstrats)
File "C:\Users\-\envs\backtrader\lib\site-packages\backtrader\cerebro.py", line 1695, in _runonce
strat._oncepost(dt0)
File "C:\Users\-\envs\backtrader\lib\site-packages\backtrader\strategy.py", line 293, in _oncepost
self._next_analyzers(minperstatus, once=True)
File "C:\Users\-\envs\backtrader\lib\site-packages\backtrader\strategy.py", line 366, in _next_analyzers
analyzer._prenext()
File "C:\Users\-\envs\backtrader\lib\site-packages\backtrader\analyzer.py", line 150, in _prenext
child._prenext()
File "C:\Users\-\envs\backtrader\lib\site-packages\backtrader\analyzer.py", line 152, in _prenext
self.prenext()
File "C:\Users\-\envs\backtrader\lib\site-packages\backtrader\analyzer.py", line 229, in prenext
self.next()
File "C:\Users\-\envs\backtrader\lib\site-packages\backtrader\analyzers\positions.py", line 78, in next
pvals = [self.strategy.broker.get_value([d]) for d in self.datas]
File "C:\Users\-\envs\backtrader\lib\site-packages\backtrader\analyzers\positions.py", line 78, in <listcomp>
pvals = [self.strategy.broker.get_value([d]) for d in self.datas]
File "C:\Users\-\envs\backtrader\lib\site-packages\backtrader\brokers\bbroker.py", line 422, in get_value
return self._get_value(datas=datas, lever=lever)
File "C:\Users\-\YandexDisk\Codes Python\_Frame\bt_external\brokers\multibroker.py", line 90, in _get_value
data.close[0])
File "C:\Users\-\envs\backtrader\lib\site-packages\backtrader\linebuffer.py", line 165, in __getitem__
return self.array[self.idx + ago]
IndexError: array index out of range
If to add checking for empty array in linebuffer.py all works correctly:
def __getitem__(self, ago):
if self.array:
return self.array[self.idx + ago]
Is this the right solution? Maybe should fix this in backtrader?
@backtrader said in Mismatch unrealized pnl in broker and in strategies, need help:
If the strategies operate on the same assets (i.e.: they are antagonistic) there is no way.
Bad news for me. But anyway thanks for reply.
@backtrader said in Mismatch unrealized pnl in broker and in strategies, need help:
Wrong. The positions are the same for all strategies for a single asset.
Yes for broker it's true, I understand. But example was for situation with separatly storing positions for each strategy (with analyzer <class Positions> above). And then comparing it with Positions in broker.
It's a little bit mysterious approach above, and the reason is in the next:
Example
We have two strategies and looking for Positions for each separately (added analyzer above), and also looking for Positions in BackBroker that summarized it. Open a position in strat1: 10 size by price 100, and open in strat2: 20 size by price 200. Same instrument in both, but there are may be different opening dates.
Broker calculated price (as in position.update() logic): (10 * 100 + 20 * 200) / (10 + 20) = 166.(6) and position size 10 + 20. i.e. broker has size 30 by price 166.(6)
Then if in strategy 1 we have to partial close position with size 5, then:
strat1 position have: size 5 by price 100
strat2 position have same: size 20 by price 200 (no updating position).
and broker after updating has: size 25 by same price 166.(6)
And now if we find average price for strat1 and strat2: (5 * 100 + 20 * 200) / (5 + 20) = 180 that obviously not equal that broker position store. Thats it.
Hello. I have two strategies and need to calculate unrealized pnl separately for each strategy. I made this functionality with Analyzers:
class Positions(bt.Analyzer):
def start(self):
self._positions = defaultdict(Position)
def notify_order(self, order):
if order.status not in [Order.Partial, Order.Completed]:
return # It's not an execution
pos = self._positions[order.data]
for exbit in order.executed.iterpending():
if exbit is None:
break # end of pending reached
pos.update(exbit.size, exbit.price)
class Unrealized(bt.Analyzer):
def start(self):
self._positions = self.strategy.analyzers.positions._positions
self.unrealized = OrderedDict()
def next(self):
unrealized = 0.0
for data in self._positions:
comminfo = self.strategy.broker.getcommissioninfo(data)
position = self._positions[data]
dunrealized = comminfo.profitandloss(position.size, position.price,
data.close[0])
unrealized += dunrealized
dtkey = self.strategy.datetime.datetime()
self.unrealized[dtkey] = unrealized
def get_analysis(self):
return self.unrealized
class BrokerValue(bt.Analyzer):
def start(self):
self.unrealized = OrderedDict()
def next(self):
dtkey = self.strategy.datetime.datetime()
self.unrealized[dtkey] = self.strategy.broker._unrealized
def get_analysis(self):
return self.unrealized
It works well for one strategy in cerebro: unrealized pnl from analyzer BrokerValue is the same as from Unrealized analyzer. But for two strategies results from strat1.analyzers.unrealized + (plus) strat2.analyzers.unrealized is different with unrealized from cerebro.broker.
Here is screen with difference (orders makes in the beginings of each month): link text
Can you please advice what may be wrong in my approach? And how to make it work properly.
I was really already exhausted with it.
p.s. There are no any commissions schemes in cerebro.broker and each strategy can make orders in same instrument.
Thanks!
Solution: using analyzer.
Maybe will be usefull for somebody: next analyzer calculate and save realized pnl and opened positions for each strategy date to date:
import backtrader as bt
from collections import OrderedDict
class TotalReturns(bt.Analyzer):
'''This analyzer reports the total pnl for strategy
Params:
- timeframe (default: ``None``)
If ``None`` then the timeframe of the 1st data of the system will be
used
- compression (default: ``None``)
Only used for sub-day timeframes to for example work on an hourly
timeframe by specifying "TimeFrame.Minutes" and 60 as compression
If ``None`` then the compression of the 1st data of the system will be
used
Methods:
- get_analysis
Returns a dictionary with returns as values and the datetime points for
each return as keys
'''
def __init__(self):
self.total_pnl = 0
self.unrealized = 0 # Unrealized pnl for all positions all strategies
self.positions = OrderedDict.fromkeys([d._name or 'Data%d' % i
for i, d in enumerate(self.datas)], 0) # Current strategy positions
def start(self):
tf = min(d._timeframe for d in self.datas)
self._usedate = tf >= bt.TimeFrame.Days
def notify_order(self, order):
if order.status in [order.Completed, order.Partial]:
self.total_pnl += order.executed.pnl - order.executed.comm
self.positions[order.data._name] += order.executed.size
def next(self):
if self._usedate:
self.rets[self.strategy.datetime.date()] = self.total_pnl
else:
self.rets[self.strategy.datetime.datetime()] = self.total_pnl
def stop(self):
for dname in self.positions:
self.unrealized += (self.strategy.dnames[dname].close[0] -
self.strategy.positionsbyname[dname].price) * \
self.strategy.positionsbyname[dname].size
@backtrader ok. After much thought, one version (for example for calc pnl for each strategy) may be like this:
def notify_order(self, order):
if order.status in [order.Completed]:
if hasattr(self, 'total_pnl'):
self.total_pnl += order.executed.pnl - order.executed.comm
else:
self.total_pnl = order.executed.pnl - order.executed.comm
Hello,
my code like this
cerebro = bt.Cerebro()
cerebro.broker.set_cash(100000.0)
cerebro.broker.set_coc(True)
cerebro.broker.addcommissioninfo(CommDivFrance())
# --- Adding some data feeds ---
# --- Observers ---
cerebro.addobserver(bt.observers.Broker)
cerebro.addobservermulti(bt.observers.BuySell)
cerebro.addobserver(bt.observers.Benchmark,
data=benchdata,
_doprenext=True,
timeframe=TimeFrame.NoTimeFrame)
# --- Analyzers ---
cerebro.addanalyzer(bt.analyzers.SharpeRatio, timeframe=TimeFrame.Years)
cerebro.addanalyzer(bt.analyzers.Calmar, timeframe=TimeFrame.Years)
cerebro.addanalyzer(bt.analyzers.DrawDown)
cerebro.addanalyzer(bt.analyzers.AnnualReturn)
cerebro.addanalyzer(bt.analyzers.PyFolio, timeframe=TimeFrame.Days, compression=1)
cerebro.addanalyzer(bt.analyzers.Transactions)
cerebro.addanalyzer(bt.analyzers.Returns)
# --- Strategies ---
cerebro.addstrategy(strategy=ValueBasket)
cerebro.addstrategy(strategy=ValueBasket2)
results = cerebro.run()
for strat in results:
print('='*79)
print(strat.__class__.__name__)
# Analyzers results
strat.analyzers.annualreturn.print()
strat.analyzers.drawdown.print()
strat.analyzers.sharperatio.print()
strat.analyzers.returns.print()
# pyfolio
pyfoliozer = strat.analyzers.getbyname('pyfolio')
returns, positions, transactions, gross_lev = pyfoliozer.get_pf_items()
# Perfomance
pf.show_perf_stats(returns=returns)
#
cerebro.plot(plotter=PlotterZoomed())
I added 2 strategies, and would like to extract performance of each ones. But for each strat I get same perfomance (annualreturn, drawdown, profit, sharpe and so on), which comes from backtasting together (printing in section # Analyzers results). There are some capital allocation logic between strategies and they are related to each other.
@ab_trader yes, it's clear, thanks.
To the question above how to use two classes CommIB and Dividends I found simple solution with inheritance like this:
class Dividends(bt.CommissionInfo):
params = (...)
def _get_credit_interest(self, data, size, price, days, dt0, dt1):
''' some logic '''
class CommIB_US(Dividends):
params = (...)
def _getcommission(self, size, price, pseudoexec):
''' some logic '''
class CommIB_France(Dividends):
params = (...)
def _getcommission(self, size, price, pseudoexec):
''' some logic '''
And then set commission scheme for each datafeed:
cerebro.broker.addcommissioninfo(CommIB_France(), name=some_france_stock_feed_name)
I'm testing portfolios on different markets (single or global, for example France, US). For each market I have class with commisions logic, for example:
import backtrader as bt
class CommIB(bt.CommInfoBase):
params = (
('commission', 0.0035),
('stocklike', True), # Stocks
('commtype', bt.CommInfoBase.COMM_FIXED), # Apply fixed Commission
('min_comm', 0.35), # USD
('max_comm', 0.005),
('on_slippage', 0.001),
)
def _getcommission(self, size, price, pseudoexec):
commis = abs(size) * self.p.commission
trade_value = abs(size) * price
max_commis = trade_value * self.p.max_comm
slipp_value = trade_value * self.p.on_slippage
if self._commtype == self.COMM_PERC:
return trade_value * self.p.commission
if commis < self.p.min_comm:
return self.p.min_comm + trade_value * self.p.on_slippage
elif commis > max_commis:
return max_commis + slipp_value
else:
return commis + slipp_value
Also there is a unified class, that apply accrues dividends on holding shares:
import backtrader as bt
import numpy as np
class Dividends(bt.CommissionInfo):
params = (
('interest_long', True),
('path_data', '..\\Data\\DIV Data')
)
def __init__(self):
super(Dividends, self).__init__()
self.div_data = div_loader(self.p.path_data)
def _get_credit_interest(self, data, size, price, days, dt0, dt1):
amount_total = 0
if (dt1 in self.div_data.index) & (data._name in self.div_data.columns):
amount = self.div_data.loc[dt1, data._name]
if ~np.isnan(amount):
amount_total = -amount * size
print('Dividend found:', data._name,
'Held shares:', size,
'Price:', price,
# 'Days:', days,
'dt0,dt1:', dt0, dt1,
'DIV Amount:', amount,
'DIV Total:', -amount_total)
return amount_total
If to test strategy only on the US market, it's obviously, that the next syntax work's well only for the Dividend accruals, (CommIB() overriden):
cerebro.broker.addcommissioninfo(CommIB())
cerebro.broker.addcommissioninfo(Dividends())
Solving of this is creating a single class with the CommIB and the Dividends logic that could works in one market (although it is inconvenient). But if to test strategy on Global market, with the different commissions for the different markets, then the task seems impossible.
It would great if the method addcommissioninfo would supports the syntax like this (doesn't overriding instance self.comminfo[name] in broker.py every time, but adding):
for stock in stock_list_US + stock_list_France:
cerebro.broker.addcommissioninfo(Dividends(), name=stock)
for stock in stock_list_US:
cerebro.broker.addcommissioninfo(CommIB(), name=stock)
for stock in stock_list_France:
cerebro.broker.addcommissioninfo(CommIB_France(), name=stock)