I'm utterly confused by this thread, sorry. If you have your datasets stashed away somewhere (loaded via broker API then saved as CSV, Pickle etc), why not do this --
for symbol in symbols:
df = broker_api.cached_price_history_1day(symbol)
data = bt.feeds.PandasData(dataname=df, plot=False, **dkwargs)
cerebro.adddata(data, name=symbol)
(...where my broker_api.cached_price_history_1day() is smart enouht to load disk-cached data if no update is needed) ?
...and an alternative implementation showing % contribution of each asset to overall portfolio value --
class PortfolioObserverPercent0(bt.observer.Observer):
_stclock = True
lines = ()
CASH_LINE = '$CASH'
REBALANCE_LINE = '_REBALANCE'
plotinfo = dict(plot=True, subplot=True, plotlinelabels=True, plotlinevalues=False, plothlines=[100.0])
plotlines = dict(_REBALANCE=dict(marker='|', markersize=8.0, color='gray'))
def next(self):
# total portfolio value
tot_value = self._owner.broker.get_value()
# add cash
cumsum = self._owner.broker.get_cash()
self.set_line(self.CASH_LINE, 0, cumsum / tot_value * 100)
# add symbol positions
for data in self.datas:
dname = data._name
cumsum += self._owner.broker.get_value([data])
self.set_line(dname, 0, cumsum / tot_value * 100)
# is this a rebalancing day?
if self._owner._orderspending:
self.set_line(self.REBALANCE_LINE, 0, 0)
def set_line(self, dname, ind, value):
"""Set named line value at index"""
getattr(self.lines, dname)[ind] = value
def make_portfolio_observer_percent(symbols: tuple):
"""Make portfolio observer with customized list of symbols"""
return type('PortfolioObserverPercent', (PortfolioObserverPercent0,), {'lines': (symbols + (PortfolioObserver0.CASH_LINE, PortfolioObserver0.REBALANCE_LINE))})
For those of us who have a habit of mixing trading with investing :), I thought that a portfolio tracking observer/plot might be of value. Sharing it to solicit feedback and further ideas. A (naive) portfolio below, rebalancing every month to have SPY, QQQ and AGG represent 33% each of the overall portfolio (actually 100% - 5% buffer then 1/3 of that each). The idea is to visualize a portion of each investment in the overall portfolio, so the $CASH line is cash, then SPY line is $CASH+SPY, then QQQ is $CASH+SPY+QQQ etc. It would look better with filled plots (since these are cumulative contributions of each investment to the overall portfolio value), but alas, no controls to fill the plot... Vertical ticks on the bottom represent rebalancing events (trades) --
Implementation --
class PortfolioObserver0(bt.observer.Observer):
_stclock = True
lines = ()
CASH_LINE = '$CASH'
REBALANCE_LINE = '_REBALANCE'
plotinfo = dict(plot=True, subplot=True, plotlinelabels=True, plotlinevalues=False)
plotlines = dict(_REBALANCE=dict(marker='|', markersize=8.0, color='gray'))
def next(self):
# add cash
cumsum = self._owner.broker.get_cash()
self.set_line(self.CASH_LINE, 0, cumsum)
# add symbol positions
for data in self.datas:
dname = data._name
cumsum += self._owner.broker.get_value([data])
self.set_line(dname, 0, cumsum)
# is this a rebalancing day?
if self._owner._orderspending:
self.set_line(self.REBALANCE_LINE, 0, 0)
def set_line(self, dname, ind, value):
"""Set named line value at index"""
getattr(self.lines, dname)[ind] = value
...then --
def make_portfolio_observer(symbols: tuple):
"""Make portfolio observer with customized list of symbols"""
return type('PortfolioObserver', (PortfolioObserver0,), {'lines': (symbols + (PortfolioObserver0.CASH_LINE, PortfolioObserver0.REBALANCE_LINE))})
...then in __main__ --
symbols = ('SPY', 'QQQ', 'AGG',)
cerebro.addobserver(make_portfolio_observer(symbols))
Thoughts? Reinventing the wheel here?
@ab_trader Thanks, soundslike a good idea, but I'm possibly missing some mechanics to make it work. Strategy init:
class TestStrategy(bt.Strategy):
def __init__(self):
self.trade_id = datetime.now().microsecond # because why not? unique enough
...then in timer-triggered portfolio re-balancing:
def notify_timer(self, timer, when, *args, **kwargs):
for data in self.datas:
self.trade_id += 1
print(f'Trade id: {self.trade_id}') # looks like it prints unique values for all trades
kwargs = {'tradeid': self.trade_id} # inject it into order_target_percent, which sends it down to order_target_value, then (eventually) to the buy/sell methods
self.order_target_percent(data, target=self.perctarget, **kwargs)
...and still the Trades observer subplot is empty. Feeling I'm close, but something is missing...
...and one last time on the topic of Trades observer --
The reason it doesn't add any trade events to its internal pnlplus and pnlminus lines (which produces a zero-length X axis, yadda, yadda) is, in my case, because I'm using exclusively self.order_target_percent() for position adjustment. As far as Trades observer sees it, the trade is never considered "closed" if I'm buying or selling just a portion of the position. So it's never added to the chart/line. Not sure I entirely agree with that logic, but at least that de-mystifies the use case.
@amf Excellent, thank you. 3.2.2 solved both issues.
As a side note (it seem like several people ran into it already), the previously suspect 'observer.Trades' is, in fact, the root case here.
In absence of trades it creates an X axis of 0 length, which leads to bt/locator.py picking MICROSECOND resolution for plotting, which leans to np.arange() exploding inside Matplotlib (because a billion of X ticks is one to many)... Haven't looked into why this still works in 3.2.2 - perhaps there are (were) additional checks against garbage input?
Experiencing the same issue. For some Python/Matplotlib versions the issue seems to be triggered by bt.observers.Trades - see https://community.backtrader.com/topic/3310/recommended-python-version-plotting-issues-on-3-8-5/2
This is NOT a memory issue.
One more note - I can somewhat successfully plot if I disable the default Observers and then re-add these back -
cerebro.addobserver(bt.observers.BuySell)
cerebro.addobserver(bt.observers.Broker)
cerebro.addobserver(bt.observers.DrawDown)
...with two weird side effects -
...so while I kind of able to make it work, I'm still hoping there's a better solution here.
Perhaps a stupid question, but I've ran into several issues with plotting with my default setup - Win 10, Python 3.8.5 32-bit, BT 1.9.76.123, Matplotlib 3.3.3 (tried 3.3.0, 3.3.1 earlier as well with the same results). Pretty sure it's something on my end but for the life of me I can't figure out.
Issue 1 (solvable) - when plotting, bt\plot\locator.py craps out on line 39 complaining about the module 'warnings' not being found:
from matplotlib.dates import (HOURS_PER_DAY, MIN_PER_HOUR, SEC_PER_MIN,
MONTHS_PER_YEAR, DAYS_PER_WEEK,
SEC_PER_HOUR, SEC_PER_DAY,
num2date, rrulewrapper, YearLocator,
MicrosecondLocator, warnings)
...solution: remove 'warnings' from the list, without (seemingly) any ill effects.
Issue 2 (NOT able to solve so far) - if I configure Cerebro like this:
cerebro = bt.Cerebro()
...or like this (note that .Trades is the culprit here):
cerebro = bt.Cerebro(stdstats=False)
cerebro.addobserver(bt.observers.Trades)
... I get the following when doing 'cerebro.plot()' later:
...
File "...\Python38\site-packages\matplotlib\dates.py", line 1748, in tick_values
ticks = self._wrapped_locator.tick_values(nmin, nmax)
File "...\Python38\site-packages\matplotlib\ticker.py", line 2009, in tick_values
locs = vmin - step + np.arange(n + 3) * step
ValueError: Maximum allowed size exceeded
...which I was able to trace to the fact that the last line is getting some insanely high values as inputs for vmin and vmax (just added a debug 'print()' in place here)
print(vmax, vmin, n)
locs = vmin - step + np.arange(n + 3) * step
>>> 91152000000.0 81648000000.0 9504000000.0
So, the question is -- what am I doing wrong AND/OR is there a different, recommended Python version (and or Matplotlib version, perhaps) that the community is using to run Backtrader? The docs say testing is done up to Python 3.5, but even 3.5.10 is end of life as of Sep 2020 (there's not even a Windows installer available for download anymore). So whats the recommendation here?
Would appreciate any help.