Recommended Python version (plotting issues on 3.8.5)

frontline

Perhaps a stupid question, but I've ran into several issues with plotting with my default setup - Win 10, Python 3.8.5 32-bit, BT 1.9.76.123, Matplotlib 3.3.3 (tried 3.3.0, 3.3.1 earlier as well with the same results). Pretty sure it's something on my end but for the life of me I can't figure out.

Issue 1 (solvable) - when plotting, bt\plot\locator.py craps out on line 39 complaining about the module 'warnings' not being found:

from matplotlib.dates import (HOURS_PER_DAY, MIN_PER_HOUR, SEC_PER_MIN,
                              MONTHS_PER_YEAR, DAYS_PER_WEEK,
                              SEC_PER_HOUR, SEC_PER_DAY,
                              num2date, rrulewrapper, YearLocator,
                              MicrosecondLocator, warnings)

...solution: remove 'warnings' from the list, without (seemingly) any ill effects.

Issue 2 (NOT able to solve so far) - if I configure Cerebro like this:

cerebro = bt.Cerebro()

...or like this (note that .Trades is the culprit here):

cerebro = bt.Cerebro(stdstats=False)
cerebro.addobserver(bt.observers.Trades)

... I get the following when doing 'cerebro.plot()' later:

...
File "...\Python38\site-packages\matplotlib\dates.py", line 1748, in tick_values
    ticks = self._wrapped_locator.tick_values(nmin, nmax)
File "...\Python38\site-packages\matplotlib\ticker.py", line 2009, in tick_values
    locs = vmin - step + np.arange(n + 3) * step
ValueError: Maximum allowed size exceeded

...which I was able to trace to the fact that the last line is getting some insanely high values as inputs for vmin and vmax (just added a debug 'print()' in place here)

print(vmax, vmin, n)
locs = vmin - step + np.arange(n + 3) * step

>>> 91152000000.0 81648000000.0 9504000000.0

So, the question is -- what am I doing wrong AND/OR is there a different, recommended Python version (and or Matplotlib version, perhaps) that the community is using to run Backtrader? The docs say testing is done up to Python 3.5, but even 3.5.10 is end of life as of Sep 2020 (there's not even a Windows installer available for download anymore). So whats the recommendation here?

Would appreciate any help.

frontline

One more note - I can somewhat successfully plot if I disable the default Observers and then re-add these back -

cerebro.addobserver(bt.observers.BuySell)
cerebro.addobserver(bt.observers.Broker)
cerebro.addobserver(bt.observers.DrawDown)

...with two weird side effects -

There are no date labels on the bottom
If I plot datas/lines for multiple symbols (multi-equity portfolio) only the first one shows any Buy/Sell indicators

...so while I kind of able to make it work, I'm still hoping there's a better solution here.

amf

It seems that the integration with Matplotlib is the culprit here. I'm using Matplotlib 3.2.2 and it works fine.
There's PR to fix this in https://github.com/mementum/backtrader/pull/418

frontline

@amf Excellent, thank you. 3.2.2 solved both issues.

As a side note (it seem like several people ran into it already), the previously suspect 'observer.Trades' is, in fact, the root case here.

In absence of trades it creates an X axis of 0 length, which leads to bt/locator.py picking MICROSECOND resolution for plotting, which leans to np.arange() exploding inside Matplotlib (because a billion of X ticks is one to many)... Haven't looked into why this still works in 3.2.2 - perhaps there are (were) additional checks against garbage input?

frontline

...and one last time on the topic of Trades observer --

The reason it doesn't add any trade events to its internal pnlplus and pnlminus lines (which produces a zero-length X axis, yadda, yadda) is, in my case, because I'm using exclusively self.order_target_percent() for position adjustment. As far as Trades observer sees it, the trade is never considered "closed" if I'm buying or selling just a portion of the position. So it's never added to the chart/line. Not sure I entirely agree with that logic, but at least that de-mystifies the use case.

vladisld

@frontline said in Recommended Python version (plotting issues on 3.8.5):

In absence of trades it creates an X axis of 0 length, which leads to bt/locator.py picking MICROSECOND resolution for plotting, which leans to np.arange() exploding inside Matplotlib (because a billion of X ticks is one to many)... Haven't looked into why this still works in 3.2.2 - perhaps there are (were) additional checks against garbage input?

Good to know. Thanks a lot.

rajanprabu

@frontline Thats a good point. Accidentally I stumbled upon this yesterday while I was writing code for something else. If my understanding is right, BT store sends signal [using oref] to broker only if trade is complete, meaning if all intended position is bought or sold.

how would you @frontline envision this is to be ? One idea would be to split the orderid with suffixes ? If order ID is 100 and 10 quantities buy order then sub order id can be 100.1, 100.2 .... to 100.10 and if 5 qty is complete then send cancel 10.6 to 10.10 and keep 10.1 to 10.5 ? Just an small idea if @vladisld wants to take it forward. I will also look at it.

ab_trader

@frontline said in Recommended Python version (plotting issues on 3.8.5):

As far as Trades observer sees it, the trade is never considered "closed" if I'm buying or selling just a portion of the position. So it's never added to the chart/line. Not sure I entirely agree with that logic, but at least that de-mystifies the use case.

You can try this approach to open multiple trades in the same ticker -
https://www.backtrader.com/blog/posts/2015-10-05-multitrades/multitrades/

Otherwise

Definition of a trade:

A Trade is open when the a position in a instrument goes from 0 to a size X which may positive/negative for long/short positions)

A Trade is closed when a position goes from X to 0.

frontline

@ab_trader Thanks, soundslike a good idea, but I'm possibly missing some mechanics to make it work. Strategy init:

class TestStrategy(bt.Strategy):
    def __init__(self):    
        self.trade_id = datetime.now().microsecond   # because why not? unique enough

...then in timer-triggered portfolio re-balancing:

 def notify_timer(self, timer, when, *args, **kwargs):
        for data in self.datas:
            self.trade_id += 1
            print(f'Trade id: {self.trade_id}')  # looks like it prints unique values for all trades
            kwargs = {'tradeid': self.trade_id}  # inject it into order_target_percent, which sends it down to order_target_value, then (eventually) to the buy/sell methods 
            self.order_target_percent(data, target=self.perctarget, **kwargs)

...and still the Trades observer subplot is empty. Feeling I'm close, but something is missing...