By "the data processing loop" do you mean the next method? The code above is inside next.

Right now I am keeping all the orders on hold until all the data are processed, then I send them to the broker like this:

    self.long_signals = []
    self.short_signals = []
    
    # Logic
    
    for i, s in enumerate(self.datas):
                    
        if not self.positions[s]:
            if s.membership[0]:
                self.long_signals.append(s)
                
            continue
            
        if self.positions[s]:
            
            if not s.membership[0]:
                self.short_signals.append(s)
                
            continue
                     
        
    # Send orders to the broker. 
    
    for s in self.short_signals:
        self.orders[s] = self.sell(s)
            
    for s in self.long_signals:
        self.orders[s] = self.buy(s)

What is the best pattern to do it? To hold all the signals in the strategy until all the data feeds are processed and then send all the sell order first?

Or we should subclass the brocker and modify its logic to process the sell orders first?

Hi,

suppose there is no cash in the account but we have 10 long positions. The next open we have to sell 5 securities at market and to buy 5 securities at market with the money we should get from the selling.

Is it possible to force the broker to execute all the sell orders (market) before the buy orders to free up cash and avoid to end with 5 rejected orders because the account has not enough cash?

I wouldn't mind being unable to execute one of the buy orders because after all the selling there is not enough cash. In reality I will do all the selling before the buying so I can assume I will able to get a fill on the majority of buy orders. The selling and the buying must be in the same bar at open, I will increase the slippage because of the delay in filling the longs.

Thanks

I am working on a simple system that uses weekly data and must have a fixed size portfolio of securities: if it sells one security, I expect it to buy another one the next week.

To size the new position I use the available cash and I make an estimation using the last available close of the cash that will be available once the position I have to close is sold.

2012-02-03, I have a SELL signal.

2012-02-10 I start the week with 1074.87 in the account. During market hours, I expected the position to be closed and get new cash for 15410.02. According to the broker the position is sold for 15407.58. I expect to have 1074.87 + 15407.58 in my account.

Once the 2012-02-10 bar is fully CLOSED, I can compute new signals and, since I sold a security for 15407.58, I have to buy the new one using this amount + the cash I had in the account.

The size is computed using the last available close and a 5% buffer, like this: (cash * 0.95) / close[0]

After the signal, when the sizer computes the size, the cash in the broker is still 1074.87!
No sign of the 15407.58 I got from the security I sold and the new trade is sized using only 1074.87 * 95%!

Here it is the full log:

2012-02-03 SELL: 63DU GY Equity
2012-02-03 SIZING 63DU GY Equity: 326 [CLOSE POSITION]

2012-02-10 ORDER SUBMITTED: SELL 326 63DU GY Equity @ Market [last: 47.2700 expected: 15410.02]
2012-02-10 ORDER ACCEPTED: SELL 326 63DU GY Equity @ Market [last: 47.2700 expected: 15410.02]
2012-02-10 ORDER EXECUTED: SOLD 326 63DU GY Equity @ 47.2625 value: 15407.58 comm: 19.00
2012-02-10 TRADE CLOSED: 63DU GY Equity gross pnl: -2120.63 comm: 38.00 net pnl: -2158.63

2012-02-10 BUY: DB1 GY Equity
2012-02-10 SIZING DB1 GY Equity: 21 [last: 47.7270 cash: 1074.87 expected: 0.00 available: 1021.13]
2012-02-17 ORDER SUBMITTED: BUY 21 DB1 GY Equity @ Market [last: 47.7270 expected: 1002.27]
2012-02-17 ORDER ACCEPTED: BUY 21 DB1 GY Equity @ Market [last: 47.7270 expected: 1002.27]
2012-02-17 ORDER EXECUTED: BOUGHT 21 DB1 GY Equity @ 47.9264 value: 1006.46 comm: 19.00
2012-02-17 TRADE OPENED: 21 DB1 GY Equity @ 47.9264 - value: 1006.46 comm: 19.00

Am I missing something?

You're right! I was adding all my weekly pandas data without specifying their timeframe because I wrongly assumed backtrader was able to identify it from the data:

for i, value in enumerate(data.items()):
    s, df = value
    
    if isinstance(df, DataFrame):
        cerebro.adddata(PandasData(dataname=df, name = df.name, plot=False))

# ......

As you said I missed this from the docs: timeframe (default: TimeFrame.Days)

Hello,

I am adding 69 Data Feeds to cerebro, each one with its own start and end date. In this example, mostly of them but not all of them, have 2010-01-01 as start date:

1 63DU GY Equity DEUTSCHE BOERSE AG-NEW 2011-05-20 2012-02-10
2 ABI BB Equity ANHEUSER-BUSCH INBEV SA/NV 2010-01-01 2019-12-27
3 ACA FP Equity CREDIT AGRICOLE SA 2010-01-01 2019-12-27
4 AD NA Equity KONINKLIJKE AHOLD DELHAIZE N 2010-01-01 2019-12-27
5 ADS GY Equity ADIDAS AG 2010-01-01 2019-12-27
6 AGN NA Equity AEGON NV 2010-01-01 2019-12-27
7 AI FP Equity AIR LIQUIDE SA 2010-01-01 2019-12-27
8 AIR FP Equity AIRBUS SE 2010-01-01 2019-12-27
9 ALO FP Equity ALSTOM 2010-01-01 2019-12-27
10 ALV GY Equity ALLIANZ SE-REG 2010-01-01 2019-12-27
11 AMS SQ Equity AMADEUS IT GROUP SA 2010-04-30 2019-12-27
12 ASML NA Equity ASML HOLDING NV 2010-01-01 2019-12-27
13 BAS GY Equity BASF SE 2010-01-01 2019-12-27
14 BAYN GY Equity BAYER AG-REG 2010-01-01 2019-12-27
15 BBVA SQ Equity BANCO BILBAO VIZCAYA ARGENTA 2010-01-01 2019-12-27
16 BMW GY Equity BAYERISCHE MOTOREN WERKE AG 2010-01-01 2019-12-27
17 BN FP Equity DANONE 2010-01-01 2019-12-27
18 BNP FP Equity BNP PARIBAS 2010-01-01 2019-12-27
19 CA FP Equity CARREFOUR SA 2010-01-01 2019-12-27
20 CRH ID Equity CRH PLC 2010-01-01 2019-12-27
21 CS FP Equity AXA SA 2010-01-01 2019-12-27
22 DAI GY Equity DAIMLER AG-REGISTERED SHARES 2010-01-01 2019-12-27
23 DB1 GY Equity DEUTSCHE BOERSE AG 2010-01-01 2019-12-27
24 DBK GY Equity DEUTSCHE BANK AG-REGISTERED 2010-01-01 2019-12-27
25 DG FP Equity VINCI SA 2010-01-01 2019-12-27
26 DPW GY Equity DEUTSCHE POST AG-REG 2010-01-01 2019-12-27
27 DTE GY Equity DEUTSCHE TELEKOM AG-REG 2010-01-01 2019-12-27
28 EL FP Equity ESSILORLUXOTTICA 2010-01-01 2019-12-27
29 ENEL IM Equity ENEL SPA 2010-01-01 2019-12-27
30 ENGI FP Equity ENGIE 2010-01-01 2019-12-27
31 ENI IM Equity ENI SPA 2010-01-01 2019-12-27
32 EOAN GY Equity E.ON SE 2010-01-01 2019-12-27
33 FP FP Equity TOTAL SA 2010-01-01 2019-12-27
34 FRE GY Equity FRESENIUS SE & CO KGAA 2010-01-01 2019-12-27
35 G IM Equity ASSICURAZIONI GENERALI 2010-01-01 2019-12-27
36 GLE FP Equity SOCIETE GENERALE SA 2010-01-01 2019-12-27
37 IBE SQ Equity IBERDROLA SA 2010-01-01 2019-12-27
38 INGA NA Equity ING GROEP NV 2010-01-01 2019-12-27
39 ISP IM Equity INTESA SANPAOLO 2010-01-01 2019-12-27
40 ITX SQ Equity INDUSTRIA DE DISENO TEXTIL 2010-01-01 2019-12-27
41 KER FP Equity KERING 2010-01-01 2019-12-27
42 LIN GY Equity LINDE PLC 2018-11-02 2019-12-27
43 LINU GY Equity LINDE AG - TENDER 2017-08-25 2018-10-26
44 MC FP Equity LVMH MOET HENNESSY LOUIS VUI 2010-01-01 2019-12-27
45 MT NA Equity ARCELORMITTAL 2010-01-01 2019-12-27
46 MUV2 GY Equity MUENCHENER RUECKVER AG-REG 2010-01-01 2019-12-27
47 NOKIA FH Equity NOKIA OYJ 2010-01-01 2019-12-27
48 OR FP Equity L'OREAL 2010-01-01 2019-12-27
49 ORA FP Equity ORANGE 2010-01-01 2019-12-27
50 PHIA NA Equity KONINKLIJKE PHILIPS NV 2010-01-01 2019-12-27
51 REP SQ Equity REPSOL SA 2010-01-01 2019-12-27
52 RWE GY Equity RWE AG 2010-01-01 2019-12-27
53 SAF FP Equity SAFRAN SA 2010-01-01 2019-12-27
54 SAN FP Equity SANOFI 2010-01-01 2019-12-27
55 SAN SQ Equity BANCO SANTANDER SA 2010-01-01 2019-12-27
56 SAP GY Equity SAP SE 2010-01-01 2019-12-27
57 SGO FP Equity COMPAGNIE DE SAINT GOBAIN 2010-01-01 2019-12-27
58 SIE GY Equity SIEMENS AG-REG 2010-01-01 2019-12-27
59 SU FP Equity SCHNEIDER ELECTRIC SE 2010-01-01 2019-12-27
60 TEF SQ Equity TELEFONICA SA 2010-01-01 2019-12-27
61 TIT IM Equity TELECOM ITALIA SPA 2010-01-01 2019-12-27
62 UCG IM Equity UNICREDIT SPA 2010-01-01 2019-12-27
63 UL FP Equity UNIBAIL-RODAMCO SE 2010-01-01 2013-03-01
64 UL NA Equity UNIBAIL-RODAMCO SE 2010-01-01 2018-06-08
65 UNA NA Equity UNILEVER NV 2010-01-01 2019-12-27
66 URW NA Equity UNIBAIL-RODAMCO-WESTFIELD 2010-01-01 2019-12-27
67 VIV FP Equity VIVENDI 2010-01-01 2019-12-27
68 VOW GY Equity VOLKSWAGEN AG 2010-01-01 2019-07-12
69 VOW3 GY Equity VOLKSWAGEN AG-PREF 2010-01-01 2019-12-27

next and prenext

If I should only use the next method the strategy will start to generate signals from the most recent common start date: 2018-11-02

I understand this is by design, so that code never breaks inside next.

To bypass this wanted behavior and generate signals from the first available start date of each data feed I am using the prenext method as suggested on several topics.

This is the code of a simple Strategy that does only some logging:

class Demo(bt.Strategy):
    """Demo Strategy"""
    
    def __init__(self):
        print('Init ', self.__class__.__name__)
        print()
        
    def prenext(self):
        self.next()
    
    def next(self):
        
        print()
        print('---------------------------')
        print('Date: {}'.format(self.datetime.date()))
        print('---------------------------')
        
        for i, s in enumerate(self.datas):
            print('{} {} {}'.format(i + 1, s.datetime.date(), s._name))

When I loop self.datas inside next I expected to have their datetime.date() in sync with self.datetime.date(), the “heartbeat”. I found this is not the case, here’s the log of the first next run:

Init Demo

Date: 2010-01-01

1 2012-02-10 63DU GY Equity
2 2010-01-01 ABI BB Equity
3 2010-01-01 ACA FP Equity
4 2010-01-01 AD NA Equity
5 2010-01-01 ADS GY Equity
6 2010-01-01 AGN NA Equity
7 2010-01-01 AI FP Equity
8 2010-01-01 AIR FP Equity
9 2010-01-01 ALO FP Equity
10 2010-01-01 ALV GY Equity
11 2019-12-27 AMS SQ Equity
12 2010-01-01 ASML NA Equity
13 2010-01-01 BAS GY Equity
14 2010-01-01 BAYN GY Equity
15 2010-01-01 BBVA SQ Equity
16 2010-01-01 BMW GY Equity
17 2010-01-01 BN FP Equity
18 2010-01-01 BNP FP Equity
19 2010-01-01 CA FP Equity
20 2010-01-01 CRH ID Equity
21 2010-01-01 CS FP Equity
22 2010-01-01 DAI GY Equity
23 2010-01-01 DB1 GY Equity
24 2010-01-01 DBK GY Equity
25 2010-01-01 DG FP Equity
26 2010-01-01 DPW GY Equity
27 2010-01-01 DTE GY Equity
28 2010-01-01 EL FP Equity
29 2010-01-01 ENEL IM Equity
30 2010-01-01 ENGI FP Equity
31 2010-01-01 ENI IM Equity
32 2010-01-01 EOAN GY Equity
33 2010-01-01 FP FP Equity
34 2010-01-01 FRE GY Equity
35 2010-01-01 G IM Equity
36 2010-01-01 GLE FP Equity
37 2010-01-01 IBE SQ Equity
38 2010-01-01 INGA NA Equity
39 2010-01-01 ISP IM Equity
40 2010-01-01 ITX SQ Equity
41 2010-01-01 KER FP Equity
42 2019-12-27 LIN GY Equity
43 2018-10-26 LINU GY Equity
44 2010-01-01 MC FP Equity
45 2010-01-01 MT NA Equity
46 2010-01-01 MUV2 GY Equity
47 2010-01-01 NOKIA FH Equity
48 2010-01-01 OR FP Equity
49 2010-01-01 ORA FP Equity
50 2010-01-01 PHIA NA Equity
51 2010-01-01 REP SQ Equity
52 2010-01-01 RWE GY Equity
53 2010-01-01 SAF FP Equity
54 2010-01-01 SAN FP Equity
55 2010-01-01 SAN SQ Equity
56 2010-01-01 SAP GY Equity
57 2010-01-01 SGO FP Equity
58 2010-01-01 SIE GY Equity
59 2010-01-01 SU FP Equity
60 2010-01-01 TEF SQ Equity
61 2010-01-01 TIT IM Equity
62 2010-01-01 UCG IM Equity
63 2010-01-01 UL FP Equity
64 2010-01-01 UL NA Equity
65 2010-01-01 UNA NA Equity
66 2010-01-01 URW NA Equity
67 2010-01-01 VIV FP Equity
68 2010-01-01 VOW GY Equity
69 2010-01-01 VOW3 GY Equity

If there was a simply buy signals on date 2010-01-01 (self.datetime.date()), the strategy would also buy 63DU GY Equity (2012-02-10 == start date), AMS SQ Equity (2019-12-27 == last date), LIN GY Equity ( 2019-12-27 == last date) and LINU GY Equity (2018-10-26 == last date).

Am I wrong (very likely) coding like this to generate signals on multiple data feeds with different start date and end date?

If I am not wrong, and this is by design, does it mean that before the logic to generate signals in the next method I should always check that s.datetime.date() == self.datetime.date() ??

Thanks

Nicola

After debugging I found the positions in self.broker.positions are not only the ones with positive size so I'm back to looping. To get the nuber of open positions inside a Sizer I am using a list comprehension:

open_positions = len([position for position in self.broker.positions if self.broker.getposition(position).size > 0])

anyone can suggest a better solution?

Hello,

I am running some backtest with a very simple system using only weekly data (pandas) and I added a few analyzers:

cerebro.addanalyzer(bt.analyzers.Returns, _name='returns')
cerebro.addanalyzer(bt.analyzers.DrawDown, _name='drawdown')
cerebro.addanalyzer(bt.analyzers.SharpeRatio, _name='sharpe', riskfreerate=0.0, annualize=True)

Portfolio results for return didn't look right to me:

Total Net Profit 1582338.10
CAGR 58.0894%
Max Drawdown 22.6135%
Max Drawdown Len 91
Sharpe Ratio 0.8554

I modified the code to add the analyzer adding the timeframe parameter:

cerebro.addanalyzer(bt.analyzers.Returns, _name='returns', timeframe=bt.TimeFrame.Weeks)
cerebro.addanalyzer(bt.analyzers.DrawDown, _name='drawdown')
cerebro.addanalyzer(bt.analyzers.SharpeRatio, _name='sharpe', riskfreerate=0.0, annualize=True, timeframe=bt.TimeFrame.Weeks)

Stats now look right:

Total Net Profit 1582338.10
CAGR 9.9116%
Max Drawdown 22.6135%
Max Drawdown Len 91
Sharpe Ratio 0.6601

In the Analyzers/Reference/Returns documentation I read:

timeframe (default: None)

If None the timeframe of the 1st data in the system will be used

Pass TimeFrame.NoTimeFrame to consider the entire dataset with no time constraints

It's not a big deal to add a parameter but maybe I am missing something here: my data (pandas) are all with weekly compression, shouldn't backtrader infer the weekly timeframe from the 1st data? Am I misunderstanding the docs?

Thanks

Nicola

found it:

len(self.broker.positions)

I needed it in Sizer