I have solved the problem thanks to @backtrader and this previous thread. The problem was in my inheritance of PandasData class. I have been using bt.feed.DataBase and bt.feed.PandasData objects for inheriting to my custom class but both were wrong it was bt.feeds.PandasData. This is my working code.

from datetime import datetime
import backtrader as bt
import pandas as pd
from pandas_datareader import data as pdr



data = pdr.get_data_yahoo('AAPL', start=datetime(2017, 8, 13), end=datetime(2018, 8, 14))

data.columns=['high', 'low', 'open', 'close', 'volume', 'adj_close']


data['pct']=data.close.pct_change(1)
data['pct2']=data.close.pct_change(5)
data['pct3']=data.close.pct_change(10)

class TestStrategy(bt.Strategy):
    def next(self):
        #print("hello")
        print(','.join(str(x) for x in [self.datetime.datetime(0), self.data.pct2[0]]))

cerebro = bt.Cerebro()
cerebro.addstrategy(TestStrategy)

class PandasData(bt.feeds.PandasData):
    lines = ('adj_close','pct','pct2','pct3')
    params = (
        ('datetime', None),
        ('open','open'),
        ('high','high'),
        ('low','low'),
        ('close','close'),
        ('volume','volume'),
        ('openinterest',None),
        ('adj_close','adj_close'),
        ('pct','pct'),
        ('pct2','pct2'),
        ('pct3','pct3'),
    )


#df = bt.feeds.PandasData(dataname=data)
df=PandasData(dataname=data)
cerebro.adddata(df)
cerebro.run()

And thank you backtrader for the help.

I am new to backtrader and I am trying to backtest a simple strategy using my custom pandas dataframe. This code fetches stock data and modifies the dataframe data by adding 3 additional columns.

from datetime import datetime
import backtrader as bt
import pandas as pd
from pandas_datareader import data as pdr
import matplotlib.pyplot as plt


data = pdr.get_data_yahoo('AAPL', start=datetime(2017, 8, 13), end=datetime(2018, 8, 14))

data.columns=['high', 'low', 'open', 'close', 'volume', 'adj close']
data['pct']=data.close.pct_change(1)
data['pct2']=data.close.pct_change(2)
data['pct3']=data.close.pct_change(3)


ax = plt.gca()
data.plot(kind='line',y='close',use_index=True,color='blue', ax=ax)
data.plot(kind='line',y='pct',use_index=True, color='red', ax=ax)
plt.show()

class PandasData(bt.feed.DataBase):
    params = (
        ('datetime', None),
        ('open', -1),
        ('high', -1),
        ('low', -1),
        ('close', -1),
        ('volume', -1),
        ('openinterest', None),
        ('adj close', -1),
        ('pct', -1),
        ('pct2', -1),
        ('pct3', -1),
    )



class MyStrategy(bt.Strategy):
    def __init__(self):
        self.sma= bt.ind.SMA(period=10)
    def next(self):
        if self.sma>self.data.close and self.data.pct>0 or self.data.pct2>0:
            print('BUY CREATE, %.2f' % self.data.close[0])
        elif self.sma<self.data.close and self.data.pct<0 or self.data.pct2<0:
            print('SELL CREATE, %.2f' % self.data.close[0])
            self.order = self.sell()

cerebro = bt.Cerebro()
cerebro.addstrategy(MyStrategy)

#data = bt.feeds.YahooFinanceData(dataname='MSFT', fromdate=datetime(2011, 1, 1),todate=datetime(2012, 12, 31))
cerebro.adddata(PandasData(data))
cerebro.broker.setcash(100000.0)

cerebro.addsizer(bt.sizers.FixedSize, stake=1)

cerebro.broker.setcommission(commission=0.05)

cerebro.addanalyzer(bt.analyzers.SharpeRatio, _name='mysharpe')
cerebro.addanalyzer(bt.analyzers.DrawDown, _name='DrawDown')

print('Starting Balance: %.2f' % cerebro.broker.getvalue())

strats =  cerebro.run(stdstats=False)
strat = strats[0]
cerebro.addobserver(bt.observers.Value)


print('Sharpe Ratio:', strat.analyzers.mysharpe.get_analysis())
print('DrawDown:', strat.analyzers.DrawDown.get_analysis())
print('Final Balance: %.2f' % cerebro.broker.getvalue())

But when I execute strats = cerebro.run(stdstats=False) it shows this error

IndexError: array assignment index out of range

How do I feed a custom pandas dataframe in backtrader?

Thank you. it cleared most of my doubts.

@ab_trader said in How to use filters properly?:
I think stop loss is not working properly

# this is my data

                  open       close
Date                              
2019-05-10  197.419998  197.179993
2019-05-13  187.710007  185.720001
2019-05-14  186.410004  188.660004
2019-05-15  186.270004  190.919998
2019-05-16  189.910004  190.080002
2019-05-17  186.929993  189.000000
2019-05-20  183.520004  183.089996
2019-05-21  185.220001  186.600006
2019-05-22  184.660004  182.779999
2019-05-23  179.800003  179.660004
2019-05-24  180.199997  178.970001
2019-05-28  178.919998  178.229996

# this is the log output
2019-05-20, SELL EXECUTED, 189.00
2019-05-20, BUY EXECUTED, 183.52
2019-05-20, SELL EXECUTED, 182.92
2019-05-21, BUY EXECUTED, 185.22
2019-05-22, Order Canceled/Margin/Rejected
2019-05-22, SELL EXECUTED, 186.60
2019-05-22, BUY EXECUTED, 184.66
2019-05-22, SELL EXECUTED, 184.06
2019-05-23, BUY EXECUTED, 179.80
2019-05-23, SELL EXECUTED, 179.20
2019-05-24, BUY EXECUTED, 180.20
2019-05-24, SELL EXECUTED, 179.60
2019-05-28, BUY EXECUTED, 178.92
2019-05-28, SELL EXECUTED, 178.32

If we look at the trading @ 2019-05-28 it has sold the stock at a higher price 178.32 than our real price 178.22.

Can you tell the rationale behing the counter and why 4? Whats this do self.datas[0].open ?

@backtrader14 said in How to use filters properly?:
lets take this example

                 open      close
2019-05-24  180.199997  178.970001

This is what I think how it works

def next(self):
    self.buy(price=self.data.open[0])
    self.sell(price=self.data.close[0])

I thought this as a loop kind of execution. when next() is called initially; In here self.buy(price=self.data.open[0]) it buys at 180.19 then in the next line self.sell(price=self.data.close[0]) (changed to sell) its sells the stock @178.97. Both buying and selling are happening in the same day ie 2019-05-24 . Then bar moves to the next position ie 2019-05-25 and again next is initialized and repeats the process. Is this logic correct?

@backtrader14 said in How to use filters properly?:
So in DaySplitter_Close for a single day it generates two bars and it alternates between OHLX and CCCC right?

Can you tell me why this doesn't work?

def next(self):
    self.buy(price=self.data.open[0])
    self.close(price=self.data.close[0])

This is acutally about two questions one is major one about filters and other is minor about buy and sell. I will start with the minor one

I am backtesting a simple strat to buy at open and sell at the close on the same day. I already got an answer here but I am curious why this didn't work in backtrader .

    def next(self):
        self.buy(price=self.data.open[0])
        self.close(price=self.data.close[0]) 
      # self.sell gives same result

some basic info

# Data
                  open       close
Date                              
2019-05-20  183.520004  183.089996
2019-05-21  185.220001  186.600006
2019-05-22  184.660004  182.779999
2019-05-23  179.800003  179.660004
2019-05-24  180.199997  178.970001

# Log details
2019-05-21, SELL EXECUTED, Price: 185.22, Cost: 185.49, Comm 0.00
2019-05-22, BUY EXECUTED, Price: 184.66, Cost: 184.66, Comm 0.00
2019-05-22, SELL EXECUTED, Price: 184.66, Cost: 185.07, Comm 0.00
2019-05-23, BUY EXECUTED, Price: 179.80, Cost: 179.80, Comm 0.00
2019-05-23, SELL EXECUTED, Price: 179.80, Cost: 182.44, Comm 0.00
2019-05-24, BUY EXECUTED, Price: 180.20, Cost: 180.20, Comm 0.00
2019-05-24, SELL EXECUTED, Price: 180.20, Cost: 181.32, Comm 0.00

Now the major question is how to properly use filters. I am using DaySplitter_Close filter to understand it.

class TestStrategy(bt.Strategy):
    def next(self):
        print("Ticks: "+str(len(self))+" Price: "+str(self.data.close[0]))

cerebro = bt.Cerebro()
cerebro.addstrategy(TestStrategy)
df=bt.feeds.PandasData(dataname=data)
df.addfilter(bt.filters.DaySplitter_Close)
cerebro.adddata(df)
cerebro.run()

I understood that it basically splits the data into two streams one with OHLX were X is the average of OHL and a second stream with only close values. When I run my strategy I get this result.

Ticks: 1901 Price: 184.30667114257812
Ticks: 1902 Price: 182.77999877929688
Ticks: 1903 Price: 179.38333129882812
Ticks: 1904 Price: 179.66000366210938
Ticks: 1905 Price: 180.31999715169272
Ticks: 1906 Price: 178.97000122070312

I understood that both streams are streaming simultaneously. How can I access the streams seperately. Like this

print("stream_1",str(self.data1.close[0]))
print("stream_2",str(self.data2.close[0]))

Thank you for the ideas. The DaySplitter_Close is little complex for me I will first try out with the first suggestion and build a custom indicator with stop loss. Lastly I really appreciate for replying to my posts :).

Thank you this makes complete sense. What is current: 0 and current: 2 means?

I am backtesting a single day trading strategy in backtrader in which I buy at current open and close at current close and no trade carry forward. This is my TradeAnalyzer's report.

# data.shape >>> (197, 6)

TradeAnalyzer:
  -----------------------------------------------------------------------------
  - total:
    - total: 196
    - open: 1
    - closed: 195
  -----------------------------------------------------------------------------
  - streak:
    +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    - won:
      - current: 0
      - longest: 5
    +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    - lost:
      - current: 2
      - longest: 5
  -----------------------------------------------------------------------------

If I am guessing it right then the total section says that I took 196 positions in which 195 are completed and 1 position is still open. The confusion is in the win/lose stat that is I have won 0 and lost 2 positions, so according to this stat I only did a total of 2 trades which is contradictory to the 196 which is given in the total. I also expected the sum of my win and lose trade should be near to 196. It also shows that my longest trading length is 5 days but my strategy involves only a single day. How to make sense of TradeAnalyzer's result?

This is my full code.

from datetime import datetime
import backtrader as bt
import pandas as pd
from pandas_datareader import data as pdr

data = pdr.get_data_yahoo('AAPL', start=datetime(2018, 8, 13), end=datetime.now())

data.columns=['high', 'low', 'open', 'close', 'volume', 'adj_close']
del data.adj_close

class TestStrategy(bt.Strategy):
    
    def __init__(self):
        pass
    
    def next_open(self):
        self.buy(coc=False)
        
    def next(self):
        pos = self.getposition()
        if pos:
            self.sell()



cerebro = bt.Cerebro(cheat_on_open=True)
cerebro.addstrategy(TestStrategy)
df=bt.feeds.PandasData(dataname=data)
cerebro.adddata(df)
cerebro.addanalyzer(bt.analyzers.TradeAnalyzer,_name="Basic_stats")
cerebro.broker.set_coc(True)


strats =  cerebro.run(stdstats=False)
strat = strats[0]
cerebro.addobserver(bt.observers.Value)#



print('Final Balance: %.2f' % cerebro.broker.getvalue())
for e in strat.analyzers:
    e.print()

cerebro.plot()

@ab_trader @momentum thank you both for the answers, I was searching for a solution for past two days. If possible can you explain the execution logic here. I want to understand how this works? Docs dont contain much information.

• Whats really is happening when we enable cheat on open or cheat on close
• cerebro.broker.set_coc(True) what is this do?
• self.buy(self.datas[0], coc=False) whats happening here?

Is there any way to put my custom stop loss with the logic I mentioned in post. Will this work?

stop_loss=self.data.open[0]*99/100 # 1% open

if stop_loss > self.data.low[0]:
    self.close(price=stop_loss)
else:
    self.close()

I am trying to backtest a strategy in which I enter at current open and exit at current close itself. That is I buy the share at start of the trading time and exit when the trading day ends. Doesn't carry forward the trade or worry about what happens in between the start and end of a trading day.

An example would be this

                  high         low        open       close      volume
Date                                                                  
2019-05-23  180.539993  177.809998  179.800003  179.660004  36501400.0
___________________________________________________________________________

STRATEGY:

Entry:  open price (179.80)

Stop_Loss(1% open): 178.01

if Stop_Loss > low:
    Exit at Stop_Loss (178.01)
else:
    Exit at close (179.66)

I tried to first understand the cheat on open sample code in the docs but I didn't get it so went this way to code without the stop loss logic.

cerebro = bt.Cerebro(cheat_on_open=True)

 def next(self):
     #order to sell at close price
     self.order = self.close(coo=False, coc=True)

 def next_open(self):
    # order to buy at open price
     self.order=self.buy(coo=True, coc=False)

The log shows that its buying and selling the stocks at the same opening price which does nothing and incur me commission charges.

2019-05-21, BUY EXECUTED, Price: 185.22, Cost: -185.22, Comm 9.26
2019-05-21, OPERATION PROFIT, GROSS 0.00, NET -18.52
2019-05-22, SELL EXECUTED, Price: 184.66, Cost: -184.66, Comm 9.23
2019-05-22, BUY EXECUTED, Price: 184.66, Cost: -184.66, Comm 9.23
2019-05-22, OPERATION PROFIT, GROSS 0.00, NET -18.47
2019-05-23, SELL EXECUTED, Price: 179.80, Cost: -179.80, Comm 8.99
2019-05-23, BUY EXECUTED, Price: 179.80, Cost: -179.80, Comm 8.99
2019-05-23, OPERATION PROFIT, GROSS 0.00, NET -17.98

Can you give a hint where I am making a mistake? In this strategy how can I add the stop loss to backtrader similar to this idea.

if Stop_Loss > low:
    exit at Stop_Loss (178.01)
else:
    exit at close (179.66)

I assume self.buy(exectype=bt.Order.Stop, price=self.data.open[0]) will not work with my above stop loss condition.