Is backtrader event driven or vectorized?

Taewoo Kim

Sorry for the complete noob question. I am new to backtrader, but it looks promising

backtrader

backtrader takes a dual approach to the problem. This is controlled with the runonce (boolean) parameter to either a instantiation of Cerebro or to cerebro.run like in

The default is runonce=True

cerebro = Cerebro(runonce=True)  # or False

or

cerebro = Cerebro()
...
...
cerebro.run(runonce=True)  # or False

1. runonce=True

This could be called a pseudo-vectorized of half-vectorized approach. Built-in operations feature a once method which calculate things in batch mode in a tight inner loop.

Data feeds are fully pre-loaded
Indicators (and sub-indicators thereof) are pre-calculated in batch-mode
Then, the Strategy instance(s) are run step-by-step

Being the goal to offer an increase in speed, but still allow for fine grained logic in the next method of the strategy

Rough calculations indicate that it is somehow between 20-30% than runonce=False

Drawback: Because indicators are precalculated (and therefore the buffers are preallocated), the data synchronization mechanism cannot pause the actual movement of a data feed when synchronizing the timestamps for the strategy, keeping the buffers to the final same length. This has no actual impact for backtesting but because matplotlib expects all things to have the same x length for plotting, it may not be possible to create a plot of the backtesting.

Drawback 2: The implementation of this mode prevented that some indicators can be fully defined in recursive terms with a single formula. A choice had to be made between having this or having the recursive formulas.

Nice Thing: If a user implements a custom Indicator and only provides a next method (intented for step-by-step, see below), the code automatically detects it and will still pre-calculate the indicator using the next method instead of the missing once method. The calculation loop will not be so tight as it could be, but users don't have to worry about implementing once

2. runonce=False

This is a 100% step-by-step mode. Also named next because only the next method of the different indicators, strategies et al., play a role.

Everything is calculated one step at a time. The reason being the addition of data feeds which would be providing the data points one step at a time (not necessarily live feeds, it could have been reading out of a socket from a database connection).

If cerebro is run with preload=False (disable the preloading of data feeds) it will switch to this mode.

shaodaodao

@backtrader
when I turn on the runonce=False,（nothing else changed） the final value is greatly reduced...
from 16700 to 2700...
Really confused why. Could you please give some explains? Many thanks!

vladisld

@shaodaodao Changing the value of runonce parameter should have no influence on the back testing / optimizations results - it's purely a performance optimization ( at least in theory )

There is a small chance that there is bug of cause - but I suspect there is something in your code that may interfere with the way the framework is supposed to work - however without seeing the full code - it is hard to tell.

If you could provide your code ( or just a sample of it that demonstrates the issue) + sample data - we'll be able to tell for sure.

shaodaodao

@vladisld
I found an interesting issue.
Because I used bt.talib.BBands and customized some parameters, then after cerebro run the final value is huge different for runonce=true or false.
But when I switched to Backtrader's own BBands and set the same parameters, this bug doesn't exist.

So I guess the connection with ta-lib has some issues for runonce selection.

vladisld

@shaodaodao said in Is backtrader event driven or vectorized?:

Because I used bt.talib.BBands and customized some parameters, then after cerebro run the final value is huge different for runonce=true or false

That's interesting. As I've said above, it would be great if you could share some code sample + data (simplified as you wish) - so we could play with it and see what causes this discrepancy.

shaodaodao

@vladisld
Hi， I copied the sample code as following:

-- coding: UTF-8 --

import backtrader as bt
import pandas as pd
import datetime
import talib
import sys # 发现脚本名字（in argv[0])
import os.path
import math

#from backtrader.trade import TradeHistory #管理路径

#1. 创建Indicator and Strategy 类
class BOLLind(bt.Indicator):
lines = ('k_high','k_low')
def init(self):
self.l.k_high = bt.Max(self.data.close,self.data.open)
self.l.k_low = bt.Min(self.data.close,self.data.open)
super(BOLLind, self).init()

class BOLLStrat(bt.Strategy):
params = dict(
period=78, # 布林线周期
nbdevup=2.0,
nbdevdn=2.0,
matype= 1,
size=20, #订单数量
risk=0.2,#风险比例，用于计算size
percA=0.28,
percB=0.04 #用于设定track ATH 直到目标比例
)
#日志函数
def log(self, txt, dt=None):
''' Logging function fot this strategy'''
dt = dt or self.data.datetime[0]
if isinstance(dt, float):
dt = bt.num2date(dt)
print('%s, %s' % (dt.isoformat(), txt))

def __init__(self):
    self.order=None #用来引用尚未执行的订单
    self.reentrytimes=0
    self.buyprice = 0.0 
    self.trail_sig = False
    #布林带上下轨
    self.BOLL = bt.talib.BBANDS(self.data, timeperiod=self.p.period, nbdevup=self.p.nbdevup, nbdevdn=self.p.nbdevdn, matype=self.p.matype)
    #在next方法中用self.BOLL.upperband middleband lowerband
    self.sma =bt.ind.SMA(self.data,period=200)
    A=BOLLind(self.data)  
    self.k_high = A.k_high 
    self.k_low = A.k_low
    self.customsize = 0.0
    self.bar_executed = 0
    self.lots_executed=0.0
    self.longstop=0

def next(self):
    
    self.crossover = self.k_low[0]>=self.BOLL.upperband[0] and self.k_low[-1]<self.BOLL.upperband[-1]
    self.crossdown = self.k_high[0]<=self.BOLL.lowerband[0] and self.k_high[-1]>self.BOLL.lowerband[-1] 
   
    if self.order:
        return 
    if not self.position: 
        self.log('多单首次入场信号')
        self.buy(size=5, exectype = bt.Order.Market) 
    if self.crossdown and self.position:
            self.log('平多单信号') 
            self.close()

def notify_order (self, order):

    if order.status in [order.Submitted, order.Accepted]:
        self.log('ORDER ACCEPTED/SUBMITTED', dt=order.created.dt)
        self.order = order
        return

    if order.status in [order.Expired]:
        self.log('买单过期')  

    if order.status in [order.Completed]:
        #订单已决，执行如下语句：
        if order.isbuy():
            self.reentrytimes += 1 
            self.log( '多单第'+str(self.reentrytimes)+'次入场买单全部完成, Price: %.6f, Size:%.3f, Cost: %.6f, Comm: %.6f' %
                (order.executed.price,
                 order.executed.size,
                 order.executed.value,
                 order.executed.comm))
            self.buyprice=order.executed.price
            self.lots_executed = order.executed.size
        else:# Sell
            self.log('卖单全部完成, Price: %.6f, Size:%.3f, Cost: %.6f, Comm: %.6f' %
                     (order.executed.price,
                      order.executed.size,
                      order.executed.value,
                      order.executed.comm)) 
        self.bar_executed = len(self)
    if order.status in [order.Partial]:  
        if order.isbuy():
            self.reentrytimes += 1 
            self.log(
                '多单第'+str(self.reentrytimes)+'次入场买单部分完成, Price: %.6f, Size:%.3f, Cost: %.6f, Comm: %.6f' %
                (order.executed.price,
                 order.executed.size,
                 order.executed.value,
                 order.executed.comm))
            self.buyprice=order.executed.price
        else:# Sell
            self.reentrytimes = self.reentrytimes 
            self.log('多单第'+str(self.reentrytimes)+'次入场卖单部分完成, Price: %.6f, Size:%.3f, Cost: %.6f, Comm: %.6f' %
                     (order.executed.price,
                      order.executed.size,
                      order.executed.value,
                      order.executed.comm))  
    elif order.status in [order.Canceled, order.Margin, order.Rejected]:
        self.log('订单 Canceled/Margin/Rejected')
    #重置未决订单为空，表示无未决订单了 
    self.order= None 
         
#记录交易收益情况（可以省略，默认不输出结果）
def notify_trade (self, trade):
    #if not trade.isclosed:
     #  return
    self.log('毛收益或损失 %.6f, 扣佣后净利润 %.6f, 佣金 %.6f' % (trade.pnl, trade.pnlcomm, trade.commission))

自定义一个佣金类允许下单量为小数

class CommInfoFractional(bt.CommissionInfo):
def getsize(self, price, cash):
'''Returns fractional size for cash operation @price'''
return self.p.leverage * (cash / price)
#######################
#主程序开始
if name == 'main':

#2. Create a Cerebro 创建大脑引擎对象
cerebro=bt.Cerebro() #如果下面自己控制observer，则(stdstats=False);否则默认broker,Trades,BuySell

#2.1 注入Observer,
cerebro.addobserver(bt.observers.DrawDown)

#3. 获取本脚本文件所在路径
modpath = os.path.dirname(os.path.abspath(sys.argv[0]))

#4. 拼接得到数据文件全路径
datapath = os.path.join(modpath, 'BTC_30min.csv')

#5. 创建行情数据对象，加载数据
data = bt.feeds.GenericCSVData(
dataname = datapath,
datetime=0, # 日期行在第0列
open=1,
high=2,
low=3,
close=4,
volume=5,
openinterest=-1, #无未平仓量列
dtformat=('%Y-%m-%d %H:%M:%S'), #日期格式读取到秒
timeframe=bt.TimeFrame.Seconds,
#timeframe=bt.TimeFrame.Minutes,
fromdate=datetime.datetime(2020,8,18), #起始日
todate=datetime.datetime(2021,7,31),
nullvalue=0.0
) #结束日

cerebro.adddata(data) 
cerebro.addstrategy(BOLLStrat) 
cerebro.broker.setcash(100000.0) #设置初始资金
cerebro.broker.setcommission(0.001) #佣金费率
cerebro.broker.set_slippage_fixed(0.025) 
print('初始市值： %.6f' % cerebro.broker.getvalue())
cerebro.run(runonce=False)
print('最终市值： %.6f' % cerebro.broker.getvalue())

cerebro.plot()

currently the difference is not that big, but still difference. And if the period is longer, and apply more usage of the indicator, the difference is higher.

BTW, I used BTC 30min data from Binance but don't know how to upload the data...
code_text