To simulate a raw tick data one can assign the value of the tick to the fields of the OHLC. See

Community - backtest with tick data, get period datas

With that in mind the value of close (for example) can be used as the tick value (And has to be added with addata)

The 2nd stream, the resampled one, is created, as already present in the code, by issuing resampledata.

The tick data would be in data0 and the resampled in data1

Together with the additions to filling in plotting and the new oanda orders, enough to make a quick release:

Community - Release 1.9.43.116

People use GitHub - ib-controller

@timzhang

Hi,
Perhaps you can give us an example of what is not being updated and why that is causing an issue?

The more information the better.

Hello

Looks like it's my mistake. I've wrongly assumed that it's rounding currency calculations to 2 decimal places - but it's not.
I've corrected numbers format printing and it's OK.
So thank You for inspirational question.
And I'm sorry for bothering with simple problem.

kind regards
silverbald

Upon further examination, the reason not to pass **kwargs with Oanda, unlike with Interactive Brokers, is:

With Interactive Brokers the order object gives a hint as to where the key in **kwargs is actually present in the order and can therefore be applied

With Oanda the order is simply a dictionary of values to be filled. The end user could actually be sending anything and this would be in the JSON sent to Oanda (whether that's simply digested with no complain by Oanda is something untested)

@backtrader Thanks again. Resampling is the behaviour I was looking for. Modified code accordingly.

Seems like the cause was pyfolio analizer

See: Community - Release 1.9.44.116

For Cheat-On-Open see Blog - Cheat On Open

@backtrader Very nice! One step closer to the quality of R's Quantstrat package. Will test it soon

@Roman-Semko said in Backtesting oanda/forex:

I guess my question is whether the size calculation takes into the account that the price/margin needed for one unit of a currency pair depends on the currency pair being traded.

That's why custom sizers can be used. You override _getsizing and then provide the actual exchange rate, which may actually be dynamically fetched from somewhere.

Or else directly in the CommissionInfo object.

@backtrader

Amazing! Thank you, that will be a big help.

By the way: There is a little bug in another blog post regarding this topic:

import pyfolio pf.create_full_tear_sheet(

should be

import pyfolio as pf pf.create_full_tear_sheet(

I didnt want to open a new topic for this..

There was another thread which already addressed this problem, which is due to the rounding around midnight.

The sample might be in need of refreshing, due to the stricter usage of sessionend when for example doing resampling operations. ibtest was created long before that was done to actually be safer in synchronizing data feeds.

@ab_trader @backtrader
Thank you for your advises. I try to use BarReplayer_Open , but I can not understand how to use it.
So I use cerebro.broker.set_coc(True), and I can get hopeful output. I put code and output in following.
・Code

from __future__ import (absolute_import, division, print_function, unicode_literals) import datetime # For datetime objects import os.path # To manage paths import sys # To find out the script name (in argv[0]) # Import the backtrader platform import backtrader as bt import backtrader.analyzers as btanalyzers import math import pandas as pd import talib import numpy as np PARAMS = ( ('maperiod', 15), #moving average period ('period', 15), # ('willperiod', 14), #moving average period ('sizer', None), ) class TestStrategy(bt.Strategy): params = PARAMS def log(self, txt, dt=None): ''' Logging function fot this strategy''' dt = dt or self.datas[0].datetime.date(0) print('%s, %s' % (dt.isoformat(), txt)) def __init__(self): # Keep a reference to the "close" line in the data[0] dataseries self.dataclose = self.datas[0].close # To keep track of pending orders and buy price/commission self.order = None # self.order = self.order_target_percent(target=0.1) self.buyprice = None self.buycomm = None self.sma = bt.indicators.SimpleMovingAverage(self.datas[0], period=self.params.maperiod) def notify_order(self, order): if order.status in [order.Submitted, order.Accepted]: # Buy/Sell order submitted/accepted to/by broker - Nothing to do return # Check if an order has been completed # Attention: broker could reject order if not enougth cash if order.status in [order.Completed, order.Canceled, order.Margin]: if order.isbuy(): self.log( 'BUY EXECUTED, Price: %.2f, Cost: %.2f, Comm %.2f' % (order.executed.price, order.executed.value, order.executed.comm)) self.buyprice = order.executed.price self.buycomm = order.executed.comm else: # Sell self.log('SELL EXECUTED, Price: %.2f, Cost: %.2f, Comm %.2f' % (order.executed.price, order.executed.value, order.executed.comm)) self.bar_executed = len(self) # Write down: no pending order self.order = None def notify_trade(self, trade): if not trade.isclosed: return self.log('OPERATION PROFIT, GROSS %.2f, NET %.2f' % (trade.pnl, trade.pnlcomm)) def next(self): # Simply log the closing price of the series from the reference # Check if an order is pending ... if yes, we cannot send a 2nd one if self.order: return # Check if we are in the market if not self.position: # Not yet ... we MIGHT BUY if ... if self.dataclose[0] > self.sma[0]: # BUY, BUY, BUY!!! (with all possible default parameters) self.log('BUY CREATE, %.2f' % self.dataclose[0]) # Keep track of the created order to avoid a 2nd order self.order = self.buy() else: if self.dataclose[0] < self.sma[0]: # SELL, SELL, SELL!!! (with all possible default parameters) self.log('SELL CREATE, %.2f' % self.dataclose[0]) # Keep track of the created order to avoid a 2nd order self.order = self.sell() class LongOnly(bt.Sizer): params = (('stake', 1),) def _getsizing(self, comminfo, cash, data, isbuy): if isbuy: divide = math.floor(cash/data.close[0]) # divide = math.floor(cash/data.open[1]) self.p.stake = divide # print(self.p.stake) # print(math.floor(cash/data.close[0])) return self.p.stake # Sell situation position = self.broker.getposition(data) if not position.size: return 0 # do not sell if nothing is open return self.p.stake if __name__ == '__main__': cerebro = bt.Cerebro() # Add a strategy cerebro.addstrategy(TestStrategy) # Add Sizer cerebro.addsizer(LongOnly) # cerebro.broker.set_coc(True) # because it could have been called from anywhere modpath = os.path.dirname(os.path.abspath(sys.argv[0])) datapath = os.path.join('datas/orcl-1995-2014.txt') # Create a Data Feed data = bt.feeds.YahooFinanceCSVData( dataname=datapath, # Do not pass values before this date fromdate=datetime.datetime(2000, 1, 1), # Do not pass values before this date todate=datetime.datetime(2000, 12, 31), # Do not pass values after this date reverse=False) # Add the Data Feed to Cerebro cerebro.adddata(data) # Set our desired cash start cerebro.broker.setcash(100000.0) # In order to buy on open you may want to # bt.filters.BarReplayer_Open(data) cerebro.broker.set_coc(True) # Set the commission cerebro.broker.setcommission(commission=0.0) # Print out the starting conditions print('Starting Portfolio Value: %.2f' % cerebro.broker.getvalue()) # Run over everything cerebro.run() # Print out the final result print('Final Portfolio Value: %.2f' % cerebro.broker.getvalue()) # Plot the result cerebro.plot()

・Output

Starting Portfolio Value: 100000.00 2000-01-25, BUY CREATE, 26.61 2000-01-26, BUY EXECUTED, Price: 26.61, Cost: 99973.77, Comm 0.00 2000-01-27, SELL CREATE, 24.43 2000-01-28, SELL EXECUTED, Price: 24.43, Cost: 99973.77, Comm 0.00 2000-01-28, OPERATION PROFIT, GROSS -8190.26, NET -8190.26 2000-02-02, BUY CREATE, 25.61 2000-02-03, BUY EXECUTED, Price: 25.61, Cost: 91786.24, Comm 0.00 2000-02-18, SELL CREATE, 27.61 2000-02-22, SELL EXECUTED, Price: 27.61, Cost: 91786.24, Comm 0.00 2000-02-22, OPERATION PROFIT, GROSS 7168.00, NET 7168.00 2000-02-22, BUY CREATE, 27.97 2000-02-23, BUY EXECUTED, Price: 27.97, Cost: 98957.86, Comm 0.00 2000-03-30, SELL CREATE, 36.98 2000-03-31, SELL EXECUTED, Price: 36.98, Cost: 98957.86, Comm 0.00 2000-03-31, OPERATION PROFIT, GROSS 31877.38, NET 31877.38 2000-04-06, BUY CREATE, 38.75 2000-04-07, BUY EXECUTED, Price: 38.75, Cost: 130820.00, Comm 0.00 2000-04-11, SELL CREATE, 36.48 2000-04-12, SELL EXECUTED, Price: 36.48, Cost: 130820.00, Comm 0.00 2000-04-12, OPERATION PROFIT, GROSS -7663.52, NET -7663.52 2000-04-18, BUY CREATE, 37.22 2000-04-19, BUY EXECUTED, Price: 37.22, Cost: 123160.98, Comm 0.00 2000-04-19, SELL CREATE, 35.16 2000-04-20, SELL EXECUTED, Price: 35.16, Cost: 123160.98, Comm 0.00 2000-04-20, OPERATION PROFIT, GROSS -6816.54, NET -6816.54 2000-04-27, BUY CREATE, 36.45 2000-04-28, BUY EXECUTED, Price: 36.45, Cost: 116348.40, Comm 0.00 2000-05-04, SELL CREATE, 35.01 2000-05-05, SELL EXECUTED, Price: 35.01, Cost: 116348.40, Comm 0.00 2000-05-05, OPERATION PROFIT, GROSS -4596.48, NET -4596.48 2000-05-05, BUY CREATE, 36.22 2000-05-08, BUY EXECUTED, Price: 36.22, Cost: 111774.92, Comm 0.00 2000-05-08, SELL CREATE, 34.10 2000-05-09, SELL EXECUTED, Price: 34.10, Cost: 111774.92, Comm 0.00 2000-05-09, OPERATION PROFIT, GROSS -6542.32, NET -6542.32 2000-05-15, BUY CREATE, 36.31 2000-05-16, BUY EXECUTED, Price: 36.31, Cost: 105226.38, Comm 0.00 2000-05-18, SELL CREATE, 34.45 2000-05-19, SELL EXECUTED, Price: 34.45, Cost: 105226.38, Comm 0.00 2000-05-19, OPERATION PROFIT, GROSS -5390.28, NET -5390.28 2000-05-30, BUY CREATE, 34.98 2000-05-31, BUY EXECUTED, Price: 34.98, Cost: 99832.92, Comm 0.00 2000-06-22, SELL CREATE, 38.43 2000-06-23, SELL EXECUTED, Price: 38.43, Cost: 99832.92, Comm 0.00 2000-06-23, OPERATION PROFIT, GROSS 9846.30, NET 9846.30 2000-06-26, BUY CREATE, 38.99 2000-06-27, BUY EXECUTED, Price: 38.99, Cost: 109678.87, Comm 0.00 2000-06-27, SELL CREATE, 38.78 2000-06-28, SELL EXECUTED, Price: 38.78, Cost: 109678.87, Comm 0.00 2000-06-28, OPERATION PROFIT, GROSS -590.73, NET -590.73 2000-06-28, BUY CREATE, 39.11 2000-06-29, BUY EXECUTED, Price: 39.11, Cost: 109077.79, Comm 0.00 2000-06-29, SELL CREATE, 38.13 2000-06-30, SELL EXECUTED, Price: 38.13, Cost: 109077.79, Comm 0.00 2000-06-30, OPERATION PROFIT, GROSS -2733.22, NET -2733.22 2000-06-30, BUY CREATE, 39.64 2000-07-03, BUY EXECUTED, Price: 39.64, Cost: 106354.12, Comm 0.00 2000-07-03, SELL CREATE, 37.81 2000-07-05, SELL EXECUTED, Price: 37.81, Cost: 106354.12, Comm 0.00 2000-07-05, OPERATION PROFIT, GROSS -4909.89, NET -4909.89 2000-07-20, BUY CREATE, 36.84 2000-07-21, BUY EXECUTED, Price: 36.84, Cost: 101457.36, Comm 0.00 2000-07-21, SELL CREATE, 35.57 2000-07-24, SELL EXECUTED, Price: 35.57, Cost: 101457.36, Comm 0.00 2000-07-24, OPERATION PROFIT, GROSS -3497.58, NET -3497.58 2000-07-25, BUY CREATE, 35.83 2000-07-26, BUY EXECUTED, Price: 35.83, Cost: 97959.22, Comm 0.00 2000-07-27, SELL CREATE, 35.39 2000-07-28, SELL EXECUTED, Price: 35.39, Cost: 97959.22, Comm 0.00 2000-07-28, OPERATION PROFIT, GROSS -1202.96, NET -1202.96 2000-07-31, BUY CREATE, 35.45 2000-08-01, BUY EXECUTED, Price: 35.45, Cost: 96743.05, Comm 0.00 2000-08-01, SELL CREATE, 34.48 2000-08-02, SELL EXECUTED, Price: 34.48, Cost: 96743.05, Comm 0.00 2000-08-02, OPERATION PROFIT, GROSS -2647.13, NET -2647.13 2000-08-03, BUY CREATE, 36.51 2000-08-04, BUY EXECUTED, Price: 36.51, Cost: 94086.27, Comm 0.00 2000-09-08, SELL CREATE, 40.81 2000-09-11, SELL EXECUTED, Price: 40.81, Cost: 94086.27, Comm 0.00 2000-09-11, OPERATION PROFIT, GROSS 11081.10, NET 11081.10 2000-09-28, BUY CREATE, 38.42 2000-09-29, BUY EXECUTED, Price: 38.42, Cost: 105155.54, Comm 0.00 2000-09-29, SELL CREATE, 37.13 2000-10-02, SELL EXECUTED, Price: 37.13, Cost: 105155.54, Comm 0.00 2000-10-02, OPERATION PROFIT, GROSS -3530.73, NET -3530.73 2000-10-19, BUY CREATE, 34.30 2000-10-20, BUY EXECUTED, Price: 34.30, Cost: 101630.90, Comm 0.00 2000-10-30, SELL CREATE, 29.82 2000-10-31, SELL EXECUTED, Price: 29.82, Cost: 101630.90, Comm 0.00 2000-10-31, OPERATION PROFIT, GROSS -13274.24, NET -13274.24 2000-11-17, BUY CREATE, 27.17 2000-11-20, BUY EXECUTED, Price: 27.17, Cost: 88384.01, Comm 0.00 2000-11-20, SELL CREATE, 23.34 2000-11-21, SELL EXECUTED, Price: 23.34, Cost: 88384.01, Comm 0.00 2000-11-21, OPERATION PROFIT, GROSS -12458.99, NET -12458.99 2000-11-30, BUY CREATE, 24.99 2000-12-01, BUY EXECUTED, Price: 24.99, Cost: 75919.62, Comm 0.00 2000-12-14, SELL CREATE, 25.93 2000-12-15, SELL EXECUTED, Price: 25.93, Cost: 75919.62, Comm 0.00 2000-12-15, OPERATION PROFIT, GROSS 2855.72, NET 2855.72 2000-12-15, BUY CREATE, 26.93 2000-12-18, BUY EXECUTED, Price: 26.93, Cost: 78770.25, Comm 0.00 2000-12-20, SELL CREATE, 26.88 2000-12-21, SELL EXECUTED, Price: 26.88, Cost: 78770.25, Comm 0.00 2000-12-21, OPERATION PROFIT, GROSS -146.25, NET -146.25 2000-12-21, BUY CREATE, 27.82 2000-12-22, BUY EXECUTED, Price: 27.82, Cost: 78619.32, Comm 0.00 2000-12-29, SELL CREATE, 27.41 Final Portfolio Value: 77478.72

@ricpruss said in Historical Data Request on IB question:

stockkwargs = dict( timeframe = bt.TimeFrame.Days, rtbar = False, # use RealTime 5 seconds bars historical = True, # only historical download what = 'TRADES', # historical - what to show TRADES MIDPOINT BID ASK BID_ASK HISTORICAL_VOLATILITY OPTION_IMPLIED_VOLATILITY useRTH = False, # historical - download only Regular Trading Hours qcheck = 0.5, # timeout in seconds (float) to check for events backfill_start = True, # do backfilling at the start backfill = True, # do backfilling when reconnecting fromdate = datetime.datetime(2017,1,2), # get data from.. latethrough = False, # let late samples through tradename = None # use a different asset as order target )

That combination of parameters seems overly complex. See Docs - Interactive Brokers (or the data feeds reference quoted above, which uses the same source)

timeframe defaults already to Days (all data feeds default actually to this) rtbar defaults already to False as has no role in historical downloads what defaults already to TRADES for an asset like *-STK-* useRTH -> default already to False qcheck defaults already to 0.5 and plays no role in historical downloads backfill_start defaults already to True and plays no roles in historical downloads backfill defaults already to True and plays no role in historical downloads latethrough defaults already to Falseand plays no role in historical downloads tradename defaults already to None and plays no role in data fetching at all

Missing and could play a role:

todate which defaults to None to indicate until now and can specified to indicate what the ending data of the historical download has to be

With that in mind and for a daily historical download like the above the parameters can be simplified to:

stockkwargs = dict( historical=True, fromdate=datetime.datetime(2017, 1, 2), # get data from )

From the docs quoted above:

- ``historical`` (default: ``False``) If set to ``True`` the data feed will stop after doing the first download of data. The standard data feed parameters ``fromdate`` and ``todate`` will be used as reference. The data feed will make multiple requests if the requested duration is larger than the one allowed by IB given the timeframe/compression chosen for the data.

Hello,

Maybe it is possible to simulate a kind of live with calling yahoo finance feed every x minutes in order to simulate a paper trading session with yahoo data...

Cheers,

remroc

@asuralm The same Docs - Plotting from above explains how to apply the subplot option, which is obviously False

That happens because the SharpeRatio is being used with less data than the timeframe with which is working.

@Maxim-Korobov said in Possible division by zero in `AnnualReturn`:

ret_free = [r - rate for r in returns] ret_free_avg = average(ret_free)

The average fails because the number of returns is 0

Change the timeframe of the analyzer to something for which returns can be generated.