@robin-dhillon said in Retrieving Executed Prices: I observed that my accepted orders (stop loss and take profit) and executed orders are the same in some cases but in other cases the difference is pretty big And what is your understanding of pretty big? Where are the logs, the associated data? Why should the matched price be any other and not the one which is returned? @robin-dhillon said in Retrieving Executed Prices: I am familiar with the fact that backtrader tries to fill in the best price You are not. backtrader tries it best to give you a realistic price, with the constraints inherent to the nature of the data (ohlc format, timeframe). It's not the best price, it is the price which it can at best be delivered. @robin-dhillon said in Retrieving Executed Prices: but is there anyway to optimize it so it doesn't make that huge of a difference in the prices. If you want to cheat yourself, you can, because that's the translation of optimize. You may try Slippage, but that will affect all orders and the ones you like know will be worse than they are. Docs - Slippage

@backtrader said in Matching of stopLimit orders: @sandro-böhme said in Matching of stopLimit orders: Somehow from the notify methods I assumed that everything will be handled asynchronously but it doesn't That's a huge mistake. You may think it's an advantage short-term ... but it isn't. The notify_xxx do simply let you know how things have happened. They imply nothing. That was indeed a big mistake. Thanks for your support @backtrader!

Docs - Broker - add_cash

Not really. The sample which is in the repository works. My suggestion is that you use that sample with as little modifications as possible, i.e.: change only ffeeds = [bt.stores. ...] and use ffeeds = [getdata(x) for x in fcodes] Where getdata is the one in where you fetch the data using GenericCSV @lampalork said in Futures rollover - data feed produced seems to be empty: month_codes = ['U18', 'V18', 'X18', 'Z18'] ffeeds = [get_data_feed('./CFE_{0}_VX.csv'.format(x)) for x in month_codes] month_codes = ['U18', 'V18', 'X18', 'Z18'] ffeeds = [get_data_feed('./CFE_{0}_VX.csv'.format(x)) for x in month_codes] In any case you may also want to remove those duplicates

This works for me def log(self, txt, dt=None, doprint=False): ''' Logging function fot this strategy''' if self.params.printlog or doprint: dt = dt or self.datas[0].datetime.datetime(0) print('%s, %s' % (dt.isoformat(), txt))

See also Medium - Custom Indicator Development in Python with backtrader

@momentum said in Multiple strategies: If I got it correctly, since there is one broker per cerebro and the broker holds positions by data and not by (data, strategy), strategies won't work independently i.e. whatever functionality is based on position will 'work' on the total position and not just the position of the strategy itself It's a single account broker. And an asset is an asset and not a strategy dependent asset. @momentum said in Multiple strategies: However, orders and trades seems to be independent per strategy. Because they are issued by independent strategies. @momentum said in Multiple strategies: you can have different cerebro No. Well yes. You can write custom code to coordinate anything you imagine. @momentum said in Multiple strategies: or have a different sizer per strategy and have each strategy keeping track of its own position, value, etc. I am not sure whether observes, analysers, etc would work on strategy level. Sizers don't keep track of position, value or other things. Analyzers do analyze what the broker does.

@backtrader said in Multi timeframe indicator plotted twice and in wrong timeframe: Use coupling: Docs - Mixing Timeframes in Indicators This functionality allows to calculate an indicator on a timeframe an automatically apply the indicator to another lower timeframe. I tried coupling the D1 indicator with the following code as suggested by the guide, but I still get the same error: ValueError: x and y must have same first dimension, but have shapes (54,) and (6,) class LevelsDisplay(bt.Strategy): params = dict(period=30) def __init__(self): # self.data1 = D1 Timeframe levels = Levels(self.data1, period=self.p.period) self.levels = levels() self.levels.plotinfo.subplot = False self.levels.plotinfo.plotmaster = self.data0 I do not see any other places in the Strategy where I could couple. The guide says See how the () is executed with no arguments (in the background a None is being supplied). The following is happening: pivotpoint.s1() is returning an internal LinesCoupler object which follows the rhythm of the larger scope. This coupler fills itself with the latest delivered value from the real s1 (starting with a default value of NaN) How does the Cerebro know which timeframe is the "the larger scope"? Does it analyze all the added data? BTW the Mixing Timeframes guide is where I started and it does not mention plotmaster at all. Maybe it could be useful for newbies if it was somehow mentioned. Just a suggestion.

Passing extra **kwwargs was supported. The rest is the usual surprise with Interactive Brokers ...

Thanks that worked .... I had an issue with kicking off the calc at the right time as this was an indicator on another indicator which is an average of another average, so had to find a way to delay the start. but the below is not very pretty but it worked. I think it might be better in a new indicator class but I kept getting a variety of errors with that and my python isn't good enough. Thanks for the help. def __init__(self): #keep a reference to the close self.dataclose = self.datas[0].close self.dataopen = self.datas[0].open self.datahi = self.datas[0].high self.datalo = self.datas[0].low self.datadate = self.datas[0].datetime self.dmi = btind.DirectionalMovement(period = self.params.fastdays) self.dyn_adx_hi = [0,] self.dyn_adx_lo = [0,] def next(self): if len(self.dmi.adx) > (self.params.fastdays)*3+1 and len(self.dmi.adx) <=(self.params.fastdays)*3+2: self.dyn_adx_hi[0] = self.dmi.adx[-1] self.dyn_adx_lo[0] = self.dmi.adx[-1] if len(self.dmi.adx) > (self.params.fastdays)*3+2: if self.dmi.adx[0] > self.dyn_adx_hi[-1] or self.dmi.adx[0] >= self.dmi.adx[-1]: self.dyn_adx_hi[0] = self.dmi.adx[0] self.dyn_adx_lo[0] = self.dyn_adx_lo[-1] elif self.dmi.adx[0] < self.dyn_adx_lo[-1] or self.dmi.adx[0] < self.dmi.adx[-1]: self.dyn_adx_lo[0] = self.dmi.adx[0] self.dyn_adx_hi[0] = self.dyn_adx_hi[-1]

There were several patches to resampling, which probably had an impact on rightedge. Use this code as a template to achieve your desired effect. rd = cerebro.resampledata(data, timeframe=bt.TimeFrame.Minutes, compression=5) def _new_load(self): ret = self._old_load() if ret: dt = self.datetime.datetime(0) self.datetime[0] = bt.date2num(dt - datetime.timedelta(minutes=5)) return ret rd._old_load, rd._load = rd._load, _new_load.__get__(rd, rd.__class__)

I see. Thanks @backtrader for the answer and for the amazing work!

Disable stdstats=False in Cerebro. Essentially your code is breaking something, and that's only the symptom. Docs - Cerebro

Although this routine is great for its simple flat-file data caching, it does have an implementation problem that can shut down the connection before the downloading of data is completed with its sleep statements. Another issue is that, even for small downloads, the sleep statements hold up processing and when one is trying to do analysis of 100-200 securities/hour these sleep statements have to be removed as below. Ideally, the callback will just terminate the connection correctly when the "finished" response is received from the api. So I've altered the code below. It's a bit hard to follow due to the callbacks but appears to work effectively. Please note the timeoutsecs argument should be specified as it is the maximum that the routine will wait for the download. from ib.opt import ibConnection, message from ib.ext.Contract import Contract from time import time, strftime from datetime import datetime import pandas as pd class IBDataCache(object): def _reset_data(self, host='127.0.0.1', port=4001, client_id=None): self._df = pd.DataFrame(columns=['Date', 'Open', 'High', 'Low', 'Close', 'Volume', 'OpenInterest']) self._s = pd.Series() #Initialize connection as long as it's not already connected: if (not hasattr(self, '_conn')) or (not self._conn.isConnected()): self._conn = ibConnection(host, port, client_id) self._conn.enableLogging() # Register the response callback function and type of data to be returned self._conn.register(self._error_handler, message.Error) self._conn.register(self._historical_data_handler, message.historicalData) self._conn.register(self._save_order_id, 'NextValidId') self._conn.register(self._nextValidId_handler, message.nextValidId) self._conn.connect() def _save_order_id(self, msg): self._next_valid_id = msg.orderId print('save order id',msg.orderId) def _nextValidId_handler(self, msg): print("nextValidId_handler: ", msg) self.inner(sec_type=self.req.m_secType, symbol=self.req.m_symbol, currency=self.req.m_currency, exchange=self.req.m_exchange, \ primaryExchange=self.req.m_primaryExch, endtime=self.req.endtime, duration=self.req.duration, \ bar_size=self.req.bar_size, what_to_show=self.req.what_to_show, use_rth=self.req.use_rth) def _error_handler(self, msg): print("error: ", msg) if not msg: print('disconnecting', self._conn.disconnect()) def __init__(self, data_path='/docker_stocks/data', date_format='%Y%m%d %H:%M:%S', host='127.0.0.1', port=4001, client_id=None): self._data_path = data_path self._date_format = date_format self._next_valid_id = 1 self._host = host self._port = port self._client_id = client_id def _historical_data_handler(self, msg): """ Define historical data handler for IB - this will populate our pandas data frame """ # print (msg.reqId, msg.date, msg.open, msg.close, msg.high, msg.low) if not 'finished' in str(msg.date): #print ("historical_data_handler: ", msg) try: self._s = ([datetime.strptime(msg.date, self._date_format), msg.open, msg.high, msg.low, msg.close, msg.volume, 0]) except: #for dates only with no time to str: self._s = ([datetime.strptime(msg.date, "%Y%m%d"), msg.open, msg.high, msg.low, msg.close, msg.volume, 0]) self._df.loc[len(self._df)] = self._s else: self._df.set_index('Date', inplace=True) def setAllWithKwArgs(self, **kwargs): #set all attributes to the kwargs to pass along for key, value in kwargs.items(): setattr(self, key, value) def inner(self, sec_type, symbol, currency, exchange, primaryExchange, endtime, duration, bar_size, what_to_show, use_rth): print ("calling inner... setting up req.") self.req = Contract() self.req.m_secType = sec_type self.req.m_symbol = symbol self.req.m_currency = currency self.req.m_exchange = exchange self.primaryExch = primaryExchange self.req.endtime = endtime self.req.duration = duration self.req.bar_size = bar_size self.req.what_to_show = what_to_show self.req.use_rth = use_rth self._conn.reqHistoricalData(self._next_valid_id, self.req, endtime, duration, bar_size, what_to_show, use_rth, 1) def get_dataframe(self, sec_type, symbol, currency, exchange, primaryExchange, endtime, duration, bar_size, what_to_show, use_rth, timeoutsecs): # build filename self.filename = symbol + '_' + sec_type + '_' + exchange + '_' + currency + '_' + \ endtime.replace(' ', '') + '_' + duration.replace(' ', '') + '_' + bar_size.replace(' ', '') + '_' + \ what_to_show + '_' + str(use_rth) + '.csv' self.filename = self.filename.replace('/', '.') self.filename = self._data_path + '/' + self.filename print ("filename: ", self.filename) try: # check if we have this cached self._df = pd.read_csv(self.filename, parse_dates=True, index_col=0) self._df.index = pd.to_datetime(self._df.index, format='%Y-%m-%d %H:%M:%S') except IOError: #set up connection: self._reset_data(self._host, self._port, self._client_id) # Not cached. Download it. # Establish a Contract object and the params for the request self.inner(sec_type, symbol, currency, exchange, primaryExchange, endtime, duration, bar_size, what_to_show, use_rth) # Make sure the connection doesn't get disconnected prior the response data return timeout = time() + timeoutsecs while self._conn.isConnected() and time() < timeout: #print(".", end="", flush=True) pass self._conn.disconnect() self._df.to_csv(self.filename) return self._df if __name__ == '__main__': date_format = '%Y%m%d %H:%M:%S' downloader_kwargs = dict( data_path='../data', date_format=date_format, host='127.0.0.1', port=4001, client_id=992 ) downloader = IBDataCache(**downloader_kwargs) stock_kwargs = dict( sec_type='STK', symbol='AAPL', currency='USD', exchange='SMART', primaryExchange='NASDAQ', endtime=datetime(2018, 10, 26, 15, 59).strftime(date_format), duration='2 D', bar_size='30 mins', what_to_show='TRADES', use_rth=1 ) df = downloader.get_dataframe(**stock_kwargs) print(df) stock_kwargs = dict( sec_type='STK', symbol='MSFT', currency='USD', exchange='SMART', primaryExchange='NASDAQ', endtime=datetime(2018, 10, 26, 15, 59).strftime(date_format), duration='1 D', bar_size='1 min', what_to_show='TRADES', use_rth=1 ) df = downloader.get_dataframe(**stock_kwargs) print ("IBCacheData") print(df) (As a side note, the int64 error mentioned above is due to the sleep statement from the original code terminating before the downloading is completed.)