Posts made by Squirrel Singing

@stevetree or other fellow traders, if you have the time, could you kindly take a look at my query?

Basically, the final delayed data bar is generated at the time of connection rather than at the time of bar complete formation. This is affecting the calculation of my indicators, such as Moving Average, as the final delayed data bar is used to arrive at the indicator value.

Has anyone noticed this?

If yes, does anyone have a solution?

For details on the code, pls refer to the earlier post.

Any response will be greatly appreciated!

Realize that there's no reply, so just want to follow up and ask if anyone has any insights on this query that I raised?

@ab_trader I've up your reputation too!

@ab_trader , thanks for providing your feedback. I was trying to imply that my code wasn't working and not bt wasn't working. Guess I should have been clearer in my post. Will definitely be more mindful of my wordings in future post.

Back to your solution, it works like a charm.

Problem solved now! Many thanks.

I totally agree. I think that the quality of the documentation is really world class, all thanks to @backtrader

Nevertheless, having an online course would probably bring traders like myself up the learning curve much faster.

And I don't mind paying a small premium for the course.

Re: How to correctly trade on data1 and data2?

Hi fellow traders,

I have two data feeds and am trying to place buy orders to buy either one of them.

I find that the buy order works on the 1st data feed but doesn't work on the second data feed. To be exact, self.position.size would be equal to zero after I place an order for the second data feed whereas it wouldn't be zero if the order were to be placed for the 1st data feed.

Can someone pls help me out here?

See the code below for reference.

import backtrader as bt
import datetime as dt

class Test_Strategy (bt.Strategy):    
    params = (('risk', 0),
             )
    
    def __init__ (self):
        self.data_risk1 = self.datas[0]
        self.data_risk2 = self.datas[1]
        
    def next (self):
        print (self.position.size)
        if self.params.risk == 1:
            self.buy (data = self.data_risk1, size = 100)  
        elif self.params.risk == 2:
            self.buy (data = self.data_risk2, size = 100)  
                       
if __name__ == '__main__':
    cerebro = bt.Cerebro()
    data_risk1 = bt.feeds.GenericCSVData (dataname = r"C:\data1.csv",
                                          timeframe = bt.TimeFrame.Days,
                                          compression = 1,
                                          fromdate = dt.datetime (1999,12,31),
                                          todate = dt.datetime (2018,12,31),
                                          headers = True,
                                          nullvalue = 0,
                                          dtformat = '%m/%d/%Y',
                                          datetime = 0,
                                          time = -1,
                                          open = 2,
                                          high = -1,
                                          low = -1,
                                          close = 1,
                                          volume = 3,
                                          openinterest = -1,
                                         )
    
    data_risk2 = bt.feeds.GenericCSVData (dataname = r"C:\data2.csv",
                                          timeframe = bt.TimeFrame.Days,
                                          compression = 1,
                                          fromdate = dt.datetime (1999,12,31),
                                          todate = dt.datetime (2018,12,31),
                                          headers = True,
                                          nullvalue = 0,
                                          dtformat = '%m/%d/%Y',
                                          datetime = 0,
                                          time = -1,
                                          open = 2,
                                          high = -1,
                                          low = -1,
                                          close = 1,
                                          volume = 3,
                                          openinterest = -1,
                                         )
    cerebro.resampledata (data_risk1, timeframe = bt.TimeFrame.Months, compression = 1)
    cerebro.resampledata (data_risk2, timeframe = bt.TimeFrame.Months, compression = 1)
    cerebro.addstrategy(Test_Strategy, risk = 2)
    cerebro.broker.setcash (1000000)
    results = cerebro.run()  

Sample data format of the two imported csv files are as follows

@stevetree , thanks for your response! Not too sure about the details of your strategy, but i find that the final delayed data bar is generated at the time of connection instead of at the appropriate time, eg 08:05:00 if i were to resample to 5 minute timeframe.

This is impacting all my indicators that are calculated based on delayed data bars.

Using the below screenshot as an example, GBPUSD closing price of 1.28984 at 8:05:00 is incorrect as it shows GBPUSD's closing price at the time of connection instead of at 08:05:00

A sample code is attached below if you wish to replicate what i showed in the screenshot.

import backtrader as bt
import datetime as dt
import pytz as pytz

class Test_Strategy (bt.Strategy):    
    def next (self):
        self.log (self.data.close[0])
    
    def notify_data(self, data, status):
        print('*' * 5, 'DATA NOTIF:', data._getstatusname(status))
    
    def log (self, text):
        print (self.data.datetime.datetime(0),": ", text)

if __name__ == '__main__':
    ibstore = bt.stores.IBStore (host = '127.0.0.1', clientId = 110, port = 7497)
    data = ibstore.getdata (dataname = 'GBP.USD-CASH-IDEALPRO',       #GBP.USD-CASH-IDEALPRO, BARC-STK-LSE-GBP
                            rtbar = False,
                            tz=pytz.timezone('US/Eastern'),
                            )
    cerebro = bt.Cerebro()
    cerebro.broker = ibstore.getbroker()                                                          ###
    cerebro.resampledata (data, timeframe = bt.TimeFrame.Minutes, compression = 5, rightedge = True, boundoff = 0)
    cerebro.addstrategy (Test_Strategy)
    cerebro.run()

@dasch , thanks for your reply.

@backtrader , can I ask for your thoughts on the issue I mentioned 25 minutes earlier?

Thanks Dasch for your comprehensive reply.

The issue that I am seeing is an incorrect final delayed bar, where it is formed before the time period is over.

Using the below screenshot as an example, GBPUSD closing price of 1.28984 at 8:05:00 is incorrect as it shows GBPUSD's closing price at the time of connection instead of at 08:05:00

Is it just me facing this problem or this data point got overlooked by others?

Creating my own data filter/feed could be an option but I just find it incredible how users could live/paper trade with this incorrect data point.

Code is attached below for reference.

import backtrader as bt
import datetime as dt
import pytz as pytz

class Test_Strategy (bt.Strategy):    
    def next (self):
        self.log (self.data.close[0])
    
    def notify_data(self, data, status):
        print('*' * 5, 'DATA NOTIF:', data._getstatusname(status))
    
    def log (self, text):
        print (self.data.datetime.datetime(0),": ", text)

if __name__ == '__main__':
    ibstore = bt.stores.IBStore (host = '127.0.0.1', clientId = 110, port = 7497)
    data = ibstore.getdata (dataname = 'GBP.USD-CASH-IDEALPRO',       #GBP.USD-CASH-IDEALPRO, BARC-STK-LSE-GBP
                            rtbar = False,
                            tz=pytz.timezone('US/Eastern'),
                            )
    cerebro = bt.Cerebro()
    cerebro.broker = ibstore.getbroker()                                                          ###
    cerebro.resampledata (data, timeframe = bt.TimeFrame.Minutes, compression = 5, rightedge = True, boundoff = 0)
    cerebro.addstrategy (Test_Strategy)
    cerebro.run()

Just want to emphasize that the workaround solution, boundoff = 1, as proposed earlier solve the problem of last delayed bar being created before time period is up but then, that workaround solution creates another problem.

The new problem is that all the live signals and execution would be brought forward by one period.

How do I resolve the problem of "last delayed bar being created before time period is up" and not have all the live signals and execution being brought forward?

Besides the different rightedge convention between delayed and live data, has anyone noticed that the last delayed data bar is created even before the time period is up?

I am using Interactive Brokers and am seeing this behaviour.

One comment that I see.

I believe that we can remove the parameter on timeframe in stockkwargs so that it becomes as follows since we will be resampling the data anyway later on

stockkwargs = dict(
rtbar=False, # use RealTime 5 seconds bars
historical=True, # only historical download
qcheck=0.5, # timeout in seconds (float) to check for events
fromdate=datetime.datetime(2019, 9, 24), # get data from..
todate=datetime.datetime(2019, 9, 25), # get data from..
latethrough=False, # let late samples through
tradename=None # use a different asset as order target

Re: Daily Resample Data

The problem is solved after I remove timeframe and compression parameter in getdata().

Can I ask why would that resolve the problem?

Is the inclusion of parameters, timeframe and compression, in ibstore.getdata() redundant if I were to specify a value for rtbar?

See below correct code.

import backtrader as bt
import datetime as dt
import pytz

class Test_Strategy (bt.Strategy):
    def next (self):
        self.log (self.data.close[0])
        
    def log (self, text):
        print (self.data.datetime.datetime(0),": ", text)
        
if __name__ == '__main__':
    ibstore = bt.stores.IBStore (host = '127.0.0.1', clientId = 100, port = 7497)
    data = ibstore.getdata (dataname = 'GBP.USD-CASH-IDEALPRO',     
                            rtbar = False,
                            historical = True,
                            fromdate = dt.datetime (2019,4,25),
                            todate = dt.datetime(2019,10,26),
                            tz=pytz.timezone('US/Eastern'),
                            )
    cerebro = bt.Cerebro()
    cerebro.resampledata (data, timeframe = bt.TimeFrame.Days, compression = 1)
    cerebro.addstrategy (Test_Strategy)
    cerebro.run()

@backtrader Been using Backtrader to do simulated trading on Interactive Brokers and realize something unique about the last bar of delayed data. The closing price of the last bar of delayed data (the bar just before the first live data) to be made available even before its time period is up? Is that intentional?

Re: Backtest vs live bars "off-by-1" discrepancy?

Hi,

For those of you who use Interactive Brokers, are you all seeing the last bar of delayed data being formed even before the time period is over?

I am aware that rightedge = False for Delayed data and rightedge = True for Live Data.

For example, using the below screenshot as an example
At 08:05:00, Closing Price of 1.28926 is the closing price for the period 08:05:00 to 08:10:00
At 08:10:00, Closing Price of 1.28935 is the closing price for the period 08:10:00 to 08:13:33 (the time when connection to IB server was established)
At 08:15:00, Closing Price of 1.2891 is the closing price for the period 08:10:00 to 08:15:00

If the last bar of delayed data is indeed formed before time period is over, aren't indicators, such as 2 periods SMA in the below screenshot being skewed by this "Last Bar of Delayed Data"?

If you want to replicate the results, pls use the below code.

import backtrader as bt
import datetime as dt
import pytz as pytz

class Test_Strategy (bt.Strategy):
    params = (('maperiod',10),
             )
    
    def __init__ (self):
        self.data_live = False
        self.sma = bt.indicators.SimpleMovingAverage (self.data.close, period = self.params.maperiod)
    
    def next (self):
        self.log (str(self.data.close[0]) + '(Closing Price) ' + str(self.sma[0]) + '(2 periods SMA)')
    
    def notify_data(self, data, status):
        print('*' * 5, 'DATA NOTIF:', data._getstatusname(status))
        if status == data.LIVE:
            self.data_live = True
    
    def log (self, text):
        print (self.data.datetime.datetime(0),": ", text)

if __name__ == '__main__':
    ibstore = bt.stores.IBStore (host = '127.0.0.1', clientId = 111, port = 7497)
    data = ibstore.getdata (dataname = 'GBP.USD-CASH-IDEALPRO',       #GBP.USD-CASH-IDEALPRO, BARC-STK-LSE-GBP
                            rtbar = False,
                            timeframe = bt.TimeFrame.Minutes,
                            compression = 1,
                            tz=pytz.timezone('US/Eastern'),
                            )
    cerebro = bt.Cerebro()
    cerebro.broker = ibstore.getbroker()                                                          ###
    cerebro.resampledata (data, timeframe = bt.TimeFrame.Minutes, compression = 5)
    cerebro.addstrategy (Test_Strategy, maperiod = 2)
    cerebro.run()

Hi fellow traders,

Can I ask whether any of you have tried resampling to daily data before?

In the below code, it seems that the output, the daily closing price, is shown once every other days.

If I want to use resampling to show the daily closing price, what should I do?

import backtrader as bt
import datetime as dt
import pytz

class Test_Strategy (bt.Strategy):
    def next (self):
        self.log (self.data.close[0])
        
    def log (self, text):
        print (self.data.datetime.datetime(0),": ", text)
        
if __name__ == '__main__':
    ibstore = bt.stores.IBStore (host = '127.0.0.1', clientId = 100, port = 7497)
    data = ibstore.getdata (dataname = 'GBP.USD-CASH-IDEALPRO',     
                            rtbar = False,
                            timeframe = bt.TimeFrame.Minutes,
                            compression = 60,
                            historical = True,
                            fromdate = dt.datetime (2019,4,25),
                            todate = dt.datetime(2019,10,26),
                            tz=pytz.timezone('US/Eastern'),
                            )
    cerebro = bt.Cerebro()
    cerebro.resampledata (data, timeframe = bt.TimeFrame.Days, compression = 1)
    cerebro.addstrategy (Test_Strategy)
    cerebro.run()

See the below screenshot for selected portion of the output based on the above code.

Just to rephrase my question,

Besides the restriction of 5s or 250ms on the RealTimeBars and tickString, can I ask what other differences are there between RealTimeBars and tickString in Interactive Brokers?

Is one more accurate than the other?

Hi Fellow Backtraders,

I am learning the live trading functionality of backtrader and came upon the following text in documentation.

Besides the restriction of 5s or 250ms on the tickbars and tickstrings, can I ask what other differences are there between tickbars and tickstrings in Interactive Brokers?

Is one more accurate than the other?

====================================================
RealTime Data in 3 flavors

tickPrice events (via IB reqMktData)
Used for CASH products (experimentation with at least TWS API 9.70 has shown no support for the other types)

Receives a tick price event by looking at the BID prices, which according to the non-official Internet literature seems to be the way to track the CASH market prices.

Timestamps are generated locally in the system. An offset to the IB Server time can be used if wished by the end user (calculated from IB reqCurrentTime)

tickString events (aka RTVolume (via IB reqMktData)
Receives a OHLC/Volume snapshot from IB approx. every 250ms (or greater if no trading has happened)

RealTimeBars events (via IB reqRealTimeBars)
Receives historical 5 seconds bars (duration fixed by IB) every 5 seconds

If the chosen timeframe/combination is below the level Seconds/5 this feature will be automatically disabled.

====================================================

Hi Folks,

Greetings from Asia.

This is my first post, so kindly be patient with me if I have asked anything that seems simple.

I have been using backtrader for backtesting and am moving on to use it for Paper Trading and have read this article written by Daniel, https://medium.com/@danjrod/interactive-brokers-in-python-with-backtrader-23dea376b2fc

The following code is from the above mentioned article and I have made the following changes

1. used GBPUSD instead of TWTR
2. imported pytz to synchronize the timezone to US/Eastern

My questions are

1. Why are the Live Resampled Data and the Delayed Resampled Data for the same code and same instrument (GBPUSD) showing different values for the same timings?

2. I also notice that the OHLC values of the Live Data 5 minutes resampled bar are available almost at the start of the bar formation, eg the closing price of 1.300370 for Live Bar 08:45:00 was available at approximately 08:45:00, whereas the OHLC values of the Delayed Data 5 minutes resampled bar follow IB charting convention, eg closing price of 1.299480 for Delayed Bar 08:45:00 shows the closing price of the bar for the period 08:45:00 to 08:49:99. Shouldn't both live data and delayed data be using the same convention?

3. What should I do to make the OHLC values of the Live Data resampled bar be available only at the end of the bar formation so that the Live Resampled Data and Delayed Resampled Data for the same instrument show the same values for the same timings?

from __future__ import (absolute_import, division, print_function,
                        unicode_literals)
import backtrader as bt
import pytz
class St(bt.Strategy):
    def logdata(self):
        txt = []
        txt.append('{}'.format(len(self)))
           
        txt.append('{}'.format(
            self.data.datetime.datetime(0).isoformat())
        )
        txt.append('{:.6f}'.format(self.data.open[0]))
        txt.append('{:.6f}'.format(self.data.high[0]))
        txt.append('{:.6f}'.format(self.data.low[0]))
        txt.append('{:.6f}'.format(self.data.close[0]))
        txt.append('{:.2f}'.format(self.data.volume[0]))
        print(','.join(txt))
        
    data_live = False
    
    def notify_data(self, data, status, *args, **kwargs):
        print('*' * 5, 'DATA NOTIF:', data._getstatusname(status),
              *args)
        if status == data.LIVE:
            self.data_live = True
    def next(self):
        self.logdata()
        
def run(args=None):
    cerebro = bt.Cerebro(stdstats=False)
    store = bt.stores.IBStore(port=7497)
    data = store.getdata(dataname='GBP.USD-CASH-IDEALPRO',
                         tz=pytz.timezone('US/Eastern'),      
                         timeframe=bt.TimeFrame.Ticks)    
    cerebro.resampledata(data, timeframe=bt.TimeFrame.Minutes, compression=5)       
    cerebro.addstrategy(St)
    cerebro.run()
if __name__ == '__main__':
    run()

I have also read the forum on related issues and have also tried to vary the values for parameters, rightedge and boundoff. However, I still cannot find the answers to the questions posted above.

Thank you for reading my post and I hope someone can help me out here.