Thanks, i have read again the definition, and i agree with you. All is working well.

You are absolutely correct. Thanks for your help.

They are plotted. The problem arises because matplotlib makes assumptions about situations where the difference between the top and bottom of a plot is too small and could lead to problems.

The plot below plots the BuySell observer on a different axis to make it clear: I am here. But cannot be seen with the data. The right axis ticks (Y) show values like 3.5 and the top of the axis (already inside the observer) indicates that everything has to be multiplied by 1000 (3.065e3)

The scale is wrong obviously because it goes from 0 to 3.065

Probably your case too.

0_1482755765918_upload-8b6b42a3-d34f-4f90-94df-55dc163bf2ad

There isn't documentation as such, given the differences found in different platforms.

Oanda (https://www.oanda.com) was the last integration and was at the same time an effort to try to streamline the interfaces and make it as generic as possible. I would have a look at it.

But being your provider Windows based and with OCX in play I guess you'll have to play with comtypes. Although there are, at least, 2 major COM Windows implementations, you are basically on your own. Visual Chart (https://www.visualchart.com) is also Windows based and uses COM and the following still holds:

win32com (pywin32) launched OutOfMemoryError exceptions because it wouldn't fully understand a type.

Although the type was properly decoded, it was clearly going to be a headache and meant fiddling directly with C++ for any needed modification (the major problem being other people compiling pywin32 from scratch or installing a DLL from a non-trusted source, not to mention the lack of proper support for GCC based compilers under Windows that Python exhibits)

comtypes being pure Python was more up to the task of being a target. It still failed to properly decode complex arrays, so a fork of comptypes was needed for full support.

See the fork: https://github.com/mementum/comtypes See the pull request (from Apr 9) quietly waiting for integration in the master: https://github.com/enthought/comtypes/pull/104

Hopefully your provider is using simpler types and nothing breaks with the standard distributions.

For implementation, the following pattern was chosen:

Store: is an entity which should be the only one talking to the data/broker provider, be it interfacing with sockets or COM

Both the data feed and the broker rely on calling the store and receiving events from it (yes it has to run in a separate thread) to make the information available to the platform.

The suggestion would be to look at the patterns in the Oanda store and associated broker and feeds.

Perfect thanks!

As mentioned by @ab_trader orders are executed on the same bar if you set coc=True in the broker. If not the bar is considered closed and order execution is deferred until the next bar. Two orders issued on the same bar with Market execution policy should be executed also on the same bar (be it the next or the current bar)

The system does a check of submitted orders before accepting them. To avoid for example having 2 consecutive orders which would go over the actual cash reserves on the same bar. If tries to take into account that if a long position is closed cash will be re-injected into the system to let you operate with another buy.

Deactivation of the check can be achieved in the broker with checksubmit=False

Sizers are called internally by buy and sell (close uses the actual existing position) but a Sizer can be subclassed to provide complex functionality.

https://www.backtrader.com/docu/sizers/sizers.html

In that documentation page there are examples of how the modification of _getsizing achieves policies, like for example reversing a position or keeping it long only.

@revelator101 with regards to pyfolio, the sample that is in the sources run seamlessly against pyfolio

A screenshot of today's run:

0_1483196265017_upload-d8f8e45b-0f13-4ae3-972c-76743bc83e36

Great, thank you!

Hi,

If your number 1 goal is:

Learn Python so I can make use of Python to code my strategy

You shouldn't probably be using btrun which is meant to abstract things and won't help with learning.

Non code reports are difficult to look into. In your case the last lines give an insight:

File "c:\users\charl\anaconda3\lib\site-packages\backtrader\feeds\btcsv.py", line 44, in _loadline
y = int(dttxt[0:4])
ValueError: invalid literal for int() with base 10: '14/0'

You are loading a csv data with the BactraderCSVData data feed and the field which is being encountered is 14/0 which cannot be parsed.

If that csv file is ok you best options:

Use GenericCSVData (see here) and configure the fields according to reality Load it with pandas.read_csv and then use a PandasData (see here)data feed also configuring the fields

But if you are learning I would recommend focusing on easy and tested samples (and the quickstart), rather than feeding in your own data sources.

thanks, all worked well

That would mean that there are actually no transactions happening during those seconds. With 2 possibilities in mind:

You have joined the data feed during a not so active market phase

or

You are testing against the demo account setup of TWS

Take into account that the data feed provided by IB for Forex is not like the data feed for equities, for example. Appearances of the BID price will be recorded as the indication for price oscillation. This is what also others do.

In any case you may also run the sample with --debug and you will have the chance to see all messages delivered by TWS to your client (literally all), which may help you follow the details of the price formation in Forex

You may change BID to ASK or MID (check the IB documentation for possible values) by manipulating the what parameter during the creation of an IBData data feed in backtrader.

Read here: Data feeds reference

Nothing prevents you from querying the Sizer which has been added to the strategy.

See the getsizer method of the strategy

Since you have subclassed Sizer (or SizerBase, which is the same), you control any extra interfaces/methods which may help you in the querying.

With that in mind, the Sizer concept was developed with the other direction in mind and that's why inside the Sizer there is a strategy member attribute, which allows the sizer to query whatever interfaces the strategy may have. The rationale behind that: abstracting the Strategy from any logic related to position sizing, allowing it to concentrate on the logic of entering/exiting the market.

The Sizerattributes are describe here: https://www.backtrader.com/docu/sizers/sizers.html#backtrader.Sizer

Over the strategy attribute you can also reach the broker, to for example query the actual net liquidation value

class MySizer(bt.Sizer): def mymethod(self): value = self.strategy.broker.get_value()

An extra fix has been pushed to the development branch, because @randyt also checked it out and found a typo in the part which services historical data downloads between 2 dates (the part initially tested, downloads as much as possible and not between two given dates, which may happend during a disconnection/reconnection cycle)

That time.sleep(5) shouldn't help at all.

The connection to TWS will be 1st done when you call cerebro.run() which happens for sure after that time.sleep(5)

But it is good if it helps.

The idea of capturing your own ticks is in theory sound, but has always practical problems. And that's why companies that collect and curate data make money out of it.

Collecting ticks or minute bars or anything else with backtrader is just a python statement away (see for example the trace that run with 2 instruments collecting 1-minute bars during approximately 35 minutes)

The data points could have been stored somewhere (hierarchically or not) instead of simply having printed them out.

It was not meant so, but there are 2 options:

1 Generic

You pass the created store as an argument to your strategy

2 IBData specific

It has an attribute named ib which is the *store. You can then do: self.data0.ib.xxxxx for example

Hopefully solved from the python side, not backtrader library. See answer in SO about one more stub class.
Overall abstract classes makes me a little bit of pain in python. Maybe sometimes I'll master metaclasses :)
Thanks!

One of the latest additions lets you do direct time comparisons directly to lines to generate signals. Probably nobody has used it yet. An example:

class MySignal(bt.Indicator): lines = ('mysignal',) def __init__(self): self.lines.mysignal = self.data.datetime > datetime.time(10, 30)

This generates an indicator which can be used as a signal to only execute orders when the daily time goes over 10:30.

But since you are probably looking for day, you'll need to work it out in the next method comparing the self.data.datetime.date() (maybe even getting the weekday) against something of your choosing

Okay looks like I can't read instructions :(.. I tried setting the timezone in the ibtest.py script to PST and voila! all the problems solved. all the datetime stamps are aligned and the last daily bar is waiting for the day to complete, I hope!

python ibtest.py --port 4003 --data0 ES-201703-GLOBEX --resample --timeframe Days --compression 1 --fromdate 2017-01-01 --timezone PST Server Version: 76 TWS Time at connection:20170116 10:34:35 UTC -------------------------------------------------- Strategy Created -------------------------------------------------- Timezone from ContractDetails: America/Belize Datetime, Open, High, Low, Close, Volume, OpenInterest, SMA ***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm connection is OK:usfuture> ***** STORE NOTIF: <error id=-1, errorCode=2106, errorMsg=HMDS data farm connection is OK:ilhmds> ***** DATA NOTIF: DELAYED Data0, 0001, 736332.0, 2017-01-03T00:00:00.000000, 2240.75, 2259.5, 2239.5, 2252.625, 1786475.0, 0, nan Data0, 0002, 736333.0, 2017-01-04T00:00:00.000000, 2252.75, 2267.25, 2251.0, 2264.25, 1383745.0, 0, nan Data0, 0003, 736334.0, 2017-01-05T00:00:00.000000, 2264.5, 2266.0, 2254.0, 2264.25, 1307080.0, 0, nan Data0, 0004, 736335.0, 2017-01-06T00:00:00.000000, 2264.25, 2277.0, 2258.25, 2271.5, 1541103.0, 0, nan Data0, 0005, 736338.0, 2017-01-09T00:00:00.000000, 2271.25, 2275.25, 2263.5, 2265.0, 1019553.0, 0, 2263.525 Data0, 0006, 736339.0, 2017-01-10T00:00:00.000000, 2264.5, 2274.0, 2259.5, 2263.75, 1299832.0, 0, 2265.75 Data0, 0007, 736340.0, 2017-01-11T00:00:00.000000, 2263.75, 2271.75, 2255.0, 2270.625, 1727931.0, 0, 2267.025 Data0, 0008, 736341.0, 2017-01-12T00:00:00.000000, 2270.5, 2270.5, 2248.5, 2263.375, 1745819.0, 0, 2266.85 Data0, 0009, 736342.0, 2017-01-13T00:00:00.000000, 2264.5, 2273.5, 2262.75, 2272.5, 1182909.0, 0, 2267.05 ***** DATA NOTIF: LIVE

PS: I fixed it in my modified script and it works there as well. thanks Daniel for your help and for backtrader!!

Thank you @backtrader for your prompt response - this is fantastic.
I will have a go today.