The code is clear but the actual problem you faced (at least on this side) is not yet clear.
To format the code use the markdown conventions. The following will do:
Indent everything 4 spaces
Code contained in a block with opening and closing lines with: ```
When you are writing a message there is a Compose ? link on the top right corner of the editing area. That will guide you to the full reference (which is here: commonmark.org)
Thanks. Now that I know that, compression makes more sense.
Would probably be of value to make it a bit more explicit that compression can convert data. I had a look at all the examples and it mostly "uses" compression, rather than stating compression is used to summarise data to different timeframes ;-)
Thanks for the pointer anyway!
Is backtesting with IB officially supported?
This is an open source packages. "Officially" may not be the right description. Yes, you can connect TWS and download data. It is seen as another data feed.
Your code seems to fail here:
which seems to indicate matplotlib may be missing.
From the docs Docs - Installation
Additional requirements if plotting is wished:
Matplotlib >= 1.4.1
It may work with previous versions, but this the one used for development
On the other hand and looking again at the chart it seems the daily variation may be rather small. Because a full year is on the chart, the 1-minute bars are compressed give a sense of continuity.
Some more data would be needed.
Withoug going into calculations (which is not relevant): Small data * times + indicators (and associated sub-indicators) * times will end up being something big.
10 processes can be broken down to:
4 core computer
2 threads per core
For a total of 4 x 2 workers, plus 2 additional python processes (there may be a master created by the multiprocessing module) for a grand total of 10. Seems right.
If you release the memory from the previous iterations you lose the results. If you don't have complex resample/replay scenarios, the suggestion is to use exactbars=1 when creating/running the cerebro, which tries to reduce the buffers to the minimum.
Or you break your optimization into different runs to make them fit within the limits of your machine.
In any case it seems that a "call a function at a given time", be it related to the session starting/ending times or totally unrelated (for example around midnight) is something people actually do. Seems like a proper add-on.
anaconda and winpython are two of the usually almost-all-in-one Python distributions mostly used.
virtualenv can be used also with them to separate different installations of the same packages (even directly installing from the git repository and pointing to a specific branch as done by @RandyT.
Okay looks like I can't read instructions :(.. I tried setting the timezone in the ibtest.py script to PST and voila! all the problems solved. all the datetime stamps are aligned and the last daily bar is waiting for the day to complete, I hope!
python ibtest.py --port 4003 --data0 ES-201703-GLOBEX --resample --timeframe Days --compression 1 --fromdate 2017-01-01 --timezone PST
Server Version: 76
TWS Time at connection:20170116 10:34:35 UTC
Timezone from ContractDetails: America/Belize
Datetime, Open, High, Low, Close, Volume, OpenInterest, SMA
***** STORE NOTIF: <error id=-1, errorCode=2104, errorMsg=Market data farm connection is OK:usfuture>
***** STORE NOTIF: <error id=-1, errorCode=2106, errorMsg=HMDS data farm connection is OK:ilhmds>
***** DATA NOTIF: DELAYED
Data0, 0001, 736332.0, 2017-01-03T00:00:00.000000, 2240.75, 2259.5, 2239.5, 2252.625, 1786475.0, 0, nan
Data0, 0002, 736333.0, 2017-01-04T00:00:00.000000, 2252.75, 2267.25, 2251.0, 2264.25, 1383745.0, 0, nan
Data0, 0003, 736334.0, 2017-01-05T00:00:00.000000, 2264.5, 2266.0, 2254.0, 2264.25, 1307080.0, 0, nan
Data0, 0004, 736335.0, 2017-01-06T00:00:00.000000, 2264.25, 2277.0, 2258.25, 2271.5, 1541103.0, 0, nan
Data0, 0005, 736338.0, 2017-01-09T00:00:00.000000, 2271.25, 2275.25, 2263.5, 2265.0, 1019553.0, 0, 2263.525
Data0, 0006, 736339.0, 2017-01-10T00:00:00.000000, 2264.5, 2274.0, 2259.5, 2263.75, 1299832.0, 0, 2265.75
Data0, 0007, 736340.0, 2017-01-11T00:00:00.000000, 2263.75, 2271.75, 2255.0, 2270.625, 1727931.0, 0, 2267.025
Data0, 0008, 736341.0, 2017-01-12T00:00:00.000000, 2270.5, 2270.5, 2248.5, 2263.375, 1745819.0, 0, 2266.85
Data0, 0009, 736342.0, 2017-01-13T00:00:00.000000, 2264.5, 2273.5, 2262.75, 2272.5, 1182909.0, 0, 2267.05
***** DATA NOTIF: LIVE
PS: I fixed it in my modified script and it works there as well. thanks Daniel for your help and for backtrader!!
One of the latest additions lets you do direct time comparisons directly to lines to generate signals. Probably nobody has used it yet. An example:
lines = ('mysignal',)
self.lines.mysignal = self.data.datetime > datetime.time(10, 30)
This generates an indicator which can be used as a signal to only execute orders when the daily time goes over 10:30.
But since you are probably looking for day, you'll need to work it out in the next method comparing the self.data.datetime.date() (maybe even getting the weekday) against something of your choosing
Hopefully solved from the python side, not backtrader library. See answer in SO about one more stub class.
Overall abstract classes makes me a little bit of pain in python. Maybe sometimes I'll master metaclasses :)