Posts made by vladisld

hmm, seems like the patch either doesn't work or missing.

it should have looked like the following log:

**20:37:31:657 -> 4-2-16777223-10225-Bust event occurred, current subscription is deactivated. Please resubscribe real-time bars immediately.-
**20:37:31:982 <- 51-1-16777223-
**20:37:31:989 <- 50-2-16777224-6248713-FFIV-STK--0.0---SMART-NASDAQ-USD-FFIV-NMS-5-TRADES-0-
20:37:32:349 -> 50-3-16777224-1600461445-120.84-120.84-120.84-120.84-0-120.84-0-
20:37:35:493 -> 50-3-16777221-1600461450-13.80-13.80-13.80-13.80-0-13.80-0-
20:37:35:494 -> 50-3-16777224-1600461450-120.84-120.84-120.84-120.84-0-120.84-0-
20:37:35:494 -> 50-3-16777217-1600461450-59.93-59.93-59.93-59.93-0-59.93-0-

here the 51 code stands for CANCEL_REAL_TIME_BARS and 50 for REQ_REAL_TIME_BARS messages

You may try to insert some prints in ibdata.py in the error processing code around lines 513-518 I guess. The code there looks like the following:

                elif msg == -10225:  # Bust event occurred...
                    self._subcription_valid = False
                    if not self._statelivereconn:
                        self._statelivereconn = self.p.backfill
                        self.reqdata()  # resubscribe
                    continue

and see if the code flow actually enters this region.

Slicing is not directly supported for lines objects.

Take a look at the following docs: https://backtrader.com/docu/concepts/#slicing

Could you provide more of the TWS API Logs: few lines before the event and at least 30 seconds after it.

You are very welcome. Currently we have a bunch of issues open in backtrader2 repository. Please take a look. Right now we are concentrating on bug fixes only - no new features. But even this goes pretty slowly to tell you the truth ☺️.

please go over the following docs:

https://www.backtrader.com/docu/cerebro/#returning-the-results
https://www.backtrader.com/docu/optimization-improvements/#results-management

By default the optreturn Cerebro parameter is set to True - meaning that run method will not return the list of Strategy objects but a stripped down OptReturn objects containing only the original strategy parameters and analyzers.

Quoting the description of the optreturn parameter:

``optreturn`` (default: ``True``)

        If ``True`` the optimization results will not be full ``Strategy``
        objects (and all *datas*, *indicators*, *observers* ...) but and object
        with the following attributes (same as in ``Strategy``):

          - ``params`` (or ``p``) the strategy had for the execution
          - ``analyzers`` the strategy has executed

        In most occassions, only the *analyzers* and with which *params* are
        the things needed to evaluate a the performance of a strategy. If
        detailed analysis of the generated values for (for example)
        *indicators* is needed, turn this off

The best way of getting the optimization statistics is to use the analyzer objects that will collect and report such statistics. There are two benefits using the analyzers :

• Modular and easy to understand design
• Some performance benefits (up to 15% according to the docs ) in execution time.

@chewbacca said in Intraday + Daily datasets Date misalignment ?:

why the buy/sell price would match a price on that bar ?

Not sure I'm following. Could you provide an example with a data feed sample? Also sharing more code (notify_timer in particular) may help.

@chewbacca said in Intraday + Daily datasets Date misalignment ?:

Why would the timer act on the secondary feed?

It doesn't necessarily. Timers are not bound to any data feed at all.

@chewbacca said in Intraday + Daily datasets Date misalignment ?:

The timer acts only once, or else I would have seen double logs (i.e. timer acting twice, once for each feed?)

Right, non-periodic timers will reset once notified, so it can't be triggered anymore.

@chewbacca said in Intraday + Daily datasets Date misalignment ?:

"0" stands for "today", -1 "yesterday"

0 stands for the last delivered bar, -1 stands for the one delivered prior to the last one.

AFAIU this is by design.

1. The timers are checked ( and notified if triggered) each time the new bar is produced by any data feed.
2. Daily data feed produces the bar as the day pass ( so that OLHC prices will be captured)

So for 2020-05-19 9:25 the last daily bar that was issued was 2020-05-18. Please correct me if my understanding is wrong.

Issue was opened in backtrader2 project: https://github.com/backtrader2/backtrader/issues/39

AFAIU, the cash position is updated upon order execution and not upon order creation.

self.close will just issue the order - it doesn't mean it will be immediately executed. IMHO it is probably better to check for the broker cash in notify_order method.

Not sure it could be easily done - but if possible will be glad to take a look :-)

Personally I'm iterating the date periods outside the Cerebro loop (just running multiple Cerebro engines - one per date period - in my case even serial ).

Given that iterating over other parameters permutations occupies all the machine resources - serially iterating the date period doesn't introduce any considerable inefficiency.

On the positive side - one may pass the analysis data from one date period to another - something that is not trivial to do in parallel execution.

The lambda function inside the GenericCSVData is only used if the dtformat parameter is set to integer value (1 or 2) indicating that the datetime column contains timestamps instead of strings

Is it really a case ? ( this is the reason I've asked to share the code )

@T-H said in Order Synchronization:

How can I get API logs?

Not using Oanda myself unfortunately. Hope other forum members may help here.

Could you provide more details:

• what live broker is used ( assuming you are talking about live trading )
• API logs if available

Backtrader updates its order objects on notifications from the broker API. So the best way to detect such discrepancies is to analyze the API logs.

@Petter_Axcell said in Easy Strategy: Begginer RSI:

2020-09-05, BUY CREATE, 10061.94

The log shows the close price IIUC:

self.log('BUY CREATE, %.2f' % self.dataclose[0])

you initialize your cash with just 1000:

cerebro.broker.set_cash(1000)

seems not to be enough to satisfy the above buy order.

Short:
I'm not sure the IBData datafeed is supported for optimization with multiple CPUs (it may work on single CPU though).

Detail:

During the optimization with multiple CPU involved, Backtrader will use the process pool to run the optimization for each parameter permutation in parallel.
For this to work the Cerebro instance will need to be copied to each worker process (it is not done if only single CPU is used).
This is implemented by using pickle mechanism, which is used to serialize the Cerebro instance (together with all the objects connected to it - like data feeds, analyzers ... ).
The problem with IBData data feed is that it is using some objects that can't be serialized (ex. locks, sockets, ...).
Usually, for optimization purposes one may use the data feeds that support preloading of data bars without keeping any sync/connection objects alive (ex. CSV or Database based datafeeds)

@Abhay-Jindal said in Error in multi-core optimization:

I am getting the following error when I run multi-core optimization.
AttributeError: Can't pickle local object 'GenericCSVData.start.<locals>.<lambda>'

It would be helpful if you could share your code.

AFAIU, the data feeds ( and other classes inherited from LineRoot ) override all the operators (including eq ) in order to support the definition of complex lines during the strategy's __init__ method. Explained in the docs: here and here

So just writing ( during the __init__ method only ):

a = self.datas[2] == self.datas[3]

will result not in boolean value, but return a LineOperation object.

However, after the data feedsstart` method is called the behavior of the above operators changes. Now instead of returning the LineOperation, the operator is applied to the current value of the first line object in the data feed. So:

a = self.datas[2] == self.datas[3]

will actually be equal to:

a = self.datas[2].lines[0][0] == self.datas[3].lines[0][0]

It may not sound intuitive at first - but this is how it seems to be working (at least in my debugger :-) ). Please correct me if I'm wrong in case you are observing something else.

Back to your second question - I would suggest to store the index of the data in the rankings array instead of storing the data feed itself. Probably this way it will be easier to compare them.

please go over the following docs:

• https://www.backtrader.com/docu/cerebro/#returning-the-results
• https://www.backtrader.com/docu/optimization-improvements/#results-management

some example from the docs:

def runstrat(pargs=None):
    args = parse_args(pargs)

    cerebro = bt.Cerebro()
    data = bt.feeds.BacktraderCSVData(dataname=args.data)
    cerebro.adddata(data)

    cerebro.addanalyzer(bt.analyzers.Returns)
    cerebro.optstrategy(StFetcher, idx=StFetcher.COUNT())
    results = cerebro.run(maxcpus=args.maxcpus, optreturn=args.optreturn)

    strats = [x[0] for x in results]  # flatten the result
    for i, strat in enumerate(strats):
        rets = strat.analyzers.returns.get_analysis()
        print('Strat {} Name {}:\n  - analyzer: {}\n'.format(
            i, strat.__class__.__name__, rets))

@censorship said in No excecution of orders during tutorial SMAcrossover:

the lines always contain only 1 value for each next.
In my usecase i will need to be able to save more values matching the last close

Indicator's next method will indeed be called for each new bar of the underlying data. However, by properly defining the minimum period of the indicator, it is ensured that previous minimum period bars of the underlying data are available and you may safely perform your calculations.

If you need to pre-calculate all the indicator values for all the available bars of the underlying data - you may provide the implementation of the once method - where all the underlying data bars are pre-loaded.

@censorship said in No excecution of orders during tutorial SMAcrossover:

i use a panda dataframe and convert to numpy array

Not sure you need pandas to implement a simple indicator at all.