@leecallen Never mind -- I figured it out.

The next bar at 7:45 hits both the SL and the TP. And then I remembered: in Backtrader the order events get processed before the next() logic. And some bug in my code (I know where it is) is posting the prev-bar datetime in my logging.

And I guess I need to set up OCO or bracket orders or something so my SL and TP can't both fire.

Great, thank you for clearing that up for me.

If I open a bracket order with SL and TP, and both the SL and the TP are hit in a single bar, which will BT process first? I am using the BT internal broker.

This must be a FAQ but I can't find it.

When I resample 5M data to 1H it seems to be wrong - offset incorrectly.

I download historical data from Dukascopy for backtesting. Here is a piece of the 5M data file:

20.10.2020 15:55:00.000 GMT-0400,0.70532,0.70535,0.70522,0.70525,0.000500640000000002
20.10.2020 16:00:00.000 GMT-0400,0.70524,0.70529,0.70514,0.70528,0.0001319599999999999
20.10.2020 16:05:00.000 GMT-0400,0.70528,0.70534,0.70526,0.70530,0.00006616999999999997
20.10.2020 16:10:00.000 GMT-0400,0.70530,0.70535,0.70529,0.70530,0.00006858
20.10.2020 16:15:00.000 GMT-0400,0.70531,0.70535,0.70529,0.70530,0.00004368999999999999
20.10.2020 16:20:00.000 GMT-0400,0.70529,0.70530,0.70528,0.70530,0.000018000000000000004
20.10.2020 16:25:00.000 GMT-0400,0.70529,0.70531,0.70523,0.70524,0.00005646999999999998
20.10.2020 16:30:00.000 GMT-0400,0.70524,0.70530,0.70496,0.70498,0.00014451999999999995
20.10.2020 16:35:00.000 GMT-0400,0.70497,0.70497,0.70445,0.70449,0.0002016899999999998
20.10.2020 16:40:00.000 GMT-0400,0.70452,0.70457,0.70440,0.70451,0.0001535199999999999
20.10.2020 16:45:00.000 GMT-0400,0.70452,0.70478,0.70452,0.70465,0.00011174999999999995
20.10.2020 16:50:00.000 GMT-0400,0.70463,0.70477,0.70459,0.70477,0.00010162999999999984
20.10.2020 16:55:00.000 GMT-0400,0.70475,0.70480,0.70466,0.70468,0.00017255000000000005
20.10.2020 17:00:00.000 GMT-0400,0.70469,0.70510,0.70462,0.70491,0.000017390000000000008
20.10.2020 17:05:00.000 GMT-0400,0.70493,0.70510,0.70488,0.70488,0.000017259999999999997

That's date-time, open, high, low, close, volume. That 16:05 row represents the 5M period from 16:05 to 16:10. I verified that by examining the raw 1M and 5M data.

When I resample to 1H I I expect the 16:00 bar to represent actual times of 16:00-17:00, or the 5M bars 16:00-16:55. Instead I get this:

2020-10-20T17:00:00-04:00, O:0.70528 H:0.70535 L:0.7044 C:0.70491 ADX:23.74952207714273

The 17:00 bar represents the 5M bars 16:05-17:00 which represents actual times 16:05-17:05. So it's wrong in two ways:

• it's off by 5 minutes, it should start at the :00 minute instead of :05
• it's also off by an hour: this bar should be labelled 16:00

Do you agree? Can this be fixed, or is there some way I can work around it?

Here is the relevant code:

# Use my 'data feed', the pre-loaded bars[]
   data = MyData(timeframe=bt.TimeFrame.Minutes, compression=5)
   cerebro.adddata(data)
   # and resample to get 1H data
   cerebro.resampledata(data, timeframe=bt.TimeFrame.Minutes, compression=60)

I am optimizing, so I do a one-time load of a CSV file into a list bars[], and then MyData reads bars[] into the dataframe:

class MyData(bt.DataBase):

    params = (
        ('timeframe', bt.TimeFrame.Minutes), 
        ('compression', 5))

    def __init__(self):
        global bars
        self.idx = 0
        self.size = len(bars)

    def start(self):
        global bars
        self.idx = 0

    def stop(self):
        pass

    def _load(self):
        global bars
        if self.idx >= self.size:
            # at end of data
            return False

        dt, open, high, low, close = bars[ self.idx ]
        self.lines.datetime[0] = dt
        self.lines.open[0] = open
        self.lines.high[0] = high
        self.lines.low[0] = low
        self.lines.close[0] = close
        
        self.idx += 1
        return True

I know the above code is good because my 'next' function prints each 5M bar and they match up with the CSV file.

That's exactly what I needed, thank you.

@vladisld that change seems to be working fine for me. Thank you.

@dasch That was EXACTLY the bit of information I needed - thank you!

Could you tell me where might I have found this information for myself?

I'm having kind of a fundamental problem with resampling.

My data is M5 and I need to make decisions based on M5, H1, and H4 bars, and indicators based on those bars. For instance an H1 EMA and several H4 EMAs.

I have always done that with resampling, and it works very well.

Here is the problem I am having:

Note 17:55 is the close of an M5 bar and the close of an H1 bar.
At 17:55 I process an M5 bar and create a SELL STOP order to sell at 1.34536. That price is below the M5 close.

My STOP order gets triggered at 17:55 - the same time I opened it!

I think this is because

• the H1 bar closing at 17:55 has an OHLC of 1.31927 1.35149 1.31914 1.34541 so my entry price is below the close but above the low
• I think the BT code to evaluate the STOP order conditions is getting executed for both the M5 and H1 bars, so it gets executed again for the H1 bar, and it sees the H1 LOW below the STOP price and opens the trade.

Is that possible?

If so, how can I prevent that BT code from executing for my resampled H1 & H4 bars?

@vladisld that change seems to be working fine for me. Thank you.

Thank you, I am going to try dasch's "ugly solution"

I need to edit that remark again. I am running 16 instances of my program against different forex currency pairs. Eventually all of them fail with this error. Most of them get through the backfill data and at least a few bars into live data before they fail.

I could really use some advice on this, I can't figure out a workaround.

@leecallen said in Oanda and resampling:

The really weird part is, this is sporadic. The program runs fine for hours and then fails with this error. Or it runs fine on some currency pairs and fails on others.

Edit:

The really weird part is, this is sporadic. The program runs fine for hours and then fails with this error. Or It runs fine on some currency pairs and fails on others.

Sorry folks but a second post with the same title - a related problem.

I am sporadically getting this error:

Traceback (most recent call last):
  File "anz48-notify.py", line 369, in <module>
    runstrategy( )
  File "anz48-notify.py", line 303, in runstrategy
    cycles = cerebro.run()
  File "/home/lallen/.local/lib/python3.6/site-packages/backtrader/cerebro.py", line 1127, in run
    runstrat = self.runstrategies(iterstrat)
  File "/home/lallen/.local/lib/python3.6/site-packages/backtrader/cerebro.py", line 1298, in runstrategies
    self._runnext(runstrats)
  File "/home/lallen/.local/lib/python3.6/site-packages/backtrader/cerebro.py", line 1557, in _runnext
    dt0 = min((d for i, d in enumerate(dts)
ValueError: min() arg is an empty sequence

I have seen this error reported a couple of times here, but in those cases the problem was determined to be the program was not identifying the timeframe in the underlying datafeed. But I am doing so:

import btoandav20
...
    cerebro = bt.Cerebro()
    broker = BrokerCls( token='my token', account='my account', practice=False )
    cerebro.setbroker(broker)
    DataFactory = DataCls
    data = DataFactory( dataname='EUR_USD', timeframe=bt.TimeFrame.Minutes, compression=5 )
    cerebro.adddata(data)
    cerebro.resampledata(data, timeframe=bt.TimeFrame.Minutes, compression=60, rightedge=True)
    cerebro.resampledata(data, timeframe=bt.TimeFrame.Minutes, compression=180, rightedge=True)
   
    cerebro.addstrategy(TestStrategy)
    cycles = cerebro.run()

The really weird part is, this is sporadic. The program runs fine for hours and then fails with this error. Or it runs fine on some currency pairs and fails on others.

@rajanprabu said in Oanda and resampling:

cerebro.adddata(data)

Omigod THANK YOU! So fast, so simple, and so right.

It's a festivus miracle!

Merry Christmas everyone.

One of my wife's gifts to me is time to work on my forex project today. So I am tackling a problem that has mystified me for weeks. Here is a snippet of my code:

def runstrategy( currpair ):
    cerebro = bt.Cerebro()
    broker = BrokerCls( token='my-token-here', account='my-account-here', practice=True )
    cerebro.setbroker(broker)
    DataFactory = DataCls
    data = DataFactory( dataname=currpair[0] + '_' + currpair[1], timeframe=bt.TimeFrame.Minutes, compression=5 )
    cerebro.resampledata(data , timeframe=bt.TimeFrame.Minutes, compression=60, rightedge=True)
    cerebro.resampledata(data , timeframe=bt.TimeFrame.Minutes, compression=240, rightedge=True)
    cerebro.addstrategy()
    cycles = cerebro.run()

and then:

    def next( self ):
        for i, d in enumerate(self.datas):
            print(i, d.datetime.datetime(0).strftime('%Y-%m-%dT%H:%M:%S.%f'))

My problem: next() gets invoked for my two resampled timeframes - 60 & 240 minutes - but never for the raw M5 'ticks'. Every output looks like this:

0 2020-12-24T11:00:00.000000
1 2020-12-24T08:00:00.000000

So self.datas[0] contains the first resample (M60) and self.datas[1] contains the second (M240). Where is my M5 data???

I have been working around this by request M1 data and resampling to M5,M60,M240. That way data0,data1,data2 reflect those timeframes. But I need more backfill data on startup, and I think I would get more if I request M5 data instead of M1.

I hope I am just misunderstanding something.

Thanks to any fellow nerds who are active here today. And yes, I realize not everyone celebrates Christmas.

In case anyone from The Future reads this, I specified rightedge=False in the resample code, that took care of the hour issue. But it did not address the 5M problem - the hourly bar still represented eg 14:05-15:05, instead of 14:00-15:00. So I made an ugly hack: I add 5M to the datetime of each 5M bar.

That's exactly what I needed, thank you.