Help with oandatest.py on samples
-
The error was meant as the previous error having an indication that an order was being canceled and the latest one not.
With regards to
Spyderit seems obvious it's not using the same installed packets as those you have in your distribution. -
sorry, i am bit lost now, does pip install not work on Anaconda python distributions?
i just tried conda install oandapy, got package not found error -
Hi, soled the problem with spyder, it's showing the same error msg as jupyter qt console. oandatest.py also ran(asked for the token, account etc). Thanks for all the help. I am coming with more help requests soon!!
-
Hi, This is my edited code so far, having problem with data being unsubscribed, please help.
from __future__ import (absolute_import, division, print_function, unicode_literals) #importing import backtrader as bt import datetime # For datetime objects #import os.path # To manage paths #import sys # To find out the script name (in argv[0]) # Datas are in a subfolder of the samples. Need to find where the script is # because it could have been called from anywhere # Create a Stratey COPIED FROM BACKTRADER/OANDATEST.PY class TestStrategy(bt.Strategy): params = dict( smaperiod=5, trade=False, stake=1, exectype=bt.Order.Market, stopafter=0, valid=None, cancel=0, donotcounter=False, sell=False, ) def __init__(self): # To control operation entries self.orderid = list() self.order = None self.counttostop = 0 self.datastatus = 0 # Create SMA on 2nd data self.sma = bt.indicators.MovAv.SMA(self.data, period=self.p.smaperiod) print('--------------------------------------------------') print('Strategy Created') print('--------------------------------------------------') def notify_data(self, data, status, *args, **kwargs): print('*' * 5, 'DATA NOTIF:', data._getstatusname(status), *args) if status == data.LIVE: self.counttostop = self.p.stopafter self.datastatus = 1 def notify_store(self, msg, *args, **kwargs): print('*' * 5, 'STORE NOTIF:', msg) def notify_order(self, order): if order.status in [order.Completed, order.Cancelled, order.Rejected]: self.order = None print('-' * 50, 'ORDER BEGIN', datetime.datetime.now()) print(order) print('-' * 50, 'ORDER END') def notify_trade(self, trade): print('-' * 50, 'TRADE BEGIN', datetime.datetime.now()) print(trade) print('-' * 50, 'TRADE END') def prenext(self): self.next(frompre=True) def next(self, frompre=False): txt = list() txt.append('Data0') txt.append('%04d' % len(self.data0)) dtfmt = '%Y-%m-%dT%H:%M:%S.%f' txt.append('{:f}'.format(self.data.datetime[0])) txt.append('%s' % self.data.datetime.datetime(0).strftime(dtfmt)) txt.append('{:f}'.format(self.data.open[0])) txt.append('{:f}'.format(self.data.high[0])) txt.append('{:f}'.format(self.data.low[0])) txt.append('{:f}'.format(self.data.close[0])) txt.append('{:6d}'.format(int(self.data.volume[0]))) txt.append('{:d}'.format(int(self.data.openinterest[0]))) txt.append('{:f}'.format(self.sma[0])) print(', '.join(txt)) if len(self.datas) > 1 and len(self.data1): txt = list() txt.append('Data1') txt.append('%04d' % len(self.data1)) dtfmt = '%Y-%m-%dT%H:%M:%S.%f' txt.append('{}'.format(self.data1.datetime[0])) txt.append('%s' % self.data1.datetime.datetime(0).strftime(dtfmt)) txt.append('{}'.format(self.data1.open[0])) txt.append('{}'.format(self.data1.high[0])) txt.append('{}'.format(self.data1.low[0])) txt.append('{}'.format(self.data1.close[0])) txt.append('{}'.format(self.data1.volume[0])) txt.append('{}'.format(self.data1.openinterest[0])) txt.append('{}'.format(float('NaN'))) print(', '.join(txt)) if self.counttostop: # stop after x live lines self.counttostop -= 1 if not self.counttostop: self.env.runstop() return if not self.p.trade: return if self.datastatus and not self.position and len(self.orderid) < 1: if not self.p.sell: # price = round(self.data0.close[0] * 0.90, 2) price = self.data0.close[0] - 0.005 self.order = self.buy(size=self.p.stake, exectype=self.p.exectype, price=price, valid=self.p.valid) else: # price = round(self.data0.close[0] * 1.10, 4) price = self.data0.close[0] - 0.05 self.order = self.sell(size=self.p.stake, exectype=self.p.exectype, price=price, valid=self.p.valid) self.orderid.append(self.order) elif self.position and not self.p.donotcounter: if self.order is None: if not self.p.sell: self.order = self.sell(size=self.p.stake // 2, exectype=bt.Order.Market, price=self.data0.close[0]) else: self.order = self.buy(size=self.p.stake // 2, exectype=bt.Order.Market, price=self.data0.close[0]) self.orderid.append(self.order) elif self.order is not None and self.p.cancel: if self.datastatus > self.p.cancel: self.cancel(self.order) if self.datastatus: self.datastatus += 1 def start(self): if self.data0.contractdetails is not None: print('-- Contract Details:') print(self.data0.contractdetails) header = ['Datetime', 'Open', 'High', 'Low', 'Close', 'Volume', 'OpenInterest', 'SMA'] print(', '.join(header)) self.done = False #MY Credentials apikey = '572....' acc = '6....' if __name__ == '__main__': cerebro = bt.Cerebro() #adding strategy cerebro.addstrategy(TestStrategy) #Getting data from oandastore oandastore = bt.stores.OandaStore(token=apikey, account=acc, practice=True) cerebro.broker = oandastore.getbroker() data0 = oandastore.getdata(dataname="GBP_USD", timeframe=bt.TimeFrame.Ticks, compression=1,fromdate=datetime.datetime(2017, 3, 20),todate=datetime.datetime(2017, 3, 26), backfill_start=False, backfill=False) #This is setting what timeframe we want to use. cerebro.resampledata(data0, timeframe=bt.TimeFrame.Minutes, compression=5) # Add the Data Feed to Cerebro cerebro.adddata(data0) # Print out the starting conditions print('Starting Portfolio Value: %.2f' % cerebro.broker.getvalue()) # Run over everything cerebro.run() # Print out the final result print('Final Portfolio Value: %.2f' % cerebro.broker.getvalue())This is the spyder output;
Exception in thread Thread-92: Traceback (most recent call last): File "C:\Users\Mahbub\Anaconda3\lib\threading.py", line 916, in _bootstrap_inner self.run() File "C:\Users\Mahbub\Anaconda3\lib\threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "C:\Users\Mahbub\Anaconda3\Lib\site-packages\backtrader\stores\oandastore.py", line 335, in _t_streaming_listener self._transaction(trans) File "C:\Users\Mahbub\Anaconda3\Lib\site-packages\backtrader\stores\oandastore.py", line 568, in _transaction ttype = trans['type'] KeyError: 'type' runfile('C:/Users/Mahbub/Documents/PythonTrading/bktrdralgo.py', wdir='C:/Users/Mahbub/Documents/PythonTrading') Starting Portfolio Value: 0.00 -------------------------------------------------- Strategy Created -------------------------------------------------- Datetime, Open, High, Low, Close, Volume, OpenInterest, SMA ***** DATA NOTIF: NOTSUBSCRIBED ***** DATA NOTIF: NOTSUBSCRIBED Final Portfolio Value: 0.00Exception in thread Thread-101: Traceback (most recent call last): File "C:\Users\Mahbub\Anaconda3\lib\threading.py", line 916, in _bootstrap_inner self.run() File "C:\Users\Mahbub\Anaconda3\lib\threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "C:\Users\Mahbub\Anaconda3\Lib\site-packages\backtrader\stores\oandastore.py", line 548, in _t_order_cancel if oref is None: NameError: name 'oref' is not defined -
From above:
@backtrader said in Help with oandatest.py on samples:
A long shot: since you have no cash in your account, there is no permission to download data and hence the NOTSUBSCRIBED.
Before engaging into buying and selling, it would seem a lot more sensible to have a very simple script with which you can confirm that you can stream the given data.
Note. Use:
```For the code blocks, and NOT
''' -
@backtrader
Hi,
I have 1000 in my demo account, showing 0.00, what is the command cerebro.broker.getvalue() wrong command for oandastore?I am not clear how backtrader saves data for live trading, do we put fromdate and todate for live trading, for backtesting todatemakes sense, but live trading shouldn't be any todate, right? or todate = now?
-
backtrader doesn't save any data. It streams the data from the data feed to your strategy.
The
fromdateandtodateparameters are mostly thought to filter out timestamps in large data collections which are not wished for the backtesting.In the case of an
Oandadata feed, a regular usage (nofromdate, notodate) will give you the maximum possible amount of backfilling (choose your own resamplingtimeframe/compressioncombinatino) and then will stream the incoming data. -
Hi,
This part of a code from backtrader docs;# Datas are in a subfolder of the samples. Need to find where the script is # because it could have been called from anywhere '''modpath = os.path.dirname(os.path.abspath(sys.argv[0])) datapath = os.path.join(modpath, 'data/msft dt.csv') # Create a Data Feed data = bt.feeds.YahooFinanceCSVData( dataname=datapath, # Do not pass values before this date fromdate=datetime.datetime(2016, 3, 20), # Do not pass values before this date todate=datetime.datetime(2017, 3, 20), # Do not pass values after this date reverse=False)Is the data from yahoo feeds beings saved on my home dir /data/msft data.csv?
For backtesting; should i use backtrader.feeds.oandastore or oandastore.getdata, what is the difference? -
That code is loading a file, it is not saving anything. Repeat: backtrader doesn't save any data.
should i use backtrader.feeds.oandastore or oandastore.getdata, what is the difference?
You probably mean
backtrader.feeds.Oanda. It's the same. The store pattern liberates you from knowing the specific name of the feed, because you deal with a single interface which is the store (you can later switch to a different store without changing the pattern) -
@backtrader
Hi,
Presently, i am trying to load a csv file from hard disk, using bt.feeds.GenericCSVData, my data is of 1 min timeframe, getting this errorFile "C:\Users\Mahbub\Anaconda3\lib\_strptime.py", line 362, in _strptime (data_string, format)) ValueError: time data '29.02.2016 00:00:00.000' does not match format '%Y-%m-%d'This is the code used:
data = btfeeds.GenericCSVData( dataname= 'data\gbpusd 1min data.csv', fromdate=datetime.datetime(2017, 1, 1), todate=datetime.datetime(2017,3 , 1), nullvalue=0.0, dtformat=('%Y-%m-%d'), datetime=0, high=1, low=2, open=3, close=4, volume=5, openinterest=-1 )do i change the dtformat? to what? help please.
-
'29.02.2016 00:00:00.000'
and
dtformat=('%Y-%m-%d'),
obviously don't match. The data feed uses the standard
datetimemodule. The format string is defined here (it's the same for Python 2): -
did this
dtformat=('%d.%m.%y %H:%M:%S.%f'), File "C:\Users\Mahbub\Anaconda3\lib\_strptime.py", line 362, in _strptime (data_string, format)) ValueError: time data '29.02.2016 00:00:00.000' does not match format '%d.%m.%y %H:%M:%S.%f' -
@backtrader
This my full code with access code to get data from onada, but the data doesn't seem to be coming, please check.# -*- coding: utf-8 -*- """ Created on Tue Mar 21 23:32:23 2017 @author: Mahbub """ from __future__ import (absolute_import, division, print_function, unicode_literals) import datetime # For datetime objects import os.path # To manage paths import sys # To find out the script name (in argv[0]) import backtrader.feeds as btfeeds # Import the backtrader platform import backtrader as bt #MY Credentials apikey = '572e1741f53d948e6e3e0a929ca2fd32-7957548542378c2ef835a5b4a56502b2' acc = '6273293' # Create a Stratey class TestStrategy(bt.Strategy): params = ( ('maperiod', 15), ) def log(self, txt, dt=None): ''' Logging function fot this strategy''' dt = dt or self.datas[0].datetime.date(0) print('%s, %s' % (dt.isoformat(), txt)) def __init__(self): # Keep a reference to the "close" line in the data[0] dataseries self.dataclose = self.datas[0].close # To keep track of pending orders and buy price/commission self.order = None self.buyprice = None self.buycomm = None # Add a MovingAverageSimple indicator self.sma = bt.indicators.SimpleMovingAverage( self.datas[0], period=self.params.maperiod) # Indicators for the plotting show bt.indicators.ExponentialMovingAverage(self.datas[0], period=25) bt.indicators.WeightedMovingAverage(self.datas[0], period=25, subplot=True) bt.indicators.StochasticSlow(self.datas[0]) bt.indicators.MACDHisto(self.datas[0]) rsi = bt.indicators.RSI(self.datas[0]) bt.indicators.SmoothedMovingAverage(rsi, period=10) bt.indicators.ATR(self.datas[0], plot=False) def notify_order(self, order): if order.status in [order.Submitted, order.Accepted]: # Buy/Sell order submitted/accepted to/by broker - Nothing to do return # Check if an order has been completed # Attention: broker could reject order if not enougth cash if order.status in [order.Completed, order.Canceled, order.Margin]: if order.isbuy(): self.log( 'BUY EXECUTED, Price: %.2f, Cost: %.2f, Comm %.2f' % (order.executed.price, order.executed.value, order.executed.comm)) self.buyprice = order.executed.price self.buycomm = order.executed.comm else: # Sell self.log('SELL EXECUTED, Price: %.2f, Cost: %.2f, Comm %.2f' % (order.executed.price, order.executed.value, order.executed.comm)) self.bar_executed = len(self) # Write down: no pending order self.order = None def notify_trade(self, trade): if not trade.isclosed: return self.log('OPERATION PROFIT, GROSS %.2f, NET %.2f' % (trade.pnl, trade.pnlcomm)) def next(self): # Simply log the closing price of the series from the reference self.log('Close, %.2f' % self.dataclose[0]) # Check if an order is pending ... if yes, we cannot send a 2nd one if self.order: return # Check if we are in the market if not self.position: # Not yet ... we MIGHT BUY if ... if self.dataclose[0] > self.sma[0]: # BUY, BUY, BUY!!! (with all possible default parameters) self.log('BUY CREATE, %.2f' % self.dataclose[0]) # Keep track of the created order to avoid a 2nd order self.order = self.buy() else: if self.dataclose[0] < self.sma[0]: # SELL, SELL, SELL!!! (with all possible default parameters) self.log('SELL CREATE, %.2f' % self.dataclose[0]) # Keep track of the created order to avoid a 2nd order self.order = self.sell() if __name__ == '__main__': # Create a cerebro entity cerebro = bt.Cerebro() # Add a strategy cerebro.addstrategy(TestStrategy) oandastore = bt.stores.OandaStore(token=apikey, account=acc, practice=True) cerebro.broker = oandastore.getbroker() data = oandastore.getdata(dataname="GBP_USD", timeframe=bt.TimeFrame.Minutes, compression=1,fromdate=datetime.datetime(2017, 3, 20), todate=datetime.datetime(2017, 3, 27)) # Add the Data Feed to Cerebro cerebro.adddata(data) # Print out the starting conditions print('Starting Portfolio Value: %.2f' % cerebro.broker.getvalue()) # Run over everything cerebro.run() # Print out the final result print('Final Portfolio Value: %.2f' % cerebro.broker.getvalue()) # Plot the result cerebro.plot() -
@mahbubk9 said in Help with oandatest.py on samples:
ValueError: time data '29.02.2016 00:00:00.000' does not match format '%d.%m.%y %H:%M:%S.%f'
There is difference between
%y(2 digits years) and%Y(4 digits years) -
The real suggestion here is to use something that already works and is delivered with backtrader which is the
oandatestsample.It is a command line tool and you can specify your account, api token and desired ticker on the command line (and for example the desire to resample and to which timeframe/compression)
That should give you a starting point as to whether you can download data or not.
-
Hi
I am working on a new connector for live trade, using as a base the Oanda connector.
Following this post, when executing the code provided by @ThatBlokeDave,
import argparse import datetime import backtrader as bt apikey = 'Insert your api key here' acc = 'insert your acc num here' # Create a Stratey class TestStrategy(bt.Strategy): def __init__(self): pass #Provides any noticficaitons about the data. def notify_data(self, data, status, *args, **kwargs): print('*' * 5, 'DATA NOTIF:', data._getstatusname(status), *args) def notify_store(self, msg, *args, **kwargs): print('*' * 5, 'STORE NOTIF:', msg) def next(self): # Simply log the closing price of the series from the reference print('O: {} H: {} L: {} C: {}'.format( self.data.open[0], self.data.high[0], self.data.low[0], self.data.close[0], )) def start(self): if self.data0.contractdetails is not None: print('-- Contract Details:') print(self.data0.contractdetails) print('Started') acc_cash = cerebro.broker.getcash() acc_val = cerebro.broker.getvalue() print('Account Cash = {}'.format(acc_cash)) print('Account Value = {}'.format(acc_val)) if __name__ == '__main__': cerebro = bt.Cerebro() oandastore = bt.stores.OandaStore(token=apikey, account=acc, practice=True) cerebro.broker = oandastore.getbroker() data0 = oandastore.getdata(dataname="GBP_USD", timeframe=bt.TimeFrame.Ticks, compression=1, backfill_start=False, backfill=False) #This is setting what timeframe we want to use. cerebro.resampledata(data0, timeframe=bt.TimeFrame.Minutes, compression=5) cerebro.addstrategy(TestStrategy) cerebro.run()I receive the following error (which is correct, by not assigning the Loguin data, can not connect to Oanda and fails to receive data).
/home/lml/Code/ymy3/venv/bin/python /home/lml/Code/ymy3/venv/code/rfxtest_oanda_short.py Exception in thread Thread-1: Traceback (most recent call last): File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner self.run() File "/usr/lib/python3.6/threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "/home/lml/Code/ymy3/venv/lib/python3.6/site-packages/backtrader/stores/oandastore.py", line 335, in _t_streaming_listener self._transaction(trans) File "/home/lml/Code/ymy3/venv/lib/python3.6/site-packages/backtrader/stores/oandastore.py", line 576, in _transaction ttype = trans['type'] KeyError: 'type' Started Account Cash = 0.0 Account Value = 0.0 ***** DATA NOTIF: NOTSUBSCRIBED Process finished with exit code 0But my problem comes, making a copy of the Oanda components (oandabroker.py, oanda.py and oandastore.py) and renaming them to build my integration gives me the following error.
I rename the files, then I change the name of the classes and their references (nothing else) and I add the references in the modules (init).
When executing it:import argparse import datetime import backtrader as bt apikey = 'Insert your api key here' acc = 'insert your acc num here' # apikey = '0000000000000000000000' # acc = '000000000000000000' # Create a Stratey class TestStrategy(bt.Strategy): def __init__(self): pass #Provides any noticficaitons about the data. def notify_data(self, data, status, *args, **kwargs): print('*' * 5, 'DATA NOTIF:', data._getstatusname(status), *args) def notify_store(self, msg, *args, **kwargs): print('*' * 5, 'STORE NOTIF:', msg) def next(self): # Simply log the closing price of the series from the reference print('O: {} H: {} L: {} C: {}'.format( self.data.open[0], self.data.high[0], self.data.low[0], self.data.close[0], )) def start(self): if self.data0.contractdetails is not None: print('-- Contract Details:') print(self.data0.contractdetails) print('Started') # acc_cash = cerebro.broker.getcash() # acc_val = cerebro.broker.getvalue() print('Account Cash = {}'.format(acc_cash)) print('Account Value = {}'.format(acc_val)) if __name__ == '__main__': cerebro = bt.Cerebro() rfxstore = bt.stores.RfxStore(token=apikey, account=acc, practice=True) cerebro.broker = rfxstore.getbroker() data0 = rfxstore.getdata(dataname="GBP_USD", timeframe=bt.TimeFrame.Ticks, compression=1, backfill_start=False, backfill=False) #This is setting what timeframe we want to use. cerebro.resampledata(data0, timeframe=bt.TimeFrame.Minutes, compression=5) cerebro.addstrategy(TestStrategy) cerebro.run()I get:
/home/lml/Code/ymy3/venv/bin/python /home/lml/Code/ymy3/venv/code/rfxtest2.py Traceback (most recent call last): File "/home/lml/Code/ymy3/venv/code/rfxtest2.py", line 48, in <module> cerebro.broker = rfxstore.getbroker() File "/home/lml/Code/ymy3/venv/lib/python3.6/site-packages/backtrader/stores/rfxstore.py", line 214, in getbroker return cls.BrokerCls(*args, **kwargs) TypeError: 'NoneType' object is not callable Process finished with exit code 1It seems that when he does (on rfxstore.py):
def getbroker (cls, * args, ** kwargs): '' 'Returns broker with * args, ** kwargs from registered `` BrokerCls``' '' return cls.BrokerCls (* args, ** kwargs)Still taking:
BrokerCls = None # broker class will autoregisterSomething that does not happen when I use Oanda's original files.
I think I'm missing a reference, but I can not find it.
From already thank you very much.
-
@labrk said in Help with oandatest.py on samples:
I think I'm missing a reference, but I can not find it.
@labrk said in Help with oandatest.py on samples:
BrokerCls = None # broker class will autoregisterNothing, neither you nor python code which is being executed, has changed
BrokerClsin the store to contain a reference to a class to be instantiated.You need to do it manually or let the underlying code from the broker modify it in the store.
-
@backtrader
Ok, I understand this.But for a better understanding of the BT process.
What is the difference that makes when I run the original code of Oanda, it detects what class to instantiate for the BrokerCls (Why did not that error occur using the Oanda code?)TK
-
@backtrader
Yes, I solved it. Apologies, there was a reference that was badly modified and that's why the 2 codes gave different error.
Thanks for the support. I keep doing tests -
@LAbrk any progress on this?
i have the same NOTSUBSCRIBED issue with the feed.
trying to execute a simple strategywhile digging into the code and error code i stumble into this:
{'code': 1, 'message': 'Invalid or malformed argument: accountId', 'moreInfo': 'http://developer.oanda.com/docs/v1/troubleshooting/#errors'}