Hi,
I've implemented IQfeed support for backtrader.
It's quite functional from my point of view. I was able to test it with IB broker.
It's in iqfeed branch of my github repo
Let me know if you're interested in testing it, reviewing the code and any other kind of collaboration.
Regards,
Ed
Project update:
Please upgrade your installations from ccxt branch of my github repository:
pip install --upgrade git+https://github.com/bartosh/backtrader.git@ccxt
As people show some interest I decided to get started.
Here is a simplified code just for the start.
I intentionally made it as simple as possible to make it understandable and easy to play with. Please, consider this only as an invitation to collaborate.
I followed suggestions and examples from this guide
This test script:
#!/usr/bin/env python
# -*- coding: utf-8; py-indent-offset:4 -*-
###############################################################################
#
# Copyright (C) 2017 Ed Bartosh
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
###############################################################################
from __future__ import (absolute_import, division, print_function,
unicode_literals)
import sys
import backtrader as bt
class TestStrategy(bt.Strategy):
def notify_data(self, data, status, *args, **kwargs):
print('*' * 5, 'DATA NOTIF:', data._getstatusname(status))
def next(self):
print('*' * 5, 'NEXT:', bt.num2date(self.data0.datetime[0]), self.data0._name, self.data0.open[0],
bt.TimeFrame.getname(self.data0._timeframe), len(self.data0))
def runstrategy(argv):
# Create a cerebro
cerebro = bt.Cerebro()
data = bt.feeds.CCXT(exchange='gdax', symbol='BTC/USD', timeframe=bt.TimeFrame.Ticks, compression=1)
cerebro.resampledata(data, timeframe=bt.TimeFrame.Seconds)
#cerebro.adddata(data)
# Add the strategy
cerebro.addstrategy(TestStrategy)
# Run the strategy
cerebro.run()
if __name__ == '__main__':
sys.exit(runstrategy(sys.argv))
produces this output:
$ ./gdaxtest.py
loaded tick time: 2017-10-28 15:39:39.726000, price: 5715.01, size: 0.01757807
loaded tick time: 2017-10-28 15:39:39.726000, price: 5715.01, size: 0.01757807
loaded tick time: 2017-10-28 15:39:42.513000, price: 5715.01, size: 1.74e-06
***** NEXT: 2017-10-28 15:39:40 5715.01 Second 1
loaded tick time: 2017-10-28 15:39:42.513000, price: 5715.01, size: 1.74e-06
loaded tick time: 2017-10-28 15:39:43.848000, price: 5715.01, size: 0.14919127
***** NEXT: 2017-10-28 15:39:43 5715.01 Second 2
loaded tick time: 2017-10-28 15:39:43.848000, price: 5715.01, size: 0.14919127
loaded tick time: 2017-10-28 15:39:43.848000, price: 5715.01, size: 0.14919127
loaded tick time: 2017-10-28 15:39:43.848000, price: 5715.01, size: 0.14919127
loaded tick time: 2017-10-28 15:39:43.978000, price: 5715.01, size: 0.01624613
loaded tick time: 2017-10-28 15:39:43.978000, price: 5715.01, size: 0.01624613
loaded tick time: 2017-10-28 15:39:43.978000, price: 5715.01, size: 0.01624613
loaded tick time: 2017-10-28 15:39:43.978000, price: 5715.01, size: 0.01624613
loaded tick time: 2017-10-28 15:39:43.978000, price: 5715.01, size: 0.01624613
loaded tick time: 2017-10-28 15:39:43.978000, price: 5715.01, size: 0.01624613
loaded tick time: 2017-10-28 15:39:43.978000, price: 5715.01, size: 0.01624613
loaded tick time: 2017-10-28 15:39:45.516000, price: 5715.01, size: 0.06103097
***** NEXT: 2017-10-28 15:39:44 5715.01 Second 3
loaded tick time: 2017-10-28 15:39:45.516000, price: 5715.01, size: 0.46466909
loaded tick time: 2017-10-28 15:39:49.408000, price: 5715.01, size: 0.17441143
***** NEXT: 2017-10-28 15:39:46 5715.01 Second 4
loaded tick time: 2017-10-28 15:39:49.408000, price: 5715.01, size: 0.17441143
loaded tick time: 2017-10-28 15:39:49.408000, price: 5715.01, size: 0.17441143
loaded tick time: 2017-10-28 15:39:49.408000, price: 5715.01, size: 0.17441143
loaded tick time: 2017-10-28 15:39:49.408000, price: 5715.01, size: 0.17441143
loaded tick time: 2017-10-28 15:40:02.295000, price: 5715.01, size: 1.74e-06
***** NEXT: 2017-10-28 15:39:50 5715.01 Second 5
loaded tick time: 2017-10-28 15:40:02.295000, price: 5715.01, size: 1.74e-06
loaded tick time: 2017-10-28 15:40:02.295000, price: 5715.01, size: 1.74e-06
loaded tick time: 2017-10-28 15:40:02.295000, price: 5715.01, size: 1.74e-06
loaded tick time: 2017-10-28 15:40:02.295000, price: 5715.01, size: 1.74e-06
loaded tick time: 2017-10-28 15:40:02.295000, price: 5715.01, size: 1.74e-06
loaded tick time: 2017-10-28 15:40:02.295000, price: 5715.01, size: 1.74e-06
loaded tick time: 2017-10-28 15:40:11.930000, price: 5715.01, size: 1.74e-06
***** NEXT: 2017-10-28 15:40:03 5715.01 Second 6
loaded tick time: 2017-10-28 15:40:15.836000, price: 5715.0, size: 0.1853
***** NEXT: 2017-10-28 15:40:12 5715.01 Second 7
loaded tick time: 2017-10-28 15:40:15.836000, price: 5715.0, size: 0.1853
loaded tick time: 2017-10-28 15:40:15.836000, price: 5715.0, size: 0.1853
Next I'm going to implement loading historical ohlc data using fetchOHLCV ccxt API.
Any suggestions and help are welcome.
Started to implement basic support of ordering.
Testing code uses gdax exchange for its data feed and gemini for ordering:
from __future__ import (absolute_import, division, print_function,
unicode_literals)
import sys
import time
from datetime import datetime, timedelta
import backtrader as bt
from pandas import bdate_range
class TestStrategy(bt.Strategy):
def next(self):
for data in self.datas:
print('*' * 5, 'NEXT:', bt.num2date(data.datetime[0]), data._name, data.open[0], data.high[0],
data.low[0], data.close[0], data.volume[0],
bt.TimeFrame.getname(data._timeframe), len(data))
if not self.getposition(data):
order = self.buy(data, exectype=bt.Order.Limit, size=10, price=data.close[0])
else:
order = self.sell(data, exectype=bt.Order.Limit, size=10, price=data.close[0])
def notify_order(self, order):
print('*' * 5, "NOTIFY ORDER", order)
def runstrategy(argv):
# Create a cerebro
cerebro = bt.Cerebro()
# Create broker
broker_config = {'urls': {
'logo': 'https://user-images.githubusercontent.com/1294454/27816857-ce7be644-6096-11e7-82d6-3c257263229c.jpg',
'api': 'https://api.sandbox.gemini.com',
'www': 'https://gemini.com',
'doc': 'https://docs.gemini.com/rest-api',},
'apiKey': 'put your api key here',
'secret': 'put your secret here',
'nonce': lambda: str(int(time.time() * 1000))
}
broker = bt.brokers.CCXTBroker(exchange='gemini', currency='USD', config=broker_config)
cerebro.setbroker(broker)
# Create data feeds
data_ticks = bt.feeds.CCXT(exchange='gdax', symbol='BTC/USD', name="btc_usd_tick",
timeframe=bt.TimeFrame.Ticks)
#cerebro.resampledata(data_sec, timeframe=bt.TimeFrame.Seconds)
cerebro.adddata(data_ticks)
#hist_start_date = bdate_range(end=datetime.now(), periods=1)[0].to_pydatetime()
#hist_start_date = datetime.utcnow() - timedelta(minutes=30)
#data_min = bt.feeds.CCXT(exchange="gdax", symbol="BTC/USD", name="btc_usd_min",
# timeframe=bt.TimeFrame.Minutes, fromdate=hist_start_date)
#cerebro.adddata(data_min)
# Add the strategy
cerebro.addstrategy(TestStrategy)
# Run the strategy
cerebro.run()
if __name__ == '__main__':
sys.exit(runstrategy(sys.argv))
output:
$ ./ccxttest.py
btc_usd_tick: loaded tick: time: 2017-11-12 12:08:21.737000, price: 6251.54, size: 1.59e-06
***** NEXT: 2017-11-12 12:08:21.736998 btc_usd_tick 6251.54 6251.54 6251.54 6251.54 1.59e-06 Tick 1
btc_usd_tick: loaded tick: time: 2017-11-12 12:08:55.247000, price: 6278.9, size: 0.01797351
***** NOTIFY ORDER Ref: 1
OrdType: 1
OrdType: Sell
Status: 0
Status: Created
Size: -10.0
Price: None
Price Limit: None
TrailAmount: None
TrailPercent: None
ExecType: 0
ExecType: Market
CommInfo: None
End of Session: 736646.0
Info: AutoOrderedDict()
Broker: None
Alive: True
***** NEXT: 2017-11-12 12:08:55.246996 btc_usd_tick 6278.9 6278.9 6278.9 6278.9 0.01797351 Tick 2
btc_usd_tick: loaded tick: time: 2017-11-12 12:08:58.981000, price: 6278.89, size: 0.02526792
***** NOTIFY ORDER Ref: 2
OrdType: 1
OrdType: Sell
Status: 0
Status: Created
Size: -10.0
Price: None
Price Limit: None
TrailAmount: None
TrailPercent: None
ExecType: 0
ExecType: Market
CommInfo: None
End of Session: 736646.0
Info: AutoOrderedDict()
Broker: None
Alive: True
@Søren-Pallesen from the top of my head:
pip install ccxt
git clone https://github.com/bartosh/backtrader.git
cd backtrader
git checkout -b ccxt origin/ccxt
python ./setup.py install
#!/usr/bin/env python
# -*- coding: utf-8; py-indent-offset:4 -*-
###############################################################################
#
# Copyright (C) 2017 Ed Bartosh
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
###############################################################################
from __future__ import (absolute_import, division, print_function,
unicode_literals)
import sys
import time
from datetime import datetime, timedelta
import backtrader as bt
class TestStrategy(bt.Strategy):
def next(self):
for data in self.datas:
print('*' * 5, 'NEXT:', bt.num2date(data.datetime[0]), data._name, data.open[0], data.high[0],
data.low[0], data.close[0], data.volume[0],
bt.TimeFrame.getname(data._timeframe), len(data))
if not self.getposition(data):
order = self.buy(data, exectype=bt.Order.Limit, size=10, price=data.close[0])
else:
order = self.sell(data, exectype=bt.Order.Limit, size=10, price=data.close[0])
def notify_order(self, order):
print('*' * 5, "NOTIFY ORDER", order)
def runstrategy(argv):
# Create a cerebro
cerebro = bt.Cerebro()
# Create broker
broker_config = {'urls': {'api': 'https://api.sandbox.gemini.com'},
'apiKey': '<your api key>',
'secret': '<your secret key>',
'nonce': lambda: str(int(time.time() * 1000))
}
broker = bt.brokers.CCXTBroker(exchange='gemini', currency='USD', config=broker_config)
cerebro.setbroker(broker)
# Create data feeds
data_ticks = bt.feeds.CCXT(exchange='gdax', symbol='BTC/USD', name="btc_usd_tick",
timeframe=bt.TimeFrame.Ticks, compression=1)
cerebro.adddata(data_ticks)
hist_start_date = datetime.utcnow() - timedelta(minutes=30)
data_min = bt.feeds.CCXT(exchange="gdax", symbol="BTC/USD", name="btc_usd_min",
timeframe=bt.TimeFrame.Minutes, fromdate=hist_start_date)
cerebro.adddata(data_min)
# Add the strategy
cerebro.addstrategy(TestStrategy)
# Run the strategy
cerebro.run()
if __name__ == '__main__':
sys.exit(runstrategy(sys.argv))
python ./bt-ccxt-test.py
Hi,
Latest project update:
Please, install it from ccxt branch of my github repo:
pip install pip install --upgrade git+https://github.com/bartosh/backtrader.git@ccxt
and install latest ccxt :
pip install ccxt
PS: Tried to reproduce issues mentioned here, but failed. Please, provide the code that I can run to see the issue if you want it to be fixed. Thanks.
Regards,
Ed
@Brad-Lloyd I'm trading US stock market. It's not always closes at 16:00, I'm afraid.
Can anybody suggest how to do it? As only current price and size attributes present in Position object I'm assuming I need to calculate cost basis when orders are filled. Is there any code examples how to do it?
Implemented support for all ccxt time frames using fetchOHLCV. The code is in ccxt branch of my github repo.
I'm probably missing something, but with the current ccxt feed code It looks like it works more or less expected way in both modes. To switch between modes I only change 'historical' parameter for the feed.
The current code is in cctx branch
I'd appreciate if you can point me out where I'm wrong in my implementation.
@backtrader thank you for the explanations.
Yes, I was trying to use one symbol that's trading on pre-market to trigger prenext. As you can see it didn't help much, i.e. it crashed backtrader.
What I was trying to ask (and apparently failed) is: if creating orders without price data supported by backtrader at all? If it's possible then it would be great if you explain how to do it without loading data as @ab_trader suggested. If it's not possible ... well I'll try to work around it using @ab_trader's suggestion.
@backtrader said in Backtrader DOES NOT CRASH on MOO orders - The USER CODE does:
prenextandnext(andnextstart) have nothing to do with market times. Docs - Operating the Platform - Minimum Period@ab_trader said in Backtrader DOES NOT CRASH on MOO orders - The USER CODE does:
@Ed-Bartosh maybe some data from the previous day loaded into bt might help?
Have I understood correctly that you're suggesting to load price data to be able to create orders?
@backtrader said in Backtrader DOES NOT CRASH on MOO orders - The USER CODE does:
Use next where guarantees are valid.
I tried this before using prenext. next is not called as securities are not traded, so there not enough bars I guess. For some securities there is no trading happening at all. Does it mean that for them it's not possible to create orders before market open?
Hi,
I'm getting this traceback when creating MOO(Market On Open) order before market open. It seems that backtrader needs price data to create orders. However, some securities do not trade during premarket.
Any ideas how to fix or work around this?
P.S. This happens in live trading.
Traceback (most recent call last):
File "./test-moo.py", line 87, in <module>
sys.exit(main(sys.argv))
File "./test-moo.py", line 84, in main
cerebro.run()
File "backtrader\cerebro.py", line 1127, in run
runstrat = self.runstrategies(iterstrat)
File "backtrader\cerebro.py", line 1295, in runstrategies
self._runnext(runstrats)
File "backtrader\cerebro.py", line 1626, in _runnext
strat._next()
File "backtrader\strategy.py", line 325, in _next
super(Strategy, self)._next()
File "backtrader\lineiterator.py", line 270, in _next
self.prenext()
File "./test-moo.py", line 53, in prenext
self.buy(data, size=100, exectype=bt.Order.Market,
File "backtrader\strategy.py", line 905, in buy
data = data or self.datas[0]
File "backtrader\lineroot.py", line 287, in __nonzero__
return self._operationown(bool)
File "backtrader\lineroot.py", line 94, in _operationown
return self._operationown_stage2(operation)
File "backtrader\lineroot.py", line 218, in _operationown_stage2
return operation(self[0])
File "backtrader\lineseries.py", line 467, in __getitem__
return self.lines[0][key]
File "backtrader\linebuffer.py", line 163, in __getitem__
return self.array[self.idx + ago]
IndexError: array index out of range
Regards,
Ed
@petens Can you check if this commit fixes the issue for you?
@petens said in Anyone use backtrader to do live trading on Bitcoin exchange?:
size=0.5
Does it work with size=1 ?
@totoro said in Anyone use backtrader to do live trading on Bitcoin exchange?:
if the solution is here https://github.com/bartosh/backtrader/pull/6, why cannot we merge ?
I believe comments in that PR explain why it's not merged.
Btw, thank you @Ed-Bartosh for this fabulous work :)
You're welcome. It's actually a community work, not just mine. Many people contributed to it.
BTW, you have chance to do it too :)
@dedeco > Cloud you accept a pull request?
Absolutely.
BTW, You don't need to ask permissions for this, just submit PR. In worse case scenario it'll be rejected :)
@jacob > can you maybe open the option for 'issues' in your repo.
done
@sfkiwi yes, that issue was reported some time ago and there are 2 PRs about it: https://github.com/bartosh/backtrader/pull/13 and https://github.com/bartosh/backtrader/pull/6
Unfortunately they're both outdated and require rebase. Can you check if any of them fixes the issue for you?