Best posts made by blonc

to read more about marketstore. MarketStore is a database server optimized for financial timeseries data. I am integrating MS because it as well allows for real-time streaming via a websocket of the bars from the db. IMO this will allow for consistency between realtime and historical testing. Data is stored in tick compression and then using On-disk Aggregate updates the downsample data compressed into any timeframe you want. Easiest way to get this to work is to build MS from source in a docker, its already integrated with GDAX and works out the box.

The next step for me is to integrate polygon.io which is REALTIME trade by trade SIP data for 200$ a month, latency is under 10ms too if you are colocated in or around ny4.

will update with the polygon connector to MS once it is complete and will post the live datafeed code as well once done. Will update this code too once all the rest is complete.

thanks,

datafeed:

        import backtrader as bt
        from ..utils import date2num
        import backtrader.feed as feed
        import datetime
        import pytz
        import pymarketstore as pymkts


        class MarketStore(feed.DataBase):

            params = (
                ('dataname', None),
                ('fromdate', datetime.date(1990, 1, 1)),
                ('todate', datetime.date(2050,1,1)),
                ('name', ''),
                ('compression', 1),
                ('timeframe', bt.TimeFrame.Days),
                ('host', '127.0.0.1'),
                ('port', '5993'),
                ('symbol', None),
                ('query_timeframe', None),

            )

            def start(self):
                super(MarketStore, self).start()

                self.ndb = pymkts.Client('http://{host}:{port}/rpc'.format(
                    host=self.p.host,
                    port=self.p.port
                ))

                qstr = pymkts.Params(self.p.symbol,
                                     self.p.query_timeframe,
                                     'OHLCV',
                                     start=self.p.fromdate.isoformat(),
                                     end=self.p.todate.isoformat())


                dbars = list(self.ndb.query(qstr).first().array)

                self.biter = iter(dbars)


            def _load(self):
                try:
                    bar = next(self.biter)
                except StopIteration:
                    return False


                self.l.datetime[0] = date2num(datetime.datetime.fromtimestamp(bar[0],pytz.utc))
                self.l.open[0] = bar[1]
                self.l.high[0] = bar[2]
                self.l.low[0] = bar[3]
                self.l.close[0] = bar[4]
                self.l.volume[0] = bar[5]

                return True

simple test strategy:

    from __future__ import (absolute_import, division, print_function,
                            unicode_literals)
    import pytz
    import datetime
    import backtrader as bt
    
    class Master(bt.Strategy):
    
        def __init__(self):
            self.dataopen = self.datas[0].open
            self.datahigh = self.datas[0].high
            self.datalow = self.datas[0].low
            self.dataclose = self.datas[0].close
    
        def next(self):
            # self.log('Line')
            print(self.datas[0].datetime.date(0), self.datas[0].datetime.time(0),
                  self.dataopen[0],
                  self.datahigh[0],
                  self.datalow[0],
                  self.dataclose[0],
                  self.datas[0]._name
                  )
    
    def runstrat():
        cerebro = bt.Cerebro()
    
        cerebro.broker.setcash(550000.0)
    
        data123 = bt.feeds.MarketStore(
            symbol='ETH',
            name='ETH',
            query_timeframe='1Hour',
            timeframe=bt.TimeFrame.Minutes,
            fromdate=datetime.date(2018,6,3),
            todate=datetime.date(2018, 6,3),
            # sessionstart=datetime.time(7),
            # sessionend=datetime.time(10),
            tz=pytz.timezone('US/Eastern'),
    
        )
    
        cerebro.adddata(data123)
    
        cerebro.addstrategy(Master)
    
        cerebro.run()
    
        print('finished')
    
    if __name__ == '__main__':
        runstrat()

I have a pairs strategy, that I want to re-use with multiple pairs. When I add the strategy as thus:

`cerebro.addstrategy(TestStrategy,d0=NTRS,d1=PNC)`

and then for my main :

`def __init__(self,d0,d1):`

with in that I try to pull the data from that

   ` self.data_0_close = self.d0.close
    self.data_1_close = self.d1.close`

however when I run the strategy I get the following error:

'Lines_LineSeries_LineIterator_DataAccessor_Strateg' object has no attribute 'd0'

@nooby_mcnoob
thanks you, that makes more sense.

for anyone in the future: if you are using multiple data sets and want to define the data that goes specifically to that strategy. You need to define the data when you import it.

data_feed_X = bt.feeds.GenericCSVData()

when you add a strategy you need to pass the data you want to that strategy IE

cerebro.addstrategy(TestStrategy,d0=data_feed_X,d1=data_feed_Y)

then with in the strategy you need to add params that pull the data into the strategy, IE:

params = ( ('d0', None ), ('d1', None), )

now you are set you can call:

self.params.d0.close[0]

and it will work just like calling data0 or datas0 or so on.

** solved **

passing a variable to the strategy params() I can then pull that that variable in my print line. The variables sit under strat.params. . or looking something like the following:

results = cerebro.run()

strats = [x[0] for x in results]  # flatten the result

for i, strat in enumerate(strats):
    rets = strat.analyzers.returns.get_analysis()
    print('Strat {} Name {}:\n  - analyzer: {}\n'.format(
        i, strat.params.example_param, rets))

@jacob gonna be a lot easier to just write a simple csv data feed handler and a lot cleaner. most the code is there to do it, just change as needed.. import the two files, do the conversion and pass it along. this will be totally reusable too for any two files you need.