MongoDB As a Data Feed

[mattmar10]

Greetings from a Backtrader newbie. I'm pretty new to Backtrader, but I was able to walk through some of the Getting Started documentation and get a backtest going reading from a CSV file. This is great, but my pricing data lives in a MongoDB time series collection. The mongo documets have a datetime field that is a unix timestamp and also a string representation that could be used as well. I've been trying with the unix timestamp, but haven't had any luck.

Here is an example of a candle in the collection.

How would I go about loading that into a data feed for Backtrader? I have been trying to get it going by reading into a pandas data frame from some other code I've found online, but Im running into an issue trying to read into the PandasData class. This is the error I'm seeing:

Traceback (most recent call last):
  File "/Users/matthewmartin/workspace/trading/bt/sample/TQQQTest.py", line 391, in <module>
    strats =  cerebro.run(stdstats=True)
  File "/usr/local/lib/python3.9/site-packages/backtrader/cerebro.py", line 1127, in run
    runstrat = self.runstrategies(iterstrat)
  File "/usr/local/lib/python3.9/site-packages/backtrader/cerebro.py", line 1212, in runstrategies
    data.preload()
  File "/usr/local/lib/python3.9/site-packages/backtrader/feed.py", line 438, in preload
    while self.load():
  File "/usr/local/lib/python3.9/site-packages/backtrader/feed.py", line 479, in load
    _loadret = self._load()
  File "/usr/local/lib/python3.9/site-packages/backtrader/feeds/pandafeed.py", line 268, in _load
    dt = tstamp.to_pydatetime()

And here is the code I've been working from is below. Any tips would be greatly appreciated.

def read_mongo(db, collection, query={}, host='localhost', port=27017, username=None, password=None, no_id=True):
    """ Read from Mongo and Store into DataFrame """

    # Connect to MongoDB
    db = _connect_mongo(host=host, port=port, username=username, password=password, db=db)

    # Make a query to the specific DB and Collection
    cursor = db[collection].find(query)

    # Expand the cursor and construct the DataFrame
    df =  pandas.DataFrame(list(cursor))

    # Delete the _id
    if no_id:
        del df['_id']

    return df

def _connect_mongo(host, port, username, password, db):
    """ A util for making a connection to mongo """

    if username and password:
        mongo_uri = 'mongodb://%s:%s@%s:%s/%s' % (username, password, host, port, db)
        conn = MongoClient(mongo_uri)
    else:
        conn = MongoClient(host, port)

    return conn[db]

if __name__ == '__main__':

    pandas_df = read_mongo("moneymaker", "candles", {"ticker":"TQQQ"})
    pandas_data = bt.feeds.PandasData(dataname=pandas_df, timeframe=bt.TimeFrame.Minutes, datetime='datetime', open='open', high='high', low='low', close='close', volume='volume', openinterest=None)

    # Create a cerebro entity
    cerebro = bt.Cerebro()

    # Add a strategy
    cerebro.addstrategy(TestStrategy)

[trombone43]

@mattmar10

I ran into a similar problem with pandafeed.py converting from csv to datetime object.

I solved it by updating pandafeed.py to check the type of the tstamp from the csv. If it is a string change it to a pandas datetime.

Add in a reference to pandas and check tstamp for type.