Best posts made by vladisld

Short:

In your class methods you may use self.p and self.params interchangeably - they are actually the same object.
(see the 'Parameters' section in the docs: https://www.backtrader.com/docu/concepts/)

Longer:

One need to distinguish between the params class variable and the self.params (or self.p for that matter) instance variable.

params class variable (list of tuples or dict) defines the custom parameters and their default values that will be later used to "magically" create and populate the self.params (or self.p) instance variable

This self.params (or self.p) instance variable will have members for each tuple originally present in params class variable but their value could be different depending of which parameters where passed during class instantiation ( or have their default values if no matching parameters were passed )

TLDR:

The params class variable may be defined in classes that use MetaParams metaclass (or its derivatives) during their instantiation sequences (like Cerebro, Strategy, Analyzer, BrokerBase, Filter, Indicator, an so on and all their inherited classes ).

During the instantiation the meta-class "hijack" the instantiation sequence and creates additional instance variables (self.p and self.params in particular, but also many others like 'datas', 'data0' and other aliazes for LineSeries inherited classes for example), intializing them in various ways. You may take a look at the with_metaclass method in utils\py3.py for technical details of how this is implemented.

For example one of such meta-classes is MetaParams class which is particularly responsible for creating the self.p and self.params instance variables ( see its donew method )

During your strategy (derived from Strategy class which uses MetaParams meta-class) instantiation process a new auto-generated class (let's call it X) will be used to hold the parameter values. For each tuple in the 'params' class variable there will be a member in X class. The instance of this X class will be created, where each member will get its value either from the parameters passed during the Cerebro.addstrategy call or from the default value in the params class variable tuples.

For example let's assume your strategy params variable defined as:

class SMACrossOver(bt.Strategy):
    params = (
        ('stake', 1),
        ('period', 30),
    )

and you add this strategy to Cerebro using:

cerebro.addstrategy(SMACrossOver, period=15)

The values for 'self.p' members inside your strategy will be:

self.p.stake == 1  # from default value in `params` class variable
self.p.period == 15 # from parameters to cerebro.addstrategy

Few things:

• Positions are maintained by the broker per data.

• position property of the Strategy class just returns the position of the data[0]. If you need to get the position for the different data, the getposition(data=...) could be used.

Now the problem in the above code seems to be related to the following logic in the next method:

        if not self.position:
            self.log('Buy Created, %.2f ' % (self.dataCloseLong[0]))
            self.order = self.buy(data = self.datas[0],size=1)
        else:
            self.log('Sell Created, %.2f' % self.dataCloseShort[0])
            self.order = self.sell(data = self.datas[1],size=1) 

If we have no position in data[0] then the 'buy' will be issued. Once this 'buy' is completed - the position will be established in data[0].

The next time the next will be called, the position in data[0] is present - so the 'sell' is issued for data[1] ( different data, right ). So the position in data[0] will never be changed from now on.

Please correct me if I'm wrong.

Ok, so the following was done:

• Created the org: https://github.com/backtrader2
• Created two teams within the org: admins, contributors
  • admins: has Admins permissions (all possible permissions)
  • contributors: has Write permissions (read, clone, push, manage issues and pull requests)
• Forked the original mementum/backtrader repository under org: https://github.com/backtrader2/backtrader
• Invited both @ab_trader and @run-out to the admin team ( those who provided their github ids)
• Enabled the Issues tab.

Questions:

• Any others want-to-be admins ? Please either apply directly in github or provide your github id in this thread

• Any suggestions/todos that I've missed ?

Notes:

• There are a 'GitHub Discussions' forum for each team - so some admin related topics could possibly be discussed there instead of polluting this community forum ( other suggestions are welcome as always )

Thanks all for the support and interest in maintaining this wonderful project.

Yes sir. This is Backtrader framework support forum.

There are other general algo and non-algo trading forums out there:

https://www.reddit.com/r/algotrading/
https://www.elitetrader.com/et/

and many others - just google it.

@André-Tavares said in How can I inherit params from a parent class and append on the child class?:

params.update({

No need to update the params dict. The parameters from the base classes are fused automagically by metaclass during the instantiation. You may take a look at the following post as well: https://community.backtrader.com/topic/2379/difference-between-params-and-p/2

In this case, I'd just run the whole Cerebro logic for each stock separately ( in a loop) - at least that's what I'm doing in my project.

Probably others may have better advice.

External blog:

https://backtest-rookies.com/2017/08/22/backtrader-multiple-data-feeds-indicators/

Internal blog:
https://www.backtrader.com/blog/posts/2017-04-09-multi-example/multi-example/

Community post(s):
https://community.backtrader.com/topic/2032/manage-multiple-assets/8

This is just one sample, but there are multiple others - try to search.

You are very welcome. Currently we have a bunch of issues open in backtrader2 repository. Please take a look. Right now we are concentrating on bug fixes only - no new features. But even this goes pretty slowly to tell you the truth ☺️.

Short:
I'm not sure the IBData datafeed is supported for optimization with multiple CPUs (it may work on single CPU though).

Detail:

During the optimization with multiple CPU involved, Backtrader will use the process pool to run the optimization for each parameter permutation in parallel.
For this to work the Cerebro instance will need to be copied to each worker process (it is not done if only single CPU is used).
This is implemented by using pickle mechanism, which is used to serialize the Cerebro instance (together with all the objects connected to it - like data feeds, analyzers ... ).
The problem with IBData data feed is that it is using some objects that can't be serialized (ex. locks, sockets, ...).
Usually, for optimization purposes one may use the data feeds that support preloading of data bars without keeping any sync/connection objects alive (ex. CSV or Database based datafeeds)

I didn't try it myself, but probably the following may work:

class MyGenericCSVData(bt.GenericCSVData):
    def start(self):
        super(MyGenericCSVData, self).start()
        if self.p.headers:
            self.f.readline()  # skip the second header (the first one was skipped in GenericCSVData start method)

class MyGenericCSV(bt.GenericCSV):
    DataCls = MyGenericCSVData

Thanks
Vlad

@Yury-Tsar said in Multiple strategies on single data:

@vladisld
question was how to run them simultaneously, not using iteration 1 after another.

This wasn't clear from the original question:

@Yury-Tsar said in Multiple strategies on single data:

how to implement different strategies on single data feed

In case you would like to backtest the same strategy with a different parameters, where each run of the strategy will have its own position, and all of them will be run simultaneously, but still having the same "big OHLC" data loaded just once - this is exactly were the cerebro.optstrategy comes to rescue. Cerebro engine could be setup to preload the data feed (so that it will be read just once) and each strategy instance will run in its own process - all of them running simultaneously as you wished.

Am I still getting your question wrong ?

If your strategies are completely different you may just add all of them to cerebro - each one will be given the same data feed upon start, also each strategy's next method will be called once for each update in the data feed.
Alternatively you may use a more advanced technique described here: (https://www.backtrader.com/blog/posts/2016-10-29-strategy-selection/strategy-selection/) and here: (https://www.backtrader.com/blog/posts/2017-05-16-stsel-revisited/stsel-revisited/)

In case the only difference between your strategies is in a single parameter (% stop loss) you may set it as a parameter of the same strategy like:

class MyStrategy(bt.Strategy):
    params = (
        ('stop_loss_percent', 0.96),
    )

and use the cerebro.optstrategy instead of using cerebro.addstrategy. The 'optstrategy' method accepts a range for each strategy's parameter and will instantiate the same strategy for each value of the parameter in the above range. See more info here: (https://www.backtrader.com/docu/cerebro/)

@Kevin-Galkov said in DataCls does autoregister:

I am confused about how the DataCls attribute [0] of the OandaStore is supposed to "auto" register?

I suppose the question is at which point the DataCls attribute of the OandaStore is initialized. This is happening during the instantiation of OandaData's metaclass (which by itself is happening during the chain of imports when the Backtrader module is imported into your code)

class MetaOandaData(DataBase.__class__):
    def __init__(cls, name, bases, dct):
        '''Class has already been created ... register'''
        # Initialize the class
        super(MetaOandaData, cls).__init__(name, bases, dct)

        # Register with the store
        oandastore.OandaStore.DataCls = cls

class OandaData(with_metaclass(MetaOandaData, DataBase)):
...

This is done in order to support two different usage scenarios:

1. The case where OandaData instance is instantiated directly, like:

data0 = OandaData(dataname=args.data0, **datakwargs)

2. The case where the OandaStore.getdata method is used for the same purpose:

data0 = bt.stores.OandaStore.getdata(dataname=args.data0, **datakwargs)

In case you need to use a different data class (like OandaV20), I'm afraid the new OandaStoreV2 should be introduced and initialized in a way similar to the original OandaStore. But probably there is another way and more experienced forum members may point it out.

Backtrader uses the floating point datetime representation internally, both for line series (data feeds, indicators and so on) and orders.

This is similar to the way the datetime is used in matplotlib. See https://matplotlib.org/_modules/matplotlib/dates.html for more info.

The float based datetime could be easily converted to python datetime by using num2date() function (see dateintern.py module). Alternatively for data feeds, one may use the data.num2date() to directly get the python datetime for current data feed bar.

Similar question was answered here: https://community.backtrader.com/topic/2458/getting-a-calculated-value-from-strateg

Could you please clarify the exact scenario:

1. Do you want to start the backtesting with the position you had while live trading using paper account ? (In this case you indeed want to somehow tell the Backtrader's simulated broker about the position you had in your paper trading account)

2. Do you want to restart the somehow terminated live-trading session using paper account ? In this case your paper trading account is probably maintaining the position for your already.

3. Are you storing some information about your current position in your session and would like to sync it with the position maintained in your paper account upon session restart ? This may be non-trivial, given the open orders leftovers, in case of session crashes.

Am I understanding correctly that there is a consensus ( @ab_trader , @run-out , @vbs ) to come up with a separate fork (we should probably integrate it back to the official one in case the "blessing" from @backtrader will be obtained ).

I've run your code and it is working ok (up to some syntax errors in accessing strategy parameters directly and not through self.params). So your problem seems to be an additional backtrader package installed somewhere, such that you are importing a wrong backtrader package.

It is easy to check - just print/debug the bt.__file__ var

@Aviral_ said in AttributeError: 'str' object has no attribute 'toordinal':

fdate = '2001-01-01'
todate = '2007-01-01'
# Data feed
data0 = bt.feeds.YahooFinanceCSVData(dataname=data0_path, fromdate=fdate, todate=todate)

It seems you are passing the fromdate and todate parameters to YahooFinanceCSVData as strings. Please try to pass it as a python datetime objects.

Talking about HW specs makes it a completely different discussion of cause, however with all that said:

i7-6500U ( 4 cores hyperthreaded) + 12GB buys you a decent machine for development and basic backtesting probably. However you'll hit its limits pretty fast (if not already) if you start optimizing.

In my case, a similar machine is utilized for development but for optimizing a more capable server is used ( old dual Xeon E5-2690 v2 - 48 core + 256 GB + small NVMe drive ). Usually I'm running optimizations with up to 4096 configs - so something that will take months to run on my dev machine may take day or less on this server.

So the answer is yes - more capable, dedicated machine may improve things for heavy loads. And it is actually not that expensive - got my for less that 1500$ on eBay.