dislib.optimization.ADMM

ADMM Lasso

@Authors: Aleksandar Armacki and Lidija Fodor @Affiliation: Faculty of Sciences, University of Novi Sad, Serbia

This work is supported by the I-BiDaaS project, funded by the European Commission under Grant Agreement No. 780787.

class dislib.optimization.admm.base.ADMM(loss_fn, k, rho=1, max_iter=100, rtol=0.01, atol=0.0001, verbose=False)

Bases: BaseEstimator

Alternating Direction Method of Multipliers (ADMM) solver. ADMM is renowned for being well suited to the distributed settings [1], for its guaranteed convergence and general robustness with respect to the parameters. Additionally, the algorithm has a generic form that can be easily adapted to a wide range of machine learning problems with only minor tweaks in the code.

Parameters:

• loss_fn (func) – Loss function.

• k (float) – Soft thresholding value.

• rho (float, optional (default=1)) – The penalty parameter for constraint violation.

• max_iter (int, optional (default=100)) – Maximum number of iterations to perform.

• atol (float, optional (default=1e-4)) – The absolute tolerance used to calculate the early stop criterion.

• rtol (float, optional (default=1e-2)) – The relative tolerance used to calculate the early stop criterion.

• verbose (boolean, optional (default=False)) – Whether to print information about the optimization process.

Variables:

• z (ds-array shape=(1, n_features)) – Computed z.

• n_iter (int) – Number of iterations performed.

• converged (boolean) – Whether the optimization converged.

References

[1]

S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein (2011). Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers. In Foundations and Trends in Machine Learning, 3(1):1–122.

fit(x, y)

Fits the model with training data.

Parameters:

• x (ds-array, shape=(n_samples, n_features)) – Training samples.

• y (ds-array, shape=(n_samples, 1)) – Class labels of x.

Returns:

self

Return type:

ADMM