Gradient-based methods¶
Proximal-Gradient¶
The proximal-gradient method [BT2009], [N2013] is a method to solve problems of the form
where $f$ is a differentiable function for which we have access to its gradient and $g$ is a potentially non-smooth function for which we have access to its proximal operator.
|
Proximal gradient descent. |
Examples
References
- BT2009
Beck, Amir, and Marc Teboulle. “Gradient-based algorithms with applications to signal recovery.” Convex optimization in signal processing and communications (2009)
- N2013
Nesterov, Yu. “Gradient methods for minimizing composite functions.” Mathematical Programming 140.1 (2013): 125-161.
Primal-dual hybrid gradient¶
The primal-dual hybrid gradient method [C2013] [V2013] [CP2016] is a method to solve problems of the form
where $f$ is a differentiable function for which we have access to its gradient and $g$ and $h$ are potentially non-smooth functions for which we have access to their proximal operator.
|
Primal-dual hybrid gradient splitting method. |
Examples
References
- C2013
Condat, Laurent. “A primal–dual splitting method for convex optimization involving Lipschitzian, proximable and linear composite terms.” Journal of Optimization Theory and Applications 158.2 (2013): 460-479.
- V2013
Vũ, Bằng Công. “A splitting algorithm for dual monotone inclusions involving cocoercive operators.” Advances in Computational Mathematics 38.3 (2013)
- CP2016
Chambolle, Antonin, and Thomas Pock. “An introduction to continuous optimization for imaging.” Acta Numerica 25 (2016)
Three-operator splitting¶
The three operator splitting [DY2017] [PG2018] is a method to solve problems of the form
where $f$ is a differentiable function for which we have access to its gradient and $g$ and $h$ are potentially non-smooth functions for which we have access to their proximal operator.
|
Davis-Yin three operator splitting method. |
Examples
References
- DY2017
Davis, Damek, and Wotao Yin. “A three-operator splitting scheme and its optimization applications.” Set-Valued and Variational Analysis, 2017.
- PG2018
Pedregosa, Fabian, and Gauthier Gidel. “Adaptive Three Operator Splitting.” Proceedings of the 35th International Conference on Machine Learning, 2018.