Abstract:
An envelope called an accelerated meta-algorithm is proposed. Based on the envelope, accelerated methods for solving convex unconstrained minimization problems in various formulations can be obtained from nonaccelerated versions in a unified manner. Quasi-optimal algorithms for minimizing smooth functions with Lipschitz continuous derivatives of arbitrary order and for solving smooth minimax problems are given as applications. The proposed envelope is more general than existing ones. Moreover, better convergence estimates can be obtained in the case of this envelope and better efficiency can be achieved in practice for a number of problem formulations.
The work by A. Gasnikov (Section 2) was supported by the Russian Foundation for Basic Research (project no. 18-31-20005 mol_a_ved), Kamzolov’s work (Section 3) was supported by the Russian Foundation for Basic Research (project no. 19-31-90170 Aspiranty), and Dvurechensky’s work (Section 3) was supported by the Russian Foundation for Basic Research (project no. 18-29-03071 mk). Dvinskikh and Matyukhin acknowledge the support of the Ministry of Science and Higher Education of the Russian Federation, state assignment no. 075-00337-20-03, project no. 0714-2020-0005.
Citation:
A. V. Gasnikov, D. M. Dvinskikh, P. E. Dvurechenskii, D. Kamzolov, V. V. Matyukhin, D. A. Pasechnyuk, N. K. Tupitsa, A. V. Chernov, “Accelerated meta-algorithm for convex optimization problems”, Zh. Vychisl. Mat. Mat. Fiz., 61:1 (2021), 20–31; Comput. Math. Math. Phys., 61:1 (2021), 17–28
This publication is cited in the following 13 articles:
M. S. Alkousa, A. V. Gasnikov, E. L. Gladin, I. A. Kuruzov, D. A. Pasechnyuk, F. S. Stonyakin, “Solving strongly convex-concave composite saddle-point problems with low dimension of one group of variable”, Sb. Math., 214:3 (2023), 285–333
Ya. D. Tominin, V. D. Tominin, E. D. Borodich, D. A. Kovalev, P. E. Dvurechenskii, A. V. Gasnikov, S. V. Chukanov, “On accelerated methods for saddle-point problems with composite structure”, Kompyuternye issledovaniya i modelirovanie, 15:2 (2023), 433–467
Artem Agafonov, Dmitry Kamzolov, Pavel Dvurechensky, Alexander Gasnikov, Martin Takáč, “Inexact tensor methods and their application to stochastic convex optimization”, Optimization Methods and Software, 2023, 1
Dmitry Kamzolov, Alexander Gasnikov, Pavel Dvurechensky, Artem Agafonov, Martin Takáč, Encyclopedia of Optimization, 2023, 1
E. L. Gladin, E. D. Borodich, “Variance reduction for minimax problems with a small dimension of one of the variables”, Kompyuternye issledovaniya i modelirovanie, 14:2 (2022), 257–275
Anastasiya Ivanova, Pavel Dvurechensky, Evgeniya Vorontsova, Dmitry Pasechnyuk, Alexander Gasnikov, Darina Dvinskikh, Alexander Tyurin, “Oracle Complexity Separation in Convex Optimization”, J Optim Theory Appl, 193:1-3 (2022), 462
Ekaterina Borodich, Vladislav Tominin, Yaroslav Tominin, Dmitry Kovalev, Alexander Gasnikov, Pavel Dvurechensky, “Accelerated variance-reduced methods for saddle-point problems”, EURO Journal on Computational Optimization, 10 (2022), 100048
A. S. Anikin, V. V. Matyukhin, D. A. Pasechnyuk, “Accelerated proximal envelopes: application to componentwise methods”, Comput. Math. Math. Phys., 62:2 (2022), 336–345
Fedor Stonyakin, Alexander Gasnikov, Pavel Dvurechensky, Alexander Titov, Mohammad Alkousa, “Generalized Mirror Prox Algorithm for Monotone Variational Inequalities: Universality and Inexact Oracle”, J Optim Theory Appl, 194:3 (2022), 988
Eduard Gorbunov, Alexander Rogozin, Aleksandr Beznosikov, Darina Dvinskikh, Alexander Gasnikov, Springer Optimization and Its Applications, 191, High-Dimensional Optimization and Probability, 2022, 253
Anastasiya Ivanova, Dmitry Pasechnyuk, Dmitry Grishchenko, Egor Shulgin, Alexander Gasnikov, Vladislav Matyukhin, Lecture Notes in Computer Science, 13078, Optimization and Applications, 2021, 20
Alexander Rogozin, Mikhail Bochko, Pavel Dvurechensky, Alexander Gasnikov, Vladislav Lukoshkin, 2021 60th IEEE Conference on Decision and Control (CDC), 2021, 3367
A. Birjukov, A. Chernov, Communications in Computer and Information Science, 1514, Advances in Optimization and Applications, 2021, 3