Abstract:
A new concept of (δ,L) -model of a function that is a generalization of the Devolder–Glineur–Nesterov (δ,L)-oracle is proposed. Within this concept, the gradient descent and fast gradient descent methods are constructed and it is shown that constructs of many known methods (composite methods, level methods, conditional gradient and proximal methods) are particular cases of the methods proposed in this paper.
Key words:
gradient descent, fast gradient descent, model of function, universal method, conditional gradient method, composite optimization.
The work by Tyurin was supported by the program of support of leading Russian universities, project no. 5-100. The work by Gasnikov was supported by the Russian Foundation for Basic Research, project no. 18-31-20005 mol_a_ved (the main part of the paper) and by the Russian Science Foundation, project 17-11-01027 (the Appendix).
Citation:
A. V. Gasnikov, A. I. Turin, “Fast gradient descent for convex minimization problems with an oracle producing a (δ,L)-model of function at the requested point”, Zh. Vychisl. Mat. Mat. Fiz., 59:7 (2019), 1137–1150; Comput. Math. Math. Phys., 59:7 (2019), 1085–1097