We say that the function belongs to the class if it is times continuously differentiable on , and the derivative has a Lipschitz constant .
The most commonly used for . Notice that:
- If , then . The higher is the order of the derivative, the stronger is the limitation (fewer functions belong to the class).
Prove that the function belongs to the class if and only if :
Prove that the last condition can be rewritten in the form without loss of generality:
Show that for gradient descent with the following stepsize selection strategies:
- constant step
- Dropping sequence .
you can get the estimation of the function decrease at the iteration of the view:
- some constant, - Lipschitz constant of the function gradient.