It is known, that antigradient is the direction of the steepest descent of the function at point . However, we can introduce another concept for choosing the best direction of function decreasing.

Given and a point . Define as the set of points with distance to . Here we presume the existence of a distance function .

Than, we can define another steepest descent direction in terms of minimizer of function on a sphere:

Let us assume that the distance is defined locally by some metric :

Let us also consider first order Taylor approximation of a function near the point :

Now we can explicitly pose a problem of finding , as it was stated above.

Using it can be written as:

Using Lagrange multipliers method, we can easily conclude, that the answer is:

Which means, that new direction of steepest descent is nothing else, but .

Indeed, if the space is isotropic and , we immediately have gradient descent formula, while Newton method uses local Hessian as a metric matrix.


Table of contents