Convex Function - Strongly Convex Functions

Strongly Convex Functions

The concept of strong convexity extends and parametrizes the notion of strict convexity. A strongly convex function is also strictly convex, but not vice-versa.

A differentiable function f is called strongly convex with parameter m > 0 if the following inequality holds for all points x,y in its domain:

Some authors, such as refer to functions satisfying this inequality as elliptic functions.

An equivalent condition is the following:

It is not necessary for a function to be differentiable in order to be strongly convex. A third definition for a strongly convex function, with parameter m, is that, for all x,y in the domain and ,

Notice that this definition approaches the definition for strict convexity as, and is identical to the definition of a convex function when m = 0. Despite this, functions exist that are strictly convex but are not strongly convex for any m > 0 (see example below).

If the function f is twice continuously differentiable, then f is strongly convex with parameter m if and only if for all x in the domain, where I is the identity and is the Hessian matrix, and the inequality means that is positive definite. This is equivalent to requiring that the minimum eigenvalue of be at least m for all x. If the domain is just the real line, then is just the second derivative, so the condition becomes . If m = 0, then this means the Hessian is positive semidefinite (or if the domain is the real line, it means that ), which implies the function is convex, and perhaps strictly convex, but not strongly convex.

Assuming still that the function is twice continuously differentiable, we show that the lower bound of implies that it is strongly convex. Start by using Taylor's Theorem:

for some (unknown) . Then by the assumption about the eigenvalues, and hence we recover the second strong convexity equation above.

The distinction between convex, strictly convex, and strongly convex can be subtle at first glimpse. If is twice continuously differentiable and the domain is the real line, then we can characterize it as follows:

convex if and only if for all

strictly convex if for all (note: this is sufficient, but not necessary)

strongly convex if and only if for all

For example, consider a function that is strictly convex, and suppose there is a sequence of points such that . Even though, the function is not strongly convex because will become arbitrarily small.

Strongly convex functions are in general easier to work with than convex or strictly convex functions, since they are a smaller class. Like strictly convex functions, strongly convex functions have unique minima.