Fisher Information Metric - Formal Definition

Formal Definition

A slightly more formal, abstract definition can be given, as follows.

Let X be an orientable manifold, and let be a measure on X. Equivalently, let be a probability space on, with sigma algebra and probability .

The statistical manifold S(X) of X is defined as the space of all measures on X (with the sigma-algebra held fixed). Note that this space is infinite-dimensional, and is commonly taken to be a Frechet space. The points of S(X) are measures.

Pick a point and consider the tangent space . The Fisher information metric is then an inner product on the tangent space. With some abuse of notation, one may write this as

Here, and are vectors in the tangent space; that is, . The abuse of notation is to write the tangent vectors as if they are derivatives, and to insert the extraneous d in writing the integral: the integration is meant to be carried out using the measure over the whole space X.

This definition of the metric can be seen to be equivalent to the previous, in several steps. First, one selects a submanifold of S(X) by considering only those measures that are parameterized by some smoothly varying parameter . Then, if is finite-dimensional, then so is the submanifold; likewise, the tangent space has the same dimension as .

With some additional abuse of language, one notes that the exponential map provides a map from vectors in a tangent space to points in an underlying manifold. Thus, if is a vector in the tangent space, then is the corresponding probability associated with point (after the parallel transport of the exponential map to .) Conversely, given a point, the logarithm gives a point in the tangent space (roughly speaking, as again, one must transport from the origin to point ; for details, refer to original sources). Thus, one has the appearance of logarithms in the simpler definition, previously given.