Formal Definition
A slightly more formal, abstract definition can be given, as follows.
Let X be an orientable manifold, and let be a measure on X. Equivalently, let be a probability space on, with sigma algebra and probability .
The statistical manifold S(X) of X is defined as the space of all measures on X (with the sigma-algebra held fixed). Note that this space is infinite-dimensional, and is commonly taken to be a Frechet space. The points of S(X) are measures.
Pick a point and consider the tangent space . The Fisher information metric is then an inner product on the tangent space. With some abuse of notation, one may write this as
Here, and are vectors in the tangent space; that is, . The abuse of notation is to write the tangent vectors as if they are derivatives, and to insert the extraneous d in writing the integral: the integration is meant to be carried out using the measure over the whole space X.
This definition of the metric can be seen to be equivalent to the previous, in several steps. First, one selects a submanifold of S(X) by considering only those measures that are parameterized by some smoothly varying parameter . Then, if is finite-dimensional, then so is the submanifold; likewise, the tangent space has the same dimension as .
With some additional abuse of language, one notes that the exponential map provides a map from vectors in a tangent space to points in an underlying manifold. Thus, if is a vector in the tangent space, then is the corresponding probability associated with point (after the parallel transport of the exponential map to .) Conversely, given a point, the logarithm gives a point in the tangent space (roughly speaking, as again, one must transport from the origin to point ; for details, refer to original sources). Thus, one has the appearance of logarithms in the simpler definition, previously given.
Read more about this topic: Fisher Information Metric
Famous quotes containing the words formal and/or definition:
“This is no argument against teaching manners to the young. On the contrary, it is a fine old tradition that ought to be resurrected from its current mothballs and put to work...In fact, children are much more comfortable when they know the guide rules for handling the social amenities. It’s no more fun for a child to be introduced to a strange adult and have no idea what to say or do than it is for a grownup to go to a formal dinner and have no idea what fork to use.”
—Leontine Young (20th century)
“Mothers often are too easily intimidated by their children’s negative reactions...When the child cries or is unhappy, the mother reads this as meaning that she is a failure. This is why it is so important for a mother to know...that the process of growing up involves by definition things that her child is not going to like. Her job is not to create a bed of roses, but to help him learn how to pick his way through the thorns.”
—Elaine Heffner (20th century)