Fisher Information Metric - Definition

Definition

Given a statistical manifold, with coordinates given by, one writes for the probability distribution. Here, is a specific value drawn from a collection of (discrete or continuous) random variables X. The probability is normalized, in that

The Fisher information metric then takes the form:

g_{jk}(\theta) = \int_X \frac{\partial \log p(x,\theta)}{\partial \theta_j} \frac{\partial \log p(x,\theta)}{\partial \theta_k} p(x,\theta) \, dx.

The integral is performed over all values x of all random variables X. Again, the variable is understood as a coordinate on the Riemann manifold. The labels j and k index the local coordinate axes on the manifold.

When the probability is derived from the Gibbs measure, as it would be for any Markovian process, then can also be understood to be a Lagrange multiplier; Lagrange multipliers are used to enforce constraints, such as holding the expectation value of some quantity constant. If there are n constraints holding n different expectation values constant, then the manifold is n-dimensional. In this case, the metric can be explicitly derived from the partition function; a derivation and discussion is presented there.

Substituting from information theory, an equivalent form of the above definition is:

g_{jk}(\theta) = \int_X \frac{\partial^2 i(x,\theta)}{\partial \theta_j \partial \theta_k} p(x,\theta) \, dx = \mathrm{E} \left[ \frac{\partial^2 i(x,\theta)}{\partial \theta_j \partial \theta_k} \right].

Read more about this topic:  Fisher Information Metric

Famous quotes containing the word definition:

    “Mothers often are too easily intimidated by their children’s negative reactions...When the child cries or is unhappy, the mother reads this as meaning that she is a failure. This is why it is so important for a mother to know...that the process of growing up involves by definition things that her child is not going to like. Her job is not to create a bed of roses, but to help him learn how to pick his way through the thorns.”
    —Elaine Heffner (20th century)

    “Beauty, like all other qualities presented to human experience, is relative; and the definition of it becomes unmeaning and useless in proportion to its abstractness. To define beauty not in the most abstract, but in the most concrete terms possible, not to find a universal formula for it, but the formula which expresses most adequately this or that special manifestation of it, is the aim of the true student of aesthetics.”
    —Walter Pater (1839–1894)

    “No man, not even a doctor, ever gives any other definition of what a nurse should be than this—”devoted and obedient.” This definition would do just as well for a porter. It might even do for a horse. It would not do for a policeman.”
    —Florence Nightingale (1820–1910)