Fisher Information Metric - Definition

Definition

Given a statistical manifold, with coordinates given by, one writes for the probability distribution. Here, is a specific value drawn from a collection of (discrete or continuous) random variables X. The probability is normalized, in that

The Fisher information metric then takes the form:

g_{jk}(\theta) = \int_X \frac{\partial \log p(x,\theta)}{\partial \theta_j} \frac{\partial \log p(x,\theta)}{\partial \theta_k} p(x,\theta) \, dx.

The integral is performed over all values x of all random variables X. Again, the variable is understood as a coordinate on the Riemann manifold. The labels j and k index the local coordinate axes on the manifold.

When the probability is derived from the Gibbs measure, as it would be for any Markovian process, then can also be understood to be a Lagrange multiplier; Lagrange multipliers are used to enforce constraints, such as holding the expectation value of some quantity constant. If there are n constraints holding n different expectation values constant, then the manifold is n-dimensional. In this case, the metric can be explicitly derived from the partition function; a derivation and discussion is presented there.

Substituting from information theory, an equivalent form of the above definition is:

g_{jk}(\theta) = \int_X \frac{\partial^2 i(x,\theta)}{\partial \theta_j \partial \theta_k} p(x,\theta) \, dx = \mathrm{E} \left[ \frac{\partial^2 i(x,\theta)}{\partial \theta_j \partial \theta_k} \right].

Read more about this topic:  Fisher Information Metric

Famous quotes containing the word definition:

    “I’m beginning to think that the proper definition of “Man” is “an animal that writes letters.””
    —Lewis Carroll [Charles Lutwidge Dodgson] (1832–1898)

    “Perhaps the best definition of progress would be the continuing efforts of men and women to narrow the gap between the convenience of the powers that be and the unwritten charter.”
    —Nadine Gordimer (b. 1923)

    “... we all know the wag’s definition of a philanthropist: a man whose charity increases directly as the square of the distance.”
    —George Eliot [Mary Ann (or Marian)