Fisher Information Metric - Definition

Definition

Given a statistical manifold, with coordinates given by, one writes for the probability distribution. Here, is a specific value drawn from a collection of (discrete or continuous) random variables X. The probability is normalized, in that

The Fisher information metric then takes the form:

g_{jk}(\theta) = \int_X \frac{\partial \log p(x,\theta)}{\partial \theta_j} \frac{\partial \log p(x,\theta)}{\partial \theta_k} p(x,\theta) \, dx.

The integral is performed over all values x of all random variables X. Again, the variable is understood as a coordinate on the Riemann manifold. The labels j and k index the local coordinate axes on the manifold.

When the probability is derived from the Gibbs measure, as it would be for any Markovian process, then can also be understood to be a Lagrange multiplier; Lagrange multipliers are used to enforce constraints, such as holding the expectation value of some quantity constant. If there are n constraints holding n different expectation values constant, then the manifold is n-dimensional. In this case, the metric can be explicitly derived from the partition function; a derivation and discussion is presented there.

Substituting from information theory, an equivalent form of the above definition is:

g_{jk}(\theta) = \int_X \frac{\partial^2 i(x,\theta)}{\partial \theta_j \partial \theta_k} p(x,\theta) \, dx = \mathrm{E} \left[ \frac{\partial^2 i(x,\theta)}{\partial \theta_j \partial \theta_k} \right].

Read more about this topic:  Fisher Information Metric

Famous quotes containing the word definition:

    “The man who knows governments most completely is he who troubles himself least about a definition which shall give their essence. Enjoying an intimate acquaintance with all their particularities in turn, he would naturally regard an abstract conception in which these were unified as a thing more misleading than enlightening.”
    —William James (1842–1910)

    “Perhaps the best definition of progress would be the continuing efforts of men and women to narrow the gap between the convenience of the powers that be and the unwritten charter.”
    —Nadine Gordimer (b. 1923)

    “According to our social pyramid, all men who feel displaced racially, culturally, and/or because of economic hardships will turn on those whom they feel they can order and humiliate, usually women, children, and animals—just as they have been ordered and humiliated by those privileged few who are in power. However, this definition does not explain why there are privileged men who behave this way toward women.”
    —Ana Castillo (b. 1953)