Partition Function (mathematics) - Information Geometry

Information Geometry

The points can be understood to form a space, and specifically, a manifold. Thus, it is reasonable to ask about the structure of this manifold; this is the task of information geometry.

Multiple derivatives with regard to the lagrange multipliers gives rise to a positive semi-definite covariance matrix

g_{ij}(\beta) = \frac{\partial^2}{\partial \beta^i\partial \beta^j} \left(-\log Z(\beta)\right) = \langle \left(H_i-\langle H_i\rangle\right)\left( H_j-\langle H_j\rangle\right)\rangle

This matrix is positive semi-definite, and may be interpreted as a metric tensor, specifically, a Riemannian metric. Equiping the space of lagrange multipliers with a metric in this way turns it into a Riemannian manifold. The study of such manifolds is referred to as information geometry; the metric above is the Fisher information metric. Here, serves as a coordinate on the manifold. It is interesting to compare the above definition to the simpler Fisher information, from which it is inspired.

That the above defines the Fisher information metric can be readily seen by explicitly substituting for the expectation value:

\begin{align} g_{ij}(\beta) & = \langle \left(H_i-\langle H_i\rangle\right)\left( H_j-\langle H_j\rangle\right)\rangle \\ & = \sum_{x} P(x) \left(H_i-\langle H_i\rangle\right)\left( H_j-\langle H_j\rangle\right) \\ & = \sum_{x} P(x) \left(H_i + \frac{\partial\log Z}{\partial \beta_i}\right) \left(H_j + \frac{\partial\log Z}{\partial \beta_j}\right) \\ & = \sum_{x} P(x) \frac{\partial \log P(x)}{\partial \beta^i} \frac{\partial \log P(x)}{\partial \beta^j} \\ \end{align}

where we've written for and the summation is understood to be over all values of all random variables . For continuous-valued random variables, the summations are replaced by integrals, of course.

Curiously, the Fisher information metric can also be understood as the flat-space Euclidean metric, after appropriate change of variables, as described in the main article on it. When the are complex-valued, the resulting metric is the Fubini-Study metric. When written in terms of mixed states, instead of pure states, it is known as the Bures metric.

Read more about this topic:  Partition Function (mathematics)

Famous quotes containing the words information and/or geometry:

    “Computers are good at swift, accurate computation and at storing great masses of information. The brain, on the other hand, is not as efficient a number cruncher and its memory is often highly fallible; a basic inexactness is built into its design. The brain’s strong point is its flexibility. It is unsurpassed at making shrewd guesses and at grasping the total meaning of information presented to it.”
    —Jeremy Campbell (b. 1931)

    “... geometry became a symbol for human relations, except that it was better, because in geometry things never go bad. If certain things occur, if certain lines meet, an angle is born. You cannot fail. It’s not going to fail; it is eternal. I found in rules of mathematics a peace and a trust that I could not place in human beings. This sublimation was total and remained total. Thus, I’m able to avoid or manipulate or process pain.”
    —Louise Bourgeois (b. 1911)