Partition Function (mathematics) - Information Geometry

Information Geometry

The points can be understood to form a space, and specifically, a manifold. Thus, it is reasonable to ask about the structure of this manifold; this is the task of information geometry.

Multiple derivatives with regard to the lagrange multipliers gives rise to a positive semi-definite covariance matrix

g_{ij}(\beta) = \frac{\partial^2}{\partial \beta^i\partial \beta^j} \left(-\log Z(\beta)\right) = \langle \left(H_i-\langle H_i\rangle\right)\left( H_j-\langle H_j\rangle\right)\rangle

This matrix is positive semi-definite, and may be interpreted as a metric tensor, specifically, a Riemannian metric. Equiping the space of lagrange multipliers with a metric in this way turns it into a Riemannian manifold. The study of such manifolds is referred to as information geometry; the metric above is the Fisher information metric. Here, serves as a coordinate on the manifold. It is interesting to compare the above definition to the simpler Fisher information, from which it is inspired.

That the above defines the Fisher information metric can be readily seen by explicitly substituting for the expectation value:

\begin{align} g_{ij}(\beta) & = \langle \left(H_i-\langle H_i\rangle\right)\left( H_j-\langle H_j\rangle\right)\rangle \\ & = \sum_{x} P(x) \left(H_i-\langle H_i\rangle\right)\left( H_j-\langle H_j\rangle\right) \\ & = \sum_{x} P(x) \left(H_i + \frac{\partial\log Z}{\partial \beta_i}\right) \left(H_j + \frac{\partial\log Z}{\partial \beta_j}\right) \\ & = \sum_{x} P(x) \frac{\partial \log P(x)}{\partial \beta^i} \frac{\partial \log P(x)}{\partial \beta^j} \\ \end{align}

where we've written for and the summation is understood to be over all values of all random variables . For continuous-valued random variables, the summations are replaced by integrals, of course.

Curiously, the Fisher information metric can also be understood as the flat-space Euclidean metric, after appropriate change of variables, as described in the main article on it. When the are complex-valued, the resulting metric is the Fubini-Study metric. When written in terms of mixed states, instead of pure states, it is known as the Bures metric.

Read more about this topic:  Partition Function (mathematics)

Famous quotes containing the words information and/or geometry:

    “So while it is true that children are exposed to more information and a greater variety of experiences than were children of the past, it does not follow that they automatically become more sophisticated. We always know much more than we understand, and with the torrent of information to which young people are exposed, the gap between knowing and understanding, between experience and learning, has become even greater than it was in the past.”
    —David Elkind (20th century)

    “... geometry became a symbol for human relations, except that it was better, because in geometry things never go bad. If certain things occur, if certain lines meet, an angle is born. You cannot fail. It’s not going to fail; it is eternal. I found in rules of mathematics a peace and a trust that I could not place in human beings. This sublimation was total and remained total. Thus, I’m able to avoid or manipulate or process pain.”
    —Louise Bourgeois (b. 1911)