Fisher Information - Matrix Form

Matrix Form

When there are N parameters, so that θ is a N × 1 vector \theta = \begin{bmatrix} \theta_{1}, \theta_{2}, \dots, \theta_{N} \end{bmatrix}^{\mathrm T}, then the Fisher information takes the form of an N × N matrix, the Fisher Information Matrix (FIM), with typical element

{\left(\mathcal{I} \left(\theta \right) \right)}_{i, j} = \operatorname{E} \left[\left. \left(\frac{\partial}{\partial\theta_i} \log f(X;\theta)\right) \left(\frac{\partial}{\partial\theta_j} \log f(X;\theta)\right) \right|\theta\right].

The FIM is a N × N positive semidefinite symmetric matrix, defining a Riemannian metric on the N-dimensional parameter space, thus connecting Fisher information to differential geometry. In that context, this metric is known as the Fisher information metric, and the topic is called information geometry.

Under certain regularity conditions, the Fisher Information Matrix may also be written as

{\left(\mathcal{I} \left(\theta \right) \right)}_{i, j} = - \operatorname{E} \left[\left. \frac{\partial^2}{\partial\theta_i \, \partial\theta_j} \log f(X;\theta) \right|\theta\right]\,.

The metric is interesting in several ways; it can be derived as the Hessian of the relative entropy; it can be understood as a metric induced from the Euclidean metric, after appropriate change of variable; in its complex-valued form, it is the Fubini-Study metric.

Read more about this topic:  Fisher Information

Famous quotes containing the words matrix and/or form:

    ““The matrix is God?”
    “In a manner of speaking, although it would be more accurate ... to say that the matrix has a God, since this being’s omniscience and omnipotence are assumed to be limited to the matrix.”
    “If it has limits, it isn’t omnipotent.”
    “Exactly.... Cyberspace exists, insofar as it can be said to exist, by virtue of human agency.””
    —William Gibson (b. 1948)

    “[One cannot express lack of knowledge in affirmative language.] This idea is more firmly grasped in the form of interrogation: “What do I know?”Mthe words I bear as a motto, inscribed over a pair of scales.”
    —Michel de Montaigne (1533–1592)