Observed Information - Definition

Definition

Suppose we observe random variables, independent and identically distributed with density f(X; θ), where θ is a (possibly unknown) vector. Then the log-likelihood of the parameters given the data is

.

We define the observed information matrix at as

\mathcal{J}(\theta^*) = - \left. \nabla \nabla^{\top} \ell(\theta) \right|_{\theta=\theta^*}
= - \left. \left( \begin{array}{cccc} \tfrac{\partial^2}{\partial \theta_1^2} & \tfrac{\partial^2}{\partial \theta_1 \partial \theta_2} & \cdots & \tfrac{\partial^2}{\partial \theta_1 \partial \theta_n} \\ \tfrac{\partial^2}{\partial \theta_2 \partial \theta_1} & \tfrac{\partial^2}{\partial \theta_2^2} & \cdots & \tfrac{\partial^2}{\partial \theta_2 \partial \theta_n} \\ \vdots & \vdots & \ddots & \vdots \\ \tfrac{\partial^2}{\partial \theta_n \partial \theta_1} & \tfrac{\partial^2}{\partial \theta_n \partial \theta_2} & \cdots & \tfrac{\partial^2}{\partial \theta_n^2} \\ \end{array} \right) \ell(\theta) \right|_{\theta = \theta^*}

In many instances, the observed information is evaluated at the maximum-likelihood estimate.

Read more about this topic:  Observed Information

Famous quotes containing the word definition:

    “Beauty, like all other qualities presented to human experience, is relative; and the definition of it becomes unmeaning and useless in proportion to its abstractness. To define beauty not in the most abstract, but in the most concrete terms possible, not to find a universal formula for it, but the formula which expresses most adequately this or that special manifestation of it, is the aim of the true student of aesthetics.”
    —Walter Pater (1839–1894)

    “The physicians say, they are not materialists; but they are:MSpirit is matter reduced to an extreme thinness: O so thin!—But the definition of spiritual should be, that which is its own evidence. What notions do they attach to love! what to religion! One would not willingly pronounce these words in their hearing, and give them the occasion to profane them.”
    —Ralph Waldo Emerson (1803–1882)

    “I’m beginning to think that the proper definition of “Man” is “an animal that writes letters.””
    —Lewis Carroll [Charles Lutwidge Dodgson] (1832–1898)