Observed Information - Definition

Definition

Suppose we observe random variables, independent and identically distributed with density f(X; θ), where θ is a (possibly unknown) vector. Then the log-likelihood of the parameters given the data is

.

We define the observed information matrix at as

\mathcal{J}(\theta^*) = - \left. \nabla \nabla^{\top} \ell(\theta) \right|_{\theta=\theta^*}
= - \left. \left( \begin{array}{cccc} \tfrac{\partial^2}{\partial \theta_1^2} & \tfrac{\partial^2}{\partial \theta_1 \partial \theta_2} & \cdots & \tfrac{\partial^2}{\partial \theta_1 \partial \theta_n} \\ \tfrac{\partial^2}{\partial \theta_2 \partial \theta_1} & \tfrac{\partial^2}{\partial \theta_2^2} & \cdots & \tfrac{\partial^2}{\partial \theta_2 \partial \theta_n} \\ \vdots & \vdots & \ddots & \vdots \\ \tfrac{\partial^2}{\partial \theta_n \partial \theta_1} & \tfrac{\partial^2}{\partial \theta_n \partial \theta_2} & \cdots & \tfrac{\partial^2}{\partial \theta_n^2} \\ \end{array} \right) \ell(\theta) \right|_{\theta = \theta^*}

In many instances, the observed information is evaluated at the maximum-likelihood estimate.

Read more about this topic:  Observed Information

Famous quotes containing the word definition:

    “According to our social pyramid, all men who feel displaced racially, culturally, and/or because of economic hardships will turn on those whom they feel they can order and humiliate, usually women, children, and animals—just as they have been ordered and humiliated by those privileged few who are in power. However, this definition does not explain why there are privileged men who behave this way toward women.”
    —Ana Castillo (b. 1953)

    “I’m beginning to think that the proper definition of “Man” is “an animal that writes letters.””
    —Lewis Carroll [Charles Lutwidge Dodgson] (1832–1898)

    “The man who knows governments most completely is he who troubles himself least about a definition which shall give their essence. Enjoying an intimate acquaintance with all their particularities in turn, he would naturally regard an abstract conception in which these were unified as a thing more misleading than enlightening.”
    —William James (1842–1910)