Covariance - Definition

Definition

The covariance between two jointly distributed real-valued random variables x and y with finite second moments is defined as

\sigma(x,y) = \operatorname{E}{\big)(y - \operatorname{E})\big]},

where E is the expected value of x, also known as the mean of x. By using the linearity property of expectations, this can be simplified to

\begin{align} \sigma(x,y) &= \operatorname{E}\left\right) \left(y - \operatorname{E}\left\right)\right] \\ &= \operatorname{E}\left - \operatorname{E}\left y + \operatorname{E}\left \operatorname{E}\left\right] \\ &= \operatorname{E}\left - \operatorname{E}\left \operatorname{E}\left - \operatorname{E}\left \operatorname{E}\left + \operatorname{E}\left \operatorname{E}\left \\ &= \operatorname{E}\left - \operatorname{E}\left \operatorname{E}\left, \end{align}

where is the correlation between x and y; if this term is zero then the random variables are orthogonal.

For random vectors and (of dimension m and n respectively) the m×n covariance matrix is equal to

\begin{align} \sigma(\mathbf{x},\mathbf{y}) & = \operatorname{E} \left) (\mathbf{y} - \operatorname{E})^\mathrm{T}\right]\\ & = \operatorname{E}\left - \operatorname{E}\operatorname{E}^\mathrm{T}, \end{align}

where mT is the transpose of the vector (or matrix) m.

The (i,j)-th element of this matrix is equal to the covariance Cov(xi, yj) between the i-th scalar component of x and the j-th scalar component of y. In particular, Cov(y, x) is the transpose of Cov(x, y).

For a vector \mathbf{x}= \begin{bmatrix}x_1 & x_2 & \dots & x_m\end{bmatrix}^\mathrm{T} of m jointly distributed random variables with finite second moments, its covariance matrix is defined as

Random variables whose covariance is zero are called uncorrelated.

The units of measurement of the covariance Cov(x, y) are those of x times those of y. By contrast, correlation coefficients, which depend on the covariance, are a dimensionless measure of linear dependence. (In fact, correlation coefficients can simply be understood as a normalized version of covariance.)

Read more about this topic:  Covariance

Famous quotes containing the word definition:

    “The definition of good prose is proper words in their proper places; of good verse, the most proper words in their proper places. The propriety is in either case relative. The words in prose ought to express the intended meaning, and no more; if they attract attention to themselves, it is, in general, a fault.”
    —Samuel Taylor Coleridge (1772–1834)

    “Beauty, like all other qualities presented to human experience, is relative; and the definition of it becomes unmeaning and useless in proportion to its abstractness. To define beauty not in the most abstract, but in the most concrete terms possible, not to find a universal formula for it, but the formula which expresses most adequately this or that special manifestation of it, is the aim of the true student of aesthetics.”
    —Walter Pater (1839–1894)

    “... we all know the wag’s definition of a philanthropist: a man whose charity increases directly as the square of the distance.”
    —George Eliot [Mary Ann (or Marian)