Variance Inflation Factor - Definition

Definition

Consider the following linear model with k independent variables:

Y = β0 + β1 X1 + β2 X 2 + ... + βk Xk + ε.

The standard error of the estimate of βj is the square root of the j+1, j+1 element of s2(X′X)−1, where s is the root mean squared error (RMSE) (note that RMSE2 is an unbiased estimator of the true variance of the error term, ); X is the regression design matrix — a matrix such that Xi, j+1 is the value of the jth independent variable for the ith case or observation, and such that Xi, 1 equals 1 for all i. It turns out that the square of this standard error, the estimated variance of the estimate of βj, can be equivalently expressed as

{\rm \hat{var}}(\hat{\beta}_j) = \frac{s^2}{(n-1)\widehat{\rm var}(X_j)}\cdot \frac{1}{1-R_j^2},

where Rj2 is the multiple R2 for the regression of Xj on the other covariates (a regression that does not involve the response variable Y). This identity separates the influences of several distinct factors on the variance of the coefficient estimate:

  • s2: greater scatter in the data around the regression surface leads to proportionately more variance in the coefficient estimates
  • n: greater sample size results in proportionately less variance in the coefficient estimates
  • : greater variability in a particular covariate leads to proportionately less variance in the corresponding coefficient estimate

The remaining term, 1 / (1 − Rj2) is the VIF. It reflects all other factors that influence the uncertainty in the coefficient estimates. The VIF equals 1 when the vector Xj is orthogonal to each column of the design matrix for the regression of Xj on the other covariates. By contrast, the VIF is greater than 1 when the vector Xj is not orthogonal to all columns of the design matrix for the regression of Xj on the other covariates. Finally, note that the VIF is invariant to the scaling of the variables (that is, we could scale each variable Xj by a constant cj without changing the VIF).

Read more about this topic:  Variance Inflation Factor

Famous quotes containing the word definition:

    “It’s a rare parent who can see his or her child clearly and objectively. At a school board meeting I attended . . . the only definition of a gifted child on which everyone in the audience could agree was “mine.””
    —Jane Adams (20th century)

    “One definition of man is “an intelligence served by organs.””
    —Ralph Waldo Emerson (1803–1882)

    “... if, as women, we accept a philosophy of history that asserts that women are by definition assimilated into the male universal, that we can understand our past through a male lens—if we are unaware that women even have a history—we live our lives similarly unanchored, drifting in response to a veering wind of myth and bias.”
    —Adrienne Rich (b. 1929)