Propensity Score - Advantages and Disadvantages

Advantages and Disadvantages

PSM, like any matching procedure, enables estimation of an average treatment effect from observational data. The key advantages of PSM were, at the time of its introduction, that by creating a linear combination of covariates into a single score it allowed researchers to balance treatment and control groups on a large number of covariates without losing a large number of observations. If units in the treatment and control were balanced on a large number of covariates one at a time, large numbers of observations would be needed to overcome the "dimensionality problem" whereby the introduction of a new balancing covariate increases the minimum necessary number of observations in the sample geometrically.

Disadvantages of PSM are many. Among the most critical disadvantage is that PSM can only account for observed (and observable) covariates. Factors that affect assignment to treatment but that cannot be observed cannot be accounted for in the matching procedure. Shadish, Cook, & Campbell (2002) additionally argue that PSM requires large samples, overlap between treatment and control groups must be substantial, and hidden bias may remain after matching because the procedure only controls for observed variables (to the extent that they are perfectly measured).

General concerns with matching have also been raised by Judea Pearl, who has argued that hidden bias may actually increase because matching on observed variables may unleash bias due to dormant unobserved confounders. Similarly, Pearl has argued that bias reduction can only be assured (asymptotically) by modeling the qualitative causal relationships between treatment, outcome, observed and unobserved covariates.