Minimax Estimator - Problem Setup

Problem Setup

Consider the problem of estimating a deterministic (not Bayesian) parameter from noisy or corrupt data related through the conditional probability distribution . Our goal is to find a "good" estimator for estimating the parameter, which minimizes some given risk function . Here the risk function is the expectation of some loss function with respect to . A popular example for a loss function is the squared error loss, and the risk function for this loss is the mean squared error (MSE).

Unfortunately in general the risk cannot be minimized, since it depends on the unknown parameter itself (If we knew what was the actual value of, we wouldn't need to estimate it). Therefore additional criteria for finding an optimal estimator in some sense are required. One such criterion is the minimax criteria.