Empirical Measure - Definition

Let be a sequence of independent identically distributed random variables with values in the state space S with probability measure P.

Definition

The empirical measure is defined for measurable subsets of S and given by
where is the indicator function and is the Dirac measure.

For a fixed measurable set A, nPn(A) is a binomial random variable with mean nP(A) and variance nP(A)(1 − P(A)). In particular, is an unbiased estimator of P(A).

Definition

is the empirical measure indexed by, a collection of measurable subsets of S.

To generalize this notion further, observe that the empirical measure Pn maps measurable functions to their empirical mean,

In particular, the empirical measure of A is simply the empirical mean of the indicator function, .

For a fixed measurable function f, is a random variable with mean and variance .

By the strong law of large numbers, converges to P(A) almost surely for fixed A. Similarly converges to almost surely for a fixed measurable function f. The problem of uniform convergence of to P was open until Vapnik and Chervonenkis solved it in 1968.

If the class (or ) is Glivenko–Cantelli with respect to P then converges to P uniformly over (or ). In other words, with probability 1 we have

Read more about this topic:  Empirical Measure

Famous quotes containing the word definition:

    Scientific method is the way to truth, but it affords, even in
    principle, no unique definition of truth. Any so-called pragmatic
    definition of truth is doomed to failure equally.
    Willard Van Orman Quine (b. 1908)

    One definition of man is “an intelligence served by organs.”
    Ralph Waldo Emerson (1803–1882)

    No man, not even a doctor, ever gives any other definition of what a nurse should be than this—”devoted and obedient.” This definition would do just as well for a porter. It might even do for a horse. It would not do for a policeman.
    Florence Nightingale (1820–1910)