Conditioning (probability) - Conditioning On The Discrete Level - Conditional Expectation

Conditional Expectation

Given that X = 1, the conditional expectation of the random variable Y is E ( Y | X = 1 ) = 0.3. More generally,

for x = 0, ..., 10. (In this example it appears to be a linear function, but in general it is nonlinear.) One may also treat the conditional expectation as a random variable, — a function of the random variable X, namely,

The expectation of this random variable is equal to the (unconditional) expectation of Y,

namely,

or simply

which is an instance of the law of total expectation E ( E ( Y | X ) ) = E ( Y ).

The random variable E(Y | X) is the best predictor of Y given X. That is, it minimizes the mean square error E ( Y - f(X) )2 on the class of all random variables of the form f(X). This class of random variables remains intact if X is replaced, say, with 2X. Thus, E ( Y | 2X ) = E ( Y | X ). It does not mean that E (Y | 2X ) = 0.3 × 2X; rather, E ( Y | 2X ) = 0.15 × 2X = 0.3 X. In particular, E (Y | 2X=2) = 0.3. More generally, E (Y | g(X)) = E ( Y | X ) for every function g that is one-to-one on the set of all possible values of X. The values of X are irrelevant; what matters is the partition (denote it α_X)

of the sample space Ω into disjoint sets {X = x_n}. (Here are all possible values of X.) Given an arbitrary partition α of Ω, one may define the random variable E ( Y | α ). Still, E ( E ( Y | α)) = E ( Y ).

Conditional probability may be treated as a special case of conditional expectation. Namely, P ( A | X ) = E ( Y | X ) if Y is the indicator of A. Therefore the conditional probability also depends on the partition α_X generated by X rather than on X itself; P ( A | g(X) ) = P (A | X) = P (A | α), α = α_X = α_g(X).

On the other hand, conditioning on an event B is well-defined, provided that P (B) ≠ 0, irrespective of any partition that may contain B as one of several parts.