AEP For Discrete-time I.i.d. Sources
Given is an i.i.d. source, its time series X1, ..., Xn is i.i.d. with entropy H(X) in the discrete-valued case and differential entropy in the continuous-valued case. The weak law of large numbers gives the AEP with convergence in probability,
since the entropy is equal to the expectation of . The strong law of large number asserts the stronger almost sure convergence,
which implies the result from the weak law of large numbers.
Read more about this topic: Asymptotic Equipartition Property
Famous quotes containing the word sources:
“Even healthy families need outside sources of moral guidance to keep those tensions from imploding—and this means, among other things, a public philosophy of gender equality and concern for child welfare. When instead the larger culture aggrandizes wife beaters, degrades women or nods approvingly at child slappers, the family gets a little more dangerous for everyone, and so, inevitably, does the larger world.”
—Barbara Ehrenreich (20th century)