Determining The Update Rule
The likelihood estimate needs to be as large as possible; because it's a lower bound, getting closer improves the approximation of the log likelihood. By substituting in the factorized version of, parameterized over the hidden nodes as above, is simply the negative relative entropy between and plus other terms independent of if is defined as
- ,
where is the expectation over all distributions except . Thus, if we set to be, the bound is maximized.
Read more about this topic: Variational Message Passing
Famous quotes containing the words determining the, determining and/or rule:
“I esteem it the happiness of this country that its settlers, whilst they were exploring their granted and natural rights and determining the power of the magistrate, were united by personal affection. Members of a church before whose searching covenant all rank was abolished, they stood in awe of each other, as religious men.”
—Ralph Waldo Emerson (1803–1882)
“The true rule, in determining to embrace, or reject any thing, is not whether it have any evil in it; but whether it have more of evil, than of good. There are few things wholly evil, or wholly good.”
—Abraham Lincoln (1809–1865)
“As an example to others, and not that I care for moderation myself, it has always been my rule never to smoke when asleep, and never to refrain from smoking when awake.”
—Mark Twain [Samuel Langhorne Clemens] (1835–1910)