Determining The Update Rule
The likelihood estimate needs to be as large as possible; because it's a lower bound, getting closer improves the approximation of the log likelihood. By substituting in the factorized version of, parameterized over the hidden nodes as above, is simply the negative relative entropy between and plus other terms independent of if is defined as
- ,
where is the expectation over all distributions except . Thus, if we set to be, the bound is maximized.
Read more about this topic: Variational Message Passing
Famous quotes containing the words determining the, determining and/or rule:
“Our civilization has decided ... that determining the guilt or innocence of men is a thing too important to be trusted to trained men.... When it wants a library catalogued, or the solar system discovered, or any trifle of that kind, it uses up its specialists. But when it wishes anything done which is really serious, it collects twelve of the ordinary men standing round. The same thing was done, if I remember right, by the Founder of Christianity.”
—Gilbert Keith Chesterton (1874–1936)
“A quality is something capable of being completely embodied. A law never can be embodied in its character as a law except by determining a habit. A quality is how something may or might have been. A law is how an endless future must continue to be.”
—Charles Sanders Peirce (1839–1914)
“We’ve only just begun to learn about the water and its secrets, just as we’ve only touched on outer space. We don’t entirely rule out the possibility that there might be some form of life on another planet. Then why not some entirely different form of life in a world we already know is inhabited by millions of living creatures?”
—Harry Essex (b. 1910)