Model
With plate notation, the dependencies among the many variables can be captured concisely. The boxes are “plates” representing replicates. The outer plate represents documents, while the inner plate represents the repeated choice of topics and words within a document. M denotes the number of documents, N the number of words in a document. Thus:
- α is the parameter of the Dirichlet prior on the per-document topic distributions.
- β is the parameter of the Dirichlet prior on the per-topic word distribution.
- is the topic distribution for document i,
- is the word distribution for topic k,
- is the topic for the jth word in document i, and
- is the specific word.
The are the only observable variables, and the other variables are latent variables. Mostly, the basic LDA model will be extended to a smoothed version to gain better results. The plate notation is shown on the right, where K denotes the number of topics considered in the model and:
- is a K*V (V is the dimension of the vocabulary) Markov matrix each row of which denotes the word distribution of a topic.
The generative process behind is that documents are represented as random mixtures over latent topics, where each topic is characterized by a distribution over words. LDA assumes the following generative process for each document in a corpus D :
1. Choose, where and is the Dirichlet distribution for parameter
2. Choose, where
3. For each of the words, where
- (a) Choose a topic
- (b) Choose a word .
(Note that the Multinomial distribution here refers to the Multinomial with only one trial. It is formally equivalent to the categorical distribution.)
The lengths are treated as independent of all the other data generating variables ( and ). The subscript is often dropped, as in the plate diagrams shown here.
Read more about this topic: Latent Dirichlet Allocation
Famous quotes containing the word model:
“Research shows clearly that parents who have modeled nurturant, reassuring responses to infants’ fears and distress by soothing words and stroking gentleness have toddlers who already can stroke a crying child’s hair. Toddlers whose special adults model kindliness will even pick up a cookie dropped from a peer’s high chair and return it to the crying peer rather than eat it themselves!”
—Alice Sterling Honig (20th century)
“If the man who paints only the tree, or flower, or other surface he sees before him were an artist, the king of artists would be the photographer. It is for the artist to do something beyond this: in portrait painting to put on canvas something more than the face the model wears for that one day; to paint the man, in short, as well as his features.”
—James Mcneill Whistler (1834–1903)
“Socrates, who was a perfect model in all great qualities, ... hit on a body and face so ugly and so incongruous with the beauty of his soul, he who was so madly in love with beauty.”
—Michel de Montaigne (1533–1592)