Formal Definition
Let denote a set of training examples, each of the form where is the value of the th attribute of example and is the corresponding class label. The information gain for an attribute is defined in terms of entropy as follows:
The information gain is equal to the total entropy for an attribute if for each of the attribute values a unique classification can be made for the result attribute. In this case the relative entropies subtracted from the total entropy are 0.
Read more about this topic: Information Gain In Decision Trees
Famous quotes containing the words formal and/or definition:
“Good gentlemen, look fresh and merrily.
Let not our looks put on our purposes,
But bear it as our Roman actors do,
With untired spirits and formal constancy.”
—William Shakespeare (15641616)
“The physicians say, they are not materialists; but they are:MSpirit is matter reduced to an extreme thinness: O so thin!But the definition of spiritual should be, that which is its own evidence. What notions do they attach to love! what to religion! One would not willingly pronounce these words in their hearing, and give them the occasion to profane them.”
—Ralph Waldo Emerson (18031882)