Constructing A Decision Tree Using Information Gain
A decision tree can be constructed top-down using the information gain in the following way:
- begin at the root node
- determine the attribute with the highest information gain which is not already used as an ancestor node
- add a child node for each possible value of that attribute
- attach all examples to the child node where the attribute values of the examples are identical to the attribute value attached to the node
- if all examples attached to the child node can be classified uniquely add that classification to that node and mark it as leaf node
- go back to step two if there are unused attributes left, otherwise add the classification of most of the examples attached to the child node
Read more about this topic: Information Gain In Decision Trees
Famous quotes containing the words constructing, decision, tree, information and/or gain:
“The very hope of experimental philosophy, its expectation of constructing the sciences into a true philosophy of nature, is based on induction, or, if you please, the a priori presumption, that physical causation is universal; that the constitution of nature is written in its actual manifestations, and needs only to be deciphered by experimental and inductive research; that it is not a latent invisible writing, to be brought out by the magic of mental anticipation or metaphysical mediation.”
—Chauncey Wright (1830–1875)
“Drug misuse is not a disease, it is a decision, like the decision to step out in front of a moving car. You would call that not a disease but an error of judgment.”
—Philip K. Dick (1928–1982)
“The body in the grave is like the tree in winter; they conceal their greenness under a show of dryness.... We too must wait for the springtime of the body.”
—Marcus Minucius Felix (2nd or 3rd cent. A.D.)
“So while it is true that children are exposed to more information and a greater variety of experiences than were children of the past, it does not follow that they automatically become more sophisticated. We always know much more than we understand, and with the torrent of information to which young people are exposed, the gap between knowing and understanding, between experience and learning, has become even greater than it was in the past.”
—David Elkind (20th century)
“The first place he went into was the Royal Exchange .... where men of all ages and all nations were assembled, with no other view than to barter for interest. The countenances of most of the people showed they were filled with anxiety; some indeed appeared pleased, but yet it was with a mixture of fear.... [David] resolved to stay no longer in a place where riches were esteemed goodness, and deceit, low cunning, and giving up all things to the love of gain were thought wisdom.”
—Sarah Fielding (1710–1768)