Constructing A Decision Tree Using Information Gain
A decision tree can be constructed top-down using the information gain in the following way:
- begin at the root node
- determine the attribute with the highest information gain which is not already used as an ancestor node
- add a child node for each possible value of that attribute
- attach all examples to the child node where the attribute values of the examples are identical to the attribute value attached to the node
- if all examples attached to the child node can be classified uniquely add that classification to that node and mark it as leaf node
- go back to step two if there are unused attributes left, otherwise add the classification of most of the examples attached to the child node
Read more about this topic: Information Gain In Decision Trees
Famous quotes containing the words constructing, decision, tree, information and/or gain:
“The very hope of experimental philosophy, its expectation of constructing the sciences into a true philosophy of nature, is based on induction, or, if you please, the a priori presumption, that physical causation is universal; that the constitution of nature is written in its actual manifestations, and needs only to be deciphered by experimental and inductive research; that it is not a latent invisible writing, to be brought out by the magic of mental anticipation or metaphysical mediation.”
—Chauncey Wright (1830–1875)
“A good decision is based on knowledge and not on numbers.”
—Plato (c. 427–347 B.C.)
“The whole tree itself is but one leaf, and rivers are still vaster leaves whose pulp is intervening earth, and towns and cities are the ova of insects in their axils.”
—Henry David Thoreau (1817–1862)
“Computers are good at swift, accurate computation and at storing great masses of information. The brain, on the other hand, is not as efficient a number cruncher and its memory is often highly fallible; a basic inexactness is built into its design. The brain’s strong point is its flexibility. It is unsurpassed at making shrewd guesses and at grasping the total meaning of information presented to it.”
—Jeremy Campbell (b. 1931)
“The honest Man has, I know, that modest Desire of Gain which is peculiar to those who understand better Things than Riches; and I dare say he would be contented with much less than what is called Wealth in that Quarter of the Town which he inhabits, and will oblige all his Customers with Demands agreeable to the Moderation of his Desires.”
—Richard Steele (1672–1729)