Information Gain in Decision Trees - Constructing A Decision Tree Using Information Gain

Constructing A Decision Tree Using Information Gain

A decision tree can be constructed top-down using the information gain in the following way:

  1. begin at the root node
  2. determine the attribute with the highest information gain which is not already used as an ancestor node
  3. add a child node for each possible value of that attribute
  4. attach all examples to the child node where the attribute values of the examples are identical to the attribute value attached to the node
  5. if all examples attached to the child node can be classified uniquely add that classification to that node and mark it as leaf node
  6. go back to step two if there are unused attributes left, otherwise add the classification of most of the examples attached to the child node

Read more about this topic:  Information Gain In Decision Trees

Famous quotes containing the words constructing, decision, tree, information and/or gain:

    “The very hope of experimental philosophy, its expectation of constructing the sciences into a true philosophy of nature, is based on induction, or, if you please, the a priori presumption, that physical causation is universal; that the constitution of nature is written in its actual manifestations, and needs only to be deciphered by experimental and inductive research; that it is not a latent invisible writing, to be brought out by the magic of mental anticipation or metaphysical mediation.”
    —Chauncey Wright (1830–1875)

    “The decision to feed the world
    is the real decision. No revolution
    has chosen it. For that choice requires
    that women shall be free.”
    —Adrienne Rich (b. 1929)

    “I expect a time when, or rather an integrity by which, a man will get his coat as honestly and as perfectly fitting as a tree its bark. Now our garments are typical of our conformity to the ways of the world, i.e., of the devil, and to some extent react on us and poison us, like that shirt which Hercules put on.”
    —Henry David Thoreau (1817–1862)

    “But while ignorance can make you insensitive, familiarity can also numb. Entering the second half-century of an information age, our cumulative knowledge has changed the level of what appalls, what stuns, what shocks.”
    —Anna Quindlen (b. 1952)

    “Let the wise also hear and gain in learning, and the discerning acquire skill, to understand a proverb and a figure, the words of the wise and their riddles.”
    —Bible: Hebrew, Proverbs 1:5-6.