Constructing A Decision Tree Using Information Gain
A decision tree can be constructed top-down using the information gain in the following way:
- begin at the root node
- determine the attribute with the highest information gain which is not already used as an ancestor node
- add a child node for each possible value of that attribute
- attach all examples to the child node where the attribute values of the examples are identical to the attribute value attached to the node
- if all examples attached to the child node can be classified uniquely add that classification to that node and mark it as leaf node
- go back to step two if there are unused attributes left, otherwise add the classification of most of the examples attached to the child node
Read more about this topic: Information Gain In Decision Trees
Famous quotes containing the words constructing, decision, tree, information and/or gain:
“The very hope of experimental philosophy, its expectation of constructing the sciences into a true philosophy of nature, is based on induction, or, if you please, the a priori presumption, that physical causation is universal; that the constitution of nature is written in its actual manifestations, and needs only to be deciphered by experimental and inductive research; that it is not a latent invisible writing, to be brought out by the magic of mental anticipation or metaphysical mediation.”
—Chauncey Wright (1830–1875)
“How could a man be satisfied with a decision between such alternatives and under such circumstances? No more than he can be satisfied with his hat, which he’s chosen from among such shapes as the resources of the age offer him, wearing it at best with a resignation which is chiefly supported by comparison.”
—George Eliot [Mary Ann (or Marian)
“There is hardly an American male of my generation who has not at one time or another tried to master the victory cry of the great ape as it issued from the androgynous chest of Johnny Weissmuller, to the accompaniment of thousands of arms and legs snapping during attempts to swing from tree to tree in the backyards of the Republic.”
—Gore Vidal (b. 1925)
“I was brought up to believe that the only thing worth doing was to add to the sum of accurate information in the world.”
—Margaret Mead (1901–1978)
“Whether a man hides his bad qualities and vices or confesses them openly, his vanity wants to gain an advantage by it in both cases: just note how subtly he distinguishes between those he will hide his bad qualities from and those he will face honestly and candidly.”
—Friedrich Nietzsche (1844–1900)