Constructing A Decision Tree Using Information Gain
A decision tree can be constructed top-down using the information gain in the following way:
- begin at the root node
- determine the attribute with the highest information gain which is not already used as an ancestor node
- add a child node for each possible value of that attribute
- attach all examples to the child node where the attribute values of the examples are identical to the attribute value attached to the node
- if all examples attached to the child node can be classified uniquely add that classification to that node and mark it as leaf node
- go back to step two if there are unused attributes left, otherwise add the classification of most of the examples attached to the child node
Read more about this topic: Information Gain In Decision Trees
Famous quotes containing the words constructing, decision, tree, information and/or gain:
“The very hope of experimental philosophy, its expectation of constructing the sciences into a true philosophy of nature, is based on induction, or, if you please, the a priori presumption, that physical causation is universal; that the constitution of nature is written in its actual manifestations, and needs only to be deciphered by experimental and inductive research; that it is not a latent invisible writing, to be brought out by the magic of mental anticipation or metaphysical mediation.”
—Chauncey Wright (1830–1875)
“Will mankind never learn that policy is not morality,—that it never secures any moral right, but considers merely what is expedient? chooses the available candidate,—who is invariably the devil,—and what right have his constituents to be surprised, because the devil does not behave like an angel of light? What is wanted is men, not of policy, but of probity,—who recognize a higher law than the Constitution, or the decision of the majority.”
—Henry David Thoreau (1817–1862)
“A pinecone does not fall far from the tree trunk.”
—Estonian. Trans. by Ilse Lehiste (1993)
“Phenomenal nature shadows him wherever he goes. Clouds in the staring sky transmit to one another, by means of slow signs, incredibly detailed information regarding him. His inmost thoughts are discussed at nightfall, in manual alphabet, by darkly gesticulating trees. Pebbles or stains or sunflecks form patterns representing in some awful way messages which he must intercept. Everything is a cipher and of everything he is the theme.”
—Vladimir Nabokov (1899–1977)
“[A man’s] moral conscience is the curse he had to accept from the gods in order to gain from them the right to dream.”
—William Faulkner (1897–1962)