Information Gain in Decision Trees - General Definition

General Definition

In general terms, the expected information gain is the change in information entropy from a prior state to a state that takes some information as given:

Where denotes the information entropy.

Read more about this topic:  Information Gain In Decision Trees

Famous quotes containing the words general and/or definition:

    A bill of rights is what the people are entitled to against every government on earth, general or particular, and what no just government should refuse, or rest on inference.
    Thomas Jefferson (1743–1826)

    I’m beginning to think that the proper definition of “Man” is “an animal that writes letters.”
    Lewis Carroll [Charles Lutwidge Dodgson] (1832–1898)