Category Utility - Information-theoretic Definition of The Category Utility

Information-theoretic Definition of The Category Utility

The information-theoretic definition of category utility for a set of entities with size- binary feature set, and a binary category is given in Gluck & Corter (1985) as follows:

CU(C,F) = \left - \sum_{i=1}^n p(f_i)\log p(f_i)

where is the prior probability of an entity belonging to the positive category (in the absence of any feature information), is the conditional probability of an entity having feature given that the entity belongs to category, is likewise the conditional probability of an entity having feature given that the entity belongs to category, and is the prior probability of an entity possessing feature (in the absence of any category information).

The intuition behind the above expression is as follows: The term represents the cost (in bits) of optimally encoding (or transmitting) feature information when it known that the objects to be described belong to category . Similarly, the term represents the cost (in bits) of optimally encoding (or transmitting) feature information when it known that the objects to be described belong to category . The sum of these two terms in the brackets is therefore the weighted average of these two costs. The final term, represents the cost (in bits) of optimally encoding (or transmitting) feature information when no category information is available. The value of the category utility will, in the above formulation, be negative (???).

Read more about this topic:  Category Utility

Famous quotes containing the words definition, category and/or utility:

    “Although there is no universal agreement as to a definition of life, its biological manifestations are generally considered to be organization, metabolism, growth, irritability, adaptation, and reproduction.”
    —The Columbia Encyclopedia, Fifth Edition, the first sentence of the article on “life” (based on wording in the First Edition, 1935)

    “Despair is typical of those who do not understand the causes of evil, see no way out, and are incapable of struggle. The modern industrial proletariat does not belong to the category of such classes.”
    —Vladimir Ilyich Lenin (1870–1924)

    “Moral sensibilities are nowadays at such cross-purposes that to one man a morality is proved by its utility, while to another its utility refutes it.”
    —Friedrich Nietzsche (1844–1900)