Category Utility - Information-theoretic Definition of The Category Utility

Information-theoretic Definition of The Category Utility

The information-theoretic definition of category utility for a set of entities with size- binary feature set, and a binary category is given in Gluck & Corter (1985) as follows:

CU(C,F) = \left - \sum_{i=1}^n p(f_i)\log p(f_i)

where is the prior probability of an entity belonging to the positive category (in the absence of any feature information), is the conditional probability of an entity having feature given that the entity belongs to category, is likewise the conditional probability of an entity having feature given that the entity belongs to category, and is the prior probability of an entity possessing feature (in the absence of any category information).

The intuition behind the above expression is as follows: The term represents the cost (in bits) of optimally encoding (or transmitting) feature information when it known that the objects to be described belong to category . Similarly, the term represents the cost (in bits) of optimally encoding (or transmitting) feature information when it known that the objects to be described belong to category . The sum of these two terms in the brackets is therefore the weighted average of these two costs. The final term, represents the cost (in bits) of optimally encoding (or transmitting) feature information when no category information is available. The value of the category utility will, in the above formulation, be negative (???).

Read more about this topic:  Category Utility

Famous quotes containing the words definition, category and/or utility:

    “Mothers often are too easily intimidated by their children’s negative reactions...When the child cries or is unhappy, the mother reads this as meaning that she is a failure. This is why it is so important for a mother to know...that the process of growing up involves by definition things that her child is not going to like. Her job is not to create a bed of roses, but to help him learn how to pick his way through the thorns.”
    —Elaine Heffner (20th century)

    “Despair is typical of those who do not understand the causes of evil, see no way out, and are incapable of struggle. The modern industrial proletariat does not belong to the category of such classes.”
    —Vladimir Ilyich Lenin (1870–1924)

    “Moral sensibilities are nowadays at such cross-purposes that to one man a morality is proved by its utility, while to another its utility refutes it.”
    —Friedrich Nietzsche (1844–1900)