Category Utility - Information-theoretic Definition of The Category Utility

Information-theoretic Definition of The Category Utility

The information-theoretic definition of category utility for a set of entities with size- binary feature set, and a binary category is given in Gluck & Corter (1985) as follows:

where is the prior probability of an entity belonging to the positive category (in the absence of any feature information), is the conditional probability of an entity having feature given that the entity belongs to category, is likewise the conditional probability of an entity having feature given that the entity belongs to category, and is the prior probability of an entity possessing feature (in the absence of any category information).

The intuition behind the above expression is as follows: The term represents the cost (in bits) of optimally encoding (or transmitting) feature information when it known that the objects to be described belong to category . Similarly, the term represents the cost (in bits) of optimally encoding (or transmitting) feature information when it known that the objects to be described belong to category . The sum of these two terms in the brackets is therefore the weighted average of these two costs. The final term, represents the cost (in bits) of optimally encoding (or transmitting) feature information when no category information is available. The value of the category utility will, in the above formulation, be negative (???).