Receiver Operating Characteristic - Basic Concept

Basic Concept

Terminology and derivations
from a confusion matrix
true positive (TP)
eqv. with hit
true negative (TN)
eqv. with correct rejection
false positive (FP)
eqv. with false alarm, Type I error
false negative (FN)
eqv. with miss, Type II error
sensitivity or true positive rate (TPR)
eqv. with hit rate, recall
false positive rate (FPR)
eqv. with fall-out
accuracy (ACC)
specificity (SPC) or True Negative Rate
positive predictive value (PPV)
eqv. with precision
negative predictive value (NPV)
false discovery rate (FDR)
Matthews correlation coefficient (MCC)
F1 score

Source: Fawcett (2006).
See also: Type I and type II errors

A classification model (classifier or diagnosis) is a mapping of instances between certain classes/groups. The classifier or diagnosis result can be a real value (continuous output), in which case the classifier boundary between classes must be determined by a threshold value (for instance, to determine whether a person has hypertension based on a blood pressure measure). Or it can be a discrete class label, indicating one of the classes.

Let us consider a two-class prediction problem (binary classification), in which the outcomes are labeled either as positive (p) or negative (n). There are four possible outcomes from a binary classifier. If the outcome from a prediction is p and the actual value is also p, then it is called a true positive (TP); however if the actual value is n then it is said to be a false positive (FP). Conversely, a true negative (TN) has occurred when both the prediction outcome and the actual value are n, and false negative (FN) is when the prediction outcome is n while the actual value is p.

To get an appropriate example in a real-world problem, consider a diagnostic test that seeks to determine whether a person has a certain disease. A false positive in this case occurs when the person tests positive, but actually does not have the disease. A false negative, on the other hand, occurs when the person tests negative, suggesting they are healthy, when they actually do have the disease.

Let us define an experiment from P positive instances and N negative instances. The four outcomes can be formulated in a 2×2 contingency table or confusion matrix, as follows:

actual value
p n total
prediction
outcome		 p' True
Positive		 False
Positive		 P'
n' False
Negative		 True
Negative		 N'
total P N

Read more about this topic:  Receiver Operating Characteristic

Famous quotes containing the words basic and/or concept:

    “The basic difference between classical music and jazz is that in the former the music is always greater than its performance—Beethoven’s Violin Concerto, for instance, is always greater than its performance—whereas the way jazz is performed is always more important than what is being performed.”
    —André Previn (b. 1929)

    “The new concept of the child as equal and the new integration of children into adult life has helped bring about a gradual but certain erosion of these boundaries that once separated the world of children from the word of adults, boundaries that allowed adults to treat children differently than they treated other adults because they understood that children are different.”
    —Marie Winn (20th century)