Use in Assessing Conformity Among Observers
The ICC is used to assess the consistency, or conformity, of measurements made by multiple observers measuring the same quantity. For example, if several physicians are asked to score the results of a CT scan for signs of cancer progression, we can ask how consistent the scores are to each other. If the truth is known (for example, if the CT scans were on patients who subsequently underwent exploratory surgery), then the focus would generally be on how well the physicians' scores matched the truth. If the truth is not known, we can only consider the similarity among the scores. An important aspect of this problem is that there is both inter-observer and intra-observer variability. Inter-observer variability refers to systematic differences among the observers — for example, one physician may consistently score patients at a higher risk level than other physicians. Intra-observer variability refers to deviations of a particular observer's score on a particular patient that are not part of a systematic difference.
The ICC is constructed to be applied to exchangeable measurements — that is, grouped data in which there is no meaningful way to order the measurements within a group. In assessing conformity among observers, if the same observers rate each element being studied, then systematic differences among observers are likely to exist, which conflicts with the notion of exchangeability. If the ICC is used in a situation where systematic differences exist, the result is a composite measure of intra-observer and inter-observer variability. One situation where exchangeability might reasonably be presumed to hold would be where a specimen to be scored, say a blood specimen, is divided into multiple aliquots, and the aliquots are measured separately on the same instrument. In this case, exchangeability would hold as long as no effect due to the sequence of running the samples was present.
Since the intraclass correlation coefficient gives a composite of intra-observer and inter-observer variability, its results are sometimes considered difficult to interpret when the observers are not exchangeable. Alternative measures such as Cohen's kappa statistic, the Fleiss kappa, and the concordance correlation coefficient have been proposed as more suitable measures of agreement among non-exchangeable observers.
Read more about this topic: Intraclass Correlation
Famous quotes containing the words conformity and/or observers:
“Belonging to a group can provide the child with a variety of resources that an individual friendship often cannot—a sense of collective participation, experience with organizational roles, and group support in the enterprise of growing up. Groups also pose for the child some of the most acute problems of social life—of inclusion and exclusion, conformity and independence.”
—Zick Rubin (20th century)
“Most observers of the French Revolution, especially the clever and noble ones, have explained it as a life-threatening and contagious illness. They have remained standing with the symptoms and have interpreted these in manifold and contrary ways. Some have regarded it as a merely local ill. The most ingenious opponents have pressed for castration. They well noticed that this alleged illness is nothing other than the crisis of beginning puberty.”
—Novalis [Friedrich Von Hardenberg] (1772–1801)