Evaluating Model-data Fit
Prior to any further analysis, the cognitive model specified must accurately reflect the cognitive attributes used by the examinees. It is expected that there will be discrepancies, or slips, between observed response patterns generated by a large group of examinees and the expected response patterns. The fit of the cognitive model relative to the observed response patterns obtained from examinees can be evaluated using the Hierarchical Consistency Index. The HCI evaluates the degree to which the observed response patterns are consistent with the attribute hierarchy. The HCI for examinee i is given by:
where J is the total number of items, Xij is examinee i ‘s score (i.e., 1 or 0) to item j, Sj includes items that require the subset of attributes of item j, and Nci is the total number of comparisons for correctly answered items by examinee i.
The values of the HCI range from -1 to +1. Values closer to 1 indicate a good fit between the observed response pattern and the expected examinee response patterns generated from the hierarchy. Conversely, low HCI values indicate a large discrepancy between the observed examinee response patterns and the expected examinee response patterns generated from the hierarchy. HCI values above 0.70 indicate good model-data fit.
Read more about this topic: Attribute Hierarchy Method
Famous quotes containing the words evaluating and/or fit:
“Evaluation is creation: hear it, you creators! Evaluating is itself the most valuable treasure of all that we value. It is only through evaluation that value exists: and without evaluation the nut of existence would be hollow. Hear it, you creators!”
—Friedrich Nietzsche (1844–1900)
“If a drunkard in a sober fit is the dullest of mortals, an enthusiast in a reason-fit is not the most lively.”
—Herman Melville (1819–1891)