Abstract: Attributes and their hierarchy may present a cognitive model. Building a cognitive model is one of key steps for cognitive diagnosis as it is directly related to the validity and usefulness of test results. It is very important to detect whether the test specification coincides with the cognitive model before administering the test. In this paper, an explicit index is given to measure the extent to which the cognitive model is represented by the test items of the diagnostic test. We call this index as the theoretical construct validity (TCV). In terms of TCV, the test reported by Tatsuoka and her colleagues (1988) is reanalyzed, and the TCV of the test is only 9/24. That is, 24 knowledge states are obtained from the theoretic cognitive model but the test specification could distinguish only 9 knowledge states. Cui and her colleagues established a person fit index named hierarchy consistency index (HCI) to detect the fitness of an examinee’s observed response pattern (ORP) to an expected response pattern (ERP). HCI is not defined well when an examinee mastered one attribute only and there is only one item measuring the same attribute in the test, and the examinee responses correctly to the item. The original HCI could not compute for the number of comparisons being zero, hence the denominator being zero. In addition, the HCI includes the slipping only. Combining the slipping and guessing in the response to an item, we proposed a new index called modified HCI (MHCI). MHCI could detect the consistency of the response data and the cognitive model that is obtained by experts.

Key words: cognitive diagnosis, cognitive model, cognitive diagnostic test