Abstract:

Item Cognitive Attribute Identification (ICAI) is the basis of Cognitive Diagnosis (CD), which is designed to measure specific knowledge structures and processing skills in students. According to the published documents, there are two methods used in ICAI.The one is to indentify item attributes by some experts of relative domains. When there are many items, it will be a huge burgen for experts to identify their attributes in the items. Especially, for some items, it’s difficult for experts to get a unified opinion about items’ attributes. As an assistant to this one, the other method is to identify items’ attributes by CD-CAT (Cognitive Diagnostic Computerized Adaptive Testing). Using CD-CAT in ICAI is an obvious breakthrough, for that it is not necessary totally depentant on manual labour. But using CD-CAT in ICAI has some heavy limitation. For example, if the items’parameters such as difficulty, are unknown, big samples of subjects and items are necessary for CD-CAT to identify item attributes. The second limit of CD-CAT is that it is based on item pool, and the development of item pool is very expensive that the cost of one item is about $1000. Cognitive diagnosis is designed to provide information about students’ cognitive strengths and weaknesses and to assist the teaching. So, the best place to use it is in classrooms. But cognitive diagnosis is just used in lager–scale examinations now for two reasons: First, most cognitive diagnosis models are based on probability models which need a large sample in estimating item parameters, and the using of these cognitive diagnosis models are also based on a large sample of subjects even the items’ parameters have been estimated. Secondary, even though the method of CD-CAT can be used in a small–scale examination once the item parameters are known, CAT has been prohibited in many kinds of examinations for other reasons. So, it is very necessary to find a new method to indentify item attributes when item parameters are unknown, examinees are less and feedbacks are timely. In the current studies, we apply a new method – Rough Set Theory (RST) to ICAI. RST can solve the uncertainty in CD caused by the size of knowledge granularity. It doesn’t require any priori knowledge. Through the knowledge reduction, RST induces decision or classification rules, and then classifies the object. At first, we verificate the application of RST in ICAI. Then, in Study One, we explore how the match ratio of subjects' knowledge states and the slippage in subjects' responses to items impact the match ratio of item attributes. The number of item attributes is a variable which impacts the accuracy of CD, so, we also examine how the number of cognitive contributes impact the match ratio of item attributes. The results show that: (1) In the absence of item parameters, the rough set theory of ICAI has fast diagnostic speed and good results even though the sample size is small. So RST can be applied to classroom assessment. (2) The lower examinee’s PMR, the lower PMR of item attribute identification is. And the higher slippage in examinee’s response, the lower item attribute identification’s PMR is. (3) The more the number of item attributes, the lower item attribute identification’s PMR is. (4) Both results are estimated by rough set software, and regardless of sample size and item number, the estimated speed is very fast (about 10 seconds). It shows the advantage of RST in ICAI.

Key words: rough set theory, cognitive diagnosis, item attribute cognitive identification