关键词: 多级评分项目, 规则空间方法, 等级反应模型, 异常反应指标, 理想反应模式

Abstract: Polytomous item has become an important item type in many large scale educational assessments. However, cognitive diagnostic models are initially proposed for dichotomous data, which limits the application and development of the Cognitive Diagnostic Assessment (CDA). For example, the rule space methodology (RSM) is still limited to the dichtomous item. The article thus extended the dichotomous RSM to that of polytomous item based on Graded Response Model, focusing on the definition and formula of ideal item-response pattern (IRP) and caution index in the polytomous item case.
In this paper, the IRP, defined as the number of attributes in an item that could be correctly responded by examinees without slippage or guessing, can be obtained by matrix multiplication. The degree of aberrant response, which indicated higher ratio score on hard items and lower ratio score on easy items, was described by the caution index, .The expectation of f(V) was zero, its variance was , and it was orthogonal to the ability of q at a given level of ability.
In the simulation study, Monte-Carlo experiment was used to test the classification accuracy under 4(attribute hierarchy)×4(slippage probability) conditions, indicated by four indexes, the pattern ratio, marginal ratio, sensitivity and specificity. Under each of 16 situations, the IRP was firstly ranked by raw score in ascending order, and the distribution of IRP in the population was made to approximate a normal distribution. A sample of 5000 observed response patterns was generated from the ideal item-response patterns, by randomly adding slips to each component of the ideal item- response patterns. For example, in order to generate 2% random errors, a random number was produced from the distribution of U(0,1). For each item, if the random number was less than 0.01, and its ideal item score was not zero, then 1 was subtracted from it, otherwise 1 was added; if random number was higher than 0.99, and its ideal item score was not full, then the 1 was added to it, otherwise 1 was subtracted. MULTILOG 7.03 was applied to estimate the item and person parameters.
The results showed that: (1) the caution index for the polytomous item was constructed properly, and it could be interpreted with semantic meaning; the caution index for the dichotomous item was a special case for that of polytomous item; (2) the accuracy of classification under each of 4 hierarchy structures decreased as the slippage probability increased; (3) the classification accuracy associated with the linear and convergent hierarchy was obviously better than that of the divergent and unstructured hierarchy. (4) The distribution of the dots mapped from the ideal item-response patterns has an effect on the classification accuracy, that is, the overlap of the dots could reduce the classification accuracy rate.

Key words: polytomous item, rule space methodology, graded response model, caution index, ideal item-response pattern