Abstract:

Q-matrix is an important component of cognitive diagnostic assessment, which represents the item-attribute relationships. Cognitive diagnostic assessment infers attribute mastery patterns of respondents in the testing field based on item responses. Item responses in the assessment are observable, but respondents attribute mastery patterns are potential, not observable. Q-matrix plays the role of a bridge in cognitive diagnostic assessment. Therefore, Q-matrix impact the accuracy of cognitive diagnostic assessment greatly. Research on the effect of parameter estimation and classification accuracy caused by the error in Q-matrix already existed, and it turned out that Q-matrix gotten from expert definition or experience was more easily subject to be affected by subjective factors, lead to a misspecified Q-matrix. Under this circumstance, it’s urgently needed to find more objective Q-matrix inference methods. This paper started from this consideration, carried out further research on the Q-matrix inference from response data based on the research of Liu, Xu and Ying (2012), and modified the Liu et al. algorithm, designed a joint estimate item parameters and Q-matrix algorithm. The joint estimate algorithm can estimate item parameters and the Q-matrix simultaneously. In simulations, considered different Q-matrix (attribute-number is 3,4 and 5), different sample size (500, 1000, 2000 and 4000), different number of error items (3,4 and 5), the attribute mastery pattern of the sample followed an uniform distribution, and the item parameters followed an uniform distribution with interval [0.05,0.25]. When item parameters were unknown, item number was 20, and item attributes was 3, 4 or 5, based on the initial Q-matrix, and the joint estimate algorithm can get the true Q-matrix with a high probability and item parameters with small deviation, even the sample size is relatively small (such as 300), and the misspecified-item number is relatively large (such as 6). Furthermore, when the number of item attribute was misspecified by experts, in other words, the Q-matrix lacked a required attribute or added a redundant attribute, this would lead to incorrectness of all items, and the joint estimate algorithm will provide reliable information to infer the true Q-matrix. The results indicated that: (1) The joint estimate algorithm had a good performance and suitable for practical application when some item attribute vectors misspecified. (2) The joint estimate algorithm could provide useful information when added a redundant attribute or lacked a required attribute in Q-matrix, and then amended and estimated the Q-matrix.

Key words: cognitive diagnostic assessment, Q-matrix, T-matrix, joint estimate, DINA model