ISSN 1671-3710
CN 11-4766/R

Advances in Psychological Science ›› 2021, Vol. 29 ›› Issue (12): 2272-2280.doi: 10.3724/SP.J.1042.2021.02272

• Research Method • Previous Articles    

Q-matrix estimation (validation) methods for cognitive diagnosis

LI Jia, MAO Xiuzhen, ZHANG Xueqin   

  1. Institute of Educational Sichuan Normal University, Chengdu 610066, China
  • Received:2021-04-02 Online:2021-12-15 Published:2021-10-26

Abstract: The Q-matrix plays the role of bridging between the observable responses, the unobservable item characteristics and the knowledge state of participants. It is of vital importance to obtain accurate Q-matrix. In the past decade, researchers have conducted extensive studies and proposed a number of methods for the estimation (validation) of Q-matrix. In general, the existing methods of Q-matrix estimation and validation are classified into: 1) parameterized methods in the CDM perspective, including item differentiation, model-data fit index and parameter estimation; 2) non-parametric methods in the statistical perspective, including the distance between observed and expected response vector, abnormal responses index and factor analysis.
The core thought of the optimal item discrimination methods is to select the attribute pattern with the optimal item discrimination from all possible attribute patterns as the q-vector of item. Among them, theδ (de la Torre, 2008) and ς 2 (de la Torre & Chiu, 2016) methods determine the q -vector based on the absolute optimal item discrimination index; the γ (Tu et al., 2012) method adopt the effect size test based on attribute discrimination; the stepwise (Ma & de la Torre, 2020) method and the likelihood ratio test (Wang et al., 2019; Wang et al., 2020) focus on the performance of search algorithm and significant difference test in Q-matrix estimation (validation). Secondly, the key to the methods of absolute fit index based on model-data is to construct the difference or consistency index that reflects the probability distribution of observed response and expected response. For instance, S statistic (Liu et al., 2012) judges the difference between the observed and the expected distribution of all participants from all the items; Likelihood ratio D2 statistics (Yu et al., 2015), RMSEA(Kang et al., 2019) and R(Yu & Cheng, 2020) are used to judge the difference between the observed and expected distributions of the item j from all the participants; the residual method (Chen, 2017) is based on the absolute error of the correlation or logarithmic ratio of the item pairs. It is still an effective way to estimate (validate) Q-matrix by taking the elements of Q-matrix as parameters to be estimated. In this line of research, MLE and MMLE methods (Wang et al., 2018) are common parameter estimation methods, which are simple and easy to understand, but iterative EM algorithm is often time-consuming. Bayesian parameter estimation methods (Chung, 2019; Chen et al., 2018; DeCarlo, 2012; Templin & Henson, 2006) obtain the posterior distribution of the parameters to be estimated based on the prior distribution, and then use the mean value of the posterior distribution or the sample mean value as the estimated value.
The non-parametric methods based on statistical analysis express the degree of fit between the observed reaction and the ideal reaction by calculating the distance between the discrete observed response vector and the ideal response vector or the abnormal response index. For example, the Euclidean distance $\sum\nolimits_{i=1}^{N}{{{({{Y}_{ij}}-{{\eta }_{ijc}})}^{2}}}$(Barnes, 2010; Chiu, 2013; Hang , 2020), the Hamming distance$\sum\nolimits_{i=1}^{N}{\text{I(}{{Y}_{ij}}\ne {{\eta }_{ijc}}\text{)}}$ (Wang et al., 2018) or the Manhattan distance$\sum\nolimits_{i=1}^{N}{\left| {{Y}_{ij}}-{{\eta }_{ijc}} \right|}$ (Liu , 2020). In addition, based on a large amount of reaction data, the non-parametric methods also regard the elements of the Q-matrix as the factor structure between item and potential attributes for factor analysis, which are essentially the estimation of the factor structure (e.g., Close, 2012; Wang et al., 2015; Wang et al., 2020).
Finally, several directions for future research are proposed. 1) The influence of item quality, characteristics of participants and test conditions on all methods should be investigated comprehensively. 2) The characteristics of existing methods should be explored based on complex models such as more general cognitive diagnostic models, response time models or higher-order cognitive diagnostic models. 3) The estimation (validation) methods of Q-matrix should be concerned when items or attributes are polytomous or the number of attributes is unknown. 4) It is possible to introduce the estimation errors of item parameters and knowledge states to improve the estimation accuracy of Q-matrix. 5) The Q-matrix estimation (validation) methods can be applied to the on-line calibration and the joint calibration of Q-matrix and item parameters.

Key words: cognitive diagnosis models, Q-matrix, Q-matrix estimation (validation) methods, model-data fit, parameter estimation

CLC Number: