ISSN 0439-755X
CN 11-1911/B

Acta Psychologica Sinica ›› 2015, Vol. 47 ›› Issue (12): 1499-1510.

Previous Articles     Next Articles

Distance Discrimination Method based on Conditional Expectation in Cognitive Diagnosis

WANG Wenyi1; DING Shuliang1; SONG Lihong2   

  1. (1 College of Computer Information Engineering, Jiangxi Normal University, Nanchang 330022, China)
    (2 Elementary Educational College, Jiangxi Normal University, Nanchang 330022, China)
  • Received:2014-08-12 Published:2015-12-25 Online:2015-12-25
  • Contact: WANG Wenyi, E-mail: wenyiwang@jxnu.edu.cn

Abstract:

The primary purpose for cognitive diagnostic assessment is to classify examinees into mutually exclusive categories. The current practice of obtaining classification categories relies on the distance between the ideal and observed response patterns, such as generalized distance discrimination method (Sun et al., 2011, 2013) and nonparametric approach (Chiu and Douglas, 2013). In these methods, an appropriate set of ideal response patterns can be computed from the universal set of knowledge states and a Q matrix (Tatsuoka, 1995; Leighton et al., 2004; Ding et al., 2009, 2010). However, the ideal response pattern is generated for each knowledge state without considering the stochastic nature of item response, which is contrary to real test situations. For example, examinees who have mastered some of attributes required to solve a particular item can have a higher probability of answering it correctly than less able examinees having mastered none of the attributes.

The purpose of this study is focused on choosing the center for each of the classification categories or clusters that is representative of the data. Since this task probably requires knowledge on the distribution of the data, as a practical matter, it is often reasonable to assume that item response pattern is a random vector with a discrete conditional distribution for each knowledge state. The conditional expected vector is considered to be class-center. Once the center is decided for each knowledge state, observed responses pattern can be assigned to the closest class according to minimum Euclidean distance classifier, well-known for measuring the distance between observed response pattern and the center. This method is called distance discrimination method.
The conditional distribution can be defined by item response function (IRF) in cognitive diagnostic models. However, many existing item banks are developed under the framework of item response theory. In this case, we propose a method utilizing the information of the item characteristic curve (ICC) or IRF of item response theory model to estimate the IRF for cognitive diagnostic model. It is based on a nonparametric regression approach to transform the IRF of item response theory model into that of cognitive diagnostic model. The resulting IRF is used to classify examinees using minimum Euclidean distance classifier.
To investigate whether this method can work under certain conditions, simulated data were generated with six attributes. Four important factors were considered: (a) the source of the attribute structure (the linear hierarchy, the convergent hierarchy, the divergent hierarchy, the unstructured hierarchy, the independent hierarchy), (b) the number of examinees (N = 300, 500, 1,000), (c) two cognitive diagnostic models (the deterministic inputs, noisy “and” gate model and the reduced reparametrized unified model);(d) the quality of the items (s, g ~ U (0.05, 0.25) or U (0.05, 0.4), ~U (0.8, 0.98) and r*~U (0.1, 0.6) or ~U (0.75, 0.95) and r*~U (0.2, 0.95)).
The results show that the estimation method of IRF is promising in terms of precision, and distance discrimination method based on conditional expectation works well in terms of accuracy, especially when the R-RUM fitted the data with low quality of test items. In addition, the fact that the IRF transform method between cognitive diagnostic model and item response model may contribute immensely to test equating of cognitive diagnostic tests. In discussion, we also explain the relationship between nonparametric approach, generalized distance discrimination method and rule space method.

Key words: cognitive diagnostic assessment, distance discrimination method, conditional expected vector, item characteristic curve, item response function