ISSN 0439-755X
CN 11-1911/B
主办：中国心理学会
　　　中国科学院心理研究所
出版：科学出版社

心理学报 ›› 2016, Vol. 48 ›› Issue (3): 318-330.doi: 10.3724/SP.J.1041.2016.00318

• 论文 • 上一篇    

重参数化的多分属性诊断分类模型及其判准率影响因素

詹沛达1;边玉芳1;王立君2   

  1. (1北京师范大学 中国基础教育质量监测协同创新中心, 北京 100875)
    (2浙江师范大学 心理系, 金华 321004)
  • 收稿日期:2015-05-26 发布日期:2016-03-25 出版日期:2016-03-25
  • 通讯作者: 边玉芳, E-mail: bianyufang66@126.com; 王立君, E-mail: frankwlj@163.com
  • 基金资助:

    全国教育科学规划教育部重点课题(主观题的多分属性认知诊断模型开发及其在物理测验中的应用)，课题批准号: DBA150236

Factors affecting the classification accuracy of reparametrized diagnostic classification models for expert-defined polytomous attributes

ZHAN Peida1; BIAN Yufang1; WANG Lijun2   

  1. (1 Collaborative Innovation Center of Assessment toward Basic Education Quality, Beijing Normal University, Beijing 100875, China)
    (2 Department of Psychology, Zhejiang Normal University, Jinhua 321004, China)
  • Received:2015-05-26 Online:2016-03-25 Published:2016-03-25
  • Contact: BIAN Yufang, E-mail: bianyufang66@126.com; WANG Lijun, E-mail: frankwlj@163.com

PDF (PC)

1757

被引次数

8

摘要：

多分属性比传统的二分属性提供更多更详细的诊断反馈信息, 符合对知识技能的多水平要求, 具有较好的应用前景。本文首先介绍了多分属性和多分Q矩阵的概念; 之后重参数化了3个分别满足连接、分离和补偿缩合规则的多分属性诊断分类模型并研究了其判准率影响因素, 结果发现它们的判准率(1)均随多分属性数量的增加而降低, 建议实际使用中不宜高于5个; (2)均随多分属性的最高水平数增加而降低, 建议实际使用中不宜高于4水平; (3)均随多分属性间统计相关性增加而增加, 但影响不大; (4)受多分属性层级结构的影响较大; (4)受被试量影响不大; (5)均随题目数量增加而增加且影响较大。最后, 针对“多分属性与多级评分的关系”和“多分属性与二分属性之间的关系”这两个问题进行了讨论。以期为实证研究者提供相关的理论支持和使用建议。

关键词: 认知诊断, 多分认知属性, 多分Q矩阵, 诊断分类模型, DINA, DINO, LLM

Abstract:

Diagnostic classification assessment (DCA) utilizes latent class models to provide fine-grained information about students’ strengths and weaknesses in the learning process. In the past decades, extensive research has been conducted in the area of DCA and many statistical models based on a probabilistic approach have been proposed. At present, several diagnostic classification models (DCMs) for dichotomous attributes exist, which include the deterministic inputs, noisy “and” gate (DINA; Junker & Sijtsma, 2001); the deterministic inputs, noisy “or” gate (DINO; Templin & Henson, 2006); and the linear logistic model (LLM; Maris, 1999). In contrast, only a few DCMs can be used to deal with the polytomous attributes, such as the model based on the ordered-category attribute coding (OCAC; Karelitz, 2004), and the polytomous generalized DINA (pG-DINA; Chen & de la Torre, 2013).
Polytomous attributes, particularly those defined as part of the test development process, can provide additional diagnostic information. The present research proposes three reparametrized reduced models of pG-DINA (Chen & de la Torre, 2013), which include the reparametrized polytomous attributes DINA (RPa-DINA), the reparametrized polytomous attributes DINO (RPa-DINO), and the reparametrized polytomous attributes LLM (RPa-LLM). Furthermore, to better understand the classification accuracy of the new models, the impact of 6 factors was investigated, namely, the number of polytomous attributes, the highest level of polytomous attributes, the correlations among polytomous attributes, the hierarchical structure, the sample size, and the number of items. Results of the simulation study indicated that:
(1) more polytomous attributes led to lower classification. Their effects, in descending order, were the RPa-LLM, the RPa-DINO, and the RPa-DINA. Less than 5 polytomous attributes used in empirical research is suggested;
(2) for the number of attribute levels, more levels resulted in worse performance. Less than 4 levels within one attribute used in empirical research is suggested;
(3) the higher the correlations among polytomous attributes, the higher the classification accuracy would be;
(4) different hierarchical structure had different influences on the classification accuracy. No matter what structure we had, the performance of RPa-DINA was quite well behaved. However, other 2 models, especially the RPa-DINO, were recommended for the analysis of response data from independent hierarchical structure;
(5) the sample size has little impact on the classification accuracy; and

(6) the number of items was inversely proportional to the classification accuracy.

Key words: cognitive diagnosis, polytomous attribute, polytomous Q matrix, diagnostic classification models, DINA, DINO, LLM