ISSN 0439-755X
CN 11-1911/B
主办:中国心理学会
   中国科学院心理研究所
出版:科学出版社

心理学报 ›› 2013, Vol. 45 ›› Issue (11): 1295-1304.doi: 10.3724/SP.J.1041.2013.01295

• 论文 • 上一篇    下一篇

五大认知诊断模型的诊断正确率比较及其影响因素:基于分布形态、属性数及样本容量的比较

蔡艳;涂冬波;丁树良   

  1. (1江西师范大学心理学院, 南昌 330022) (2江西师范大学计算机信息工程学院, 南昌 330022)
  • 收稿日期:2013-02-04 发布日期:2013-11-25 出版日期:2013-11-25
  • 通讯作者: 涂冬波
  • 基金资助:

    教育部人文社科基金(11YJC190002), 国家自然科学基金(31100756, 31160203), 高等院校博士点基金项目(20103604120001, 20123604120001), 江西省教育科学规划项目(13YB029, 12YB088), 江西师范大学青年英才培育资助计划项目资助。

A Simulation Study to Compare Five Cognitive Diagnostic Models

CAI Yan;TU Dongbo;DING Shuliang   

  1. (1 School of Psychology, Jiangxi Normal University, Nanchang 330027, China) (2 School of Computer and Information Engineering, Jiangxi Normal University, Nanchang 330027, China)
  • Received:2013-02-04 Online:2013-11-25 Published:2013-11-25
  • Contact: TU Dongbo

摘要:

本研究主要在Leighton, Gierl和Hunka (2004)的四种属性层级结构测验情景下, 考察不同被试知识状态(knowledge states)分布形态、不同样本容量和不同认知属性数3种实验条件下, 分别比较、分析五种常用认知诊断模型的属性诊断正确率(含边际判准率和模式判准率)及其影响因素, 从而深入探讨每种模型的计量性能及模型属性诊断正确率的影响因素等, 试图为实际应用者在模型比较与选用上提供借鉴和指导。

关键词: 认知属性, 认知诊断, 认知诊断模型, 属性诊断正确率

Abstract:

Cognitive diagnosis is an important topic in modern psychometric area. Now more than 70 cognitive diagnosis models (CDMs) are developed. There are some questions among these models: (1) When the attribute hierarchy structure is known, how to choose the most suitable model? (2) When the attribute hierarchy structure is unknown, and cognitive diagnosis is required, how to do it? These problems seems especially more puzzled for the practice workers. This paper only paid main attention on three international popular models. Therefore, five cognitive diagnosis models (RSM, AHM_A, GDD, DINA and DINA_HC) were compared corresponding to the above questions from psychometric opinion. In this paper, Monte Carlo simulation study was used. Although the number of slips and the hierarchy structure are two important factors that affect the performance on corrected match ratio of cognitive diagnosis, this study would pay attention on other three factors: the distribution of cognitive pattern, the sample size, the number of attributes. The findings identified: (1) When the characteristic of data was known, focusing on specific factor, the five methods had different advantages. a) For the distributions of cognitive pattern, although they have different effects on different methods, the same conclusion could find that the performance on negative bias distribution was the best, and that of DINA_HC and DINA were better than the rest methods on any discussed distributions. b) Considering the sample size, the performance of GDD with small scale assessment (100/20, persons/items)was the best one; with medium and large scale assessment (1000/60, 5000/100, persons/items), the performance of DINA_HC and DINA were better than the rest c) For the number of attributes, the more the attributes are the worse the performance will be. But for the methods, the performance the performance of DINA_HC and DINA were also better than the rest. All these reflected that the most suitable method could be adapt from the three methods: GDD, DINA_HC and DINA, corresponding to the real scenario. And the RSM was the worst cognitive diagnosis method. (2) When the characteristic of data was unknown, an unstructured attribute hierarchy is treated as a coarsened version of a structured one, and the DINA method has a similar performance under unstructured hierarchy with the GDD, DINA_HC methods under structured hierarchy. Thus if the hierarchy structure could not be identified clearly, and the test Q matrix was clear, then the DINA could be adapt.

Key words: cognitive attributes, cognitive diagnosis, cognitive diagnosis models, attributes Match Ratio