ISSN 0439-755X
CN 11-1911/B
主办:中国心理学会
   中国科学院心理研究所
出版:科学出版社

心理学报 ›› 2025, Vol. 57 ›› Issue (10): 1849-1866.doi: 10.3724/SP.J.1041.2025.1849 cstr: 32110.14.2025.1849

• 研究报告 • 上一篇    下一篇

认知诊断测验最简完备Q矩阵统一设计方法

唐小娟1(), 毛萌萌2, 李瑜3(), 丁树良4, 彭志霞5   

  1. 1江西师范大学教育学院, 南昌 330022
    2南昌大学公共政策与管理学院, 南昌 330036
    3浙江工商大学马克思主义学院心理健康教育中心, 杭州 310018
    4江西师范大学计算机信息工程学院, 南昌 330022
    5浙江工商大学统计与数学学院, 杭州 310018
  • 收稿日期:2024-09-13 发布日期:2025-08-15 出版日期:2025-10-25
  • 通讯作者: 唐小娟, E-mail: 137622064@qq.com;
    李瑜, E-mail: mingliyuduo@126.com
  • 基金资助:
    江西省社会科学规划课题(22JY04);国家自然科学基金项目(31860278);国家自然科学基金项目(62467003);国家自然科学基金项目(62067005);国家自然科学基金项目(61967009);浙江工商大学高教课题(1270KU224034)

A unified design method of the simplest complete Q matrix for cognitive diagnostic tests

TANG Xiaojuan1(), MAO Mengmeng2, LI Yu3(), DING Shuliang4, PENG Zhixia5   

  1. 1School of Education, Jiangxi Normal University, Nanchang 330022, China
    2School of Public Policy and Administration, Nanchang University, Nanchang 330036, China
    3Mental Health Education Center of School of Marxism, Zhejiang Gongshang University, Hangzhou 310018, China
    4College of Computer Information Engineering, Jiangxi Normal University, Nanchang 330022, China
    5School of Statistics and Mathematics, Zhejiang Gongshang University, Hangzhou 310018, China
  • Received:2024-09-13 Online:2025-08-15 Published:2025-10-25

摘要:

属性水平(二分属性和多分属性)和项目理想评分方式(0-1评分与多级评分)是认知诊断测验设计的两个重要维度。其中, 多分属性测验能提供更详细的诊断信息, 而多级评分测验能提高判准率, 但现有认知诊断测验缺乏对多分属性和多级评分的整合设计。借鉴二分属性多级评分结构化/非结构化最简完备Q矩阵(SSCQM/USCQM)的概念, 本文提出统一的认知诊断测验最简完备Q矩阵设计方法, 解决不同属性水平和不同项目理想评分方式的各种组合情境下的认知诊断测验设计问题, 并在长测验和短测验两种条件下, 以(拟)可达阵为参照, 通过模拟研究对各种SSCQM和USCQM准确率进行了比较。结果表明,总体而言, SSCQM和USCQM具有更高的判准率。实证研究数据进一步验证了SSCQM和USCQM测验的优势。

关键词: 认知诊断测验, 测验设计, 最简完备Q矩阵, 统一设计方法

Abstract:

The quality of cognitive diagnose test (CDT) directly influences the performance of the diagnostic results and its remedial function. Thus, CDT design plays a vital role in the cognitive diagnosis process. A key challenge is maximizing differentiation among participants while minimizing the number of items. Although CDT is commonly applied in contexts involving polytomous attributes and responses, prior research often yields fragmented results, limiting practical application and theoretical development. A complete Q matrix represents a valuable tool that can be employed as a high-quality test for various types of tests. The (quasi-) reachability matrix is widely utilized in various testing procedures. This study aims to develop a unified, simplified testing design that is simpler than (quasi-) reachability matrix and improves usability and applicability across various cognitive diagnosis scenarios.

The simplest complete Q-matrix (SCQM) is a key concept in test design, offering a straightforward yet effective way to organize test items using fewer questions. This study proposes a unified construction model of SCQM applicable to various cognitive diagnostic test types. The Unstructured SCQM (USCQM) is developed from the Structured SCQM (SSCQM) to enhance organization. The proposed unified SSCQM model is based on the item attribute total score and selecting the SSCQM in the (quasi-) reachability matrix, specifically: (1) retaining the maximum column from each group; (2) keeping columns in each group with the distinct highest average attribute scores from the maximum columns. Following Tang Xiaojuan et al. (2024), a unified USCQM model for dichotomous attributes and polytomous responses is also proposed.

A simulation study was conducted to evaluate the effects of the number of attributes, attribute hierarchy, attribute levels, and complete Q matrices on the classification capabilities of SSCQM and USCQM in both long and short tests. Their performance was compared with the (quasi-) reachability and identity matrices. Results showed that classification accuracy generally declined with increasing attribute hierarchy, number of attributes, and attribute levels, except in long tests with multilevel attributes. Accuracy improved with more complete Q matrices. For short tests, SSCQM performed better with dichotomous responses, while USCQM was superior with polytomous responses. In long tests, the trend reversed. When attribute levels were 3 or 4, tests with polytomous attributes and responses achieved the highest accuracy. In other cases, binary attribute tests performed better. Empirical findings confirmed SSCQM and USCQM outperform the (quasi-) reachability matrix.

Polytomous attribute and response tests provide richer diagnostic information and higher accuracy, making them valuable in educational assessments. However, research on their design remains limited and fragmented. This study introduces a unified design model of the Simplest Complete Q Matrix (SSCQM and USCQM) for cognitive diagnostic tests based on dichotomous attributes and polytomous responses. Classification accuracy was assessed in both long and short tests. Simulation and empirical results show SSCQM and USCQM outperform or match the (quasi-) reachability matrix, with especially strong performance in short test scenarios.

Key words: cognitive diagnose test, the simplest complete Q matrix, test design, unified design method

中图分类号: