考虑题目选项信息的非参数认知诊断计算机自适应测验

doi:10.3724/SP.J.1041.2022.01137

摘要/Abstract

摘要：

选择题中的作答选项能提供额外诊断信息, 为充分利用选项信息, 研究提出认知诊断计算机自适应测验(CD-CAT)中两种处理选择题选项信息的非参数选题策略和变长终止规则。模拟研究的结果发现：(1)定长条件下两种非参数选题策略的分类准确性整体要高于参数选题策略; (2)两种非参数选题策略较参数选题策略具有更加均衡的题库使用情况; (3)非参数选题策略在两种新的变长终止规则下具有更高的分类准确率; (4)两种非参数选题策略均适用于选择题CD-CAT情境, 使用者可任选其一进行测验分析。

关键词: 认知诊断计算机自适应测验, 题目选项信息, 非参数选题策略, 变长终止规则

Abstract:

Most existing cognitive diagnostic computerized adaptive testing (CD-CAT) item selection methods ignore the diagnostic information that distractors provide for multiple-choice (MC) items. Consequently, some useful information is missed and resources are wasted. To overcome this, researchers proposed the Jensen-Shannon divergence (JSD) strategy to select items with the MC-DINA model. However, the JSD strategy needs large samples to obtain reliable estimates of the item parameters before the formal test, and this could compromise the items in the bank. By contrast, the nonparametric method does not require any parameter calibration before the formal test and can be used in small educational programs.
The current study proposes two nonparametric item selection methods (i.e., HDDmc and JDDmc) for CD-CAT with MC items as well as two termination rules (i.e., MR and DR) for variable-length CD-CAT with MC items. Two simulation studies were conducted to examine the performance of these nonparametric item selection methods and termination rules.
The first study examined the performance of the HDDmc and JDDmc with fixed-length CD-CAT. In this study, six factors were manipulated: the number of attributes (K = 4 vs. 6), the structure of the Q-matrix (simple vs. complex), the quality of the item bank (high vs. low vs. mixed), the distribution of the attribute profile (multivariate normal threshold model vs. discrete uniform distribution), the test length (two vs. three vs. four times of K), and the item selection methods (HDDmc vs. JDDmc vs. JSD). Of these, item selection method was the within-group variable, and the rest were between-group variables. The results showed that: (1) the HDDmc and JDDmc produced higher attribute pattern matched ratios (PMRs) than the JSD method for most conditions; (2) the HDDmc and JDDmc produced similar PMRs for all conditions; (3) the HDDmc and JDDmc produced more even distributions of item exposure than the JSD method.
The second simulation study investigated the performance of the MR and DR with variable-length CD-CAT. Six factors were also manipulated in this study: the settings for the number of attributes, the structure of the Q-matrix, the quality of the item bank, and the distribution of the attribute profile were the same as in the first study; the other two factors were termination rules (MR, DR, D1, and D3) and item selection methods (HDDmc and JDDmc). Again, the first four were between-group variables, while termination rules and item selection methods were within-group variables. The results showed that: (1) the HDDmc and JDDmc yielded higher PMRs for MR and DR rules than for the D1 and D3 rules; (2) the HDDmc and JDDmc yielded longer test lengths for MR and DR rules than for the D1 and D3 rules, especially for the JDD rule.
In sum, both nonparametric item selection methods and the two new termination rules proved appropriate for CD-CAT with MC items, which means they can be used to balance the trade-off between measurement accuracy and item exposure rate.

Key words: cognitive diagnostic computerized adaptive testing, multiple-choice items, nonparametric item selection method, termination rule

中图分类号:

B841

孙小坚, 郭磊. (2022). 考虑题目选项信息的非参数认知诊断计算机自适应测验. 心理学报, 54(9), 1137-1150.

SUN Xiaojian, GUO Lei. (2022). Nonparametric cognitive diagnostic computerized adaptive testing using multiple-choice option information. Acta Psychologica Sinica, 54(9), 1137-1150.

图/表 5

参考文献 43

[1]	Bradshaw, L., & Templin, J. (2014). Combining item response theory and diagnostic classification models: A psychometric model for scaling ability and diagnosing misconceptions. Psychometrika, 79(3), 403-425. doi: 10.1007/s11336-013-9350-4 pmid: 25205005
[2]	Chang, H.-H. (2015). Psychometrics behind computerized adaptive testing. Psychometrika, 80(1), 1-20. doi: 10.1007/s11336-014-9401-5 URL
[3]	Chang, Y.-P., Chiu, C.-Y., & Tsai, R.-C. (2019). Nonparametric CAT for CD in educational settings with small samples. Applied Psychological Measurement, 43(7), 543-561. doi: 10.1177/0146621618813113 URL
[4]	Chen, P., Li, Z., & Xin, T. (2011). A note on the uniformity of item bank usage in cognitive diagnostic computerized adaptive testing. Studies of Psychology and Behavior, 9(2), 125-132.
	[陈平, 李珍, 辛涛. (2011). 认知诊断计算机化自适应测验的题库使用均匀性初探. 心理与行为研究, 9(2), 125-132.]
[5]	Cheng, Y. (2009). When cognitive diagnosis meets computerized adaptive testing: CD-CAT. Psychometrika, 74(4), 619-632. doi: 10.1007/s11336-009-9123-2 URL
[6]	Chiu, C. Y., & Chang, Y. P. (2021). Advances in CD-CAT: The general nonparametric item selection method. Psychometrika, 86(4), 1039-1057. doi: 10.1007/s11336-021-09792-z URL
[7]	Chiu, C. Y., Douglas, J. A., & Li, X. D. (2009). Cluster analysis for cognitive diagnosis: Theory and applications. Psychometrika, 74(4), 633-665. doi: 10.1007/s11336-009-9125-0 URL
[8]	Chiu, C. Y., Sun, Y., & Bian, Y. H. (2018). Cognitive diagnosis for small educational programs: The general nonparametric classification method. Psychometrika, 83(2), 355-375. doi: 10.1007/s11336-017-9595-4 URL
[9]	de la Torre, J. (2009). A cognitive diagnosis model for cognitively based multiple-choice options. Applied Psychological Measurement, 33(3), 163-183. doi: 10.1177/0146621608320523 URL
[10]	de la Torre, J. (2011). The generalized DINA model framework. Psychometrika, 76, 179-199. doi: 10.1007/s11336-011-9207-7 URL
[11]	Gao, Y., Zhai, X., Cui, Y., Xin, T., & Bulut, O. (2021). Re-validating a learning progression of buoyancy for middle school students: A longitudinal study. Research in Science Education. Advance online publication. https://doi.org/10.1007/s11165-021-10021-x.
[12]	Guo, L., Yang, J., & Song, N. Q. (2018). Application of spectral clustering algorithm under various attribute hierarchical structures for cognitive diagnostic assessment. Journal of Psychological Science, 41(3), 735-742.
	[郭磊, 杨静, 宋乃庆. (2018). 谱聚类算法在不同属性层级结构诊断评估中的应用. 心理科学, 41(3), 735-742.]
[13]	Guo, L., Zheng, C., Bian, Y. (2015). Exposure control methods and termination rules in variable-length cognitive diagnostic computerized adaptive testing. Acta Psychologica Sinica, 47(1), 129-140. doi: 10.3724/SP.J.1041.2015.00129 URL
	[郭磊, 郑蝉金, 边玉芳. (2015). 变长CD-CAT中的曝光控制与终止规则. 心理学报, 47(1), 129-140.]
[14]	Guo, L., Zheng, C., Bian, Y., Song, N., & Xia, L. (2016). New item selection methods in cognitive diagnostic computerized adaptive testing: Combining item discrimination indices. Acta Psychologica Sinica, 48(7), 903-914. doi: 10.3724/SP.J.1041.2016.00903 URL
	[郭磊, 郑蝉金, 边玉芳, 宋乃庆, 夏凌翔. (2016). 认知诊断计算机化自适应测验中新的选题策略: 结合项目区分度指标. 心理学报, 48(7), 903-914.]
[15]	Guo, L., & Zhou, W. (2021). Nonparametric methods for cognitive diagnosis to multiple-choice test items. Acta Psychologica Sinica, 53(9), 1032-1043. doi: 10.3724/SP.J.1041.2021.01032 URL
	[郭磊, 周文杰. (2021). 基于选项层面的认知诊断非参数方法. 心理学报, 53(9), 1032-1043.]
[16]	He, M. (2021). Research on nonparametric cognitive diagnosis method and item selection strategy of nonparametric CD-CAT (Unpublished master's thesis). Sichuan Normal University, Chengdu, China.
	[何明霜. (2021). 非参数认知诊断方法与非参数 CD-CAT 选题策略研究 (硕士毕业论文). 四川师范大学, 成都.]
[17]	Hsu, C.-L., Wang, W.-C., & Chen, S.-Y. (2013). Variable- length computerized adaptive testing based on cognitive diagnosis models. Applied Psychological Measurement, 37(7), 563-582. doi: 10.1177/0146621613488642 URL
[18]	Huang, H.-Y. (2018). Effects of item calibration errors on computerized adaptive testing under cognitive diagnosis models. Journal of Classification, 35(3), 437-465. doi: 10.1007/s00357-018-9265-y URL
[19]	Jaccard, P. (1912). The distribution of the flora in the alpine zone. New Phytologist, 11(2), 37-50. doi: 10.1111/j.1469-8137.1912.tb05611.x URL
[20]	Kang, C. H., Ren, P., & Zeng, P. F. (2015). Nonparametric cognitive diagnosis: A cluster diagnostic method based on grade response items. Acta Psychologica Sinica, 47(8), 1077-1088. doi: 10.3724/SP.J.1041.2015.01077 URL
	[康春花, 任平, 曾平飞. (2015). 非参数认知诊断方法:多级评分的聚类分析. 心理学报, 47(8), 1077-1088.]
[21]	Kang, C. H., Yang, Y. K., & Zeng, P. F. (2019). Approach to cognitive diagnosis: The Manhattan distance discriminating method. Journal of Psychological Science, 42(2), 455-462.
	[康春花, 杨亚坤, 曾平飞. (2019). 一种混合计分的非参数认知诊断方法:曼哈顿距离判别法. 心理科学, 42(2), 455-462.]
[22]	Kaplan, M., de la Torre, J., & Barrada, J. R. (2015). New item selection methods for cognitive diagnosis computerized adaptive testing. Applied Psychological Measurement, 39(3), 167-188. doi: 10.1177/0146621614554650 URL
[23]	Kosub, S. (2019). A note on the triangle inequality for the Jaccard distance. Pattern Recognition Letters, 120, 36-38. doi: 10.1016/j.patrec.2018.12.007
[24]	Liu, C., & Cheng, Y. (2018). An application of the support vector machine for attribute-by-attribute classification in cognitive diagnosis. Applied Psychological Measurement, 42(1), 58-72. doi: 10.1177/0146621617712246 pmid: 29881112
[25]	Liu, C.-W., Andersson, B., & Skrondal, A. (2020). A constrained Metropolis-Hastings Robbins-Monro algorithm for Q matrix estimation in DINA models. Psychometrika, 85(2), 322-357. doi: 10.1007/s11336-020-09707-4 URL
[26]	Liu, H. Y., You, X. F., Wang, W.-Y., Ding, S.-L., & Chang, H.-H. (2013). The development of computerized adaptive testing with cognitive diagnosis for an English achievement test in China. Journal of Classification, 30(2), 152-172. doi: 10.1007/s00357-013-9128-5 URL
[27]	Liu, T. (2016). Using distractor information in computerized adaptive testing (Unpublished doctoral dissertation). Beijing Normal University, Beijing.
	[刘拓. (2016). 干扰项信息在计算机化自适应测验中的利用 (博士学位论文). 北京师范大学, 北京.]
[28]	Luo, Z., Li, Y., Yu, X., Gao, C., & Peng, Y. (2015). A simple cognitive diagnosis method based on Q-matrix theory. Acta Psychologica Sinica, 47(2), 264-272. doi: 10.3724/SP.J.1041.2015.00264 URL
	[罗照盛, 李喻骏, 喻晓锋, 高椿雷, 彭亚风. (2015). 一种基于 Q 矩阵理论朴素的认知诊断方法. 心理学报, 47(2), 264-272.]
[29]	Mao, X. Z., & Xin, T. (2013). A comparison of item selection methods for controlling exposure rate in cognitive diagnostic computerized adaptive testing. Acta Psychologica Sinica, 45(6), 694-703. doi: 10.3724/SP.J.1041.2013.00694 URL
	[毛秀珍, 辛涛. (2013). 认知诊断CAT中项目曝光控制方法的比较. 心理学报, 45(6), 694-703.]
[30]	Ozaki, K. (2015). DINA models for multiple-choice items with few parameters: Considering incorrect answers. Applied Psychological Measurement, 39(6), 431-447. doi: 10.1177/0146621615574693 URL
[31]	Sun, X., Andersson, B., & Xin, T. (2021). A new method to balance measurement accuracy and attribute coverage in cognitive diagnostic computerized adaptive testing. Applied Psychological Measurement, 45(7-8), 463-476.
[32]	Sun, X., Liu, Y., Xin, T., & Song, N. (2020). The impact of item calibration error on variable-length cognitive diagnostic computerized adaptive testing. Frontiers in Psychology, 141(11), Article e575141. https://10.3389/fpsyg.2020.575141
[33]	Sun, X., Mao, X., Song, N., & Xin, T. (2021). New methods for item exposure control in cognitive diagnostic computerized adaptive testing. Journal of Psychological Science, 44(1), 205-213.
	[孙小坚, 毛秀珍, 宋乃庆, 辛涛. (2021). 定长CD-CAT中两种新的题目曝光控制方法. 心理科学, 44(1), 205-213.]
[34]	Sun, X., Wang, Y., Zhang, S., & Xin, T. (2019). New methods to balance attribute coverage for cognitive diagnostic computerized adaptive testing. Journal of Psychological Science, 42(5), 1236-1244.
	[孙小坚, 王钰彤, 张世夷, 辛涛. (2019). 认知诊断计算机自适应测验中平衡属性收敛的新方法. 心理科学, 42(5), 1236-1244.]
[35]	Wang, W. Y., Ding, S. L., Song, L. H., Kuang, Z., & Cao, H. Y. (2016). Application of neural networks and support vector machines to cognitive diagnosis. Journal of Psychological Science, 39(4), 777-782.
	[汪文义, 丁树良, 宋丽红, 邝铮, 曹慧媛. (2016). 神经网络和支持向量机在认知诊断中的应用. 心理科学, 39(4), 777-782.]
[36]	Xin, T., Le, M., & Guo, Y., & Jiang, Y. (2015). The approach to establishing achievement standard: The learning progressions based on cognition diagnostic. Journal of Educational Studies, 5, 72-79.
	[辛涛, 乐美玲, 郭艳芳, 姜宇. (2015). 学业质量标准的建立途径:基于认知诊断的学习进阶方法. 教育学报, 5, 72-79.]
[37]	Xu, G., Wang, C., & Shang, Z. (2016). On initial item selection in cognitive diagnostic computerized adaptive testing. British Journal of Mathematical and Statistical Psychology, 69(3), 291-315. doi: 10.1111/bmsp.12072 URL
[38]	Xu, X. L., Chang, H.-H., & Douglas, J. (2003). A simulation study to compare CAT strategies for cognitive diagnosis. Paper presented at the Paper presented at the annual meeting of National Council on Measurement in Education, Montreal, Canada.
[39]	Yamaguchi, K. (2020). Variational Bayesian inference for the multiple-choice DINA model. Behaviormetrika, 47(1), 159-187. doi: 10.1007/s41237-020-00104-w URL
[40]	Yang, J., Chang, H.-H., Tao, J., & Shi, N. (2020). Stratified item selection methods in cognitive diagnosis computerized adaptive testing. Applied Psychological Measurement, 44(5), 346-361. doi: 10.1177/0146621619893783 URL
[41]	Yigit, H. D., Sorrel, M. A., & de la Torre, J. (2019). Computerized adaptive testing for cognitively based multiple-choice data. Applied Psychological Measurement, 43(5), 388-401. doi: 10.1177/0146621618798665 URL
[42]	Zhang, S. (2019). Applying npCD-CAT based on MDD to the field of number and algebra (Unpublished master's thesis). Zhejiang Normal University, Jinhua, China.
	[张淑君. (2019). 基于MDD的npCD-CAT研究及其在数与代数领域的应用 (硕士毕业论文). 浙江师范大学, 金华.]
[43]	Zheng, C. J., & Chang, H.-H. (2016). High-efficiency response distribution-based item selection algorithms for short-length cognitive diagnostic computerized adaptive testing. Applied Psychological Measurement, 40(8), 608-624. doi: 10.1177/0146621616665196 URL

题目质量	测验长度	诊断方法	简单Q矩阵				复杂Q矩阵
题目质量	测验长度	诊断方法	χ²	TOR	UIR	OIR	χ²	TOR	UIR	OIR
高	2K	HDDmc	1.343	0.018	0.663	0.000	1.214	0.017	0.663	0.000
		JDDmc	1.325	0.017	0.667	0.000	1.210	0.017	0.659	0.000
		JSD	148.184	0.324	0.931	0.022	102.493	0.229	0.899	0.022
	3K	HDDmc	1.620	0.026	0.277	0.000	1.432	0.026	0.265	0.000
		JDDmc	1.638	0.026	0.281	0.000	1.431	0.026	0.259	0.000
		JSD	171.692	0.381	0.905	0.045	121.801	0.277	0.869	0.043
	4K	HDDmc	1.932	0.035	0.096	0.000	1.659	0.035	0.078	0.000
		JDDmc	1.932	0.035	0.093	0.000	1.660	0.035	0.077	0.000
		JSD	187.661	0.423	0.875	0.059	133.228	0.310	0.839	0.060
低	2K	HDDmc	1.236	0.017	0.669	0.000	1.167	0.017	0.666	0.000
		JDDmc	1.216	0.017	0.660	0.000	1.135	0.017	0.668	0.000
		JSD	187.023	0.405	0.934	0.023	139.115	0.305	0.909	0.023
	3K	HDDmc	1.392	0.026	0.256	0.000	1.273	0.026	0.242	0.000
		JDDmc	1.413	0.026	0.256	0.000	1.250	0.026	0.239	0.000
		JSD	194.582	0.429	0.902	0.039	152.600	0.342	0.873	0.043
	4K	HDDmc	1.581	0.035	0.064	0.000	1.418	0.034	0.064	0.000
		JDDmc	1.581	0.035	0.065	0.000	1.425	0.034	0.065	0.000
		JSD	199.925	0.449	0.872	0.050	165.428	0.377	0.840	0.053
混合	2K	HDDmc	1.260	0.017	0.664	0.000	1.169	0.017	0.665	0.000
		JDDmc	1.292	0.017	0.663	0.000	1.143	0.017	0.662	0.000
		JSD	173.083	0.376	0.938	0.028	142.062	0.311	0.918	0.021
	3K	HDDmc	1.509	0.026	0.264	0.000	1.343	0.026	0.249	0.000
		JDDmc	1.537	0.026	0.268	0.000	1.378	0.026	0.255	0.000
		JSD	191.497	0.423	0.901	0.035	173.288	0.385	0.898	0.037
	4K	HDDmc	1.737	0.035	0.082	0.000	1.546	0.035	0.072	0.000
		JDDmc	1.763	0.035	0.083	0.000	1.589	0.035	0.071	0.000
		JSD	192.640	0.434	0.868	0.051	181.812	0.411	0.867	0.052

题目质量	测验长度	诊断方法	简单Q矩阵				复杂Q矩阵
题目质量	测验长度	诊断方法	χ²	TOR	UIR	OIR	χ²	TOR	UIR	OIR
高	2K	HDDmc	1.343	0.018	0.663	0.000	1.214	0.017	0.663	0.000
		JDDmc	1.325	0.017	0.667	0.000	1.210	0.017	0.659	0.000
		JSD	148.184	0.324	0.931	0.022	102.493	0.229	0.899	0.022
	3K	HDDmc	1.620	0.026	0.277	0.000	1.432	0.026	0.265	0.000
		JDDmc	1.638	0.026	0.281	0.000	1.431	0.026	0.259	0.000
		JSD	171.692	0.381	0.905	0.045	121.801	0.277	0.869	0.043
	4K	HDDmc	1.932	0.035	0.096	0.000	1.659	0.035	0.078	0.000
		JDDmc	1.932	0.035	0.093	0.000	1.660	0.035	0.077	0.000
		JSD	187.661	0.423	0.875	0.059	133.228	0.310	0.839	0.060
低	2K	HDDmc	1.236	0.017	0.669	0.000	1.167	0.017	0.666	0.000
		JDDmc	1.216	0.017	0.660	0.000	1.135	0.017	0.668	0.000
		JSD	187.023	0.405	0.934	0.023	139.115	0.305	0.909	0.023
	3K	HDDmc	1.392	0.026	0.256	0.000	1.273	0.026	0.242	0.000
		JDDmc	1.413	0.026	0.256	0.000	1.250	0.026	0.239	0.000
		JSD	194.582	0.429	0.902	0.039	152.600	0.342	0.873	0.043
	4K	HDDmc	1.581	0.035	0.064	0.000	1.418	0.034	0.064	0.000
		JDDmc	1.581	0.035	0.065	0.000	1.425	0.034	0.065	0.000
		JSD	199.925	0.449	0.872	0.050	165.428	0.377	0.840	0.053
混合	2K	HDDmc	1.260	0.017	0.664	0.000	1.169	0.017	0.665	0.000
		JDDmc	1.292	0.017	0.663	0.000	1.143	0.017	0.662	0.000
		JSD	173.083	0.376	0.938	0.028	142.062	0.311	0.918	0.021
	3K	HDDmc	1.509	0.026	0.264	0.000	1.343	0.026	0.249	0.000
		JDDmc	1.537	0.026	0.268	0.000	1.378	0.026	0.255	0.000
		JSD	191.497	0.423	0.901	0.035	173.288	0.385	0.898	0.037
	4K	HDDmc	1.737	0.035	0.082	0.000	1.546	0.035	0.072	0.000
		JDDmc	1.763	0.035	0.083	0.000	1.589	0.035	0.071	0.000
		JSD	192.640	0.434	0.868	0.051	181.812	0.411	0.867	0.052

题目质量	终止规则	诊断方法	简单Q矩阵^a					复杂Q矩阵^a
题目质量	终止规则	诊断方法	M	Min	Max	UIR	PMR	M	Min	Max	UIR	PMR
高	MR	HDDmc	9.274	7	25.033	0.520	0.712	9.369	7	26.333	0.661	0.775
	MR	JDDmc	9.300	7	25.367	0.511	0.710	9.402	7	25.600	0.659	0.768
	DR	HDDmc	13.785	5	30.000	0.134	0.724	11.767	5	30.000	0.473	0.738
	DR	JDDmc	14.876	5	30.000	0.086	0.745	12.853	5	30.000	0.363	0.752
	D1	HDDmc	5.308	5	8.733	0.849	0.496	5.289	5	9.400	0.751	0.514
	D1	JDDmc	5.303	5	8.667	0.846	0.490	5.293	5	9.200	0.752	0.508
	D3	HDDmc	7.939	7	12.833	0.650	0.648	7.914	7	13.433	0.728	0.703
	D3	JDDmc	7.964	7	12.900	0.651	0.651	7.920	7	13.600	0.726	0.702
低	MR	HDDmc	10.204	7	28.667	0.414	0.450	10.482	7	29.167	0.577	0.509
	MR	JDDmc	10.199	7	28.367	0.415	0.441	10.460	7	29.133	0.582	0.514
	DR	HDDmc	19.536	5	30.000	0.009	0.629	17.237	5	30.000	0.097	0.648
	DR	JDDmc	20.838	5	30.000	0.003	0.641	18.545	5	30.000	0.068	0.663
	D1	HDDmc	5.431	5	9.333	0.839	0.288	5.430	5	10.300	0.750	0.303
	D1	JDDmc	5.423	5	9.333	0.841	0.293	5.418	5	10.367	0.751	0.310
	D3	HDDmc	8.308	7	13.833	0.612	0.396	8.319	7	14.900	0.716	0.445
	D3	JDDmc	8.315	7	13.800	0.608	0.397	8.303	7	14.733	0.719	0.434
混合	MR	HDDmc	9.762	7	26.400	0.463	0.591	9.961	7	27.233	0.620	0.666
	MR	JDDmc	9.765	7	25.867	0.466	0.595	9.915	7	27.733	0.619	0.665
	DR	HDDmc	16.321	5	30.000	0.042	0.720	13.902	5	30.000	0.277	0.711
	DR	JDDmc	17.570	5	30.000	0.027	0.729	15.141	5	30.000	0.192	0.724
	D1	HDDmc	5.368	5	8.833	0.839	0.379	5.364	5	10.033	0.750	0.416
	D1	JDDmc	5.368	5	9.000	0.845	0.391	5.352	5	9.700	0.750	0.418
	D3	HDDmc	8.138	7	13.200	0.633	0.521	8.135	7	14.467	0.722	0.585
	D3	JDDmc	8.131	7	13.233	0.629	0.530	8.125	7	13.867	0.723	0.589

题目质量	终止规则	诊断方法	简单Q矩阵^a					复杂Q矩阵^a
题目质量	终止规则	诊断方法	M	Min	Max	UIR	PMR	M	Min	Max	UIR	PMR
高	MR	HDDmc	9.274	7	25.033	0.520	0.712	9.369	7	26.333	0.661	0.775
	MR	JDDmc	9.300	7	25.367	0.511	0.710	9.402	7	25.600	0.659	0.768
	DR	HDDmc	13.785	5	30.000	0.134	0.724	11.767	5	30.000	0.473	0.738
	DR	JDDmc	14.876	5	30.000	0.086	0.745	12.853	5	30.000	0.363	0.752
	D1	HDDmc	5.308	5	8.733	0.849	0.496	5.289	5	9.400	0.751	0.514
	D1	JDDmc	5.303	5	8.667	0.846	0.490	5.293	5	9.200	0.752	0.508
	D3	HDDmc	7.939	7	12.833	0.650	0.648	7.914	7	13.433	0.728	0.703
	D3	JDDmc	7.964	7	12.900	0.651	0.651	7.920	7	13.600	0.726	0.702
低	MR	HDDmc	10.204	7	28.667	0.414	0.450	10.482	7	29.167	0.577	0.509
	MR	JDDmc	10.199	7	28.367	0.415	0.441	10.460	7	29.133	0.582	0.514
	DR	HDDmc	19.536	5	30.000	0.009	0.629	17.237	5	30.000	0.097	0.648
	DR	JDDmc	20.838	5	30.000	0.003	0.641	18.545	5	30.000	0.068	0.663
	D1	HDDmc	5.431	5	9.333	0.839	0.288	5.430	5	10.300	0.750	0.303
	D1	JDDmc	5.423	5	9.333	0.841	0.293	5.418	5	10.367	0.751	0.310
	D3	HDDmc	8.308	7	13.833	0.612	0.396	8.319	7	14.900	0.716	0.445
	D3	JDDmc	8.315	7	13.800	0.608	0.397	8.303	7	14.733	0.719	0.434
混合	MR	HDDmc	9.762	7	26.400	0.463	0.591	9.961	7	27.233	0.620	0.666
	MR	JDDmc	9.765	7	25.867	0.466	0.595	9.915	7	27.733	0.619	0.665
	DR	HDDmc	16.321	5	30.000	0.042	0.720	13.902	5	30.000	0.277	0.711
	DR	JDDmc	17.570	5	30.000	0.027	0.729	15.141	5	30.000	0.192	0.724
	D1	HDDmc	5.368	5	8.833	0.839	0.379	5.364	5	10.033	0.750	0.416
	D1	JDDmc	5.368	5	9.000	0.845	0.391	5.352	5	9.700	0.750	0.418
	D3	HDDmc	8.138	7	13.200	0.633	0.521	8.135	7	14.467	0.722	0.585
	D3	JDDmc	8.131	7	13.233	0.629	0.530	8.125	7	13.867	0.723	0.589

题目质量	终止规则	诊断方法	简单Q矩阵^a					复杂Q矩阵^b
题目质量	终止规则	诊断方法	M	Min	Max	UIR	PMR	M	Min	Max	UIR	PMR
高	MR	HDDmc	11.814	9	29.067	0.381	0.536	12.226	9	29.000	0.381	0.536
	MR	JDDmc	11.845	9	28.967	0.380	0.529	12.218	9	29.400	0.380	0.529
	DR	HDDmc	12.628	7	30.000	0.351	0.485	11.571	7	30.000	0.351	0.485
	DR	JDDmc	16.169	7	30.000	0.113	0.603	14.532	7	30.000	0.113	0.603
	D1	HDDmc	7.320	7	11.167	0.574	0.319	7.295	7	11.900	0.574	0.319
	D1	JDDmc	7.317	7	11.300	0.576	0.316	7.287	7	11.700	0.576	0.316
	D3	HDDmc	9.988	9	15.767	0.493	0.440	10.015	9	17.533	0.493	0.440
	D3	JDDmc	9.984	9	15.600	0.496	0.442	9.992	9	16.533	0.496	0.442
低	MR	HDDmc	12.946	9	29.900	0.307	0.273	13.599	9	30.000	0.307	0.273
	MR	JDDmc	12.934	9	29.867	0.308	0.274	13.463	9	30.000	0.308	0.274
	DR	HDDmc	16.994	7	30.000	0.098	0.353	15.286	7	30.000	0.098	0.353
	DR	JDDmc	21.679	7	30.000	0.014	0.457	19.653	7	30.000	0.014	0.457
	D1	HDDmc	7.428	7	12.000	0.572	0.141	7.434	7	12.900	0.572	0.141
	D1	JDDmc	7.436	7	11.800	0.569	0.146	7.427	7	12.967	0.569	0.146
	D3	HDDmc	10.332	9	16.533	0.484	0.210	10.396	9	17.833	0.484	0.210
	D3	JDDmc	10.352	9	16.867	0.482	0.213	10.404	9	18.367	0.482	0.213
混合	MR	HDDmc	12.468	9	29.633	0.335	0.400	12.834	9	29.833	0.335	0.400
	MR	JDDmc	12.438	9	29.633	0.337	0.389	12.954	9	29.967	0.337	0.389
	DR	HDDmc	14.683	7	30.000	0.204	0.423	12.770	7	30.000	0.204	0.423
	DR	JDDmc	19.093	7	30.000	0.036	0.559	16.560	7	30.000	0.036	0.559
	D1	HDDmc	7.385	7	11.533	0.570	0.212	7.376	7	12.733	0.570	0.212
	D1	JDDmc	7.390	7	11.533	0.573	0.210	7.367	7	12.933	0.573	0.210
	D3	HDDmc	10.202	9	16.233	0.486	0.314	10.198	9	17.533	0.486	0.314
	D3	JDDmc	10.179	9	16.500	0.488	0.309	10.234	9	17.800	0.488	0.309