Abstract:

Due to the limitation of the Parameter Diagnosis model, in recent years, researchers begin to explore the nonparametric diagnosis method which is simpler and more efficient, such as SVM, the machine learning method based on statistical learning theory which is raised by Vapnik according to the risk minimization principle. SVM not only has simple structure, but can also use small sample which is quite time saving and efficient. Chiu and other fellows raised clustering method of 0-1 grading based on the idea of Sum-Scores. In order to match the practical evaluation, researchers developed clustering method of 0-1 grading into multi- grading, and discussed how sample size, percentage of random errors and attribute hierarchy structure impact on the class accuracy. The result indicates that GRCDM shows quite high class accuracy in both simulation and practice situation, little dependence on the sample size and compactness of attribute hierarchy structure, and is adaptable to small scale evaluation. All those characteristics show the advantage of nonparametric method. Nevertheless, the recent research on the nonparametric method is still superficial, so further efforts are needed to explore influence factor, investigate deep into the advantages and features of GRCDM, and enrich the study of nonparametric method with the help of the existing achievement of parameter method. By analyzing the influence factors of the diagnostic on evaluation class accuracy through three well- designed simulate studies from both test and subject layer, this research studies the five factors including the number of attribute, sample distribute, attribute hierarchy structure, attribute hierarchy structure misspecification and Q-matrix misspecification which cause the influence to the GRCDM, tries to investigate its performance comprehensively, and drives the research of the nonparametric diagnostic method. The result indicates thatGRCDM shows no dependence on the number of attribute and the more attribute the test has, the higher class accuracy this method has. Secondly, the sample distribution shows no influence on the class accuracy of GRCDM, reflecting the advantage that nonparametric method has no requirement of ability distribution. Thirdly, GRCDM has a great sensitivity to the attribute hierarchy structure misspecification, especially when for divergent or unstructured attribute hierarchy structure and the disorder of attribute hierarchy structure will lead to the maximum decreasing. Lastly, the influence on GRCDM caused by Q-matrix misspecification is differentiated by the attribute hierarchy structure, of which divergent and convergent will be influenced less, and the decreasing amplitude of the class accuracy of unstructured and linear attribute hierarchy structure will be maximum when both over and under specification of Q-matrix entries for items. This research is the penetration and extension based on the former research. Through the three simulation studies, some significant conclusions have been got, some of them being peculiar to nonparametric method, and some shared with parameter method. All in all, through this research, we can know more about the features of GRCDM. Knowing the advantage of nonparametric method and the difference from parameter method will provide useful information to both theoretical research and practical application of cognitive diagnostic assessment.

Key words: grade response cluster diagnostic method, attributes number, subject’s ability distribute, attribute hierarchy misspecification, Q matrix misspecification