关键词: 多题多做模型, EM算法, 参数估计精度

Abstract: Three one-parameter item response theory (IRT) models were proposed by Spray to describe score probabilities of an examinee who takes a multiple attempts, single-item (MASI) test of a psychomotor skill. However, if students are encouraged to check and modify their answers in a test, the phenomenon could be regarded as multiple-attempt, multiple-item (MAMI) test. To describe the MAMI test, a two-parameter IRT MAMI model (Binomial trails model) was proposed and an item parameter estimation procedure was formulated in this paper. Three assumptions about the model were made. The first two were the same as in the ordinary IRT, the unidimesionality and the local independence. The third assumption was another kind of local independence, which required that the individual trails or attempts be independent for a given examinee.
The model and the estimation procedure developed in this article were evaluated using simulated data. Test consisted 60 items and sample size was 1000 in this simulation. The simulated data were generated 50 times. The ability parameters, difficulty parameters, the logarithm of the discrimination parameters were drawn from the standard normal distribution N(0,1). Three different methods estimated procedure (MMLE/EM for MAMI model, MMLE/EM for BILOG, MMLE/EM for ordinary IRT) were used to analyze this simulation data. MMLE/EM for MAMI model means that the elements in the score matrix were the sum of the original score and the modified score (cumulating score scheme) when the examinees modified their answers. MMLE/EM for ordinary IRT means that the score matrix was the original score matrix and the modified score matrix obtained form (from?) repeated response were lengthened in row (as if the number of the examinees were double, in brief, lengthened score matrix) and were widened in column (as if the length of the test were double, in brief, widened score matrix) separately when the examinees modified their answers.
The mean of the absolute difference between the estimated and the correspondent simulated value of the parameters (ABS), the bias and root mean square error (SD) of the estimated values of the parameters were computed for each item parameter across 50 replications.
The results of simulations showed that:
1. The accuracy in terms of ABS and SD of estimating the ability parameters in MAMI model was higher than that obtained by MMLE/EM used in ordinary IRT for the score matrix being lengthened.
2. The accuracy of estimating the item parameters in MAMI model was higher than that obtained by MMLE/EM used in ordinary IRT for the score matrix being widened.
These findings indicate that when MAMI appears, the cumulating score scheme is more reasonable than the traditional scoring scheme, in which only the last response is collected. This finding may motivate researchers to consider how to score when the skill test allows reviewing and changing answers

Key words: multiple-attempt multiple-item model, EM algorithm, accuracy of estimating parameter