Abstract:

Comparing to the nonadaptive testing, the major advantage of computerized adaptive testing (CAT) is that the examinees achieve the same degree of measurement precision (i.e., fixed precision). But few studies are devoted to the termination rules in variable-length cognitive diagnostic computerized adaptive testing (CD-CAT). Inspired by the termination rule research in traditional CAT, this paper proposed four termination rules for variable-length CD-CAT. The new termination rules were standard error of attribute method (SEA), difference of the adjacent posterior probability method (DAPP), halving algorithm (HA) and hybrid method (HM), respectively. Then, the four new termination rules were compared with the HSU and KL method under two scenarios: with and without item exposure control. Three exposure control methods were considered, i.e., simple, modified restrictive progressive (MRP) and modified restrictive threshold (MRT) method. The MRP and MRT methods were extension of the Wang et al.’s (2011) work to the variable-length CD-CAT scenario. The results indicated that: (1) When the criterion of variable-length termination rule was conservative, the mean of the test length and the percentage of examinees reaching the maximum test length were large, and the classification accuracy rate for examinees who finished the CAT using fixed precision was high. (2) Without the item exposure control, the four new variable-length termination rules had a similar performance compared to the HSU method. With the increase of maximum posterior probability and the decrease of e, the classification accuracy rate and the mean test length presented a increasing trend. But the item pool usage was unsatisfactory. (3) With the item exposure control, item pool usage was greatly improved in the six variable-length termination rules while the classification accuracy rates were maintained. Different exposure control methods had a different effect on the different variable-length termination rules. The relative criterion termination rules such as DAPP and KL methods were easily affected by the item exposure control. (4) Taken all together, the SEA, HM, and HA methods were comparable to the HSU method, and followed by the KL and DAPP method. Some future directions were suggested in the end of this paper.

Key words: cognitive diagnostic computerized adaptive testing, variable-length termination rule, exposure control, classification accuracy rate, DINA model