Abstract:

Abnormal response behavior in psychological and educational tests compromises the reliability of the test and the validity of the resulting scores. In the context of academic achievement tests, such behaviors may result in inaccurate assessments of students' learning levels by teachers. Similarly, in questionnaires, these behaviors can impact the reliability of the questionnaires and the interpretation of the results. The potential negative consequences of these abnormal behaviors pose a significant threat to the security of the tests and the quality of the screening of the test administrators. At present, the prevailing approach to addressing the issue of test security is through the application of statistics. However, the increasing prevalence of diverse testing formats and the generation of substantial volumes of real-time process data have introduced novel considerations to the domain of test security. The incorporation of diverse test security detection processes with complex interactions poses a significant challenge for statistics. The analysis of these unstructured process data calls for the development of novel approaches that extend beyond latent feature modeling.

The application of machine learning methods is becoming increasingly prevalent in psychological and educational measurement research. Machine learning algorithms can learn from data and make predictions or decisions about unknown events without explicit instructions. These algorithms offer several advantages over traditional methods. Firstly, they are not limited by specific theories or assumptions and are designed to identify generalizable predictive patterns. Secondly, they can jointly model all variables related to the participants as input features, thus utilizing all available information. Thirdly, the training of machine learning models is often based on real data, reducing the problem of misfit between statistical models and empirical data. Finally, machine learning algorithms are highly efficient and capable of modeling and analyzing large amounts of assessment data in real time.

The review was divided into three principal sections. First, machine learning algorithms were classified into three principal categories: supervised, unsupervised, and semi-supervised learning methods. These categories were further subdivided into three subcategories: ensemble learning, deep learning, and transfer learning. Each study was included in a different broad category based on the underlying model used for the review. The theory of each machine learning method was first introduced, and then the application of the method is reviewed. The test security issues addressed in this study could be broadly classified into two categories: cheating in educational tests and careless responding in questionnaires. We then proceeded to examine the applicability of various machine learning methods across different test types and anomaly types. To conclude, we presented three practical recommendations for researchers and practitioners. (1) Obtaining high-quality labeled data for test security studies is challenging. There are three methods for obtaining labeled data: the simulation emulation method, the manual labeling method, and the SMOTE method. (2) Other techniques for initial data include missing value interpolation, data encoding, and feature scaling. (3) The selection of input features is also an important consideration. Finally, prospective avenues for future research were identified from the following perspectives: machine learning-based person-fit research, machine learning test security research based on multimodal data, test security research based on generative adversarial networks, and the interpretability of research results.

Key words: machine learning, psychological tests, educational tests, test security, statistics