Abstract: Both item response theory (IRT) and a multilevel model are used in a variety of social science research applications. The use of IRT allows researchers to link the observed categorical responses provided by students with an underlying unobservable trait, such as ability or attitude. A multilevel model allows the natural multilevel structure that is widely present in social science data to be represented formally in data analysis. In some cases, researchers may wish to study the effects of the covariates on the latent trait. These covariates may include information pertaining to responses as well as contextual information. Traditionally, manifest variables are used in a multilevel analysis as fixed and known entities. An important deficiency is that the measurement error associated with the test scores is ignored. In general, the use of unreliable test scores leads to a biased estimation of the regression coefficients; consequently, the resulting statistical inference can be rather misleading.
In this paper, a multilevel IRT model is presented where some of the variables cannot be observed directly but are measured using tests or questionnaires. A two-level model can be defined using a two-level formulation in which level 1 is the item-level model and level 2 is the person-level model. In such a model, the latent ability and item parameters can be estimated simultaneously. Further, we can consider the effects of the person-level covariates on the person-level ability. Similar to a traditional two-level model, the two-level IRT model can be expanded to a three-level model, which includes a higher site level. The three-level IRT model proposed in this paper yields the additional benefit of being able to accommodate data that are collected in a hierarchical setting. This expansion of multilevel IRT models to three levels allows not only the dependency typically found in hierarchical data to be accommodated but also the estimation of latent traits at different levels as well as the estimation of relationships between predictor variables and latent traits at different levels.
The purpose of this paper is to provide both a theoretic description and a practical application of the multilevel IRT model. First, we demonstrate the algebraic equivalence between the parameterizations of an IRT model and the multilevel IRT model. Second, we illustrate the method, using an application that involves students’ achievements on a mathematics test and test results regarding the characteristics of students and schools. Third, we expand the multilevel IRT model to the identification of differential item functioning (DIF). Finally, we present a discussion on the advantages and disadvantages of using multilevel IRT models in applied research and provide the scope for future research

Key words: Multilevel model, Item Response Theory Model, Multilevel Item Response Theory Model