Abstract: The Chinese College Entrance Comprehensive Examination (CCECE), which consists of Geography, History, Political Studies, Biology, Chemistry and Physics, is a comprehensive achievement test to measure an examinee's academic readiness for tertiary education. This study used multivariate generalizability theory to estimate generalizability (reliability) coefficients of the CCECE. The G-study and D-study results suggested that the generalizability coefficient of the whole test was 0.78, which was considered moderate but acceptable. Doubling the items for all the subtests would increase generalizability coefficient to 0.88. Except for geography and political studies, reliability estimates for the subtests were acceptable. However, a composite universe score approach was not ideal both because variance distributions among subtests were inconsistent with the planned score weighting and because correlations among some subtests were low. For example, chemistry and history accounted for more variance than other subtests combined. Among the problematic subtests, geography contributed negatively to reliability of the whole test.

Key words: Multivariate Generalizability Theory, reliability, College Entrance Comprehensive Examination