Please wait a minute...
   2011, Vol. 43 Issue (07) : 821-829     DOI:
|
Evaluating Test Reliability:From Coefficient Alpha to Internal Consistency Reliability
WEN Zhong-Lin;YE Bao-Juan
(1 Center for Studies of Psychological Application, South China Normal University, Guangzhou 510631, China)
(2 Hong Kong Examinations and Assessment Authority, Hong Kong, China)
Download: PDF(407 KB)  
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks    
Abstract  In the research of psychology and other social sciences, test reliability is often used to reflect measurement stability and consistency. Coefficient α is the most popular indicator of test reliability. Recent years, however, coefficient α was challenged now and again. Is coefficient α still recommended for evaluating test reliability? If not, what should replace it?
With the classical concept of reliability, which is defined as the ratio of true variance to observed variance on a test under consideration, we introduced the relationship between test reliability and coefficient α, and the limitations of coefficient α. The concepts closely related to coefficient α were considered. We clearly defined homogeneity reliability and internal consistency reliability. Homogeneity reflects the presence of a general factor, whereas internal consistency relates the presence of common factors (including a general factor and local factors). For unidimensional tests, homogeneity and internal consistency are the same concept. Investigating the relationship between test reliability, coefficient α, homogeneity reliability, and internal consistency reliability, we showed that homogeneity reliability is not larger than internal consistency reliability, and that the latter is not larger than test reliability; coefficient α usually underestimates internal consistency reliability, and the latter is closer to test reliability.
For ordinary use, the errors of items in a test are reasonably uncorrelated. Under the assumption that the total score of the test is meaningful, we proposed a guideline for evaluating test reliability. If coefficient α is high enough to be accepted, then the test reliability is also acceptable whether the test is unidimensional or not. In this case, using coefficient α to evaluate test reliability is the first choice. If the coefficient α is not large enough, we should calculate internal consistency reliability which is also known as composite reliability in literatures. If the internal consistency reliability is high enough to be accepted, then the test reliability is also acceptable. An operational procedure was summarized for reliability analysis based on the above guideline.
We illustrated how to calculate homogeneity reliability and internal consistency reliability by using factor analysis. A LISREL program was developed to calculate them for a multidimensional test. A simplified version of the program was also made for a unidimensional test. The programs are almost the same as those for ordinary confirmatory factor analysis and can be managed easily by empirical researchers.
Keywords reliability      coefficient a      homogeneity      internal consistency      composite reliability     
Corresponding Authors: WEN Zhong-Lin   
Issue Date: 30 July 2011
Service
E-mail this article
E-mail Alert
RSS
Articles by authors
WEN Zhong-Lin
YE Bao-Juan
Cite this article:   
WEN Zhong-Lin,YE Bao-Juan. Evaluating Test Reliability:From Coefficient Alpha to Internal Consistency Reliability[J]. , 2011, 43(07): 821-829.
URL:  
http://journal.psych.ac.cn/xlxb/EN/      OR     http://journal.psych.ac.cn/xlxb/EN/Y2011/V43/I07/821
[1] WANG Wenyi;SONG Lihong;DING Shuliang. Classification accuracy and consistency indices for complex decision rules in multidimensional item response theory[J]. Acta Psychologica Sinica, 2016, 48(12): 1612-1624.
[2] LUO Zhaosheng;GUO Xiaojun. The Optimal Size of Material in Psychological Experiment: The Applications of Multivariate Generalizability Theory[J]. Acta Psychologica Sinica, 2014, 46(6): 876-884.
[3] YE Baojuan;WEN Zhonglin. Estimating Homogeneity Coefficient and Its Confidence Interval[J]. Acta Psychologica Sinica, 2012, 44(12): 1687-1694.
[4] YE Bao-Juan,WEN Zhong-Lin. A Comparison of Three Confidence Intervals of Composite Reliability of A Unidimensional Test[J]. , 2011, 43(04): 453-461.
[5] LIU Qiang,HU Zhong-Hua,ZHAO Guang,TAO Wei-Dong,ZHANG Qing-Lin,SUN Hong-Jin. The Prior Knowledge of the Reliability of Sensory Cues Affects the Multisensory Integration in the Early Perceptual Processing Stage[J]. , 2010, 42(02): 227-234.
[6] WANG Hui,WU Chao-Yan,ZHANG Yan,Chao C. CHEN. The Dimensionality and Measure of Empowering Leadership Behavior in the Chinese Organizations[J]. , 2008, 40(12): 1297-1305.
[7] Xiao Lijun,Miao Danmin,Xiao Wei,Wu Shengjun,Li Hongzheng. Personality Measurement for Recruit Selection[J]. , 2007, 39(02): 362-370.
[8] Li-Tonggui,Kato-Kazuo. Measuring Adult Attachment: Chinese Adaptation of the ECR Scale[J]. , 2006, 38(03): 399-406.
[9] Liu Xuefeng,Zhang Zhixue. PROCESS OF INTERACTION AMONG MEMBERS IN SIMULATED WORK TEAMS[J]. , 2005, 37(02): 253-259.
[10] Chen Tianyong,Li Deming. THE DIVERSITY OF EXECUTIVE FUNCTIONS IN NORMAL ADULTS: A LATENT VARIABLE ANALYSIS[J]. , 2005, 37(02): 210-217.
[11] Wang Dengfeng, Cui Hong. RELIABILITIES AND VALIDITIES OF THE CHINESE PERSONALITY SCALE[J]. , 2004, 36(03): 347-358.
[12] Yang Zhiming,Chang Lei,Ma Shiye. MULTIVARIATE GENERALIZABILITY ANALYSIS OF THE CHINESE COLLEGE ENTRANCE COMPREHENSIVE EXAMINATION[J]. , 2004, 36(02): 195-200.
[13] Wan-Zhonglin,Hou Jietai. REGRESSION TOWARD THE MEAN IN PSYCHOLOGICAL TESTING AND THE REOCCURRENCE PROBABILITY OF EXTREME SCORES[J]. , 2003, 35(03): 419-425.
[14] Yan Fang, Li Weiming (Department of Psychology, East China Normal University, Shanghai 200062). USING STRUCTURAL EQUATION MODELS TO ESTIMATE RATER RELIABILITY IN GENERALIZABILITY THEORY[J]. , 2002, 34(05): 92-97.
[15] Shi Chengsun 1 , Dong Yan 1 , Hou Yubo 2 , Hou Guizhi 1 , Zhou Xiaomei 1 ( 1 Department of clinical Psychology, General Hospital of Chinese Air Force, Beijing 100036) ( 2 Department of Psychology Peking University, Beijing. PRELIMINARY DEVELOPMENT OF THE COPE SCALE[J]. , 2002, 34(04): 84-90.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
Copyright © Acta Psychologica Sinica
Support by Beijing Magtech