Please wait a minute...
   2011, Vol. 43 Issue (04) : 453-461     DOI:
|
A Comparison of Three Confidence Intervals of Composite Reliability of A Unidimensional Test
YE Bao-Juan;WEN Zhong-Lin
(1 Center for Studies of Psychological Application, South China Normal University, Guangzhou 510631, China)
(2 Hong Kong Examinations and Assessment Authority, Hong Kong, China)
Download: PDF(254 KB)  
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks    
Abstract  The widely used coefficient a may underestimate or overestimate reliability when its premise assumption is violated and therefore is not a good index to evaluate reliability. Composite reliability can better estimate reliability by using confirmatory factor analysis (see e.g., Bentler, 2009; Green & Yang, 2009). As is well known, point estimate contains limited information about a population parameter and could not give how far it could be from the population parameter. The confidence interval of the parameter could provide more information. In evaluating the quality of a test, the confidence interval of composite reliability has received more and more attention in recent years.
There are three approaches to estimate the confidence interval of composite reliability of a unidimensional test: Bootstrap method, Delta method and directly using the standard error in the output of an SEM software (e.g., LISREL). Each of the three approaches produces a standard error of composite reliability. Then the confidence interval can be easily formed based on the standard error. Bootstrap method provides an empirical result of the standard error of composite reliability and is the most credible, but the method needs data simulation technique and is not be easily mastered by general applied researchers. Delta method computes the standard error of composite reliability by approximate calculation, and the method is much simpler than Bootstrap method. LISREL software can directly give the standard error of composite reliability, and this method is the simplest among the three methods.
To evaluate the standard errors of composite reliability obtained by Delta method and LISREL software, we compared them with that obtained by Bootstrap method, because the latter can be treated as the true value in theory. A simulation study was conducted to the comparison. Four factors were considered in the simulation design: (a) the number of items on each test (k=3, 6, 10, and 15); (b) factor loading (high, medium and low); (c) sample size (N=100, 300, 500, and 1000); (d) the method for calculating the standard error of composite reliability (Bootstrap, Delta, and LISREL). Totally, 48 treatment conditions were generated in terms of the above 4-factor simulation design (i.e., 48=4×3×4×3).
The simulation results indicated that the difference between the standard errors obtained by Delta method and Bootstrap method was ignorable under each designed condition, except when sample size was small (less than 200)and standardized factor loadings were not high (less than 0.7). However, there was substantial difference between the standard errors obtained by LISREL software directly and Bootstrap method under each designed condition. Noting that the result from Bootstrap method can be treated as the true value, we recommended that Delta method could be adopted to estimate the confidence interval of composite reliability of a unidimensional test. At the same time we revealed that the standard error directly obtained by LISREL software is severely biased.
We used an example of a unidimensional test to illustrate how to calculate composite reliability and its confidence interval by using Delta method based on LISREL output. We also showed that the same results could be directly obtained by using SEM software Mplus that automatically calculates the confidence interval with Delta method and presents the confidence interval.
Keywords composite reliability      confidence interval      Bootstrap method      Delta method      LISREL     
Corresponding Authors: WEN Zhong-Lin   
Issue Date: 30 April 2011
Service
E-mail this article
E-mail Alert
RSS
Articles by authors
YE Bao-Juan
WEN Zhong-Lin
Cite this article:   
YE Bao-Juan,WEN Zhong-Lin. A Comparison of Three Confidence Intervals of Composite Reliability of A Unidimensional Test[J]. , 2011, 43(04): 453-461.
URL:  
http://journal.psych.ac.cn/xlxb/EN/      OR     http://journal.psych.ac.cn/xlxb/EN/Y2011/V43/I04/453
[1] YE Baojuan;WEN Zhonglin. A Discussion on Testing Methods for Mediated Moderation Models: Discrimination and Integration[J]. Acta Psychologica Sinica, 2013, 45(9): 1050-1060.
[2] LI Guangming;ZHANG Minqiang. Using Adjusted Bootstrap to Improve the Estimation of Variance Components and Their Variability for Generalizability Theory[J]. Acta Psychologica Sinica, 2013, 45(1): 114-124.
[3] YE Baojuan;WEN Zhonglin. Estimating Homogeneity Coefficient and Its Confidence Interval[J]. Acta Psychologica Sinica, 2012, 44(12): 1687-1694.
[4] FANG Jie;ZHANG Min-Qiang. Assessing Point and Interval Estimation for the Mediating Effect: Distribution of the Product, Nonparametric Bootstrap and Markov Chain Monte Carlo Methods[J]. Acta Psychologica Sinica, 2012, 44(10): 1408-1420.
[5] WEN Zhong-Lin,YE Bao-Juan. Evaluating Test Reliability:From Coefficient Alpha to Internal Consistency Reliability[J]. , 2011, 43(07): 821-829.
[6] Bai Ligang,Lin Wenquan,Fang Liluo(Institute of Psychology Academia Sinica,Beijing,100012). CONFIRMITORY FACTOR ANALYSIS TO THE CONSTRUCT VALIDITY OF THE CHINESE VOCATIONAL INTEREST INVENTORY OF HOLLAND TYPE(Ⅱ):THE CONTRAST OF THE MTMM-LISREL,MODELS'[J]. , 1996, 28(02): 120-125.
[7] Meng Hongwei(Central Education Iastitute,Beijing 100088). MODEL BUILDING AND STRUCTURAL EQUATION MODELLING──A DISCUSSION WITH ZHANG JIANPING[J]. , 1994, 26(04): 437-440.
[8] Zhang Jianping Institute of Psychology, Academic Sinica. STRUCTURAL EQUATION MODELLING: A NEW KIND OF STATISTICAL AND RESEARCH METHOD[J]. , 1993, 25(01): 94-102.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
Copyright © Acta Psychologica Sinica
Support by Beijing Magtech