Abstract:

Since Preacher and Kelley (2011) proposed kappa-squared (k²) as a mediation effect size measure, it has become popular in mediation analyses, as shown by its appearance in research literature (e.g., Athay, 2012; Field, 2013). Furthermore, a special on-line calculator for computing kappa-squared also became available, making its use in research practice very convenient. Unfortunately, Wen and Fan (2015) recently demonstrated both logically and mathematically that kappa-squared has fatal flaws in its definition and calculation, which should put an end to its use in mediation analysis. This article evaluates the appropriateness of the current mediation effect size measures, based on the considerations of the expected characteristics of an effect size.

Effect size plays at least two roles in research practice. First, it provides supplemental information that compensates for the limitation of null hypothesis significance testing (NHST). Second, it makes the research findings comparable across studies in which different measures may have been used. For example, in the context of difference analysis involving two groups, the mean group difference is often the quantity of our research interest. When statistically “significant” difference is revealed by NHST, we are informed that the difference between the two group means is statistically different beyond what would be expected as a result of sampling error; but we are not entirely clear about how large the difference is. Primarily for this reason, it has been advocated that an effect size measure be used to supplement the statistical NHST (Fan & Konold, 2010; Wilkinson & the Task Force on Statistical Inference, 1999). Why can’t we directly report the effect (such as the mean group difference) that represents the original quantity of interest? It turns out that the original quantity (e.g., mean group difference) is usually not comparable across studies because different measures across the studies usually have different and arbitrary measurement scales (e.g., 5-point difference on two different tests may have very different meanings). Because of these difficulties, an effect size is often constructed as a scale-free index to represent the original quantity of interest. When the NHST result is supplemented by an effect size, it is more likely that both statistical and practical meanings of an analysis finding can be better understood and conveyed.

To serve its purpose, an effect size should have some basic characteristics, including being scale-free, being monotonic with respect to the effect that it represents, and being independent of sample size. It was the lack of monotonicity that kappa-squared was called into question by Wen and Fan (2015). They showed that the problem of kappa-squared is due to (1) the improper calculation of the maximum possible value of the indirect effect, and (2) mathematically, the maximum possible indirect effect is infinity, implying that the definition of kappa-squared is mathematically incorrect.

Several R²-type effect size measures for mediation effect have been proposed, such as De Heus’s (2012) , MacKinnon’s (2008) ,  and . But all these measures are not monotonic with respect to the mediation effect. Lachowicz’s (2015)  is obviously a monotonically increasing function of the mediation effect in absolute value. However, it is more difficult to understand and explain than the original mediation effect ab itself.

The traditional mediation effect size  (the ratio of the indirect effect to the total effect) is not perfect as a mediation effect size by itself. But when accompanied by the total effect and indirect effect in standardized form, it is meaningful for a basic mediation model where the indirect effect ab and the direct effect  have the same sign. For inconsistent mediation models where the indirect effect ab and the direct effect  have opposite signs,  is not appropriate as a mediation effect size measure, and what is a suitable effect size in this situation is still an issue to be addressed in future research.

Key words: mediation effect, indirect effect, effect size, kappa-squared