A Monte Carlo Based Test for Between-study Heterogeneity in Meta-Analysis

This R package mc.heterogeneity provides functions for testing between-study heterogeneity in meta-analysis of standardized mean differences (d), Fisher-transformed Pearson’s correlations (r), and natural-logarithm-transformed odds ratio (OR).

Inclusion of moderators is an option for researchers who are interested in measuring the between-study heterogeneity per se and exploring factors that can explain the systematic between-study heterogeneity.

In the following examples, we describe how to use our package mc.heterogeneity to test the between-study heterogeneity for each of the effect sizes considered in the current study. Datasets, R codes, and output are provided for each example so that applied researchers can easily replicate the examples and modify the codes for their own datasets.

We also include a “Empirical Illustration” section in the main text of the article to discuss the following examples.

0. Installation of the package

Install the released version of mc.heterogeneity from CRAN with:

install.packages("mc.heterogeneity")

Or install the development version from GitHub with:

#install.packages("devtools")
library(devtools)
devtools::install_github("gabriellajg/mc.heterogeneity", force = TRUE, build_vignettes = TRUE)
library(mc.heterogeneity)

1. Standardized Mean Differences (d)

mc.d() is the function to test the between-study heterogeneity in meta-analysis of standardized mean differences (d).

1.1 Without moderators

Load the example dataset selfconcept first:

Extract the required arguments from selfconcept:

If g is a list of biased estimates of standardized mean differences in the meta-analytical study, a small-sample adjustment must be applied:

Run the heterogeneity test using mc.d():

Alternatively, such an adjustment could be performed within the function by specifying adjust = TRUE:

mc.run and mc.run2 will return the same results:

1.2 With moderators

Load an hypothetical dataset hypo_moder first:

Three moderators (cov.z1, cov.z2, cov.z3) are included:

Again, run the heterogeneity test using mc.d() with all moderators included in a matrix mods and model type specified as model = 'mixed':

In the presence of moderators, the results in mc.run3 will be different from those in mc.run and mc.run2:

In the presence of moderators, the function above tests whether the variability in the true standardized mean differences after accounting for the moderators included in the model is larger than sampling variability alone (Viechtbauer, 2010).

For the following two examples (Fisher-transformed Pearson’s correlations r; Natural-logarithm-transformed odds ratio OR), no moderators are included, but one can simply include moderators as in section 1.2.

2. Fisher-transformed Pearson’s correlations (r)

mc.fcor() is the function to test the between-study heterogeneity in meta-analysis of Fisher-transformed Pearson’s correlations (r).

Load the example dataset sensation first:

Extract the required arguments from sensation:

Run the heterogeneity test using mc.fcor():

The test of between-study heterogeneity has the following results:

3. Natural-logarithm-transformed odds ratio (OR)

mc.lnOR() is the function to test the between-study heterogeneity in meta-analysis of Natural-logarithm-transformed odds ratio (OR).

Load the example dataset smoking from R package HSAUR2:

Extract the required arguments from smoking:

The log odds ratios can be computed, but they are not needed by mc.lnOR():

Run the heterogeneity test using mc.lnOR():

The test of between-study heterogeneity has the following results: