Multilevel Meta-Analysis

library(tidyverse)
library(metafor)

theme_set(theme_minimal())

color_primary <- "#2171b5"
color_secondary <- "#888888"
color_reference <- "gray50"

When to use a multilevel model

In a typical meta-analysis, each study contributes one effect size and all variance is treated as a single quantity: how much do studies differ from each other? When studies contribute multiple effect sizes — from different outcomes, subgroups, or time points — there are actually two distinct sources of variance:

• Within-study variance — effect sizes vary across outcomes within the same study
• Between-study variance — average effects vary from study to study

These are separate phenomena with different explanations. Within-study variance might reflect that some outcomes are more sensitive to a treatment than others. Between-study variance might reflect differences in sample characteristics, context, or methodology across labs. A multilevel model estimates them separately; a standard model conflates them into one heterogeneity estimate.

Once variance is decomposed this way, you can model each source with predictors at the appropriate level — outcome-level variables to explain within-study variance, study-level variables to explain between-study variance. In a standard model, any moderator analysis is ambiguous about which level it’s really operating at.

A typical dataset has multiple rows per study:

set.seed(42)

n_studies <- 20
effects_per_study <- sample(1:5, n_studies, replace = TRUE)

dat <- tibble(
  study = rep(1:n_studies, times = effects_per_study),
  outcome = sequence(effects_per_study),
  yi = rnorm(sum(effects_per_study), mean = 0.3, sd = 0.25),
  vi = runif(sum(effects_per_study), 0.01, 0.06)
)

dat

Each row is one effect size; multiple rows share the same study. The multilevel model treats this nesting as a real feature of the data to be modeled, not a statistical nuisance to be corrected.