Hierarchical Meta-Analysis: A Simulation Study Comparing Classical Random Effects and Fully Bayesian Methods

Author

Nancy R. Andiloro, The Graduate Center, City University of New YorkFollow

Date of Degree

5-2018

Document Type

Dissertation

Degree Name

Ph.D.

Program

Educational Psychology

Advisor

David Rindskopf

Committee Members

Hannah Rothstein

Philip Ramsey

Subject Categories

Educational Assessment, Evaluation, and Research | Educational Psychology | Social and Behavioral Sciences | Social Statistics

Keywords

meta-analysis, metaanalysis, random effects, Bayesian methods, simulation

Abstract

Meta-analytic data have a natural hierarchical structure to them, where individuals are nested within studies, and have both within-and between-study variation to model. A random-effects hierarchical linear model is useful to conduct a meta-analysis because it allows one to appropriately parse out the two components of variation that exist within and across studies to determine an observed effect. Empirical Bayes estimation considers the reliability of variance estimates; when the reliability of the effect size estimate for a study is high, substantial weight is placed on that estimate. However, problems with estimation arise when the number of studies and their sample size is small. Although time-consuming to employ, fully Bayesian methods offer a solution, but few studies systematically compare random-effects to fully Bayesian methods. A simulation study was performed varying certain characteristics of meta-analyses, such as the number of studies, their sample size and level of heterogeneity across studies, to determine under which condition(s) a fully Bayesian method improves meta-analytic findings. Results are unexpectedly inconsistent, whereby certain scenarios in which the number of studies is small and level of heterogeneity large, show that empirical Bayes performs better than the fully Bayesian method. Despite this, bias and mean-squared error are lower, on average, among the fully Bayesian models, with a model specifying a Cauchy prior on τ performing best for the most favorable scenario. Implications and areas for future study are discussed.

Recommended Citation

Andiloro, Nancy R., "Hierarchical Meta-Analysis: A Simulation Study Comparing Classical Random Effects and Fully Bayesian Methods" (2018). CUNY Academic Works.
https://academicworks.cuny.edu/gc_etds/2579