Date of Degree

5-2018

Document Type

Dissertation

Degree Name

Ph.D.

Program

Educational Psychology

Advisor

David Rindskopf

Committee Members

Hannah Rothstein

Philip Ramsey

Subject Categories

Educational Assessment, Evaluation, and Research | Educational Psychology | Social and Behavioral Sciences | Social Statistics

Keywords

meta-analysis, metaanalysis, random effects, Bayesian methods, simulation

Abstract

Meta-analytic data have a natural hierarchical structure to them, where individuals are nested within studies, and have both within-and between-study variation to model. A random-effects hierarchical linear model is useful to conduct a meta-analysis because it allows one to appropriately parse out the two components of variation that exist within and across studies to determine an observed effect. Empirical Bayes estimation considers the reliability of variance estimates; when the reliability of the effect size estimate for a study is high, substantial weight is placed on that estimate. However, problems with estimation arise when the number of studies and their sample size is small. Although time-consuming to employ, fully Bayesian methods offer a solution, but few studies systematically compare random-effects to fully Bayesian methods. A simulation study was performed varying certain characteristics of meta-analyses, such as the number of studies, their sample size and level of heterogeneity across studies, to determine under which condition(s) a fully Bayesian method improves meta-analytic findings. Results are unexpectedly inconsistent, whereby certain scenarios in which the number of studies is small and level of heterogeneity large, show that empirical Bayes performs better than the fully Bayesian method. Despite this, bias and mean-squared error are lower, on average, among the fully Bayesian models, with a model specifying a Cauchy prior on τ performing best for the most favorable scenario. Implications and areas for future study are discussed.

Share

COinS
 
 

To view the content in your browser, please download Adobe Reader or, alternately,
you may Download the file to your hard drive.

NOTE: The latest versions of Adobe Reader do not support viewing PDF files within Firefox on Mac OS and if you are using a modern (Intel) Mac, there is no official plugin for viewing PDF files within the browser window.