Towards more scientific meta-analyses

by   Lily H. Zhang, et al.

Meta-analysis can be a critical part of the research process, often serving as the primary analysis on which the practitioners, policymakers, and individuals base their decisions. However, current literature synthesis approaches to meta-analysis typically estimate a different quantity than what is implicitly intended; concretely, standard approaches estimate the average effect of a treatment for a population of imperfect studies, rather than the true scientific effect that would be measured in a population of hypothetical perfect studies. We advocate for an alternative method, called response-surface meta-analysis, which models the relationship between the quality of the study design as predictor variables and its reported estimated effect size as the outcome variable in order to estimate the effect size obtained by the hypothetical ideal study. The idea was first introduced by Rubin several decades ago, and here we provide a practical implementation. First, we reintroduce the idea of response-surface meta-analysis, highlighting its focus on a scientifically-motivated estimand while proposing a straightforward implementation. Then we compare the approach to traditional meta-analysis techniques used in practice. We then implement response-surface meta-analysis and contrast its results with existing literature-synthesis approaches on both simulated data and a real-world example published by the Cochrane Collaboration. We conclude by detailing the primary challenges in the implementation of response-surface meta-analysis and offer some suggestions to tackle these challenges.


Causally-interpretable meta-analysis: clearly-defined causal effects and two case studies

Meta-analysis is commonly used to combine results from multiple clinical...

Estimating the sample mean and standard deviation from commonly reported quantiles in meta-analysis

Researchers increasingly use meta-analysis to synthesize the results of ...

Standard error estimation in meta-analysis of studies reporting medians

We consider the setting of an aggregate data meta-analysis of a continuo...

Content Analysis Application in Nursing: A Synthetic Knowledge Synthesis Meta-Study

Theoretical issues: With the explosive growth in the research literature...

Combining dependent p-values resulting from multiple effect size homogeneity tests in meta-analysis for binary outcomes

Testing effect size homogeneity is an essential part when conducting a m...

The reliability of a nutritional meta-analysis study

Background: Many researchers have studied the relationship between diet ...

Generalized inferential models for meta-analyses based on few studies

Meta-analysis based on only a few studies remains a challenging problem,...

Please sign up or login with your details

Forgot password? Click here to reset