Standard meta-analysis methods are not robust

by   S. Stanley Young, et al.

P values or risk ratios from multiple, independent studies, observational or randomized, can be computationally combined to provide an overall assessment of a research question in meta-analysis. There is a need to examine the reliability of these methods of combination. It is typical in observational studies to statistically test many questions and not correct the analysis results for multiple testing or multiple modeling, MTMM. The same problem can happen for randomized, experimental trials. There is the additional problem that some of the base studies may be using fabricated or fraudulent data. If there is no attention to MTMM or fraud in the base studies, there is no guarantee that the results to be combined are unbiased, the key requirement for the valid combining of results. We note that methods of combination are not robust; even one extreme base study value can overwhelm standard methods of combination. It is possible that multiple, extreme (MTMM or fraudulent) results can feed from the base studies to bias the combined result. A meta-analysis of observational (or even randomized studies) may not be reliable. Examples are given along with some methods to evaluate existing base studies and meta-analysis studies.


page 1

page 2

page 3

page 4


The reliability of an environmental epidemiology meta-analysis, a case study

Summary Background Claims made in science papers are coming under incr...

Particulate Matter Exposure and Lung Cancer: A Review of two Meta-Analysis Studies

The current regulatory paradigm is that PM2.5, over time causes lung can...

Bioenergetics modelling to analyse and predict the joint effects of multiple stressors: Meta-analysis and model corroboration

Understanding the consequences of the combined effects of multiple stres...

Background information for meta-analysis evaluation, Info 00-Info 06

Massive numbers of meta-analysis studies are being published. A Google S...

Case Study: Evaluation of a meta-analysis of the association between soy protein and cardiovascular disease

It is well-known that claims coming from observational studies most ofte...

A systematic approach to improving the reliability and scale of evidence from health care data

Concerns over reproducibility in science extend to research using existi...

Measurement Error in Meta-Analysis (MEMA) – a Bayesian framework for continuous outcome data

Ideally, a meta-analysis will summarize data from several unbiased studi...

Please sign up or login with your details

Forgot password? Click here to reset