The reliability of a nutritional meta-analysis study

by   Karl E. Peace, et al.

Background: Many researchers have studied the relationship between diet and health. There are papers showing an association between the consumption of sugar-sweetened beverages and Type 2 diabetes. Many meta-analyses use individual studies that do not adjust for multiple testing or multiple modeling and thus provide biased estimates of effect. Hence the claims reported in a meta-analysis paper may be unreliable if the primary papers do not ensure unbiased estimates of effect. Objective: Determine the statistical reliability of 10 papers and indirectly the reliability of the meta-analysis study. Method: Ten primary papers used in a meta-analysis paper and counted the numbers of outcomes, predictors, and covariates. We estimated the size of the potential analysis search space available to the authors of these papers; i.e. the number of comparisons and models available. Since we noticed that there were differences between predictors and covariates cited in the abstract and in the text, we applied this formula to information found in the abstracts, Space A, as well as the text, Space T, of each primary paper. Results: The median and range of the number of comparisons possible across the primary papers are 6.5 and (2-12,288) for abstracts, and 196,608 and (3,072-117,117,952) the texts. Note that the median of 6.5 for Space A is misleading as each primary study has 60-165 foods not mentioned in the abstract. Conclusion: Given that testing is at the 0.05 level and the number of comparisons is very large, nominal statistical significance is very weak support for a claim. The claims in these papers are not statistically supported and hence are unreliable. Thus, the claims of the meta-analysis paper lack evidentiary confirmation.


page 1

page 2

page 3

page 4


The reliability of an environmental epidemiology meta-analysis, a case study

Summary Background Claims made in science papers are coming under incr...

Reliability of meta-analysis of an association between ambient air quality and development of asthma later in life

Claims from observational studies often fail to replicate. A study was u...

Evaluation of a meta-analysis of air quality and heart attacks, a case study

It is generally acknowledged that claims from observational studies ofte...

Statistical reliability of meta_analysis research claims for gas stove cooking_childhood respiratory health associations

Odds ratios or p_values from individual observational studies can be com...

Case Study: Evaluation of a meta-analysis of the association between soy protein and cardiovascular disease

It is well-known that claims coming from observational studies most ofte...

Evaluation of a meta-analysis of the association between red and processed meat and selected human health effects

Background: Risk ratios or p-values from multiple, independent studies, ...

Towards more scientific meta-analyses

Meta-analysis can be a critical part of the research process, often serv...

Please sign up or login with your details

Forgot password? Click here to reset