Statistical Significance and Biological Relevance

The Scientific Committee (SC) developed an opinion addressing the issue of statistical significance and biological relevance. The objective of the document is to help EFSA Scientific Panels and Committee in the assessment of biologically relevant effects.

The SC considered the distinction between the concepts of biological relevance and statistical significance and produced descriptions of the terms. It is suggested that EFSA Experts and Staff members should use the terminology of biological relevance and statistical significance as interpreted by the SC in their considerations.

The SC recommends that the nature and size of biological changes or differences seen in studies that would be considered relevant should be defined before studies are initiated. The size of such changes should be used to design studies with sufficient statistical power to be able to detect effects of such size if they truly occurred.

Statistical significance is considered as just one part of an appropriate statistical analysis of a well designed experiment or study. Identifying statistical significance should not be the primary objective of a statistical analysis. The relationship of statistical significance to the concept of hypothesis testing was considered and the limitations on the use of hypothesis testing in the risk assessment process when interpreting data were noted.

The SC therefore recommended that less emphasis should be placed upon the reporting of statistical significance and more on statistical point estimation and associated interval estimations (e.g. Confidence Interval) as more information can be presented using the latter.

In addition, the SC recommends that a complete description of the methods used, the programming code and the raw data are made available to the assessors so that alternative analyses could be conducted to test the robustness of any conclusions drawn.