Are experiment sample sizes adequate to detect biologically important interactions between multiple stressors?

Abstract

AbstractAs most ecosystems are being challenged by multiple, co-occurring stressors, an important challenge is to understand and predict how stressors interact to affect biological responses. A popular approach is to design factorial experiments that measure biological responses to pairs of stressors and compare the observed response to a null model expectation. Unfortunately, we believe experiment sample sizes are inadequate to detect most non-null stressor interaction responses, greatly hindering progress. Determination of adequate sample size requires (i) knowledge of the detection ability of the inference method being used, and (ii) a consideration of the smallest biologically meaningful deviation from the null expectation. However, (i) has not been investigated and (ii) is yet to be discussed. Consequently, typical experimental sample sizes are small (3-4 replicates), except for some individual-level studies where replicates may exceed 100. Re-analysis of recently published data highlights how small sample sizes can only detect very large deviations from the null model, implying many important non-null stressor-pair interactions are being missed, and how sample size only improves detection ability slowly. Critically, researchers still need to define the smallest interaction of interest, i.e. the lower limit for a biologically important interaction, which is likely to be system specific, meaning that a general guide is unavailable. However, we suspect that most experiments may require 20 or more replicates per treatment. Sample sizes could potentially be increased by focussing on individual-level responses to multiple stressors, or by forming coordinated networks of researchers to repeat experiments in larger-scale studies. Our analyses relate to the additive and multiplicative null models and we encourage similar analyses on other null models and inference methods for stressor interactions. Without knowledge of the detection abilities of the statistical tools at hand, or definition of the smallest meaningful interaction, we will undoubtedly continue to miss important ecosystem stressor interactions.