The core of the matter: How do scientists judge trustworthiness of physical samples?

Abstract

In recent years, research funding agencies, universities, and governments have become increasingly concerned with promoting the reuse of research datasets. Enabling researchers to evaluate the trustworthiness and fitness-for-use of research datasets produced by others is critical for facilitating the reuse of these datasets. Understanding how researchers make these evaluations is crucial for developing digital infrastructure and tools, such as data repositories and metadata schema, in a way that better supports researchers in making these evaluations. Physical samples such as rocks are critical for generating datasets in many scientific domains. Often, samples are collected on field expeditions conducted by large infrastructural projects. These projects comprise many human and non-human components that affect the quality and integrity of samples. However, little is known about whether and how prospective dataset users evaluate the samples' trustworthiness and sample collection processes underlying these datasets. Researchers‘strategies for evaluating sample trustworthiness are explored through a longitudinal qualitative case study (ethnographic observation, interviews (n = 66), and document analysis) of subseafloor biosphere research, an earth sciences domain. Domain researchers use rock samples collected on research cruises conducted by the International Ocean Discovery Program (IODP). Subseafloor biosphere researchers are primarily concerned about samples being compromised by microbiological contamination. Researchers vary regarding the components of IODP infrastructure they consider when evaluating sample trustworthiness. These components include methods to process samples, people handling samples, IODP policies and procedures, and IODP organizational politics. Researchers‘strategies vary according to their disciplinary background, with microbiologists employing more fine-grained judgments about methods; whether researchers have participated in IODP expeditions, with those who have employing more fine-grained judgments about people involved; and whether researchers have ever been involved in organizing cruises or serving on IODP committees, with those who have employing more fine-grained judgments about many aspects of cruises. Researchers who make less complex decisions may be prone to erroneously trusting contaminated samples; researchers who make more complex decisions may be prone to erroneously discarding uncontaminated samples. The paper concludes by considering implications for the design of digital infrastructures to support researchers in evaluating sample trustworthiness.