Affiliation:
1. Department of Computer Science, Vanderbilt University, Nashville, Tennessee
2. School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, Texas
Abstract
Objective
This article aims to assess the reproducibility of Manufacturer and User Facility Device Experience (MAUDE) data-driven studies by analyzing the data queries used in their research processes.
Methods
Studies using MAUDE data were sourced from PubMed by searching for “MAUDE” or “Manufacturer and User Facility Device Experience” in titles or abstracts. We manually chose articles with executable queries. The reproducibility of each query was assessed by replicating it in the MAUDE Application Programming Interface. The reproducibility of a query is determined by a reproducibility coefficient that ranges from 0.95 to 1.05. This coefficient is calculated by comparing the number of medical device reports (MDRs) returned by the reproduced queries to the number of reported MDRs in the original studies. We also computed the reproducibility ratio, which is the fraction of reproducible queries in subgroups divided by the query complexity, the device category, and the presence of a data processing flow.
Results
As of August 8, 2022, we identified 523 articles from which 336 contained queries, and 60 of these were executable. Among these, 14 queries were reproducible. Queries using a single field like product code, product class, or brand name showed higher reproducibility (50%, 33.3%, 31.3%) compared with other fields (8.3%, P = 0.037). Single-category device queries exhibited a higher reproducibility ratio than multicategory ones, but without statistical significance (27.1% versus 8.3%, P = 0.321). Studies including a data processing flow had a higher reproducibility ratio than those without, although this difference was not statistically significant (42.9% versus 17.4%, P = 0.107).
Conclusions
Our findings indicate that the reproducibility of queries in MAUDE data-driven studies is limited. Enhancing this requires the development of more effective MAUDE data query strategies and improved application programming interfaces.
Publisher
Ovid Technologies (Wolters Kluwer Health)
Reference338 articles.
1. Analysis: using the FDA MAUDE and medical device recall databases to design better devices;Biomed Instrum Technol,2020
2. Adverse events and complications associated with intrathecal drug delivery systems: insights from the Manufacturer and User Facility Device Experience (MAUDE) database;Neuromodulation,2021
3. Postmarket surveillance and returned product analysis: success but not transparency;Heart Rhythm,2013
4. Conflicting information from the Food and Drug Administration: missed opportunity to lead standards for safe and effective medical artificial intelligence solutions [published correction appears in J Am Med Inform Assoc. 2021 May 5];J Am Med Inform Assoc,2021
5. Trust and transparency in medical device regulation;BMJ,2019