ARGminer: A web platform for crowdsourcing-based curation of antibiotic resistance genes

Abstract

ABSTRACTCuration of antibiotic resistance gene (ARG) databases is labor intensive and requires expert knowledge to manually collect, correct, and/or annotate individual genes. Consequently, most existing ARG databases contain only a small number of ARGs (~5k genes) and updates to these databases tend to be infrequent, commonly requiring years for completion and often containing inconsistencies. Thus a new approach is needed to achieve a truly comprehensive ARG database while also maintaining a high level of accuracy. Here we propose a new web-based curation system, ARGminer, that supports the annotation and inspection of several key attributes of potential ARGs, including gene name, antibiotic category, resistance mechanism, evidence for mobility and occurrence in clinically-important bacterial strains. Here we employ crowdsourcing as a novel strategy to overcome limitations of manual curation and expand curation capacity towards achieving a truly comprehensive and perpetually up-to-date database. Further, machine learning is employed as a powerful means to validate database curation, drawing from natural language processing to infer correct and consistent nomenclature for each potential ARG. We develop and validate the crowdsourcing approach by comparing performances of multiple cohorts of curators with varying levels of expertise, demonstrating that ARGminer is a time and cost efficient means of achieving accurate ARG curation. We further demonstrate the reliability of a trust validation filter for rejecting input generated by spammers. Crowdsourcing was found to be as accurate as expert annotation, with an accuracy >90% for the annotation of a diverse test set of ARGs. The ARGminer public search platform and database is available at http://bench.cs.vt.edu/argminer.