Abstract
AbstractBackgroundMassive growth in the amount of research data and computational analysis has led to increased utilization of pipeline managers in biomedical computational research. However, each of more than 100 such managers uses its own way to describe pipelines, leading to difficulty porting workflows to different environments and therefore poor reproducibility of computational studies. For this reason, the Common Workflow Language (CWL) was recently introduced as a specification for platform-independent workflow description, and work began to transition existing pipelines and workflow managers to CWL.FindingsHere, we present CWL-Airflow, an extension for the Apache Airflow pipeline manager supporting CWL. CWL-Airflow utilizes CWL v1.0 specification and can be used to run workflows on standalone MacOS/Linux servers, on clusters, or on a variety of cloud platforms. A sample CWL pipeline for processing of ChIP-Seq data is provided.ConclusionsCWL-Airflow will provide users with the features of a fully-fledged pipeline manager and an ability to execute CWL workflows anywhere Airflow can run—from a laptop to cluster or cloud environment.AvailabilityCWL-Airflow is available under Apache license v.2 and can be downloaded from https://barski-lab.github.io/cwl-airflow, http://doi.org/10.5281/zenodo.2669582, RRID: SCR_017196.
Publisher
Cold Spring Harbor Laboratory
Reference28 articles.
1. A review of bioinformatic pipeline frameworks
2. Existing Workflow Systems [Internet]. Available from: https://s.apache.org/existing-workflow-systems
3. Common Workflow Language [Internet]. Available from: http://www.commonwl.org/
4. Docker [Internet]. Available from: https://www.docker.com/why-docker
5. Kurtzer GM , Sochat V , Bauer MW . Singularity: Scientific containers for mobility of compute. PLoS One. 2017;12.
Cited by
5 articles.
订阅此论文施引文献
订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献