Evaluation of the Distributed Strategies for Data Parallel Deep Learning Model in TensorFlow

Abstract

Distributed deep learning is a branch of machine intelligence in which the runtime of deep learning models may be dramatically lowered by using several accelerators. Most of the past research reports the performance of the data parallelism technique of DDL. Nevertheless, additional parallelism solutions in DDL must be investigated, and their performance modeling for specific applications and application stacks must be reported. Such efforts may aid other researchers in making more informed judgments while creating a successful DDL algorithm. Distributed deep learning strategies are becoming increasingly popular as they allow for training complex models on large datasets in a much shorter time than traditional training methods. TensorFlow, a popular open-source framework for building and training machine learning models, provides several distributed training strategies. This chapter provides a detailed evaluation of the different TensorFlow strategies for medical data. The TensorFlow distribution strategy API is utilized to perform distributed training in TensorFlow.