DRD-GAN: A Novel Distributed Conditional Wasserstein Deep Convolutional Relativistic Discriminator GAN with Improved Convergence

Abstract

Generative Adversarial Network (GAN) exhibited significant capabilities in many applications including image enhancement and manipulation, language translation, generating images/videos from text, creating art and music, and so on. However, train ing GANs using large datasets remains highly computationally intensive for most of the standalone systems. Additionally, standalone GANs often exhibit poor synchronization between their generator and discriminator with unstable training , poor convergence along with a large number of mode collapse s and vanishing / exploding gradients. Standalone GANs also failed to learn in a decentralized environment, where the data is distributed among several client machines. Some researchers have lately used the most prevalent decentralized setting available today, called Federated Learning (FL) to develop distributed -GAN strategies as the possible solutions, although their implementations mostly failed to address the above issues mainly because of: the training instability within the distributed component s, which eventually leads to the poor synchronization among the generator s and discriminator s scattered over several machines. In this work, we developed a computationally inexpensive Wasserstein conditional Distributed Relativistic Discriminator-GAN or DRD-GAN to alleviate the above issues. DRD-GAN stabilizes its train ing (with non-convex losses) by keeping a global generator in the central server and relativistic discriminator s in the local client s (one discriminator per client ), and uses Wasserstein-1 for computing local and global losses. It eventually avoids mode collapse s, vanishing/exploding gradient s (both in the presence of iid and non-iid samples) and helps DRD-GAN to produce high-quality fake images. Apart from that, the sheer unavailability of a capable conditional distributed -GAN model has become another motivation behind the current work. Essentially, we revisited the FL paradigms, locating one discriminator per client , and a generator in the central server that aggregates the updates coming from multiple discriminator s. Relativistic discriminator s in the client s are train ed on both iid and non-iid private data. We presented a detailed mathematical formulation of DRD-GAN and empirically evaluated our claims using CIFAR-10, MNIST, EuroSAT, and CelebA datasets.