Contrastive language and vision learning of general fashion concepts

Abstract

AbstractThe steady rise of online shopping goes hand in hand with the development of increasingly complex ML and NLP models. While most use cases are cast as specialized supervised learning problems, we argue that practitioners would greatly benefit from general and transferable representations of products. Inthiswork, we build on recent developments in contrastive learning to trainFashionCLIP, aCLIP-like model adapted for the fashion industry. We demonstrate the effectiveness of the representations learned byFashionCLIPwith extensive tests across a variety of tasks, datasets and generalization probes. We argue that adaptations of large pre-trained models such as CLIP offer new perspectives in terms of scalability and sustainability for certain types of players in the industry. Finally, we detail the costs and environmental impact of training, and release the model weights and code as open source contribution to the community.