On Leveraging Large Language Models for Multilingual Intent Discovery

Abstract

Intent discovery is vital for any real-world dialogue systems such as chatbot. Since the intents of users naturally change over time, models only trained on a static training set of intents will inevitably fail to detect new intents. While this topic has been widely studied, existing work only focuses on monolingual datasets, rendering it less practical for international businesses where it is far more common to work with multilingual data. In this work, we present a method for multilingual intent discovery through leveraging the multilingual capabilities of recent large language models. By performing joint extraction of intent and keyphrases, as well as a chain-of-thought styled reasoning, our method is able to efficiently produce clustering results that are easy to interpret. Experimental results on two different datasets show that our proposed method consistently surpasses all baselines, with up to 15% gain in adjusted rand index.