Exploring Bi-Directional Context for Improved Chatbot Response Generation Using Deep Reinforcement Learning

Abstract

The development of conversational agents that can generate relevant and meaningful replies is a challenging task in the field of natural language processing. Context and predictive capabilities are crucial factors that humans rely on for effective communication. Prior studies have had a significant limitation in that they do not adequately consider the relationship between utterances in conversations when generating responses. This study aims to address this limitation by proposing a novel method that comprehensively models the contextual information of the current utterance for response generation. A commonly used approach is to rely on the information of the current utterance to generate the corresponding response, and as such it does not take advantage of the context of a multi-turn conversation. In our proposal, different from other studies, we will use a bi-directional context in which the historical direction helps the model remember information from the past in the conversation, while the future direction enables the model to anticipate its impact afterward. We combine a Transformer-based sequence-to-sequence model and the reinforcement learning algorithm to achieve our goal. Experimental results demonstrate the effectiveness of the proposed model through qualitative evaluation of some generated samples, in which the proposed model increases 24% average BLEU score and 29% average ROUGE score compared to the baseline model. This result also shows that our proposed model improves from 5% to 151% for the average BLEU score compared with previous related studies.