Summary Augmenter: A Text Augmentation Framework to Improve Summarization Quality

Abstract

Data augmentation in Natural Language Processing (NLP) faces various challenges that hinder its widespread adoption, unlike its ever-present usage in the field of vision. It is even more the case for the text summarization task where one should focus on both article and summary. In this paper, we review the effect of back translation augmentation, present the diverse beam search decoding strategy, and masking as a method to generate synthetic data for text summarization. The approaches will be evaluated by ROUGE score, novelty, summary length, and GPT-4 to analyze their effectiveness. Our proposed framework presents multiple combinations of back translation and masking for articles, along with diverse augmentation for summaries. Although applicable to networks of any size, we decided to use BART-large, a relatively smaller model, in order to conduct a larger number of experiments. The experiments demonstrated superior performance across all specified metrics when compared to fine-tuning BART-large on the CNN/Dailymail dataset. Specifically, we showed a significant improvement in novelty; 158% and 56% increase rate for bigrams and unigrams, respectively. It could eliminate some copyright concerns around generating content similar to human writing. Additionally, the GPT-4 assessment indicates that models trained using the augmentation technique tend to capture important information more effectively than the baseline model.