Abstract
AbstractStyle transfer is a natural language processing generation task, it consists of substituting one given writing style for another one. In this work, we seek to perform informal-to-formal style transfers in the English language by using a style transfer model that takes advantage of the GPT-2. This process is shown in our web interface where the user input a informal message by text or voice. Our target audience are students and professionals in the need to improve the quality of their work by formalizing their texts. A style transfer is considered successful when the original semantic meaning of the message is preserved after the independent style has been replaced with a formal one with a high degree of grammatical correctness. This task is hindered by the scarcity of training and evaluation datasets alongside the lack of metrics. To accomplish this task, we opted to utilize OpenAI’s GPT-2 Transformer-based pre-trained model. To adapt the GPT-2 to our research, we fine-tuned the model with a parallel corpus containing informal text entries paired with the equivalent formal ones. We evaluate the fine-tuned model results with two specific metrics, formality and meaning preservation. To further fine-tune the model, we integrate a human-based feedback system where the user selects the best formal sentence out of the ones generated by the model. The resulting evaluations of our solution exhibit similar to improved scores in formality and meaning preservation to state-of-the-art approaches.
Publisher
Springer Science and Business Media LLC
Subject
Computer Science Applications,Computer Networks and Communications,Computer Graphics and Computer-Aided Design,Computational Theory and Mathematics,Artificial Intelligence,General Computer Science
Reference33 articles.
1. Rao S, Dear Tetreault JR. Sir or madam, may i, introduce the GYAFC dataset: corpus, benchmarks and metrics for formality style transfer. In: NAACL-HLT. ACL; 2018.
2. Radford A, Wu J, Child R, Luan D, Amodei D, Sutskever I. Language models are unsupervised multitask learners. OpenAI Blog. 2019;1(8):9.
3. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, et al. Attention is all you need. In: NIPS; 2017.
4. de Rivero M, Tirado C, Ugarte W. FormalStyler: GPT based model for formal style transfer based on formality and meaning preservation. In: KDIR; 2021.
5. Serban IV, Klinger T, Tesauro G, Talamadupula K, Zhou B, Bengio Y, et al. Multiresolution recurrent neural networks: an application to dialogue response generation. In: AAAI; 2017.
Cited by
1 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献
1. NoHateS: A Transformers-based Approach for Real-Time Hate Speech Detection in Spanish;2023 IEEE XXX International Conference on Electronics, Electrical Engineering and Computing (INTERCON);2023-11-02