Towards an understanding and explanation for mixed-initiative artificial scientific text detection
-
Published:2024-04-07
Issue:3
Volume:23
Page:272-291
-
ISSN:1473-8716
-
Container-title:Information Visualization
-
language:en
-
Short-container-title:Information Visualization
Author:
Weng Luoxuan1ORCID,
Liu Shi1,
Zhu Hang1,
Sun Jiashun1,
Kam-Kwai Wong2,
Han Dongming1,
Zhu Minfeng1,
Chen Wei1
Affiliation:
1. State Key Lab of CAD&CG, Zhejiang University, Hangzhou, Zhejiang, China
2. The Hong Kong University of Science and Technology, Hong Kong, China
Abstract
Large language models (LLMs) have gained popularity in various fields for their exceptional capability of generating human-like text. Their potential misuse has raised social concerns about plagiarism in academic contexts. However, effective artificial scientific text detection is a non-trivial task due to several challenges, including (1) the lack of a clear understanding of the differences between machine-generated and human-written scientific text, (2) the poor generalization performance of existing methods caused by out-of-distribution issues, and (3) the limited support for human-machine collaboration with sufficient interpretability during the detection process. In this paper, we first identify the critical distinctions between machine-generated and human-written scientific text through a quantitative experiment. Then, we propose a mixed-initiative workflow that combines human experts’ prior knowledge with machine intelligence, along with a visual analytics system to facilitate efficient and trustworthy scientific text detection. Finally, we demonstrate the effectiveness of our approach through two case studies and a controlled user study. We also provide design implications for interactive artificial text detection tools in high-stakes decision-making scenarios.
Funder
National Natural Science Foundation of China
Zhejiang Provincial Natural Science Foundation of China
“Pioneer” and “Leading Goose” R&D Program of Zhejiang
Publisher
SAGE Publications
Reference102 articles.
1. OpenAI. Chatgpt, https://chat.openai.com/ (2023, accessed 20 March 2024).