ChatGPT: "To be or not to be" ... in academic research. The human mind's analytical rigor and capacity to discriminate between AI bots' truths and hallucinations

Abstract

Background. ChatGPT can generate increasingly realistic language, but the correctness and integrity of implementing these models in scientific papers remain unknown. Recently published literature emphasized the ”three faces of the coin” of ChatGPT: the negative impact on academic writing, limitations in analyzing and conducting extensive searches of references across multiple databases, and the superiority of the human mind. Method. The present study assessed the chatbot's ability for improvement and its propensity for self-correction at various points in 2023. Starting from previous papers published in our clinic, the authors repeatedly challenged the ChatGPT to conduct extensive searches for references across multiple databases at different time intervals (in March and September 2023). The bot was asked to find recent meta-analyses on a particular topic. Results. The replies (print screens) generated in March and September 2023 serve as evidence of the OpenAI platform's qualitative development and improvement. During the first contact with ChatGPT-3, one noticed significant content flows and drawbacks. ChatGPT provided references and short essays, but none of them were real, despite ChatGPT's clear affirmative response. When searching PubMed IDs, all DOI numbers indicated by the chatbot correlated to various unconnected manuscripts. After a few months, the authors repeated the same interrogative provocations and observed a significant shift in the replies. The ChatGPT-3.5 delivered balanced responses, emphasizing the superiority of the human intellect and advocating traditional academic research techniques and methods. Discussion. A recent comparative systematic analysis using the PRISMA method using the same keyword syntactic correlations to search for systematic literature or open sources has revealed the superiority of the classical scholarly method of research. In contrast, every document (title, authors, doi) that ChatGPT-3 initially delivered was erroneous and associated with a different field or topic. Literature published during the first trimester of 2023 emphasized ChatGPT`s hallucinatory tendency to supply fake ”bibliographic resources” and confabulatory attempts to paraphrase nonexistent ”research papers” presented as authentic articles. A second inquiry was realized six months later generated reserved and cautious solutions, indicating the researcher should analyze and carefully verify the information from specialized academic databases. Conclusions. The paper succinctly describes the flows and initial limitations of the ChatGPT-3 version and the process of updating and improving the GPT-3.5 system during 2023. ChatGPT might be a possible adjunct to academic writing and scientific research, considering any limitations that might jeopardize the study. The new perspective from ChatGPT claims that human intelligence and thought must thoroughly assess any AI information.