Evaluating Intelligence and Knowledge in Large Language Models

Abstract

AbstractIn recent times, particularly in the last few years, we have observed the rise of numerous Artificial Intelligence and Natural Language Processing semantic technologies. These advancements have subtly yet profoundly transformed our understanding of knowledge and truth, and the mechanisms for expressing, preserving, and disseminating them. This article aims to explore the dual challenge of assessing the effects of Large Language Models and associated semantic technologies on text dissemination and production, especially across the Internet. It specifically examines the implications for trust in online knowledge repositories, the creation of indirect or deliberate forms of ignorance, and the general perception of AI as a critical component of autonomous systems from the users’ viewpoint. The discussion will also consider potential strategies to mitigate the epistemic risks posed by the employment of AI semantic tools, in both suitable and unsuitable scenarios. The suggested approach contributes to the debate on AI intelligence measurement, proposing the evaluation of an AI system’s expected intelligence (as perceived by users) as a means to address the challenges associated with the “knowledge” generated by these systems. My claim is that measuring the expected intelligence in AI systems places humans at the forefront of the issue without necessitating a precise definition of intelligence for AI systems. This approach preserves therefore the essential attribute of these systems: intelligence.