Large Language Models and the Gaming Problem

Abstract

Abstract When an artificial agent can intelligently draw upon huge amounts of human-generated training data, the result can be gaming of our criteria for sentience. Gaming occurs when systems mimic human behaviours that are likely to persuade human users of their sentience without possessing the underlying capacity. The gaming problem leads initially to the thought that we should ‘box’ AI systems when assessing their sentience candidature, denying them access to a large corpus of human-generated training data. However, this would destroy the capabilities of any LLM. What we really need in the AI case are deep computational markers, not behavioural markers. If we find signs that an LLM has implicitly learned ways of recreating a global workspace or perceptual/evaluative reality monitoring system, this should lead us to regard it as a sentience candidate. Unfortunately, at the time of writing, we lack the sort of understanding of the inner workings of LLMs that is needed to ascertain which algorithms they have implicitly acquired during training.