Interrater Reliability and Coding Guide for Nonpsychotic Formal Thought Disorder

Abstract

Reliable and valid assessment of abnormal speech patterns may enable earlier recognition of nonpsychotic disorders through characteristic speech patterns. This study sought to establish interrater reliability using a standardized guide for scoring. A scoring guide defining 27 elements (e.g., inappropriate self-reference, simple loss of goal, circumstantiality) of disordered thought was developed. The seminal work of Andreasen's and Holzman's groups provided 12 elements, and 15 new elements were suggested by clinical literature. Audiotaped interviews with 12 psychiatric inpatients, adults of both sexes and various ages hospitalized for acute management of nonpsychotic psychiatric disorders, provided speech samples for observation of disordered thought by two independent raters. Using the guide's definitions and accompanying examples of elements of disordered thought, reliability in scoring was high (kappa of .85 for agreement on the presence of any abnormal speech element and kappa values from .66 to 1.00 for agreement on the presence of individual elements).