The Procedural Validity of Retrospective Case Note Diagnosis

Abstract

Objective: Deriving diagnoses from retrospective case note examination is a common practice in psychiatric research. The Operational Criteria (OPCRIT) diagnostic checklist is essentially a checklist built up of operational criteria defined by a comprehensive glossary and is designed to assign reliable diagnoses from case notes. However, the validity of such a procedure compared with procedures involving prospective assessment has never been tested. We examined the procedural validity of the OPCRIT diagnostic system in relation to four other diagnostic procedures mostly employing prospectively gathered information. Method: Three experienced psychopathology raters rated the case notes and clinical abstracts, using the OPCRIT method of diagnostic assignment, of 50 subjects who had participated in an early procedural validity study as an adjunct to the DSM-IV Field Trial for psychotic disorders. The setting was the Early Psychosis Prevention and Intervention Centre (EPPIC), which focuses on first episode psychosis. Results: The pairwise concordance with the other procedures for DSM-III-R diagnoses assigned by OPCRIT using ratings derived from either the clinical abstracts or the case notes was found to be only poor to moderate when compared with the pairwise concordance of the four other procedures. The per cent agreement between OPCRIT clinical abstracts diagnoses and the other procedures ranged from 49% to 60% with kappa values between 0.30 and 0.45, and for OPCRIT case note diagnoses and the other procedures the per cent agreement range was between 44% and 57% and the kappa values were between 0.35 and 0.49. Conclusions: The procedural validity of diagnoses assigned via the application of checklists of operational criteria to case notes and clinical abstracts alone is unacceptably poor. Such sources need to be buttressed by other data, particularly direct patient interview and informant material.