Direct observation of depression screening: identifying diagnostic error and improving accuracy through unannounced standardized patients

Abstract

Abstract Background Depression is substantially underdiagnosed in primary care, despite recommendations for screening at every visit. We report a secondary analysis focused on depression of a recently completed study using unannounced standardized patients (USPs) to measure and improve provider behaviors, documentation, and subsequent claims for real patients. Methods Unannounced standardized patients presented incognito in 217 visits to 59 primary care providers in 22 New Jersey practices. We collected USP checklists, visit audio recordings, and provider notes after visits; provided feedback to practices and providers based on the first two visits per provider; and compared care and documentation behaviors in the visits before and after feedback. We obtained real patient claims from the study practices and a matched comparison group and compared the likelihood of visits including International Classification of Diseases, 10th Revision (ICD-10) codes for depression before and after feedback between the study and comparison groups. Results Providers significantly improved in their rate of depression screening following feedback [adjusted odds ratio (AOR), 3.41; 95% confidence interval (CI), 1.52–7.65; p = 0.003]. Sometimes expected behaviors were documented when not performed. The proportion of claims by actual patients with depression-related ICD-10 codes increased significantly more from prefeedback to postfeedback in the study group than in matched control group (interaction AOR, 1.41; 95% CI, 1.32–1.50; p < 0.001). Conclusions Using USPs, we found significant performance issues in diagnosis of depression, as well as discrepancies in documentation that may reduce future diagnostic accuracy. Providing feedback based on a small number of USP encounters led to some improvements in clinical performance observed both directly and indirectly via claims.