Impact of Unannounced Standardized Patient Audit and Feedback on Care, Documentation, and Costs: an Experiment and Claims Analysis

Abstract

Abstract Background Meaningful variations in physician performance are not always discernible from the medical record. Objective We used unannounced standardized patients to measure and provide feedback on care quality and fidelity of documentation, and examined downstream effects on reimbursement claims. Design Static group pre-post comparison study conducted between 2017 and 2019. Setting Fourteen New Jersey primary care practice groups (22 practices) enrolled in Horizon BCBS’s value-based program received the intervention. For claims analyses, we identified 14 additional comparison practice groups matched on county, practice size, and claims activity. Participants Fifty-nine of 64 providers volunteered to participate. Intervention Unannounced standardized patients (USPs) made 217 visits portraying patients with 1–2 focal conditions (diabetes, depression, back pain, smoking, or preventive cancer screening). After two baseline visits to a provider, we delivered feedback and conducted two follow-up visits. Measurements USP-completed checklists of guideline-based provider care behaviors, visit audio recordings, and provider notes were used to measure behaviors performed and documentation errors pre- and post-feedback. We also compared changes in 3-month office-based claims by actual patients between the intervention and comparison practice groups before and after feedback. Results Expected clinical behaviors increased from 46% to 56% (OR = 1.53, 95% CI 1.29–1.83, p < 0.0001), with significant improvements in smoking cessation, back pain, and depression screening. Providers were less likely to document unperformed tasks after (16%) than before feedback (18%; OR = 0.74, 95% CI 0.62 to 0.90, p = 0.002). Actual claim costs increased significantly less in the study than comparison group for diabetes and depression but significantly more for smoking cessation, cancer screening, and low back pain. Limitations Self-selection of participating practices and lack of access to prescription claims. Conclusion Direct observation of care identifies hidden deficits in practice and documentation, and with feedback can improve both, with concomitant effects on costs.