Validating a Vignette-Based Instrument to Study Physician Decision Making in Trauma Triage

Abstract

Background. The evidence supporting the use of vignettes to study physician decision making comes primarily from the study of low-risk decisions and the demonstration of good agreement at the group level between vignettes and actual practice. The validity of using vignettes to predict decision making in more complex, high-risk contexts and at the individual level remains unknown. Methods. We had previously developed a vignette-based instrument to study physician decision making in trauma triage. Here, we measured the retest reliability, internal consistency, known-groups performance, and criterion validity of the instrument. Thirty-two emergency physicians, recruited at a national academic meeting, participated in reliability testing. Twenty-eight trauma surgeons, recruited using personal contacts, participated in known-groups testing. Twenty-eight emergency physicians, recruited from physicians working at hospitals for which we had access to medical records, participated in criterion validity testing. We measured rates of undertriage (the proportion of severely injured patients not transferred to trauma centers) and overtriage (the proportion of patients transferred with minor injuries) on the instrument. For physicians participating in criterion validity testing, we compared rates of triage on the instrument with rates in practice, based on chart review. Results. Physicians made similar transfer decisions for cases (κ = 0.42, P < 0.01) on 2 administrations of the instrument. Responses were internally consistent (Kuder-Richardson, 0.71–0.91). Surgeons had lower rates of undertriage than emergency physicians (13% v. 70%, P < 0.01). No correlation existed between individual rates of under- or overtriage on the vignettes and in practice ( r = −0.17, P = 0.4; r = −0.03, P = 0.85). Conclusions. The instrument developed to assess trauma triage decision making performed reliably and detected known group differences. However, it did not predict individual physician performance.