Dear Program Director: An Evaluation of Implicit Bias in Letters of Recommendation for Neurosurgery Residency

Abstract

BACKGROUND AND OBJECTIVES: Despite comprising half of medical students, women represent only 29.6% of neurosurgery applicants and 17% of residents, suggesting a “leak” in the career pipeline for women neurosurgeons. Surveys persistently show that neurosurgery programs identify United States Medical Licensing Examination (USMLE®) Step 1 score and letters of recommendation (LORs) as the most important factors in selecting applicants to interview. A previous study in neurosurgery found no differences in LORs. However, multiple studies in other specialties have demonstrated implicit gender bias in LORs, which may influence resident selection. Our objective is to evaluate neurosurgery residency LORs for evidence of implicit gender bias. METHODS: Retrospective analysis of LORs for interviewed neurosurgery applicants at a single institution during the 2014 to 2020 National Residency Matching Program (NRMP®) match cycles. Letters were evaluated using Linguistic Inquiry & Word Count (LIWC) software (Pennebaker Conglomerates), and additional applicant data were obtained from candidate applications. LIWC (Pennebaker Conglomerates) output data included custom dictionary categories and terms that were analyzed using Prism 10 and Rstudio. RESULTS: Two hundred eighteen applications were reviewed for a total of 827 letters. LIWC (Pennebaker Conglomerates) analysis showed significant differences in word count (331 vs 297, difference = 34, 95% CI: 9-61, P = .008). LORs for applicants who were men were more likely to mention Alpha Omega Alpha Honor Medical Society (1.17 vs 0.778, difference = 0.4, 95% CI: 0.13-0.67, P = .023). USMLE® Step 1 scores were significantly lower for women (241 vs 247, difference = 6, 95% CI: 2-10, P = .004). There was no significant difference between letters for men and women for all categories evaluated in the linguistic evaluation. CONCLUSION: LORs are vital to the neurosurgical residency application process. The data exhibit some differences between the men and women applicants but few differences in their LORs, consistent with the results of the previous neurosurgical study.