Development and Pilot Testing of a Programmatic System for Competency Assessment in US Anesthesiology Residency Training-Reference-Cited by-同舟云学术

Development and Pilot Testing of a Programmatic System for Competency Assessment in US Anesthesiology Residency Training

Published:2023-10-06 Issue: Volume: Page:
ISSN:0003-2999
Container-title:Anesthesia & Analgesia
language:en
Short-container-title:

Author:

Woodworth Glenn E.¹,Goldstein Zachary T.²,Ambardekar Aditee P.³,Arthur Mary E.⁴,Bailey Caryl F.⁴,Booth Gregory J.⁵,Carney Patricia A.⁶,Chen Fei⁷,Duncan Michael J.⁸,Fromer Ilana R.⁹,Hallman Matthew R.¹⁰,Hoang Thomas¹,Isaak Robert¹¹,Klesius Lisa L.¹²,Ladlie Beth L.¹³,Mitchell Sally Ann¹⁴,Miller Juve Amy K.¹,Mitchell John D.¹⁵,McGrath Brian J.¹⁶,Shepler John A.¹²,Sims Charles R.¹⁷,Spofford Christina M.¹⁸,Tanaka Pedro P.¹⁹,Maniker Robert B.²⁰

Affiliation:

1. Department of Anesthesiology and Perioperative Medicine, Oregon Health & Science University, Portland, Oregon

2. Department of Anesthesiology, Cedars Sinai Medical Center, Los Angeles, California

3. Department of Anesthesiology and Pain Management, University of Texas, Southwestern Medical Center, Dallas, Texas

4. Department of Anesthesiology and Perioperative Medicine, Medical College of Georgia at Augusta University, Augusta, Georgia

5. Uniformed Services University of the Health Sciences, Department of Anesthesiology and Pain Medicine, Naval Medical Center Portsmouth, Portsmouth, Virginia

6. Division of Hospital Medicine, Department of Family Medicine and Internal Medicine, Oregon Health & Science University, Portland, Oregon

7. Department of Anesthesiology, University of North Carolina at Chapel Hill School of Medicine, Chapel Hill, North Carolina

8. Department of Anesthesiology, University of Missouri-Kansas City, Kansas City, Missouri

9. Department of Anesthesiology, University of Minnesota, Minneapolis, Minnesota

10. Department of Anesthesiology and Pain Medicine, University of Washington, Seattle, Washington

11. Department of Anesthesiology, University of North Carolina, Chapel Hill, North Carolina

12. Department of Anesthesiology, University of Wisconsin School of Medicine and Public Health, Madison, Wisconsin

13. Department of Anesthesiology, Mayo Clinic, Rochester, Minnesota

14. Department of Anesthesiology, Indiana University, Indianapolis, Indiana

15. Department of Anesthesiology, Critical Care, and Perioperative Medicine, Henry Ford Health, Detroit, Michigan

16. Department of Anesthesiology, University of Florida College of Medicine-Jacksonville, Jacksonville, Florida

17. Department of Anesthesiology & Perioperative Medicine, Mayo Clinic, Rochester, Minnesota

18. Department of Anesthesiology, Medical College of Wisconsin, Milwaukee, Wisconsin

19. Department of Anesthesiology, Stanford University, Stanford, California

20. Department of Anesthesiology, Columbia University, New York, New York.

Abstract

BACKGROUND: In 2018, a set of entrustable professional activities (EPAs) and procedural skills assessments were developed for anesthesiology training, but they did not assess all the Accreditation Council for Graduate Medical Education (ACGME) milestones. The aims of this study were to (1) remap the 2018 EPA and procedural skills assessments to the revised ACGME Anesthesiology Milestones 2.0, (2) develop new assessments that combined with the original assessments to create a system of assessment that addresses all level 1 to 4 milestones, and (3) provide evidence for the validity of the assessments. METHODS: Using a modified Delphi process, a panel of anesthesiology education experts remapped the original assessments developed in 2018 to the Anesthesiology Milestones 2.0 and developed new assessments to create a system that assessed all level 1 through 4 milestones. Following a 24-month pilot at 7 institutions, the number of EPA and procedural skill assessments and mean scores were computed at the end of the academic year. Milestone achievement and subcompetency data for assessments from a single institution were compared to scores assigned by the institution’s clinical competency committee (CCC). RESULTS: New assessment development, 2 months of testing and feedback, and revisions resulted in 5 new EPAs, 11 nontechnical skills assessments (NTSAs), and 6 objective structured clinical examinations (OSCEs). Combined with the original 20 EPAs and procedural skills assessments, the new system of assessment addresses 99% of level 1 to 4 Anesthesiology Milestones 2.0. During the 24-month pilot, aggregate mean EPA and procedural skill scores significantly increased with year in training. System subcompetency scores correlated significantly with 15 of 23 (65.2%) corresponding CCC scores at a single institution, but 8 correlations (36.4%) were <30.0, illustrating poor correlation. CONCLUSIONS: A panel of experts developed a set of EPAs, procedural skill assessment, NTSAs, and OSCEs to form a programmatic system of assessment for anesthesiology residency training in the United States. The method used to develop and pilot test the assessments, the progression of assessment scores with time in training, and the correlation of assessment scores with CCC scoring of milestone achievement provide evidence for the validity of the assessments.

Publisher

Ovid Technologies (Wolters Kluwer Health)

Subject

Anesthesiology and Pain Medicine

Reference38 articles.

1. Competency-based postgraduate medical education: past, present and future.;Ten Cate;GMS J Med Educ,2017

2. Competency-based medical education: theory to practice.;Frank;Med Teach,2010

3. The next GME accreditation system––rationale and benefits.;Nasca;N Engl J Med,2012

4. The anesthesiology milestone project.;J Grad Med Educ,2014

5. Reflections on the first 2 years of milestone implementation.;Holmboe;J Grad Med Educ,2015