A practical guide to the implementation of AI in orthopaedic research, Part 6: How to evaluate the performance of AI research?

Author:

Oettl Felix C.12ORCID,Pareek Ayoosh3ORCID,Winkler Philipp W.456,Zsidai Bálint56ORCID,Pruneski James A.7ORCID,Senorski Eric Hamrin68ORCID,Kopf Sebastian9,Ley Christophe10,Herbst Elmar11ORCID,Oeding Jacob F.512ORCID,Grassi Alberto13,Hirschmann Michael T.1415ORCID,Musahl Volker16ORCID,Samuelsson Kristian5617ORCID,Tischer Thomas1819,Feldt Robert20ORCID,

Affiliation:

1. Hospital for Special Surgery New York New York USA

2. Schulthess Klinik Zurich Switzerland

3. Sports Medicine and Shoulder Institute, Hospital for Special Surgery New York New York USA

4. Department for Orthopaedics and Traumatology, Kepler University Hospital GmbH Johannes Kepler University Linz Linz Austria

5. Department of Orthopaedics, Institute of Clinical Sciences, Sahlgrenska Academy University of Gothenburg Gothenburg Sweden

6. Sahlgrenska Sports Medicine Center Göteborg Sweden

7. Department of Orthopaedic Surgery Tripler Army Medical Center Honolulu Hawaii USA

8. Department of Health and Rehabilitation, Institute of Neuroscience and Physiology, Sahlgrenska Academy University of Gothenburg Gothenburg Sweden

9. Center of Orthopaedics and Traumatology, University Hospital Brandenburg an der Havel, Brandenburg Medical School Theodor Fontane Germany

10. Department of Mathematics University of Luxembourg Esch‐sur‐Alzette Luxembourg

11. Department of Trauma, Hand and Reconstructive Surgery University Hospital Muenster Muenster Germany

12. Mayo Clinic Alix School of Medicine, Mayo Clinic Rochester Minnesota USA

13. IIa Clinica Ortopedica e Traumatologica, IRCCS Istituto Ortopedico Rizzoli Bologna Italy

14. Department of Orthopaedic Surgery and Traumatology Kantonsspital Baselland Bruderholz Switzerland

15. University of Basel Basel Switzerland

16. Department of Orthopaedic Surgery, UPMC Freddie Fu Sports Medicine Center University of Pittsburgh Pittsburgh Pennsylvania USA

17. Department of Orthopaedics Sahlgrenska University Hospital Mölndal Sweden

18. Department of Orthopaedic Surgery Universitymedicine Rostock Rostock Germany

19. Department of Orthopaedic and Trauma Surgery Malteser Waldkrankenhaus Erlangen Erlangen Germany

20. Department of Computer Science and Engineering Chalmers University of Technology Gothenburg Sweden

Abstract

AbstractArtificial intelligence's (AI) accelerating progress demands rigorous evaluation standards to ensure safe, effective integration into healthcare's high‐stakes decisions. As AI increasingly enables prediction, analysis and judgement capabilities relevant to medicine, proper evaluation and interpretation are indispensable. Erroneous AI could endanger patients; thus, developing, validating and deploying medical AI demands adhering to strict, transparent standards centred on safety, ethics and responsible oversight. Core considerations include assessing performance on diverse real‐world data, collaborating with domain experts, confirming model reliability and limitations, and advancing interpretability. Thoughtful selection of evaluation metrics suited to the clinical context along with testing on diverse data sets representing different populations improves generalisability. Partnering software engineers, data scientists and medical practitioners ground assessment in real needs. Journals must uphold reporting standards matching AI's societal impacts. With rigorous, holistic evaluation frameworks, AI can progress towards expanding healthcare access and quality.Level of EvidenceLevel V.

Publisher

Wiley

Cited by 1 articles. 订阅此论文施引文献 订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献

同舟云学术

1.学者识别学者识别

2.学术分析学术分析

3.人才评估人才评估

"同舟云学术"是以全球学者为主线,采集、加工和组织学术论文而形成的新型学术文献查询和分析系统,可以对全球学者进行文献检索和人才价值评估。用户可以通过关注某些学科领域的顶尖人物而持续追踪该领域的学科进展和研究前沿。经过近期的数据扩容,当前同舟云学术共收录了国内外主流学术期刊6万余种,收集的期刊论文及会议论文总量共计约1.5亿篇,并以每天添加12000余篇中外论文的速度递增。我们也可以为用户提供个性化、定制化的学者数据。欢迎来电咨询!咨询电话:010-8811{复制后删除}0370

www.globalauthorid.com

TOP

Copyright © 2019-2024 北京同舟云网络信息技术有限公司
京公网安备11010802033243号  京ICP备18003416号-3