Author:
Jin Di,Wang Qing,Peng Dezhi,Wang Jiajia,Li Bijuan,Cheng Yating,Mo Nanxun,Deng Xiaoyan,Tao Ran
Abstract
Abstract
Background
Validation of the autoverification function is one of the critical steps to confirm its effectiveness before use. It is crucial to verify whether the programmed algorithm follows the expected logic and produces the expected results. This process has always relied on the assessment of human–machine consistency and is mostly a manually recorded and time-consuming activity with inherent subjectivity and arbitrariness that cannot guarantee a comprehensive, timely and continuous effectiveness evaluation of the autoverification function. To overcome these inherent limitations, we independently developed and implemented a laboratory information system (LIS)-based validation system for autoverification.
Methods
We developed a correctness verification and integrity validation method (hereinafter referred to as the "new method") in the form of a human–machine dialog. The system records personnel review steps and determines whether the human–machine review results are consistent. Laboratory personnel then analyze the reasons for any inconsistency according to system prompts, add to or modify rules, reverify, and finally improve the accuracy of autoverification.
Results
The validation system was successfully established and implemented. For a dataset consisting of 833 rules for 30 assays, 782 rules (93.87%) were successfully verified in the correctness verification phase, and 51 rules were deleted due to execution errors. In the integrity validation phase, 24 projects were easily verified, while the other 6 projects still required the additional rules or changes to the rule settings. Taking the Hepatitis B virus test as an example, from the setting of 65 rules to the automated releasing of 3000 reports, the validation time was reduced from 452 (manual verification) to 275 h (new method), a reduction in validation time of 177 h. Furthermore, 94.6% (168/182) of laboratory users believed the new method greatly reduced the workload, effectively controlled the report risk and felt satisfied. Since 2019, over 3.5 million reports have been automatically reviewed and issued without a single clinical complaint.
Conclusion
To the best of our knowledge, this is the first report to realize autoverification validation as a human–machine interaction. The new method effectively controls the risks of autoverification, shortens time consumption, and improves the efficiency of laboratory verification.
Publisher
Springer Science and Business Media LLC
Subject
Health Informatics,Health Policy,Computer Science Applications
Reference16 articles.
1. Li J, Cheng B, Ouyang H, Xiao T, Hu J, Cai Y. Designing and evaluating autoverification rules for thyroid function profiles and sex hormone tests. Ann Clin Biochem. 2018;55(2):254–63.
2. Wang Z, Peng C, Kang H, Fan X, Mu R, Zhou L, He M, Qu B. Design and evaluation of a LIS-based autoverification system for coagulation assays in a core clinical laboratory. BMC Med Inform Decis Mak. 2019;19(1):123.
3. Wu J, Pan M, Ouyang H, Yang Z, Zhang Q, Cai Y. Establishing and evaluating autoverification rules with intelligent guidelines for arterial blood gas analysis in a clinical laboratory. SLAS Technol. 2018;23(6):631–40.
4. Randell EW, Short G, Lee N, Beresford A, Spencer M, Kennell M, Moores Z, Parry D. Strategy for 90% autoverification of clinical chemistry and immunoassay test results using six sigma process improvement. Data Brief. 2018;18:1740–9.
5. Randell EW, Short G, Lee N, Beresford A, Spencer M, Kennell M, Moores Z, Parry D. Autoverification process improvement by Six Sigma approach: clinical chemistry & immunoassay. Clin Biochem. 2018;55:42–8.
Cited by
9 articles.
订阅此论文施引文献
订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献