Correction of Misspellings and Typographical Errors in a Free-Text Medical English Information Storage and Retrieval System

Abstract

The errors studied are misspellings and typographical errors made by the physician house staff, surgical pathologists, and secretary/typists of a large teaching hospital. The 6,019 errors studies were encountered in the compilation of a LEXICON now containing 24,135 medical and non-medical terms (including errors) from Tissue Examination Request Forms and Surgical Pathology Reports. An automated error correction algorithm was sought to reduce the tedious task of manual encoding of errors, and eliminate the need for storing errors occupying 24.9% of the LEXICON storage space. The errors were classified into 23 types, and it was found that 84.2% of the errors were in the 11 first order categories.Existing error correction algorithms were analyzed with respect to possible application to our medical sample. Two were selected for experimentation, the Baskin-Selfridge algorithm and SOUNDEX. Results showed that Baskin-Selfridge worked quite well, but was too slow to be applied singularly. SOUNDEX was reasonable in speed, but had too many mismatches to be applied singularly in a non-interactive application. SOUNDEX was modified phonologically and with respect to code length in various ways and some experimental data showed improvements.The optimal design for the medical LEXICON sample appears to be a two-step process. The modified version of SOUNDEX will quickly select the most likely corrections for the error (experimental average is 2.38 choices/error). Then the Baskin-Selfridge will decide which, if any, is the actual correct form of the error. By only considering a very small number of choices, the time required for the Baskin-Selfridge algorithm becomes trivial.On the basis of experimental results, it is estimated that this combination will reduce manual encoding of errors by 60—70% and reduce the storage required for the LEXICON by approximately 15%.