Development and evaluation of a computerized algorithm for the interpretation of pulmonary function tests

Abstract

Pulmonary function tests (PFTs) are usually interpreted by clinicians using rule-based strategies and pattern recognition. The interpretation, however, has variabilities due to patient and interpreter errors. Most PFTs have recognizable patterns that can be categorized into specific physiological defects. In this study, we developed a computerized algorithm using the python package (pdfplumber) and validated against clinicians’ interpretation. We downloaded PFT reports in the electronic medical record system that were in PDF format. We digitized the flow volume loop (FVL) and extracted numeric values from the reports. The algorithm used FEV1/FVC<0.7 for obstruction, TLC<80%pred for restriction and <80% or >120%pred for abnormal DLCO. The algorithm also used a small airway disease index (SADI) to quantify late expiratory flattening of the FVL to assess small airway dysfunction. We devised keywords for the python Natural Language Processing (NLP) package (spaCy) to identify obstruction, restriction, abnormal DLCO and small airway dysfunction in the reports. The algorithm was compared to clinicians’ interpretation in 6,889 PFTs done between March 1st, 2018, and September 30th, 2020. The agreement rates (Cohen’s kappa) for obstruction, restriction and abnormal DLCO were 94.4% (0.868), 99.0% (0.979) and 87.9% (0.750) respectively. In 4,711 PFTs with FEV1/FVC≥0.7, the algorithm identified 190 tests with SADI < lower limit of normal (LLN), suggesting small airway dysfunction. Of these, the clinicians (67.9%) also flagged 129 tests. When SADI was ≥ LLN, no clinician’s reports indicated small airway dysfunction. Our results showed the computerized algorithm agreed with clinicians’ interpretation in approximately 90% of the tests and provided a sensitive objective measure for assessing small airway dysfunction. The algorithm can improve efficiency and consistency and decrease human errors in PFT interpretation. The computerized algorithm works directly on PFT reports in PDF format and can be adapted to incorporate a different interpretation strategy and platform.