Affiliation:
1. Vanderbilt University Medical Center, Nashville, TN
2. Portland State University, OR
3. Oregon Health & Science University, Portland
Abstract
Purpose:
A preliminary version of a paraphasia classification algorithm (henceforth called ParAlg) has previously been shown to be a viable method for coding picture naming errors. The purpose of this study is to present an updated version of ParAlg, which uses multinomial classification, and comprehensively evaluate its performance when using two different forms of transcribed input.
Method:
A subset of 11,999 archival responses produced on the Philadelphia Naming Test were classified into six cardinal paraphasia types using ParAlg under two transcription configurations: (a) using phonemic transcriptions for responses exclusively (
phonemic-only
) and (b) using phonemic transcriptions for nonlexical responses and orthographic transcriptions for lexical responses (
orthographic-lexical
). Agreement was quantified by comparing ParAlg-generated paraphasia codes between configurations and relative to human-annotated codes using four metrics (positive predictive value, sensitivity, specificity, and F1 score). An item-level qualitative analysis of misclassifications under the best performing configuration was also completed to identify the source and nature of coding discrepancies.
Results:
Agreement between ParAlg-generated and human-annotated codes was high, although the
orthographic-lexical
configuration outperformed
phonemic-only
(weighted-average F1 scores of .78 and .87, respectively). A qualitative analysis of the
orthographic-lexical
configuration revealed a mix of human- and ParAlg-related misclassifications, the former of which were related primarily to phonological similarity judgments whereas the latter were due to semantic similarity assignment.
Conclusions:
ParAlg is an accurate and efficient alternative to manual scoring of paraphasias, particularly when lexical responses are orthographically transcribed. With further development, it has the potential to be a useful software application for anomia assessment.
Supplemental Material:
https://doi.org/10.23641/asha.22087763
Publisher
American Speech Language Hearing Association
Subject
Speech and Hearing,Linguistics and Language,Language and Linguistics
Cited by
4 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献