Which Behaviors Generate The Best Reviews? A Sentiment Analysis of Online Reviews on AOSSM Surgeons

Abstract

Background Online surgeon reviews can significantly influence a patient’s selection of a provider, and are important in the movement towards quality-based physician compensation models. Written reviews, however, are subjective and are thus difficult to quantitatively analyze. Sentiment analysis using artificial intelligence (AI) provides the ability to quantitatively assess surgeon reviews to provide actionable feedback. Purpose The objective of this study is to quantitatively analyze the online written reviews of AOSSM surgeons utilizing sentiment analysis and report trends in the most frequently used words in the best and worst reviews. Study Design Cross-sectional study using publicly-available online reviews Methods Online reviews and star-ratings of AOSSM surgeons were obtained from healthgrades.com and zocdoc.com. A sentiment analysis algorithm was used to compute sentiment analysis scores of each written review. Sentiment scores were validated against star-ratings. Positive and negative word and word-pair frequency analysis was performed to identify common items associated with high and low scores. A multiple logistic regression was run on clinically relevant phrases. Results Following the inclusion and exclusion criteria, 18,386 AOSSM surgeon reviews were analyzed for 2071 surgeons. There was no significant difference in sentiment scores by provider gender. Surgeons who are younger than 50 years old had more positive reviews (mean sentiment = +0.536 versus +0.458, p < 0.01). The most frequently used and meaningful bi-grams used to describe top-rated surgeons are words correlating with kindness, caring personalities, and efficiency in pain management; whereas, those with the worst reviews are often characterized as unable to relieve the pain of their patients. The multiple logistic regression was significant for several clinically relevant words that confer greater or less odds of an improved score. Pain is significantly correlated with a decreased odds of receiving a positive review and positive behavioral factors confer a greater odds of receiving a positive review. Conclusion Sentiment analysis provides a means of quantifying written reviews of surgeons, and analysis of the reviews. This study provides insight into factors contributing to positive reviews, especially surgeon confidence, staff friendliness, warm disposition, and pain relief. Clinical Relevance This study delineates factors that impact the public reviews on AOSSM providers.