A Typology of Twitter Use by Human Service Nonprofits: the application of machine learning (Preprint)

Abstract

Current research has shown that nonprofit organizations, generally, are increasingly using social media to enhance their communication strategies with the wider population. Research on social media use within human service nonprofits specifically has identified some hesitancy to use social media education or useful tools to focus on social media use, and there is limited scope among organizational personnel in applying its usefulness beyond promoting one’s organization and its services. There is a need for greater conceptual clarity to support education and training on the varied reasons for using social media to increase organizational outcomes.

This study aims to leverage the potential of Twitter as a tool for examining the online communication of a sample of nonprofit sexual assault centers. To achieve this, we developed a supervised machine learning model to automatically classify tweets posted by Canadian sexual assault centers based on their organizational outcomes and the sentiment expressed in the tweets.

Employing a mixed-methods approach that combines machine learning and qualitative analysis, we manually coded 10,809 tweets from 133 sexual assault centers in Canada, spanning the period from March 2009 to March 2023. This analysis led to the identification of the following thematic categories: (1) Community Engagement, (2) Organization Administration, (3) Public Awareness, (4) Political Advocacy, (5) Support for Others, (6) Partnerships, and (7) Appreciation. These manually labeled tweets were used as the training dataset for the supervised machine learning process, which allowed us to classify 286,551 organizational tweets. The classification model based on supervised machine learning yielded satisfactory results, prompting the utilization of unsupervised machine learning to classify the topics within each thematic category and identify latent topics. The qualitative thematic analysis, in combination with topic modeling, provided a contextual understanding of each theme. Additionally, sentiment analysis was conducted to reveal the emotions conveyed in the tweets.

Manual annotation of 10,809 tweets identified seven thematic categories, with Organization Administration being the most frequent and Political Advocacy and Partnerships being the smallest segments. The supervised machine learning model achieved an accuracy of 63.4% in classifying tweets. The sentiment analysis revealed a prevalence of neutral sentiment across all categories. The emotion analysis indicated that fear was predominant, while joy was associated with Partnership and Appreciation tweets. Topic modeling identified distinct themes within each category, providing valuable insights into the prevalent discussions surrounding sexual assault and related issues.

This research contributes an original theoretical model that sheds light on how human service nonprofits utilize Twitter to achieve their organizational outcomes. The study significantly advances our comprehension of Twitter use by human service nonprofits, presenting a comprehensive typology that captures the diverse communication objectives of these organizations. This study provides valuable insights into the content and prevalent themes in nonprofit tweets. This research serves as a foundation for further investigations into the broader applications of social media use within the context of human service nonprofits.