Methodological Problems of Using Arabic-Language Twitter as a Gauge for Arab Attitudes Toward Politics and Society

Abstract

Public opinion research in the Middle Eastern context has seen a proliferation of data research that aims to track changes in the state of Arab attitudes. Yet, it remains unclear what is being measured and how representative the data is. This article aims to address this question by reviewing the extent to which and how the existing literature has addressed the issues of representativeness of Twitter communities and the validity of opinion measures derived from sentiment analysis in the Middle Eastern context. While some studies aim no further than to gauge the dynamics of Twitter debates, many others seek to generalize the larger public opinion trends. This raises questions about how representative Twitter users are of the general public, how asymmetric social media use is, and how participation routines vary. Furthermore, to what extent can sentiment analysis translate individual tweets into meaningful measures of expressed opinion, and how sensitive are means of aggregating opinions from sentiment analysis to variation in terms of tone and amount of text within and across individuals? Following a comprehensive review of existing research in the context of the Middle East, this article also aims to derive a clearer understanding of how collective Twitter opinion relates to public opinion and make some suggestions of ways to design sampling and coding procedures, as well as validation exercises to address measurement bias and error.