Research note: Examining potential bias in large-scale censored data-Reference-Cited by-同舟云学术

Research note: Examining potential bias in large-scale censored data

Published:2021-07-26 Issue: Volume: Page:
ISSN:
Container-title:Harvard Kennedy School Misinformation Review
language:
Short-container-title:HKS Misinfo Review

Author:

Allen Jennifer¹,Mobius Markus²,Rothschild David M.²,Watts Duncan J.³

Affiliation:

1. Sloan School of Management, Massachusetts Institute of Technology, USA

2. Microsoft Research, USA

3. Department of Computer and Information Science, University of Pennsylvania, USA

Abstract

We examine potential bias in Facebook’s 10-trillion cell URLs dataset, consisting of URLs shared on its platform and their engagement metrics. Despite the unprecedented size of the dataset, it was altered to protect user privacy in two ways: 1) by adding differentially private noise to engagement counts, and 2) by censoring the data with a 100-public-share threshold for a URL’s inclusion. To understand how these alterations affect conclusions drawn from the data, we estimate the preva-lence of fake news in the massive, censored URLs dataset and compare it to an estimate from a smaller, representative dataset. We show that censoring can substantially alter conclusions that are drawn from the Facebook dataset. Because of this 100-public-share threshold, descriptive statis-tics from the Facebook URLs dataset overestimate the share of fake news and news overall by as much as 4X. We conclude with more general implications for censoring data.

Funder

Nathan Cummings Foundation

Carnegie Corporation of New York

Publisher

Shorenstein Center for Media, Politics, and Public Policy

Reference13 articles.

1. Allen, J., Howland, B., Mobius, M., Rothschild, D., & Watts, D. J. (2020). Evaluating the fake news problem at the scale of the information ecosystem. Science Advances, 6(14), eaay3539. https://doi.org/10.1126/sciadv.aay3539

2. Dwork C. (2008). Differential privacy: A survey of results. In Agrawal M., Du D., Duan Z., & Li A. (Eds.), Theory and applications of models of computation (TAMC 2008) (pp. 1–19). Springer. https://doi.org/10.1007/978-3-540-79228-4_1

3. Evans, G. & King, G. (2020). Statistically valid inferences from differentially private data releases, with application to the Facebook URLs dataset [Manuscript submitted for publication]. Har-vard University. https://gking.harvard.edu/dpd

4. Gibney, E. (2019). Privacy hurdles thwart Facebook democracy research. Nature, 574(7777), 158–159. https://doi.org/10.1038/d41586-019-02966-x

5. Grinberg, N., Joseph, K., Friedland, L., Swire-Thompson, B., & Lazer, D. (2019). Fake news on Twitter during the 2016 U.S. presidential election. Science, 363(6425), 374–378. https://doi.org/10.1126/science.aau2706

Cited by 7 articles. 订阅此论文施引文献订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献

1. Visual misinformation on Facebook;Journal of Communication;2023-02-28

2. Misinformation on Misinformation: Conceptual and Methodological Challenges;Social Media + Society;2023-01

3. Temporal Dynamics of User Engagement with U.S. News Sources on Facebook;2022 Ninth International Conference on Social Networks Analysis, Management and Security (SNAMS);2022-11-29

4. FbMultiLingMisinfo: Challenging Large-Scale Multilingual Benchmark for Misinformation Detection;2022 International Joint Conference on Neural Networks (IJCNN);2022-07-18

5. Quantifying partisan news diets in Web and TV audiences;Science Advances;2022-07-15