Identifying Careless Responding in Web-Based Surveys

Abstract

Abstract: The increasing use of web-based surveys in social sciences research has brought forth the challenge of effectively identifying and managing inattentive/careless responding. The existing detection methods have shown limited success, highlighting the need for improved methodologies. This study introduces a novel approach that utilizes time-stamped action sequence data of mouse movements and employs deep learning models to detect careless responding. It introduces the concept of Approximate Areas of Interest (AAOIs) along with the application of Gated Recurrent Units (GRUs) and Bidirectional Long Short-Term Memory (BiLSTM) models. This research presents a flexible and efficient tool that can be applied across different scales and survey contexts. The results demonstrate the superior performance of the proposed approach in identifying group membership, achieving up to 95% accuracy when tested on experimental data with induced inattentiveness. The presented approach offers a potentially promising tool for overcoming the pervasive challenge of detecting careless responding in computer-based surveys.