Big Data is an emerging field in organizational research as it provides new types of data, and technologies like digitization and web scraping allow to study huge amounts of data. Since large parts of digital data consist of unstructured text, text classification - assigning texts (or parts of texts) to predefined categories - is a central task. Text classification not only allows to identify relevant texts in a jumble of data but also to extract information from texts, such as sentiments, topics, and intentions. However, large amounts of textual data require the use of automated text mining methods, which is mostly uncharted territory in organizational research. We, therefore, outline and discuss the two existing approaches to text classification, one originating from social science (dictionary content analysis) the other from computer science (supervised machine learning). Since both approaches have advantages and disadvantages, we combine ideas from both to develop a hybrid approach that reduces existing issues and requires significantly less knowledge in programming and computer science than supervised machine learning. To illustrate our approach, we develop a classifier that identifies critical media coverage of organizational actions.