Applications of automated text analysis measuring topics, ideology, sentiment or even personality are booming in fields like political science and political psychology. These developments are to be applauded as they bring about novel insights about politics using new sources of (unstructured) data. However, a divide exists between work in both disciplines using text as data. In this paper we argue in favor of more integration across disciplinary boundaries, structuring our case around four key issues in the research process: (i) sampling text; (ii) authorship as meta data; (iii) pre-processing text; (iv) analyzing text. Along the way we demonstrate that an assessment of speaker characteristics may crucially depend on the text sources under study, and that the use of sentiment words correlates with estimates of policy positions, with implications for interpretation of the latter. As such, this paper contributes to a critical discussion about the merits of automated text analysis methods in political psychology and political science, with an eye towards advancing the considerable potential of text as data in the study of politics.