Addressing Methodological Challenges in Culture-Comparative Research

Abstract

We address methodological challenges in cross-cultural and cultural psychology. First, we describe weaknesses in (quasi-)experimental designs, noting that cross-cultural designs typically do not allow any conclusive evidence of causality. Second, we argue that loose adherence to methodological principles of psychology and a focus on differences, while neglecting similarities, is distorting the literature. We highlight the importance of effect sizes and discuss the role of Bayesian statistics and meta-analysis for cross-cultural research. Third, we highlight issues of measurement bias and lack of equivalence, but note that recent large-scale projects involving researchers across many countries from the beginning of a study have much potential for overcoming biases and improving standards of equivalence. Fourth, we address some implications of multilevel models. Cultural processes are multilevel by definition and recent statistical advances can be used to explore these issues further. We believe this is an area where much theoretical work needs to be done and more rigorous methods applied. Fifth, we argue that the definition of culture and the psychological organization of cross-cultural differences as well as the definition of cultural populations to which research findings are generalized requires more attention. Sixth, we address the scope for anchoring cross-cultural research in biological variables and by asking multiple questions simultaneously, as advocated by Tinbergen for classical ethology. Bringing these discussions together, we provide recommendations for enhancing the methodological strength of culture-comparative studies to advance cross-cultural psychology as a scientific discipline.