Interobserver agreement and the effects of ethnicity on observational coding of affect

Abstract

Objectives: The Specific Affect Coding System (SPAFF) is a reliable system for coding observed affect but few studies have tested its generalizability and susceptibility to bias. Methods: The current study compared highly trained African American and Caucasian coders’ scores when coding the same videos of African American and Caucasian American couples. Results: While it was hypothesized that Caucasian Americans may code African America couples as being more aggressive and less positive based on stereotypes, results revealed a significant Ethnicity of Coder × Ethnicity of Couple interaction on the “neutral” code only. Both African American and Caucasian coders tended to rate out-group couples as being less neutral and showing less interest than in-group couples. No other emotion was identified as being over-coded in lieu of neutral, however. Conclusion: Results suggest a subtle bias in SPAFF coding. Adding a diversity component to the SPAFF training and recruiting diverse coders is recommended.