Recent studies have highlighted the importance of personality in electoral politics. With the rise of populist and atypical personalities across stable and established democracies, pundits, journalists and other political experts often rely on their assessments of politicians’ personalities to explain their behavior. Additionally, numerous citizens depend on their expertise and assessments to form their own opinion. Given that most political experts have never personally met these politicians, how reliable are their assessments of high-profile politicians’ personality? We address this question by analyzing inter-rater reliability of ratings of US President Trumps’ personality by seven Belgian political experts. Using the NEO-FFI, our analyses indicate low inter-rater agreement on most of the Big Five personality traits and the facets of Trumps’ personality. Therefore, the excessive use of analyses based on third party assessments and interpretations of politicians’ personality should be regarded with caution given their potential impact on the wider public.