Dealing with potentials and drawbacks of peer review panels: About the intertwined layers of determinacy and indeterminacy

Abstract

Abstract To improve evaluation processes in science, scholars of grant peer review and science policy often problematize that factors such as emotions, group dynamics, and informal arrangements influence panel discussions. They emphasize their negative effects on scientifically grounded deliberations and search for solutions to decrease such impacts. By doing this, these scholars easily play down positive effects of emotion work and informal talks during panel sessions and seem less encouraged to study the interplay of formally organized and more informal exchanges. To take both into consideration, in this essay I outline a concept of review processes as intertwined layers of determinacy and indeterminacy. It proposes that determinate outcomes of panel discussions (e.g. definite judgements, funding recommendations) can only be understood relative to the process’s indeterminacy (contextual vagueness such as informal talk, emotion work, tacit compromises). This theoretical framework (1) will help analytically to investigate indeterminacy in review panel processes as a whole as well as in panelists’ situated scientific reasonings and (2) will generate knowledge for more effective evaluation management.