Modulation of perception by visual, auditory, and audiovisual reward predicting cues

Abstract

AbstractRewards influence information processing in the primary sensory areas specialized to process stimuli from a specific sensory modality. In real life situations, we receive sensory inputs not only from one single modality, but stimuli are often multisensory. It is however not known whether the reward-driven modulation of perception follows the same principles when reward is cued through a single or multiple sensory modalities. We previously showed that task-irrelevant reward cues modulate perception both intra- as well as cross-modally, likely through a putative enhancement in the integration of the stimulus parts into a coherent object. In this study, we explicitly test this possibility by assessing whether reward enhances the integration of unisensory components of a multisensory object in accordance with thesupra-additiveprinciple of multisensory integration. Towards this aim, we designed a simple detection task using reward predicting cues that were either unisensory (auditory or visual, both above the detection threshold) or multisensory (audiovisual). We conducted two experiments, behavioral (experiment 1) and simultaneous behavioral and neuroimaging testing (experiment 2). We expected that reward speeds up reaction times in response to all stimulus configurations, and that additionally the reward effects in multisensory cues fulfill thesupra-additiveprinciple of multisensory integration. We observed that reward decreased response times in both experiments with the strongest effect found for the multisensory stimuli in experiment 1. However, this behavioral effect did not fulfill thesupra-additiveprinciple. Neuroimaging results demonstrated sensory supra-additivity at the classical areas involved in multisensory integration such as the Superior Temporal areas (STS), while reward modulation was found in the midbrain and fronto-parietal areas, reflecting the typical areas that receive dopaminergic projections. However, reward did not enhance thesupra-additivityin the STS compared to a no reward condition. Instead, we observed that some of the reward-related areas showed asub-additivemodulation by rewards and areas exhibiting a weakersupra-additiveresponse to audiovisual stimuli, namely the fusiform gyrus, were modulated by rewards of audiovisual stimuli as measured by a conjunction analysis. Overall, our results indicate that reward does not enhance the multisensory integration through asupra-additiverule. These findings inspire a model where reward and sensory integration processes are regulated by two independent mechanisms, where sensory information is integrated at an early stage in asupra-additivemanner, while reward modulates perception at a later stagesub-additively. Moreover, an associative area in the Fusiform gyrus exhibits a convergence of both reward and multisensory integration signals, indicating that it may be ahubto integrate different types of signals including rewards to disambiguate the information from different sensory modalities.