Using rapid invisible frequency tagging (RIFT) to probe the attentional distribution between speech planning and comprehension

Abstract

AbstractInterlocutors often use the semantics of comprehended speech to inform the semantics of planned speech. Do representations of the comprehension and planning stimuli interact on a neural level? We used rapid invisible frequency tagging (RIFT) and EEG to probe the attentional distribution between spoken distractor words and target pictures in the picture-word interference (PWI) paradigm. We presented participants with auditory distractor nouns (auditory (f1); tagged at 54Hz) together with categorically related or unrelated pictures (visual (f2); tagged at 68Hz), which had to be named after a delay. RIFT elicits steady-state evoked potentials, which reflect attentional allocation to the tagged stimuli. When representations of the tagged stimuli interact, integrative effects have been observed at the intermodulation frequency resulting from an interaction of the base frequencies (f2±f1; Drijvers et al., 2021). Our results showed clear power increases at 54Hz and 68Hz during the tagging window, but no differences between related or unrelated conditions. More interestingly, we observed a larger power difference in the unrelated compared to the related condition at the intermodulation frequency (68Hz – 54Hz: 14Hz), indicating stronger interaction between the auditory and visual representations when they were unrelated. Our results go beyond standard PWI results (e.g., Bürki et al., 2020) by showing that participants do not have more difficulty visually attending to the related pictures or inhibiting the related auditory distractors. Instead, processing difficulties arise when the representations of the stimuli interact, meaning that participants might be trying to prevent integration between the auditory and visual representations in the related condition.Significance statementStudying speech planning during comprehension with EEG has been difficult due to a lack of appropriate methodology. This study demonstrates that rapid invisible frequency tagging (RIFT) can explore attentional allocation to speech planning and comprehension stimuli, as well as their interaction. Our results show that the content of the speech planning and comprehension representations affects their interaction in the neural signal, which should always be considered when these processes are studied jointly. In future work, RIFT could be used to investigate speech planning and comprehension in more conversational settings, as tagging can be added to videos or speech segments. This is the first study that demonstrates that RIFT can be used together with EEG to study cognitive phenomena.