Evidence in favor of Crossmodal Correspondences (CCs) have been collected both in infants and adults for different sensory modalities and dimensions in the perceptual domain, using speeded categorization or comparison tasks. However, it is still largely unexplored whether CCs can also modulate post-perceptual processes, such as Working Memory (WM). We investigated this issue in 2 Experiments, using a bimodal (audio-visual) 2-back task. In Experiment 1 we considered three kinds of CCs typically used in the perceptual domain: pitch/shape, pitch/elevation, and audio/visual numerosity, each presented congruently (e.g., for numerosity: three tones along with three shapes) or incongruently (e.g., three tones along with two shapes). Participants were divided in three groups according to the attended modality: vision, audition, or both. The results showed an overall impact of the different CCs on the n-back task performance, highlighting faster target-detection latencies following congruent vs. incongruent CCs. Experiment 2 focused on numerosity congruence/incongruence and confirmed these results, but only when the audiovisual correspondence remained stable and congruent between the sample and target display (i.e., when the two displays were audiovisually congruent and identical). This indicates that the enhancement of WM performance reflects the on-line maintenance of an integrated audiovisual representation involving the auditory and visual components of the CC, and not a mere arousal effect driven by the current congruent CC. Moreover, the two Experiments present evidence of an asymmetrical congruence effect across different sensory modalities (e.g. congruence effect on WM is maximized when attention is focused on a specific modality). Overall, these findings extend the existing knowledge about the influence of CCs on post-perceptual processing. Importantly, the impact of CCs in enhancing memory representation can contribute in shedding new light on the link between perceptual and post-perceptual, learning-mediated, stages of processing.
The impact of crossmodal correspondences on working memory performance
Mastroberardino, Serena;SANTANGELO, Valerio
2015
Abstract
Evidence in favor of Crossmodal Correspondences (CCs) have been collected both in infants and adults for different sensory modalities and dimensions in the perceptual domain, using speeded categorization or comparison tasks. However, it is still largely unexplored whether CCs can also modulate post-perceptual processes, such as Working Memory (WM). We investigated this issue in 2 Experiments, using a bimodal (audio-visual) 2-back task. In Experiment 1 we considered three kinds of CCs typically used in the perceptual domain: pitch/shape, pitch/elevation, and audio/visual numerosity, each presented congruently (e.g., for numerosity: three tones along with three shapes) or incongruently (e.g., three tones along with two shapes). Participants were divided in three groups according to the attended modality: vision, audition, or both. The results showed an overall impact of the different CCs on the n-back task performance, highlighting faster target-detection latencies following congruent vs. incongruent CCs. Experiment 2 focused on numerosity congruence/incongruence and confirmed these results, but only when the audiovisual correspondence remained stable and congruent between the sample and target display (i.e., when the two displays were audiovisually congruent and identical). This indicates that the enhancement of WM performance reflects the on-line maintenance of an integrated audiovisual representation involving the auditory and visual components of the CC, and not a mere arousal effect driven by the current congruent CC. Moreover, the two Experiments present evidence of an asymmetrical congruence effect across different sensory modalities (e.g. congruence effect on WM is maximized when attention is focused on a specific modality). Overall, these findings extend the existing knowledge about the influence of CCs on post-perceptual processing. Importantly, the impact of CCs in enhancing memory representation can contribute in shedding new light on the link between perceptual and post-perceptual, learning-mediated, stages of processing.I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
