The interplay between emotions and high-level semantics on working memory encoding and retrieval

Introduction: A common view holds that emotionally-arousing stimuli are attention capturing, irrespective of whether they are or are not in the current focus of attention (see, for reviews, Pessoa and Ungerleider, 2004; Vuilleumier, 2005). However, the prioritization of emotionally-arousing stimuli in complex scenes, which entail a high-level of competition and selective attention, is unexplored. Here we asked whether the encoding and retrieval of emotional stimuli simply reflect processing of emotional valence, or rather valence interacts with selective attention associated with the processing of the scene context. For this, we used fMRI and manipulated emotion- and context-related factors during working memory encoding and retrieval of complex scenes. Methods: Twenty subjects were presented with natural scenes that included also an emotionally-arousing (positive or negative) object. This could be either congruent or incongruent with the scene context. Following a retention interval, on half of the trials the participants were asked to judge the location of the emotional object. On the other half of the trials, the retrieval test involved a different object that was also present in the initial scene but was not emotionally-arousing (neutral object) and was always congruent with the scene context. If emotionally-arousing stimuli are selected irrespective of context-processing we would expect a recollection advantage for all emotional objects (vs. neutral objects), irrespective of whether these were or were not congruent with the scene context. Results: Retrieval accuracy was better for emotionally-arousing than for neutral targets. For emotional targets we found greater accuracy for context-incongruent than congruent trials, indicating that the selection of emotional objects was not independent of other, context-related factors. The positive/negative valence of the tested target did not modulate the effect of context-congruency. However, the valence of the emotional stimuli at encoding was found to modulate the retrieval of neutral targets: participants were less accurate to retrieve neutral objects when the scene contained a positive object, and more so when this positive object was context-congruent. These results suggest that the encoding of emotional and out-of context stimuli is overall facilitated, and that the presence of objects with a positive valence reduces the likelihood of encoding other (neutral) objects of the scene. Retrieval-related fMRI analyses revealed an interaction between memory performance and target type: Successful retrieval of emotional targets involved the activation of the posterior cingulated cortex and the angular gyri, bilaterally. By contrast, successful retrieval of neutral targets activated the dorsal fronto-parietal attention network (Corbetta et al., 2008), but less so when the scene included a positive object. Analyses of the fMRI data at encoding highlighted an interaction between consistency and performance on emotional targets: the right dorsal fronto-parietal network was found to activate when the scene included an out-of-context object, specifically when this was then successfully retrieved. Conclusions: We found that context congruency rather than emotional valence affected encoding-related activation and subsequent memory performance for emotional stimuli. By contrast, the presence of a positive object in the scene was found to affect the processing of the neutral objects, with a lower retrieval performance and lower activation of fronto-parietal regions associated with the retrieval of the neutral targets. We conclude that both valence and context signals contribute to the selection of stimuli in complex scenes, but with distinctive effects on the processing of emotional vs. neutral objects.