Adaptive Changes in the Viewing Behavior of Faces Following Aversive Learning....

I decided to write few paragraphs about papers I will be publishing from now on. This will be targeted for the non-technical audience and I hope will increase the accessibility to the published results. 

Here is our latest work that shows how eye-movement patterns during viewing of faces are modified when people learn to associate faces with an aversive outcome.

Eye movements can be effortlessly recorded while humans are engaged in different situations. This can provide important insights on what the nervous system tries to achieve, as eye-movements represent the final behavioral outcome of many complex neuronal processes, which are difficult to record and understand. 

We measured eye-movements while humans were viewing faces, and analyzed the resulting exploration patterns. These faces were calibrated to have a known similarity relationship. For example, faces A, B and C were physically organised in such a way that B was perceived equally similar to A and C, whereas A and C were the most dissimilar pairs. First, using novel similarity-based analyses we show that exploration patterns are dominated by physical aspects of faces. That is, the physical similarity relationship between A, B and C could be estimated to a good degree from the similarity of eye-movement patterns that were generated during viewing of these faces. 

Following in the experiment, we selected one face to be a nasty one by associating its presentation with a mild electric current on the hand of volunteers to generate an unpleasant feeling without hurting them. Participants learnt to associate this unpleasant outcome with only one face while other faces were kept the same as before. This resulted in a gradient of unpleasantness that was not present before, and led volunteers to generalize this unpleasant association to other faces to the extent they were perceived similar with the nasty face. This is a classic phenomenon known as generalization, since early times of Pavlov.

How does this new situation modify the similarity relationships between exploration patterns? Following learning, similarity relationships of eye movement patterns started to mirror this newly learnt categories of nasty vs. safe faces, even though there were no physical changes associated with faces. This is compatible with the idea that following learning along an arbitrary continuum of stimuli, a categorization process occurs internally that distinguishes safe from nasty faces. This then biases eye-movement patterns during viewing of faces in such a way to collect information specifically associated with the safe and nasty prototypes, leading faces resembling to these prototypes to be scanned similarly.  

This study provides a nice illustration on how eye movements patterns can shed light onto neuronal processes and help us understand what the brain is trying to achieve during learning.

Eye movement patterns on 8 different, but similar faces that were carefully calibrated to form a similarity continuum. These maps show the most attended locations for a single participant before learning. Similarity analysis of these heatmaps using FPSA method can detect learning induced changes in the scanning behavior.

Reference:

Aversive Learning Changes Face-Viewing Strategies, as Revealed by Model-Based Fixation-Pattern Similarity Analysis. Lea Kampermann, Niklas Wilming, Arjen Alink, Christian Buechel, Selim Onat.

All content in this post released under CC-BY 4.0.

Effect of aversive learning on discrimination of faces

In her Msc thesis, Lea Kampermann shows that humans can perceptually discriminate faces better, when these are paired with an aversive outcome. This effect was specific to the face, which was paired with an aversive outcome and was not observed for the one which was kept neutral throughout the experiment. Furthermore the effect was strongest when these faces were presented at shorter durations (~.6 s) allowing participants to make no more than two fixations per trial.

Her thesis contains also a detailed account on the methodology for generating face-stimuli that are perceptually calibrated to form a two-dimensional similarity gradient with equal perceptual steps between faces. The methodology is an extension of work from Yue et al. (Vision Research, 2012). If you wish to use these stimuli for your experiment they are available upon request.

Perceptually calibrated set of faces according to a simple primary visual cortex forming a circular similarity gradient. Details on their production can be read in Msc Thesis of Lea (please contact any of us for a pdf) .

Categorical, yet graded--single-image activation profiles of human category-selective cortical regions.

Mur M et al. investigated the selectivity of activity levels in parahippocampal place area (PPA) and fusiform face area (FFA) evoked by single images. They focus here only on the average BOLD activity within carefully selected ROIs.

The paper is very creative in terms of new analyses methods, relies heavily on rank orders and hypothesis testing with bootstrapping.

First it establishes the fact that PPA and FFA behaves as expected, that is face stimuli for FFA and place stimuli for PPA rank highest in terms of evoked activity. Overall PPA responses are more selective than FFA responses, reaching AUC values of 1 in both hemispheres. This results from the fact that faces evokes really high activity levels in the FFA, whereas, in the case of PPA inactivation by faces contribute to the PPA selectivity.

The rest of the report focuses on characterizing the category selectivity of these areas.

If an area is category selective in an ideal sense, non-preferred stimuli should never evoke higher activity levels than any other preferred stimulus, and if so, then only by chance due to noise.

The number of inverted pairs measures exactly the number of times one could identify violation of this rule by counting the number of times a stimulus from outside the category is ranked higher than a stimulus from within category. If these inverted pairs survive across multiple sessions (as measured by PRIP metric), this would be an evidence against ideal category selectivity. However as such, PRIP is not a very sensitive metric. For example one single preferred stimulus failing by chance to evoke any activity at all would be sufficient to generate very many inverted pairs, thus the metric seems to fluctuate highly non-linearly with respect to distance between inverted pairs. Therefore the authors, used the first sessions to identify preferred-nonpreferred pairs with largest activity difference, with the idea that an inversion with the largest activity difference would be the observation with least chance level. If these pairs survive across sessions, the difference in activity would then decrease only marginally and remain positive, thus providing evidence for stable inversions (as such, however this measure is also influenced by the noise on both the preferred and non-preferred stimulus). These analyses provide supporting evidence that FFA and PPA behave like an ideal category selective area, with the exception of left FFA, in line with the fact that left FFA is the ROI where smallest AUC values were observed (only though at ROI size of 128 voxels).

Categorical Representation of Visual Stimuli in the Primate Prefrontal Cortex

Freedman et al. generated pictures of complex objects using parametric combinations of 6 base images. These base images represented different kinds of felines or dogs, therefore their combinations gave rise to images that were graded in their category membership (e.g. whereas some combinations were clearly dog- or feline-like, others pictures were somewhere in between) while guaranteeing diversity within each category. These images were shown in a delayed-match-to-category task to monkeys. Solving this task requires a level of abstraction from the sheer appearances of the stimuli. Even at category boundaries where the discrimination is most difficult, the performance was high. They recorded activity of single neurons from prefrontal cortex, more precisely from the ventral part of the principal sulcus. Their results show evidence for neurons that are able to distinguish between these two, supposedly learnt categories. That is the responses are 1/ not gradual as the stimuli  and 2/ characterized by a sharp step-like function at the category boundary. The data is clear, the interpretation is inline with the data. I find it unfortunately that pictures which are at the category boundary were not presented. And the study would gain if the similarity measure was in the perceptual space rather than in the stimulus domain. The 6 base images could in principle be tested on humans using perceptual mapping techniques.