Showing posts with label brainstorming. Show all posts
Showing posts with label brainstorming. Show all posts

Let's think about a new culture of science.

Academic production before internet

Today, science is produced by the following basic infrastructure. This simplistic picture depicts how humans in a lab interact with the nodes making up the infrastructure. Humans basically use computational resources to analyze data by writing code. The input to these nodes overall is extremely sparse. That is we generally do not have other peoples' data available, we generally cook our own code specifically for our own data, we generally use our own computational resources.


Unit infrastructure for scientific production

This unit is the fundament for academia. Research is carried out by the same infrastructure that is simply replicated across geographies and time. Of course, there could be labs collaborating with each other, of course we could be using an external grid engine to run our tasks, of course we might download a toolbox to run analyses. These are all connections that are not shown in this picture. 

But the emphasis here is that we spent most of our time to reinvent the wheel by
-writing the same piece of code that many people had done it in the past, 
-collecting yet a new dataset instead of generating a new hypothesis compatible with available datasets, 
-buying large computers that could be used by other people
-hiring system administrators that are doing exactly the same work as in another lab
the list is long...


Each lab in this culture becomes a specialized idiosyncratic creature with its own way of doing things. Politically this implies committing in long-term fixed-costs to maintain an academic infrastructure that is short-sighted and benefits mainly to the labs short-term agenda.  Academia mainly benefits from the contributions of labs in the form of publications, which is considered as the unique currency in academic reward system. What is the impact of this system on the society? In the light of current replication crises in science, it is hard to be optimistic.

Infrastructure for academic work. Culture of mine.

This type of infrastructure organization has mainly historical reasons. This model is archaic, and has been a good model for the pre-internet era where people and systems were connected sparsely with each other, where it made sense to travel to a conference and meet other people.

A new way of doing science at the age of cloud-based systems

We have to rethink about how to place boxes shown in the previous pictures, how to set novel incentive mechanisms, and how to organize the work flow across scientists and nodes. Let's talk about this simple picture.

A novel infrastructure for academic work. Culture of sharing.

Outsourcing the storage and compute resources to a cloud service (e.g. AWS, GDC or some supranational public cloud service yet to be put in place) are for the benefit of the society in terms of reducing overall costs. 

However the main point here is not about outsourcing storage and compute resources. The real reason for this move is for making datasets accessible to other scientists. And in the long-term this actually means making data to be publicly available to all citizens.

The only thing that is specific to a given lab is the data that is collected there. That's what labs should do: collect data. Most importantly data must to be stored according to strict standardization. That is to each data set, a map has to be associated, that will help people on how to navigate this data set. Furthermore, every dataset should be stored with a minimal code that ensure basic access to data. Also, most often datasets spans multiple modalities. For example, my fMRI datasets are typically bundled together with pupil recordings and heart-beat recordings. We therefore need not only a standardization for storage of specific kind of datasets, but we also need a way to create dataset-bundles that represents an experiment in a flexible manner. A principled way to bundle standardized datasets. Let's call this step 1.

The other thing that labs do is to write code to process their data. To my opinion this is where the biggest challenge is located, namely on finding a system where people can collaborate and create something together. Assuming that the step 1 is solved, the code that is written will also be publicly available. Therefore, code that is written will be directly connected to a dataset type. 

For example, if I am trying to detect peaks in a more or less periodical physiological recording, I will not start looking for literature, find someone's algorithm, implement my version of it. I will simply search for code that is compatible with this type of data, browse among alternative codes, read comments to figure out strengths and weaknesses, consider ratings and incorporate that code to my pipeline.

Basically putting up an analysis will be about creating a pipeline using previously coded nodes or coding new nodes when the analysis has not been previously carried out. When something doesn't work as expected, code needs to be improved via collaboration. Writing good quality code will be one great novel incentive for scientists.

Another challenge is to find a way how to fund this novel system. This is certainly beyond the capacity of a single start-up. This is also beyond the scope of a single lab or institute. I also don't think today's national states are visionary enough to take such moves. To my opinion this could only be established by some tech giants who have the know-how required to solve all these problems.






What is so bad and great in academia?

When you are a scientist working in public service i.e. in academia, a major and constant question you ask will be about identifying deep-rooted problems in academia. While there are actually many bad aspects of academic working conditions, there are also great things about it. This post tries to give an objective pros vs. cons perspective to this question.

I started a list of bad and ugly things that are constantly deteriorating academic life. A list of great things follows below...









  • Pressure for your own future

    Well, no surprise here. If your study fails to find an effect, you are the first one that is affected. You wasted two years of your life and didn't make any progress, your boss lost interest in your project. Depending on the workplace culture you might have absorbed such risks if your project list was diverse enough, but generally the rule is that one person = one project. There is no mechanism currently that absorbs this type of risks. Result: Scientists are pressured to find something, creating overall a bias for falsely finding positive results.

  • Backward publication system


    We publish more or less as Fisher or even Newton. Well we do not really send a post mail but an electronic mail to the editorial office, that is true. But besides that the publication system has not seen a major improvement incorporating anything positive from the era of internet. This model is hackable by short-term benefit seekers and introduces biases in findings. A. Gelman discusses here how changing the publication system by shifting towards an open-post publication reviewing system can create novel incentives for the community.

  • Lack of crowd-based knowledge systems 
    Ratings: ★★★★☆
    412 scientists rated
    this broom with 4.12 stars.
    It is a great broom!

    We scientists read many many papers. But we somehow cannot give any opinions on them. Whereas a person selling a simplest plastic broom could receive harsh comments on Amazon, a person who is writing a bad paper in an high impact journal can easily get away with it. Why can we not simply rate papers online, why can't we create a crowd-sourced reputation system for papers that is fair and transparent in the same time?

  • Culture of mine

    As soon as you are born as a scientist, say when you start you master thesis, you will be assigned to one single project. When you grow up and get your own grant you will do the same, you will assign one project to one person. This makes you live in a bubble and cuts you completely from all sort of sharing tools and mindset that are simply the standard in industry. Unless you dedicate your own time you will never learn great practices of code sharing, writing a code for others, encapsulating your analysis as a toolbox. This is because you work alone for long long periods of time.

  • Pressure for publishing

    Publishing is a key activity in academia. The problem starts as soon as your qualities are judged solely by your publication track record. What about teaching, supervision, peer-reviewing, code sharing, diversity of your publication track record? Nope. At the end of the day, only the impact factor of the journals you published counts. If you started your career in a great lab, but has never published in high-impact journals, you have already started with one leg missing. Combined with the "culture of mine", this opens the way to authorship disputes, that are everywhere.

  • Pressure for short-term thinking 

    Most of the position in academia, are short-term contracts. Great masses are hired by a few professional elites. And these elites are equally free to either plan with you and invest in your career, or to exploit you like a vampire until your contract expires. There is no mechanism that evaluates supervisors in terms of the success of their students.

  • Lack of transparency in evaluation of your work 
    Reviewing papers anonymously.

    It takes on average few years to work on a project and finalize it as a manuscript. The evaluation takes 45 minutes per reviewer that are free too evaluate your manuscript as a crocodile or a sweet hedgehog. Accountability of reviewers is not in the equation, same for inter-reviewer reliability.

  • Predatory seniors

    Academia is a social service, the person who has the title of professor is a civil servant. However, there are no mechanisms that evaluate professors on their performance with this respect. A question like "What did you do to improve scientific practices last year and make the system more efficient ?" is missing.

  • Lack of recognition

    The fact that you can spend a lot of time working in academia and collect lots of expertise and experience, doesn't entitle you with anything significant. As far as I can talk for Germany, you are just an employee, not a hair-dresser, not a pharmacist, you are just an employee. Why don't we have a profession called "scientist"? This is mainly due to the lack of long-term contracts. And I believe very strongly that third party funding is damaging the academic sphere in favor of few strong elites and at the expense talented young scientists.


An article that focuses on only the bad sides cannot be useful for anything. So let's actually talk about what is great in academia.

  • Relative Freedom

    Not being constrained by a final product that needs to do something precise gives us great freedom in the way we work our way through something. As a professor, one reaches the peak of this freedom and can work on any topic at any time. It is just a great thing to be able to start a project any time on any topic without somebody telling "you have not published enough on this topic yet!".

  • Great colleagues

    Working with alike-minded colleagues around you, who are curious and have low thresholds for brainstorming on random topics is a great positive thing. Having a constant hunger for curiosity as a social norm is certainly a positive thing.

  • Publishing a great paper

    Dedicating your efforts on something and walking step by step on that direction is a great source of happiness. And additionally, crowning your final work with a great publication is a priceless reward. It is something you can show your grand (mo/fa)ther and evoke interest on totally random people.

  • Learning doing new things, creativity

    Being able to do things that the overwhelming majority of human population cannot is a great feeling. You have spent all your day doing this weird analysis and it turned out to be completely useless, but well you were unique and used your creativity. To the extent working in academia nourishes this craftsmanship it is an extremely pleasurable occupation.

  • Flexibility in working

    As an academic we are most of the time free on where and when we want to work. You can wake up at 11 and work until midnight or take the opposite approach, it is completely normal to not expect people to conform the regular working class habits.

This is a list I will constantly update and improve. However, this is a great point in time to hear about what people think about the good and bad sides of academia.

On the complexity of product ratings...

Imagine the following scenarios... Say you need to buy an earphone set and you have access on how other people have rated different earphones in a scale of one to five. How would these ratings influence your decision? Or, say you send your manuscript to an editor, who forwards it to a set of friendly reviewers. A day later, you got the reviews back for your manuscript and they are mostly positive. What is the chance that reviewers' opinion reflect the true value of your manuscript? Here is another example... Would you go to see a movie in the theater just because your colleague has recommended it? How would that change if you know your colleague since 15 years?

These questions represent situations where one could potentially take into account opinions of others during a decision process, where estimating the true value of a product (earphone, manuscript, movie) is essential. When we decide to buy product A instead of product B, we believe that the intrinsic value of product A is higher than B. However, what we have is just an estimate, and the true value is most often unknown, or uncertain. By using opinions of other people, for example observing ratings associated with products, we can attempt to reduce this uncertainty. While these ratings can potentially be very informative, it is not clear how customers should use them. Therefore, the question I am wondering is how should a rational agent that relies exclusively on information provided by the ratings should actually act?

Earphones sorted by Amazon's Average Customer Rating.


 Let's stick to the example with products. Product A had an average of 4.1 stars (out of 5) received from 100 people, whereas product B had 5 stars from 12 people. Assuming that one initially had no prior preference for either A or B, and have no other source of information than these ratings, how should a customer make his/her choice?

I would personally go with the product A because it seems to me that achieving 4.1 stars with 100 people provides me a more reliable source of information about the quality of the product. Product B having an average of 5 stars based on only 12 people, could just be there as a pure streak of luck. That's why I believe that 4.1 is more close to the true value of the product, whereas the average rating of product 5 is a riskier choice. This is just an intuition and I think the question is actually an empirical one, that could be tested pretty straightforwardly in the laboratory.

What about the situation where two products have exactly same ratings and the same number of raters? Would you randomly pick one of them and consider them equal? Intuition says in such situations one would need to consider the distribution of ratings in order to glean further insights. Ratings could consist of a pile of stars around one single rating value (i.e. uni-modally distributed), or alternatively could consists of lots of 5s and 2s present simultaneous. While both scenarios return the same average rating, I would personally avoid the second product as the bimodal distribution would be an indication of a significant cluster of customers with a bad experience with the product.
Hamburger Restaurants sorted by Yelp's Highest Rated.


Both Yelp and Amazon provides distributions of ratings in the form of simple histograms, which I often rely upon to select products. However neither Yelp nor Amazon provides intuitive summary metrics on what customers should closely pay attention to in these histograms. Furthermore, it is actually not very clear how a person without a quantitative understanding of histograms should interpret this data which has a fair level of complexity. Even assuming that average customer have a good understanding of data presented in the form of an histogram, then still the question is about finding the best way to integrate this information into a rational/optimal decision process. I think at this point many customers are left alone with their own choice and their own way of interpretation. They are heavily biased to use the average number of stars as the single metric to achieve a decision.

This decision could also result from a strategical thinking (I honestly don't know). While providing clear metrics on how to interpret histograms might help customers to make better choices and thus increase their long-term satisfaction. It is also possible that different customers may actually have different tolerance levels on how they deal with uncertainty about the quality of a product. In this case, this lack of support may be a good choice as it would leave individual customers with their own decision metrics, no matter how they achieve this. And it would certainly give customers the possibility to integrate other sources of information about the product.

From the perspective of companies relying on crowd-sourced knowledge, the only position where they can assert their own opinion is certainly when they present products in the form of a sorted list. Even if the customer sorts products based on ratings of other customers, it is still largely under the control of the company how this sorting metric has to be computed. Therefore, it is in the interest of the company to use a metric that actually reflects the uncertainty about the quality of the products. This would avoid pushing risky or uncertain products higher up in the ranking. I think it would certainly help to sort products differently for different users to balance out the associated uncertainty about the quality of the products.

I presented in this article few idea at the intuition level. How could we translate this onto a mathematical language. I believe a Bayesian framework can be of help here. In this Bayesian perspective, we can detach the true value (i.e. intrinsic quality) of a product from its observations (i.e. user ratings) and treat observed ratings as data that is generated from an underlying (unknown) probability distribution that characterizes the true value of a product. Using the Bayesian mechanics we can work backwards and obtain the most likely probability distribution that could have generated the observed rating scores. And finally, once we know these parameters we incorporate uncertainty into the way how different products are listed or presented. This will be the topic of the next article.

The culture of "mine" in science at the age of cloud-based analysis systems*

Scientists needs computing power and storage space for their data sets. For scientific institutions, this translates onto long-term fixed-costs that are relatively high. Resources required for buying hardware, keeping network infrastructure, paying system administrators to take care of these masses of electronics is a considerable overhead. As a result, public scientific institutes spend lots of money and human resources to create and maintain infrastructures for storing and analyzing scientific data sets.

Yet, one scientific institute is pretty much the exact replica of another one when it comes to hardware demands. That is, resources that are needed in one place should in theory be very similar to another place. Therefore, instead of investing money for system administrators, storage and computational resources, scientific institutes may actually lease these services from cloud-based infrastructures with more flexible pricing opportunities and lack of overhead. Replacing your system administrator with two PhD students is an appealing idea after all.

There is actually nothing illuminating in this view because this has been actually happening already since more than 10 years in the corporate world. Many hosting companies offers as also VNC based system to connect to their servers and use software on powerful machines. Beyond simple hosting companies, Google Cloud Computing and Amazon AWS making the transformation real by integrating all sort of compute, storage, parallelization tools and selling it as a service.

Where are we in neuroscience? Some important milestones are becoming finally a reality in natural sciences, I think that the point of no return is also being slowly reached for neuroscience. I believe this because standardization procedures on how to store and share datasets is becoming more and more mainstream, and this shift has the potential to change day-to-day scientific enterprise radically. For example, Open Neuro is one such platform, where you can upload your brain imaging dataset using the BIDS format, and let analyses run on these servers. I think this is just start of a big scale transformation on how we do science.

Here is how I think how:

(1) Scientific publication

The way we publish our reports didn't change probably since the times of Fisher or even Newton. The world today is a very different place, but many of the novel tools that have been invented in the internet-based communication era have not been incorporated into the way how we conduct science today. OK, instead of sending a manuscript to the editor's office via post, we are today using emails, fine.

For example, the scientific reviewing system did not incorporate crowd-sourcing mechanisms to evaluate the quality of scientific papers. The decision of whether a manuscript or research proposal is worth being published stays largely within the hands of few not-randomly selected referees and an editor. The process is opaque, prone to biases and has no means to stop formation of small-world cartels that mutually benefit from positive biases.

The re-distribution of reputation is not based on metrics that reflect the long-term value of person for science in general. In the best case, reputation is equivalent to your h-index, which is heavily biased by the random success of your publication track, not how good a scientist your are. Metrics that ensures long-term advancements of science are typically not included. For example, we lack a metric that judges a professor based on the number of students that became also professors in the last 5-10 years. The infrastructure to achieve a better and more democratic system is in place since more than a decade. I believe this change will come faster with cloud-based systems decreasing the cost for storage and computational resources.

In the very near future, I believe any serious publication will also need to contain the related datasets, the analysis pipeline and make it publicly available to all scientific community (but also other citizens). This is already happening, and many journals let you agree with their terms of sharing data promptly when requested. However, the definition of "prompt" is also very subjective. For example, you may want to read this twit-storm to see a recent example. Even if it was obligatory to upload the dataset, the re-evaluation of the data is not within the responsibilities of the referees. This means that modifying an existing system incrementally to make it more and more suitable for the current demands of scientific democratization is not enough, we need a radically different way of publishing science.

When the data is stored and analysis ran in a cloud-based system, there will be no more excuses for reviewers for not being involved in the data analysis, as the time it will take for them to have a closer look on the data and the analysis pipelines will be insignificant. Therefore, I believe that any serious publications will take the concerted efforts of, on the one hand authors who designed the experiment, collected data and wrote the initial draft of the paper, and on the other, reviewers who will be required to contribute in the data analysis using infra-structures provided by the cloud-based storage and computational infrastructures. There will possibly be not much difference between collaborators of today and reviewers of tomorrow.

(2) Cloud-based analysis

Most of published reports use similar methods, which are re-invented again and again by generations of PhD and postdoc crew, which is a complete waste of time and resources. I believe actually there could possibly not be a more inefficient system than today's science. A large company would not be able to function like this.

Once we start talking about cloud-based storage and analysis pipelines, it will also be possible to run these analyses automatically on a server. You will need to tick the checkbox for this or that analysis and receive the results as an email in the form of a presentation or a web page (example) to click/browse around. This is of course an over simplification, but what I would like to say is that scientists will spend more time on (1) standardizing their datasets to be able to run analyses on the cloud-based system and (2) making analysis pipelines that are compatible with standardized datasets. Therefore, many scientists will use this time to record more data.

(3) End of culture of "mine"

One of the most intriguing anthropological traits of the daily scientific enterprise, is what I call the culture of "mine". This is not something that is somewhere out there, it is right inside our offices. By this I mean the way how students, PhDs, postdocs and professors (the whole crew basically) are closed to the idea of sharing and opening their projects to external influences. Most often if not always, a project is assigned to a single person in the lab, and this person is expected to run this project until the end. Because the person believes that it is her/his project, he/she can control the monopoly together with his/her boss on how this project has to run and adjust the level of external factors (politics). This results in a very conservative set of interactions between people, as any request of help, or any communication can be seen as a contribution to the project. The culture of mine, will of course be there and start the appropriate set of behaviors to not let this happen. Unfortunately, there are countless examples of authorship disputes which appear exactly from this type of culture.

Once the opportunity to upload your dataset and run your analysis in a cloud-based system is within the reach, there will be no reason to not open your data and let other people analyze it in ways different than what you have actually thought would be most appropriate. In a crow-sourced science, you will own your data, but will actually allow other people to look into it. Pretty much the same way, when people are allowed to look at you when you are walking in the street. The constructive discussions that follows during this process belongs to all parties and can be moderated by the person who created the dataset. I believe there will be a shift in the way how people conceptualize the way how they own projects and data, replacing culture of mine with crowd-sourced intelligence.

I found this article from Jeremy Freeman, entitled "Open source tools for large-scale neuroscience" which made me super happy as it expresses many of the thoughts I scratched on this post in a systematic and professional manner.

*This article has a bias from the perspective of a neuroscientist.

Towards a new understanding of fear generalization and its neural origin

This is an opinion article examining potential research directions to make progress on the topic of fear generalization.

This is based on the grant proposal I have written a year ago. 


Introduction

One way of dealing with the ungraspable complexity of the environment consists of making generalizations (1,2). Previously learnt regularities of the environment can be useful when applied to novel situations. For example, a novel nutriment can be categorized as inedible based on past experiences with truly harmful ones.

This competence called fear generalization (FG) is a remarkably high-level cognitive ability that builds upon more basic skills such as object recognition and categorization, statistical learning, perceptual learning, memory, affective processing and conceptual learning. FG provides an important opportunity to study how basic cognitive abilities, which are typically studied in isolation, function collectively to generate adaptive behavior in a complex world.

Notably, dissonance between these abilities manifests as maladaptive behavior and may result in mental health disorders (3–8), such as specific phobia. These are characterized by an overgeneralization of previous harmful encounters, leading to the perception of truly safe situations as harmful. Therefore, understanding the neuronal and computational mechanisms of FG is crucial both for basic, as well as clinical neuroscience.


Figure 1 Faces form a circular similarity gradient (color: distance from the CS+, see color wheel). BOLD responses form a fear tuning profile, which can be parametrically characterized16. 

The study of human FG has benefited enormously from well-established experimental paradigms dating back to Pavlov (9–14). The rationale behind these paradigms consists of characterizing how learning generalizes to other events based on their perceptual similarity with a harmful item. During conditioning, humans learn the characteristics of truly harmful (CS+) and safe (CS–) events. The harmful quality of the CS+ is established by pairing it with an aversive outcome (UCS; e.g. mild electric shock on the hand) using well-established conditioning paradigms (15), where learning can be objectively monitored.

Empirically, FG is characterized by measuring fear-related responses to other stimuli organized to form a continuous similarity gradient (Fig. 1). Typically responses decay with decreasing similarity to the CS+ resulting in graded fear tuning profiles (1). The strength of this paradigm consists of parametric characterization of behavioral and neuronal fear tuning profiles based on their peak positions and widths (16) (Fig. 1). Hence, it provides a powerful paradigm to investigate neuronal mechanisms responsible for enacting adaptive and selective fear responses.

Roadmap of Objectives for Progress in Fear Generalization
Accounts of FG differ on how they attempt to explain graded fear tuning profiles. According to different perceptual models, graded responses are a mere reflection of perceptual similarity to the behaviorally relevant stimulus (17,18). However, for FG to be an adaptive process in a complex world, it must be flexible and be regulated independently from perceptual factors. Several findings suggest that this is indeed the case;

(1) Fear tuning profiles do not always peak on the objectively most harmful stimulus19. Such peak-shift are well-documented (20,21) and indicate that FG is prone to subjective biases. 

(2) Patients with anxiety disorders typically show wider fear-tuning than healthy controls (3,4,6–8,22,23), even though they have been presented with the same perceptual stimulus material. 

(3) It has been shown that participants readily generalize to semantically related objects that are part of the same category but which do not necessarily bear close physical resemblance (e.g. a hammer and a saw) (24). 

Despite these observations, there are up to date no theoretical frameworks to understand how flexibility and adaptivity emerge during FG. With the FearGen project, I aim to go beyond the current conception of FG as a sole result of physical similarity. Compelling empirical evidence and a unifying framework will deliver ground-breaking insights on how FG can be adaptively tailored to ensure survival in a complex world. 

To realize this conceptual shift, I propose the following roadmap to be incorporated in my research agenda:

1. Descriptive to Normative Transition (WP1): Two factors presumably contribute to fear tuning profiles. (1) My previous work provided evidence that uncertainty about the occurrence of harmful events is an important factor for FG (16). However, the precise role of uncertainty in tailoring FG strategies in an adaptive manner is still largely unknown. (2) There is evidence that prior belief about the harmfulness of different stimuli along the generalization gradient plays an important role that might lead to biases in fear tuning profiles (19). The term “bias“ implicitly suggests an error in detection performance. However, this behavior can be compatible with an agent behaving optimally while integrating different sources of knowledge with the aim of disambiguating the source of harmful event in uncertain conditions. Normative models can characterize optimality of behavior. I will therefore use modern theoretical framework of predictive coding (25–27), where Bayesian inference (28–30) takes a central role, as a powerful tool to advance our understanding of FG as an optimal, adaptive phenomenon.

2. Role of categorization during fear generalization (WP2): Categorization constitutes a fundamental mechanism for organizing and transferring knowledge (31). It can therefore support FG (24). Yet, our knowledge on its contribution to FG is scarce (32). The role of categorical knowledge can be investigated in two forms. (A) Categorical knowledge can be used to transfer knowledge in abstract ways between events independently from perceptual similarity (31,33). Hence, the emergence of categorical knowledge predicts a qualitative change in fear tuning (34). This transition can be investigated in humans with representational similarity analysis (35) of multivariate activity patterns recorded in EEG and fMRI. This can inform us where, when and how abstract aversive representations form during FG. (B) Categorical knowledge can also lead to ambiguity, especially when a given item simultaneously belongs to multiple categories. Understanding how ambiguity is resolved has clinical relevance (36–38). Using the correct category for FG among many competing ones can only be achieved by collecting statistical regularities about the occurrence of harmful events. I will investigate whether this ambiguity is resolved faster in healthy humans in comparison to groups suffering from anxiety disorders.

3. Establishing causation between neural activity and fear generalization (WP3): Understanding neuronal underpinnings of FG requires ultimately establishing causality between neuronal activity and fear tuning profiles. Since most of the research about neuronal mechanisms of FG is of correlational nature, studies establishing causality can provide crucial insights. In particular, parametrically organized stimulus gradients offer the possibility to quantify effects of causal interventions by biasing FG profiles. Knowing which brain structures shape FG is not only of high interest for the progress of field but also of utmost importance for research into the aetiology of anxiety disorders and the development of clinical applications.

4. Space and time of neuronal dynamics (WP2 & 3): Neuronal activity unfolds both in time and space. In the past, investigations of FG has benefited enormously from fMRI (4,6,16,18,22), but much less so from EEG (but see 39). Consequently, our understanding of fast temporal dynamics of FG during a single trial is scarce. For a thorough characterization of rapid neuronal mechanisms of FG I will recourse to EEG and iEEG methods in addition to fMRI. Representational similarity analysis (35,40) (RSA) of multivariate activity patterns is an appropriate method to investigate FG both with EEG41 and fMRI (42). This is because FG relies ultimately on a subjective metric that evaluates the relatedness of different stimuli to CS+. Hence, similarity of activity patterns between conditions can capture subjective strategies used during FG. In combination with EEG, RSA is a powerful method (42) to investigate temporal dynamics of FG.

5. Clinical relevance (WP1 & 2 & 3): Ultimately research in FG must elucidate why anxiety disorders are associated with wide fear tuning profiles. Therefore, experiments proposed here will also be conducted in groups with anxiety disorders. For this objective, points 1, 2 and 3 on the roadmap will provide key insights: In (1), I will bring a normative approach that will give a computational account of wider fear tuning profiles. In (2), I will probe patients for a compromised strategy of updating their internal hypothesis about the source of threat. And in (3) I will conduct an intervention study elucidating causal neuronal mechanisms responsible for tailoring fear tuning.

Work Package 1: Fear Generalization as Harm Prediction: A Bayesian Integration

This work package addresses points 1 (“Normative Framework”) and 5 (“Clinical Relevance”) of the roadmap. The goal of WP1 is to cast FG as cognitive ability used to predict future harm. To do so, I conceive FG as a Bayesian inference problem for recovering the cause of threat in an uncertain environment. Here, fear tuning reflects the degree of belief about the potential harm of different stimuli. In order to reduce uncertainty about the source of threat, humans integrate different sources of available knowledge: (1) Learnt threat likelihood and (2) prior beliefs. Likelihoods reflect the probability of different stimuli to predict objectively harmful outcomes; therefore it reflects the conditioning regime imposed by the experimenter. Prior beliefs reflect one’s previous opinion (i.e. before conditioning) on different stimuli to be harmful. The integration of these sources results in the observed fear tuning, which reflects the posterior, the integrated high-level FG.

1.1 A new Bayesian framework for fear generalization

As a first step I will establish a new experimental paradigm, which explicitly manipulates uncertainty levels associated with the prediction of harmful events. This framework predicts that humans will rely more heavily on their prior beliefs when the sensory evidence for the prediction of harmful events is less reliable. This requires controlling uncertainty and using stimulus material where humans can use prior knowledge.

Controlling uncertainty: To introduce uncertainty as an experimental factor I will control UCS administration with a probability distribution along the generalization gradient during the conditioning phase. Depending on the width of the probability distribution (min: only one item along the gradient predicts harmful outcome; max: all items equally predict harmful outcome), participants will be provided with more or less reliable sensory information about the source of threat. This allows us to parametrically control uncertainty associated with the delivery of harmful outcomes and provides an elegant extension of the classical conditioning paradigms.

Introducing prior knowledge: Prior beliefs consist of the accumulated experience reflecting regularities acquired across longer time-scales in real life. Therefore, to bring prior knowledge into the game we must use stimuli that have ecological validity. Faces are an excellent choice for this objective, as they are a rich source of information in social situations. Even though prior beliefs are not directly accessible, we can observe their influence by using social priors that people commonly associate with faces. I will use gender (43), emotional expression (19), pupil size (44), ethnicity (45) and gaze direction (46). Independent evidence from fear learning literature indicates how these features modulate fear learning (e.g. males perceived more dangerous than females (43)). Here, I will bring these disparate observations about social priors in an encompassing theoretical framework. These features can be (1) manipulated parametrically, and (2) used as facial elements without interfering with the identity of a face that was previously learnt to predict UCS. By parametrically introducing prior biases along the generalization gradient, I will therefore test the predictions of the Bayesian framework at different uncertainty levels. The availability of good software support to generate faces makes this a feasible goal.

Predictions: The optimal Bayesian integration makes two clear predictions on empirical fear tuning profiles depending on how threat-likelihood and prior knowledge are aligned with each other, and the uncertainty levels in the threat likelihood. (i) With increasing uncertainty in threat-likelihood, FG will rely more strongly on prior beliefs. Hence, stronger deviations away from the objectively most harmful stimulus (i.e. where the likelihood peaks) with increasing uncertainty are expected (Fig. 2). (ii) At a given uncertainty-level, fear tuning must become increasingly sharper with increasing alignment of prior beliefs and threat-likelihood (i.e. smaller separation between the peaks of likelihood and prior). This is because the evidence from threat-likelihood and prior beliefs will add up to produce highly selective fear tuning. Learning with fear-relevant stimuli47 can be taken as an illustration of this effect. When objects that are also dangerous in real life are used as stimuli stronger and more persistent learning can be established. The predictions will be statistically tested based on the parameters (e.g peak position, tuning width) describing the empirical fear tuning profiles.


Fig. 2. Optimal Bayesian integration for FG predicts larger deviations in fear tuning with increasing uncertainty levels in the likelihood. 

Experiment: During the conditioning phase, faces along the generalization gradient will be probabilistically paired with UCSs based on a Gaussian distribution (i.e. likelihood). While the width parameter of this Gaussian controls uncertainty, its peak position determines which face predicts best UCS, hence the alignment between prior and likelihood. I will measure autonomic nervous system activity in the form of skin-conductance responses to monitor the establishment of learning. I will complement this measure with explicit ratings of UCS likelihood, which will be gathered at the end of the conditioning and test phases. During the test phase, same faces will be presented, but additionally they will exhibit facial features of social priors.

1.2 Computational characterization of neuronal fear tuning profiles

In our previous work16, we found that fear tuning in prefrontal cortex is significantly wider than in insula, which is characterized by a sharp fear tuning. The framework I outlined above allows us to understand these descriptive observations in computational terms. Wide prefrontal fear tuning is compatible with a fear representation that results from the integration of different sources of information. On the other hand, sharp insular fear tuning can reflect stimulus-UCS contingencies before the integration of prior knowledge. Hence, it is compatible with a representation based on threat-likelihood.

Once the experimental paradigm is established, I will use fMRI to obtain neuronal fear tuning profiles based on BOLD responses. Hence, testing the predictions (i) and (ii) of the Bayesian framework is a feasible objective by building on my previous expertise. It will advance our understanding of how brain forms aversive representations and whether these can be understood in terms of optimal aversive representations.

1.3. Clinical study

The Bayesian framework will also provide a basis to advance our understanding of anxiety disorders. We will use this paradigm to investigate a widespread anxiety disorder. Patients with specific phobias (ICD10 - F40.2) exhibit intense fear responses that are triggered by very specific situations47,48 and exhibit overgeneralization behavior during FG7,49. This offers an optimal scenario to study predictions from the Bayesian framework. In this framework, this type of behavior can be obtained by an inability to form optimal threat representations, or alternatively by the presence of over-precise prior beliefs. By comparing the width of the threat-likelihood and the recovered priors with healthy individuals, we will be able to identify which of these effects cause intense fear responses in phobia patients.

Work Package 2: Role of Categorization during Fear Generalization

This work package addresses points 2 (“Categorical Knowledge”), 4 (“Neurodynamics”) and 5 (“Clinical Relevance”) of the road map. The first part is concerned with the emergence of categorical knowledge during FG and the associated changes in neuronal representations and dynamics (34). In the last part, I will use hierarchically organized categories of commonly-known objects (50) to induce ambiguity and investigate FG strategies in anxiety patients and healthy controls (36–38).

2.1 Two-stage model of fear generalization

FG has been mainly investigated with stimuli organized along a similarity gradient. FG based on perceptual similarity could simply constitute one specific form. My working hypothesis is that this type of similarity-based generalization is the predecessor of category-based generalization, which instead requires an abstraction from superficial perceptual aspects. However, as long as the organism has not yet experienced enough aversive events, it will not be possible to extract features that can abstractly describe these events. At this initial stage, grouping different stimuli based on their perceptual similarity could be the best available strategy. However, gradual learning leads to the emergence of harmful and safe categories. I predict that this “Aha!” moment will be marked by a transition from fuzzy generalization profiles across perceptually similar stimuli to a binary yes-or-no type generalization profile reflecting their category membership.

To capture this transition, I will establish a novel experimental paradigm where the CS+ and CS– will characterize two probabilistic category structures31,51,52 defined across two facial features (e.g. gender and age). Across interleaved conditioning and test phases, I will give participants the possibility to extract the underlying category structure53. Importantly, I will pit category membership of faces against perceptual similarities to investigate their independent contributions over the course of the experiment. To this end, all stimuli will be characterized both (1) by their similarity to previous harmful faces, and (2) by their category membership. By modeling fear-related responses (i.e. SCR, explicit ratings) with these two predictors I will quantify the contribution of perceptual and categorical factors. I predict that with the emergence of categorical knowledge the contribution of perceptual factors will diminish. This will therefore establish an important link between two cognitive abilities that were so far studied separately.

2.2 fMRI on category learning during fear generalization

Bringing this paradigm to fMRI, I will identify neuronal mechanisms responsible for the emergence of categorical knowledge during FG. This will allow me to address the extent to which perceptual and categorical FG shares common neuronal mechanisms. As category-based FG relies on the use of more abstract knowledge, it is possible that it depends on different neuronal mechanisms than perceptual FG. This echoes an important dichotomy in the categorization research regarding abstract vs. similarity based categorical learning54. RSA40 is powerful and sensitive method that can contribute to the elucidation of neuronal mechanisms. As it evaluates between-condition similarity of multi-voxel activity patterns, it predicts different similarity geometries depending on whether FG proceeds with categorical24,55 or perceptual factors. It is therefore the appropriate tool for the identification of neuronal mechanisms of FG when multiple factors are available.

2.3 EEG on category learning during fear generalization fear generalization

Temporal dynamics of neuronal activity during FG within a single trial are largely unknown. Using the same experimental paradigm in an EEG setting, I will investigate fast neuronal dynamics (e.g time-frequency analysis) of FG and characterize temporal unfolding of perceptual and categorical factors. I will use RSA across EEG sensors41, an analysis that is appropriate for the identification of perceptual and categorical factors. Therefore, this EEG experiment will bring a different but synergistic perspective to the insights gathered in 2.2 with fMRI. In particular, using RSA we will be able to merge insights gathered from EEG and fMRI modalities42.

2.4 Overgeneralization across hierarchically organized categories

Overgeneralization in individuals with anxiety disorders has been observed in FG paradigms using perceptual gradients3,6,8,23. There is evidence that this finding can be accounted, at least in part, by perceptual confusion22. Moreover, perceptual performance is influence by learning56,57. It is therefore crucial to test the finding of overgeneralization in situations that do not require fine perceptual discrimination.

In hierarchically organized taxonomies, a single item simultaneously belongs to multiple categories (e.g. sub-ordinate, basic-level). Therefore, during a FG experiment with such stimuli, it should be impossible to unambiguously assign a CS+ to a given hierarchical level. Hence, with such stimuli one can measure overgeneralization independent of perceptual factors. In this experiment we will test FG performance of patients with specific phobias. Overgeneralization, if true, predicts that patients will consistently tune in on higher levels in the hierarchy in comparison to healthy individuals for the prediction of harmful events.

Work Package 3: Intracranial Recordings during Fear Generalization

This work package addresses point 3 (“Establishing causation”) and 4 (“Neurodynamics”) of the roadmap. Presurgical epilepsy patients with implanted deep electrodes offer a unique possibility to directly investigate activity of neuronal populations during complex cognitive tasks58,59, such as FG. These recordings will provide a detailed picture of population dynamics during FG in the form of local field potentials at precisely known neuronal sites. Most importantly, through stimulation of neuronal activity with subthreshold electrical currents60,61, we will aim to establish causality between neuronal activity and fear tuning profiles.

Electrodes are targeted to specific neuronal sites depending on each patient’s requirements. Typical locations include amygdala, hippocampus, insula and temporal cortices. Hence, even though electrodes have customized locations for every patient, the distribution of localizations is suitable for an analysis of how limbic and perceptual areas work in concert during FG.

I recently developed a paradigm to investigate the dynamic emergence of fear tuning (Fig. 3A). Pairing the CS+ face with UCS at unpredictable moments resulted in the emergence of fear tuning that dynamically grew along the course of the experiment (Fig. 3A, shock symbols). This paradigm is well suited for introducing biases in fear tuning profiles during learning with subthreshold electrical stimulation. Hence, my know-how from this previous fMRI experiment will contribute collecting best quality data with an already tested paradigm.

3.1 Dynamic emergence of fear generalization profiles in affective brain structures

Fear tuning in human brain has been almost exclusively shown with fMRI4,16,18,19. Given the sluggish nature of BOLD responses, our knowledge on neuronal signatures responsible for encoding fear responses is largely unknown. Local field potentials capture population dynamics at high temporal resolution, and thus provide a great source of information. The parametric nature of the FG experiment makes it possible to identify fear tuning across different frequency channels. This will provide important insights for understanding neuronal mechanisms of FG.

3.2 Establishing causality

Understanding how neuronal activity causally contributes to the regulation of behavioral fear tuning profiles is crucial for an understanding of neuronal mechanisms implicated in anxiety disorders62. Causal intervention through electrical stimulation is a powerful method that can be used to investigate neuronal sites that can potentially bias fear tuning profiles61. The parametric nature of fear tuning profiles provides an objective and quantitative method to investigate these biases at the behavioral level.
Fig. 3 A. Emergence of FG across time and faces in Insula. (color: BOLD amplitude, shock symbols: UCS delivery with CS+ presentation). FG is shown only for trials where no UCS is administrated. B. Causal intervention with electric stimulation biases fear tuning in a reversible manner. C. Deep implanted electrodes targeting amygdala. 

To achieve this objective, I will aim to introduce biases on fear tuning profiles via subthreshold electrical stimulation. I will use the loudness of white noise auditory bursts as UCSs with presurgical patients. Therefore along the generalization gradient faces will be paired with increasing loudness levels. The CS+ face will be paired with the loudest UCS. I will aim to increment the aversive quality of faces closely neighboring the CS+ face with electrical stimulation in a reversible manner across two different runs (Fig. 3B). For stimulation, we will use electrode contacts that are functionally related to FG, which will be characterized previously. Using this methodology I will investigate the causal contribution of different neuronal sites to the production of fear tuning profiles.

References


1. Shepard RN. Toward a universal law of generalization for psychological science. Science. 1987;237(4820):1317-1323.
2. Tenenbaum JB, Griffiths TL. Generalization, similarity, and Bayesian inference. Behav Brain Sci. 2001;24(4):629–640. doi:10.1017/S0140525X01000061.
3. Lissek S, Rabin S, Heller RE, et al. Overgeneralization of conditioned fear as a pathogenic marker of panic disorder. Am J Psychiatry. 2009;167(1):47-55. doi:10.1176/appi.ajp.2009.09030410.
4. Greenberg T, Carlson JM, Cha J, Hajcak G, Mujica-Parodi LR. Ventromedial Prefrontal Cortex Reactivity Is Altered in Generalized Anxiety Disorder During Fear Generalization. Depress Anxiety. 2013;30(3):242-250. doi:10.1002/da.22016.
5. Lissek S, Kaczkurkin AN, Rabin S, Geraci M, Pine DS, Grillon C. Generalized Anxiety Disorder Is Associated With Overgeneralization of Classically Conditioned Fear. Biol Psychiatry. 2014;75(11):909-915. doi:10.1016/j.biopsych.2013.07.025.
6. Cha J, Greenberg T, Carlson JM, DeDora DJ, Hajcak G, Mujica-Parodi LR. Circuit-Wide Structural and Functional Measures Predict Ventromedial Prefrontal Cortex Fear Generalization: Implications for Generalized Anxiety Disorder. J Neurosci. 2014;34(11):4043-4053. doi:10.1523/JNEUROSCI.3372-13.2014.
7. Dymond S, Dunsmoor JE, Vervliet B, Roche B, Hermans D. Fear Generalization in Humans: Systematic Review and Implications for Anxiety Disorder Research. Behav Ther. http://www.sciencedirect.com/science/article/pii/S0005789414001348. Accessed March 2, 2015.
8. Jasnow AM, Lynch JF, Gilman TL, Riccio DC. Perspectives on fear generalization and its implications for emotional disorders. J Neurosci Res. 2017;95(3):821-835. doi:10.1002/jnr.23837.
9. Anrep GV. The irradiation of conditioned reflexes. Proc R Soc Lond Ser B Contain Pap Biol Character. 1923:404–426.
10. Bass MJ, Hull CL. The irradiation of a tactile conditioned reflex in man. J Comp Psychol. 1934;17(1):47-65.
11. Hovland CI. The Generalization of Conditioned Responses: I. The Sensory Generalization of Conditioned Responses with Varying Frequencies of Tone. J Gen Psychol. 1937;17(1):125-148. doi:10.1080/00221309.1937.9917977.
12. Guttman N, Kalish HI. Discriminability and stimulus generalization. J Exp Psychol. 1956;51(1):79-88.
13. Hull CL. The problem of primary stimulus generalization. Psychol Rev. 1947;54(3):120-134. doi:10.1037/h0061159.
14. Lashley KS, Wade M. The Pavlovian theory of generalization. Psychol Rev. 1946;53(2):72-87.
15. Lonsdorf TB, Menz MM, Andreatta M, et al. Don’t fear “fear conditioning”: Methodological considerations for the design and analysis of studies on human fear acquisition, extinction, and return of fear. Neurosci Biobehav Rev. 2017;77:247-285. doi:10.1016/j.neubiorev.2017.02.026.
16. Onat S, Büchel C. The neuronal basis of fear generalization in humans. Nat Neurosci. 2015;advance online publication. doi:10.1038/nn.4166.
17. Lissek S. Toward an account of clinical anxiety predicated on basic, neurally mapped mechanisms of Pavlovian fear-learning: the case for conditioned overgeneralization. Depress Anxiety. 2012;29(4):257–263. doi:10.1002/da.21922.
18. Lissek S, Bradford DE, Alvarez RP, et al. Neural substrates of classically conditioned fear-generalization in humans: a parametric fMRI study. Soc Cogn Affect Neurosci. 2014;9(8):1134-1142. doi:10.1093/scan/nst096.
19. Dunsmoor JE, Prince SE, Murty VP, Kragel PA, LaBar KS. Neurobehavioral mechanisms of human fear generalization. NeuroImage. 2011;55(4):1878-1888. doi:10.1016/j.neuroimage.2011.01.041.
20. Purtle RB. Peak shift: A review. Psychol Bull. 1973;80(5):408-421.
21. Thomas DR, Mood K, Morrison S, Wiertelak E. Peak shift revisited: a test of alternative interpretations. J Exp Psychol Anim Behav Process. 1991;17(2):130-140.
22. Laufer O, Israeli D, Paz R. Behavioral and Neural Mechanisms of Overgeneralization in Anxiety. Curr Biol. 2016;26(6):713-722. doi:10.1016/j.cub.2016.01.023.
23. Dunsmoor JE, Paz R. Fear Generalization and Anxiety: Behavioral and Neural Mechanisms. Biol Psychiatry. 2015;78(5):336-343. doi:10.1016/j.biopsych.2015.04.010.
24. Dunsmoor JE, Kragel PA, Martin A, LaBar KS. Aversive Learning Modulates Cortical Representations of Object Categories. Cereb Cortex. 2014;24(11):2859-2872. doi:10.1093/cercor/bht138.
25. Rao RP, Ballard DH. Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nat Neurosci. 1999;2(1):79–87. doi:10.1038/4580.
26. Friston K. The free-energy principle: a unified brain theory? Nat Rev Neurosci. 2010;11(2):127-138. doi:10.1038/nrn2787.
27. Vilares I, Kording K. Bayesian models: the structure of the world, uncertainty, behavior, and the brain. Ann N Y Acad Sci. 2011;1224(1):22-39. doi:10.1111/j.1749-6632.2011.05965.x.
28. Knill DC, Pouget A. The Bayesian brain: the role of uncertainty in neural coding and computation. TRENDS Neurosci. 2004;27(12):712–719.
29. Ernst MO, Bulthoff HH. Merging the senses into a robust percept. Trends Cogn Sci. 2004;8(4):162–169. doi:10.1016/j.tics.2004.02.002.
30. Koerding KP, Wolpert DM. Bayesian integration in sensorimotor learning. Nature. 2004;427(6971):244–247. doi:10.1038/nature02169.
31. Ashby FG, Maddox WT. Human Category Learning. Annu Rev Psychol. 2005;56(1):149-178. doi:10.1146/annurev.psych.56.091103.070217.
32. Dunsmoor JE, Murphy GL. Categories, concepts, and conditioning: how humans generalize fear. Trends Cogn Sci. 2015;19(2):73-77. doi:10.1016/j.tics.2014.12.003.
33. Medin DL. Concepts and conceptual structure. Am Psychol. 1989;44(12):1469.
34. Ohl FW, Scheich H, Freeman WJ. Change in pattern of ongoing cortical activity with auditory category learning. Nature. 2001;412(6848):733-736. doi:10.1038/35089076.
35. Kriegeskorte N, Mur M, Ruff DA, et al. Matching Categorical Object Representations in Inferior Temporal Cortex of Man and Monkey. Neuron. 2008;60(6):1126-1141. doi:10.1016/j.neuron.2008.10.043.
36. Nader K, Balleine B. Ambiguity and anxiety: when a glass half full is empty. Nat Neurosci. 2007;10(7):807-808. doi:10.1038/nn0707-807.
37. Grupe DW, Nitschke JB. Uncertainty and anticipation in anxiety: an integrated neurobiological and psychological perspective. Nat Rev Neurosci. 2013;14(7):488-501. doi:10.1038/nrn3524.
38. Lake JI, LaBar KS. Unpredictability and Uncertainty in Anxiety: A New Direction for Emotional Timing Research. Front Integr Neurosci. 2011;5. doi:10.3389/fnint.2011.00055.
39. McTeague LM, Gruss LF, Keil A. Aversive learning shapes neuronal orientation tuning in human visual cortex. Nat Commun. 2015;6:7823. doi:10.1038/ncomms8823.
40. Kriegeskorte N, Mur M, Bandettini P. Representational Similarity Analysis – Connecting the Branches of Systems Neuroscience. Front Syst Neurosci. 2008;2. doi:10.3389/neuro.06.004.2008.
41. Kietzmann TC, Gert AL, Tong F, König P. Representational Dynamics of Facial Viewpoint Encoding. J Cogn Neurosci. 2017;29(4):637-651. doi:10.1162/jocn_a_01070.
42. Cichy RM, Pantazis D, Oliva A. Resolving human object recognition in space and time. Nat Neurosci. 2014;17(3):455-462. doi:10.1038/nn.3635.
43. Navarrete CD, Olsson A, Ho AK, Mendes WB, Thomsen L, Sidanius J. Fear Extinction to an Out-Group Face: The Role of Target Gender. Psychol Sci. 2009;20(2):155-158. doi:10.1111/j.1467-9280.2009.02273.x.
44. Demos KE, Kelley WM, Ryan SL, Davis FC, Whalen PJ. Human Amygdala Sensitivity to the Pupil Size of Others. Cereb Cortex. 2008;18(12):2729-2734. doi:10.1093/cercor/bhn034.
45. Olsson A, Ebert JP, Banaji MR, Phelps EA. The role of social groups in the persistence of learned fear. Science. 2005;309(5735):785-787. doi:10.1126/science.1113551.
46. N’Diaye K, Sander D, Vuilleumier P. Self-relevance processing in the human amygdala: gaze direction, facial expression, and emotion intensity. Emotion. 2009;9(6):798.
47. Öhman A, Mineka S. Fears, phobias, and preparedness: toward an evolved module of fear and fear learning. Psychol Rev. 2001;108(3):483.
48. Ipser JC, Singh L, Stein DJ. Meta-analysis of functional brain imaging in specific phobia. Psychiatry Clin Neurosci. 2013;67(5):311-322. doi:10.1111/pcn.12055.
49. Dymond S, Schlund MW, Roche B, Whelan R. The spread of fear: Symbolic generalization mediates graded threat-avoidance in specific phobia. Q J Exp Psychol. 2014;67(2):247-259. doi:10.1080/17470218.2013.800124.
50. Xu F, Tenenbaum JB. Word learning as Bayesian inference. Psychol Rev. 2007;114(2):245.
51. Nosofsky RM. Attention, similarity, and the identification–categorization relationship. J Exp Psychol Gen. 1986;115(1):39.
52. Sigala N, Logothetis NK. Visual categorization shapes feature selectivity in the primate temporal cortex. Nature. 2002;415(6869):318-320. doi:10.1038/415318a.
53. Kietzmann TC, König P. Perceptual learning of parametric face categories leads to the integration of high-level class-based information but not to high-level pop-out. J Vis. 2010;10(13):20-20. doi:10.1167/10.13.20.
54. Sloman SA. The empirical case for two systems of reasoning. Psychol Bull. 1996;119:3–22.
55. Visser RM, Scholte HS, Beemsterboer T, Kindt M. Neural pattern similarity predicts long-term fear memory. Nat Neurosci. 2013;16(4):388-390. doi:10.1038/nn.3345.
56. Resnik J, Sobel N, Paz R. Auditory aversive learning increases discrimination thresholds. Nat Neurosci. 2011;14(6):791-796. doi:10.1038/nn.2802.
57. Holt DJ, Boeke EA, Wolthusen RPF, Nasr S, Milad MR, Tootell RBH. A parametric study of fear generalization to faces and non-face objects: relationship to discrimination thresholds. Front Hum Neurosci. 2014;8:624. doi:10.3389/fnhum.2014.00624.
58. Lachaux J-P, Axmacher N, Mormann F, Halgren E, Crone NE. High-frequency neural activity and human cognition: past, present and possible future of intracranial EEG research. Prog Neurobiol. 2012;98(3):279-301. doi:10.1016/j.pneurobio.2012.06.008.
59. Vidal JR, Perrone-Bertolotti M, Kahane P, Lachaux J-P. Intracranial spectral amplitude dynamics of perceptual suppression in fronto-insular, occipito-temporal, and primary visual cortex. Front Psychol. 2015;5. doi:10.3389/fpsyg.2014.01545.
60. Cogan SF. Neural Stimulation and Recording Electrodes. Annu Rev Biomed Eng. 2008;10(1):275-309. doi:10.1146/annurev.bioeng.10.061807.160518.
61. Guillory SA, Bujarski KA. Exploring emotions using invasive methods: review of 60 years of human intracranial electrophysiology. Soc Cogn Affect Neurosci. 2014;9(12):1880-1889. doi:10.1093/scan/nsu002.
62. Etkin A. Neurobiology of Anxiety: From Neural Circuits to Novel Solutions? Depress Anxiety. 2012;29(5):355-358. doi:10.1002/da.21957.