How do our brains extract information from faces?
Most of us can tell how someone might be feeling just by looking at their face. But, how do we do this? At the University of Waterloo in Canada, Dr Roxane Itier is using eye trackers and brain imaging techniques to study how we extract information from faces, and how different contexts and personality traits might affect our ability to do this.
Talk like a cognitive and social neuroscientist
Autism spectrum disorder (ASD) — a neurodevelopmental condition impacting social interaction and communication
Cognitive neuroscience — the study of the brain mechanisms underlying mental processes such as reasoning and remembering
Electroencephalography (EEG) — a technique for measuring electrical activity in the brain using electrodes placed on the scalp
Empathy — affective empathy is the ability to relate emotionally to the feelings of others, while cognitive empathy is the ability to take the perspective of and to feel concern for another without experiencing their emotion
Event related potentials (ERPs) — brain responses that are directly triggered by specific sensory, cognitive or motor events detected using EEG
Neurons — specialised cells in the brain and wider nervous system that transmit information through electrical and chemical signals
Social neuroscience — the study of how the brain processes and understands social behaviours and interactions
Theory of mind (ToM) — the ability to understand that other people have their own thoughts and feelings
Imagine you are meeting a friend outside your local shopping centre on a sunny Saturday morning. You spot them in the crowd of faces, greet them, and start to chat about your plans for the weekend. However, you notice they are frowning slightly, and their eyes keep darting elsewhere. You wonder what is on their mind.
Dr Roxane Itier, from the University of Waterloo, is interested in situations like this. “My research focuses on understanding how the brain extracts relevant information from the face to allow us to process identity, gender, emotion and social attention,” says Roxane. She is studying how factors such as empathy, personality traits and context influence this ability, and how this may change during our lives.
How does Roxane study the brain?
Roxane uses electroencephalography (EEG), a brain imaging technique, to study the temporal dynamics of cognitive processes – that is, when these processes happen. Her participants wear caps (that look like swimming caps) fitted with electrodes. These electrodes record people’s electrical brain activity while they look at pictures of faces on a screen and complete tasks such as recognising gender, emotions and familiar faces.
“EEG records the electrical fields created by large populations of hundreds of thousands to millions of neurons,” explains Roxane. “These electrical fields spread in real time through the brain and skull to the surface of the scalp, where the electrodes record them.” In addition to neural signals, EEG electrodes also record unwanted ‘noise’, caused by signals from activity such as eye movement, muscle activity and heart rate. Researchers spend a lot of time getting rid of this noise.
Roxane uses event related potentials (ERPs) to analyse the EEG signal. Roxane explains, “ERPs allow us to track cognitive processes as they unfold across time, from the moment a stimulus, such as a face, is presented onscreen, to the moment people press a button to register their decision, categorising the face as ‘new face’, ‘angry’ or ‘woman’, for example.”
How can eye tracking be combined with EEG?
One tool for tracking where on a face people look is a remote eye tracker, positioned at the base of the screen. “We ask participants not to move their eyes, and instead, we present a little cross that we ask them to fixate on,” explains Roxane. “When the eye tracker registers that the eyes are fixated on the cross, we present the picture of the face so that a specific feature, such as the mouth, is now located where the cross used to be.” The neural activity that is recorded when the participant fixates on one facial feature can then be compared with the activity that is recorded when they are looking at other features.
Roxane is interested in how changing the features that we look at impacts our facial processing. The experiments using the eye tracker have shown that where we look may influence how we recognise faces and how we identify other people’s emotions.
How do we identify emotions from facial expressions?
After our brain recognises that we are looking at a face, it looks for clues in facial features to identify emotions. For example, a smile may indicate happiness, whereas wide eyes can show fear. “The integration of all these pieces of information leads to the understanding of the emotion conveyed by the face, which we often categorise verbally as joy, anger, fear or disgust, for example,” explains Roxane.
This all takes place in less than half a second. Roxane’s research has found that our brains recognise emotions automatically, regardless of the task or context. However, these factors can affect later cognitive processes related to the motor or verbal response “The early neural activity related to emotion is automatic, but task demands do influence the behavioural response that follows,” explains Roxane.
How does the brain process theory of mind and empathy?
“Theory of mind (ToM) and empathy are very complex processes that are not fully understood,” says Roxane. Researchers have identified areas of the brain and neural networks related to ToM and empathy, but not how they unfold over time. “What we know is that extracting information about the face promotes the emergence of these complex processes,” says Roxane. “Someone looking sad or in pain will typically trigger a response of empathy from another person. However, ToM and empathy are modulated by a lot of factors including context and individual traits.”
Roxane is studying how different factors change the neural activity shown during empathy. She says, “These studies will help us determine whether the neural activity we record on the scalp can be used as a neural signature of empathy or not.”
What are the real-life implications of this work?
Not everyone finds it easy to identify faces or emotions. For example, individuals with an autism spectrum disorder (ASD) can find this difficult. “People with ASD often avoid looking at faces, and when they do, they typically avoid the eyes,” says Roxane. “They also often show impairments in theory of mind, but not in affective empathy.”
Previous research has not combined EEG with eye tracking, so whether differences in neural activity are due to participants with ASD avoiding faces or due to differences in their neural processes is unclear. Roxane believes that the use of eye trackers alongside EEG will help provide answers.
What are the next steps for the research?
“I am starting research on self-relevance and its impact on cognitive processes,” says Roxane. “For example, do we remember faces of individuals who express opinions of us better than faces of people who express opinions about others?”
Roxane is also exploring how individual differences, such as anxious tendencies, psychopathic traits, ASD behaviours and attention deficit hyperactivity disorder (ADHD), can impact neural activity, as many of us are affected by some of these conditions.
Reference
https://doi.org/10.33424/FUTURUM537
All images © Roxane Itier
Dr Roxane Itier
Face Processing and Social Cognition Lab, Department of Psychology, University of Waterloo, Ontario, Canada
Field of research: Cognitive and social neuroscience
Research project: Combining eye tracking with brain imaging techniques to understand how we extract information from faces
Funders: Natural Sciences and Engineering Research Council of Canada (NSERC), Canada Foundation for Innovation (CFI), Canada Research Chairs (CRC)
About cognitive and social neuroscience
Cognitive and social neuroscience explores the brain activity that underlies cognitive processes such as memory, problem solving, attention and communication, as well as our social experiences and interactions. Research methods include the use of brain imaging techniques such as electroencephalography (EEG), in addition to computational modelling and behavioural techniques used in psychology. Cognitive and social neuroscience research can help to diagnose and treat various conditions, and to explore the reasons why people behave as they do and perceive the world in certain ways.
“For me, the reward of working in cognitive and social neuroscience is always the excitement that goes with discoveries and the feeling that our contribution to understanding the brain will help us better understand mental health disorders in the long run,” says Roxane. “Working with students of various levels is very rewarding, and I love the relationships I form with them and the exchange of new ideas.”
Equipment such as EEG and eye tracking devices cause many of the challenges faced by researchers. These devices can be expensive, they can break, and they cannot be used on everyone. “Because electrodes need to be in contact with the scalp to work properly, people who have very thick hair or specific hairstyles cannot participate, which limits generalisability,” explains Roxane. “The technique itself limits what we can study and conclude. For instance, briefly flashing 2D pictures of faces on the screen is unnatural, and we prevent people from moving their eyes, which is not what happens in real life. Therefore, we are likely not capturing the exact neural mechanisms that are occurring in the brain in everyday social interactions.”
The field is fast-moving, with rapid advances in the technology and the statistics used by researchers. “It is challenging to constantly keep up with all these changes – but exciting,” says Roxane.
Pathway from school to cognitive and social neuroscience
At school and post-16 years, study subjects related to cognitive and social neuroscience including biology, psychology and mathematics. At university, studying a degree in psychology or neuroscience will give you an excellent foundation to build on, and will give you experience in a variety of research methods.
Conducting research generally involves analysing data, so understanding data analysis and statistics will be valuable. Computational research methods are common, so becoming familiar with coding will give you confidence in this area.
University research departments often need volunteers to participate in their research. See if you can get involved as a volunteer or research apprentice in departments that give you experience of neuroimaging techniques and relevant research areas, such as memory, language or social cognition.
Explore careers in cognitive and social neuroscience
Learn more about the research being carried out by Roxane and her colleagues in the Department of Psychology at the University of Waterloo.
There are a number of relevant professional societies with websites where you can explore articles covering the latest research. Examples include the Cognitive Neuroscience Society, the Social and Affective Neuroscience Society and the Society for Neuroscience.
Universities often offer outreach activities which allow students to experience university and research. The University of Waterloo takes part in the University Cooperative Education Program (UCEP).
Meet Roxane
I have always been fascinated by the brain. Realising that neuroimaging allows us to study cognitive processes as they happen in the living brain was electrifying. I fell in love with faces and social cognition during my master’s degree after I read Mindblindness: An Essay on Autism and Theory of Mind by Simon Baron-Cohen. I was also in awe of neuropsychology and completely mesmerised by brain disorders including prosopagnosia (or face blindness), which is the inability to recognise people from their faces. I have been passionate about this field ever since.
My path has been atypical as I did my undergraduate degree in molecular biology and genetics. I then switched to cognitive sciences for my master’s degree and, finally, to cognitive neuroscience for my PhD, before working as a postdoctoral researcher in cognitive and social neuroscience in Toronto. This atypical training exposed me to different ways of thinking and many different approaches, including working with rats during my undergraduate degree, with babies in daycare during my master’s, and with children and adults during my PhD and postdoc. As a woman in cognitive and social neuroscience, a field still predominantly male, the path has not been easy, but I am very proud of where I am now.
I was fortunate to be involved in the field of face-related event related potentials (ERPs) from the start, and I have witnessed its drastic evolution over the last 20 years. I think the current work we are doing combining ERP and eye tracking, and using sophisticated and stronger statistics, is probably my best work so far. The cognitive and social neuroscience fields are currently in what is called a ‘replication crisis’, where many of the research findings from the last 20 years are proving hard to repeat; we are trying to understand what went wrong. It appears to come down to unverified assumptions (such as the idea that where on the face someone fixates is irrelevant for neural recordings), low power studies (for example, too small sample sizes) and inadequate statistical tools.
Roxane’s top tips
1. Always question assumptions, think outside the box and don’t be afraid to ask questions.
2. Know that it is all possible for you, whoever you are.
3. Travel the world for inspiration, keep dreaming and thrive with passion.
Do you have a question for Roxane?
Write it in the comments box below and Roxane will get back to you. (Remember, researchers are very busy people, so you may have to wait a few days.)
1 Comment
Submit a Comment
Learn how visual cognition researchers are investigating colour associations and emotions:
www.futurumcareers.com/are-there-rules-for-conveying-emotion-through-art
I was truly inspired by your article DR. Roxane. I strongly feel there is an acute awareness to in-person contact and the typical sense we are told we have go beyond the 5 that we have known all our lives. I have posted your article on Linked in because I want more people to be aware of your studies and findings. “Know that it is all possible for you, whoever you are.” I am not exactly sure why this particular tip made me a bit emotional reading it. Who knows, another untapped sense? May you continue your work with blessing and support.