Close your eyes and imagine being a passport officer at London Heathrow. There is a long line of slightly tired, irritated, and often impatient passengers aching to get to the end of their journeys. One by one they come to your counter and hand over passports with rather outdated photographs, looking at you for approval to reach their final destination. Different hair, glasses, sun tan, and a bit of weight gain or loss – the list goes on. It should be simple, though, to tell whether the ID and ID holder match. You have them both in front of you, the person and the image. Easy, right?
Well, not so much. Studies show that people are poor at recognising unfamiliar faces (ones we encounter for the first time, or have seen only briefly). This is true not only when we have to memorise the faces of new people, but also when the task is to compare images presented at the same time (such as matching a passport photo with a driver’s license photo). People also struggle to sort photographs by identity: For example, when shown a deck of forty photographs depicting only two different people, participants tend to think that the images show, on average, seven different people. Studies of a fairly simple “matching” task, where participants must decide whether two photographs show the same person or two different people, tell us that under optimal conditions people make mistakes on at least 10% of the trials. The situation is further complicated by the fact that photos in identity documents are often valid for ten years, during which one’s appearance can change substantially. Look at your driver’s license, work badge, and passport. Do they all resemble the person you see in the mirror?
One reason why lay people are not good at face matching might be that they don’t have to do this task on a day to day basis. Following this logic, those who do it frequently as a part of their job should be much better at deciding whether a document and its holder match or not. Surprisingly, research suggests otherwise. A study conducted in 2014 by David White and colleagues in Australia showed that in a mock border control scenario, passport officers accepted 14% of fraudulent documents, the same amount as untrained students who acted as a control group. Moreover, how well officers performed was unrelated to the length of their occupational experience. Put simply, twenty years of matching people and documents does not make one good at it. These findings are consistent with other studies investigating perceptual expertise. Portrait artists, who are presumably well in tune with the fine details of human faces, are not better at face recognition tasks than the general population. In the same vein, a Swiss group of researchers reported that memory champions for name-face associations are also not superior to control participants on standardized face recognition tasks, so becoming a super-memorizer does not equal being a face super-recognizer.
On supporting science journalism
If you're enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.
So what does all that tell us about how face recognition works? Studies with twins have consistently shown that face recognition is highly heritable and independent from other abilities such as general intelligence. The range of ability is remarkable: ‘super-recognisers’, are able to recognise someone in the street that they met once, some years earlier, at a party. These people are also much better than average at matching unfamiliar faces.
A key question for researchers is whether such enhanced abilities are purely innate, or whether they can be taught. In the absence of an army of face experts to guard national frontiers, scientists have made concerted efforts to improve the performance of typical perceivers on these applied tasks. Some studies have shown that continuous feedback, in this case telling participants after every trial whether they were correct or not, can slow the decline in face matching accuracy when it must be done over long periods, such as in airport-like scenarios, but does not improve face matching per se. Others have shown that feedback can indeed aid accuracy. Arguably, this strategy is not viable beyond laboratory settings, because the truth is always unknown on the streets – who would provide feedback in real world situations when the only one who knows the truth is the ID holder? Another commonly used face classification strategy is encouraging people to decide whether two faces are the same or different based on their shape, but this was also found ineffective when tested in a laboratory setting.
Other interventions have shown more promise. Incentives such as chocolate lead to improvements in accuracy on mismatched trials, suggesting it would be a viable strategy for detecting fraudulent IDs, though one can see problems with implementing this for passport controllers. Working in pairs and being free to communicate also appears to have a benefit for both partners and generalises beyond the time of joined decision making. Instructing novice participants to pay attention to specific features and evaluate them in terms of similarity to another image also leads to better performance. Finally, when the task is difficult, masking external features, hair, ears, and the face shape offers some help and leads to improvements in discriminability.
An alternative solution is to be more selective in personnel allocation. Studies have shown that super-recognisers are a long way ahead of the general population at face matching accuracy. They appear to use something called “holistic processing” where faces are perceived as a whole rather than a sum of their parts. It requires paying attention to the internal features of the faces (the eyes, the nose, and the mouth) and accurate assessment of the relative distances between these features, which is one possible explanation for how super-recognizers’ do it. Holistic processing has been repeatedly shown to be positively associated with face processing ability in the general population, but it is yet unclear whether training people to perceive faces more holistically would generalise to improved face matching.
Despite adopting a completely different strategy, forensic examiners, people who compare face images and later provide expert evidence in police investigations and court cases, have also been found to be extremely accurate at face matching. They rely heavily on feature by feature comparison to make their decisions, so they process faces less holistically. It is possible that factors other than pure face recognition ability, such as attention and motivation, play a part. Indeed, the advantage of forensic examiners becomes apparent when the time allowed to examine images is long, but not at shorter intervals.
The take home message remains: unfamiliar face recognition is an extremely difficult task that only few get almost right. And this brings us back to the original question: can we improve national security with science? The answer is yes, in many ways. Firstly, we could do away with facial photo identification and use some other form of biometric. Secondly, when faces are used as the main biometric, human decision makers should be aware of their own limitations and apply the laboratory findings in their duty to keep us safe, for example working in pairs and receiving specific instructions. Finally, there is clear potential in national security agencies to use super-recognisers to keep dangerous criminals off the streets and away from our borders.
