"Disguised face identification" technology could make staying anonymous in public a bigger challenge (and make it easier to digitally unmask dissenters).
To evade the risk of being IDed by authorities, protesters, looters, and rioters may obscure their faces with hats, scarves, or sunglasses. But for better or worse, dissenting in disguise may no longer mean dissenting anonymously.
New research introduces an artificial intelligence-powered framework for “disguised face identification” – abbreviated by the researchers as DIC. The work, which considered how to identify masked faces in a crowd, addresses two related concerns at once: Most facial recognition technologies not only lack disguised-detection abilities, they also struggle to separate individuals from their backgrounds.
By training a deep-learning system to recognize and identify 14 different keypoints on the human face, the DIC framework was able to identify persons concealed by scarves with a 67% accuracy rate.
Though the DIC research has notable flaws (including a small data set and far-from perfect success rates), the work raises timely ethical questions about anonymity, privacy, and dissent. If similar face-recognition algorithms were developed and tested with broader data sets, “disguised face identification” could make protesters easier targets for online vigilantes.
Led by a team of engineering grad students from the UK and India, the DIC research addressed the innate limitation of facial recognition technology for IDing suspects in crowds: To outsmart detection, people can simply cover their faces.
To make disguised face identification possible, the researchers had to first determine a way to uncover an individual’s obscured facial features. This is the facial keypoints framework comes into play.
The DIC research paper (developed for the 2017 IEEE International Conference on Computer Vision Workshops) works from the assumption that every person possesses a unique arrangement of 14 different “keypoints” across the brow, eyes, nose, and mouth.
To varying degrees, those 14 points can be identified in photographs even if the person’s face is obscured. (Eyes can be detected behind glasses, for example, and the position of a nose is usually still viewable under a scarf.)
The researchers trained their network to detect the 14 keypoints on photos of individuals with and without face-obstructing disguises, against both simple and complex backgrounds. In tests, their system could detect the keypoints with 85% accuracy with simple backgrounds, though that went down to 56% with complex backgrounds.
After detecting the keypoints, the system connects them into a “star-net structure” reflective of the unique configuration of their facial features. The star-net is key to identifying the individual in disguise: The DIC system would compare a disguised face against five different, nondisguised faces to assess the similarity of star-net orientations.
Like other AI technologies, the disguised face identification system relies on a neural network, which is essentially a computer system designed to recognize patterns and think like the human brain. To train their neural network to assess facial keypoint similarities, the team seeded their system with a pre-existing data set – a process referred to as “transfer learning.”
The DIC’s neural network was trained with two annotated data-sets containing 2,000 images of 25 male and female subjects (ranging in age from 18 to 30 years old) wearing ten different disguises, per the image at right. One data-set had simple background images, the other had more complex backgrounds.
From there, the system classified the identities of the 25 individuals in the data set based on star-net similarity.
The effectiveness of the classification framework varied depending on the extent of the disguise and the complexity of the background image behind the individual.
For example, the AI correctly identified people wearing both hats and scarves 69% of the time when the background was “simple,” but just 56% of the time when the background was complex. When the subjects were wearing glasses as well as scarves and hats, the success rates dropped to 55% for simple backgrounds and 43% for complex.
Clearly, there’s much more work to be done before rioters or criminals in a crowded environment could be IDed under (literal) cover. As critics of the research have noted, the transfer learning approach and 4,000-picture data set are far from ideal for a “deep learning” technology.
Industry standards for facial identification technologies are much higher than 43%-55%: Many of today’s state-of-the-art face matching systems claim as high as 95% accuracy.
Yet those high-accuracy accurate systems also have the luxury of executing facial recognition in highly controlled environments – for example, comparing faces IDed in surveillance to those in mugshot databases.
The DIC system, as presented in the IEEE research paper, isn’t designed to explicitly match disguised faces to mugshots or portraits; rather, it uses the star-net as a proxy for determining an individuals’ identity.
In fact, the proxy consideration is exactly what makes the research so applicable – and troubling – in protest environments. If the DIC algorithms (or similar ones) were applied to a national data set to recognize the unique “star-net” configurations of both disguised and uncovered faces, then all people become identifiable in disguise.
And if things like scarves, hats, and sunglasses can no longer protect dissenters from ID detection, protesters could become easier targets for hate groups seeking to uncover their identities by comparing crowd photos to online information.
The researchers have emphasized that they desire to see their work serve the public good: Lead researcher Amarjot Singh has said the idea for this work came out of facial recognition shortcomings observed in high-profile crimes like the Boston bombings.
But Singh also recognizes that it could be used for more nefarious purpose. and acknowledges that disguised face identification “has to be regulated somehow” to stay out of the hands of vigilante citizens – like recent action by those on 4chan, for example – interested in unmasking anonymous protesters.
The original research ‘Disguised Face Identification (DFI) with Facial KeyPoints using Spatial Fusion Convolutional Network’ was led by Amarjot Singh of the University of Cambridge; G Meghana Reddy and Devendra Patil of the National Institute of Technology in Inida; and SN Omkar of the Indian Institute of Science. Full information is available here.