From screening patients for clinical trials to assessing the emotional state of drivers, we dive in to how facial recognition technology is shaping the future.
Today, people most associate facial recognition with unlocking an iPhone or a laptop.
The biometric software behind facial recognition applications can identify facial structures, contours, and expressions, making it a no-brainer for security and identification purposes.
But it can also lead to creative applications that serve a different purpose. Listerine, for example, created an app that uses facial recognition to notify people who are blind that they were being smiled at.
While the technology is still developing, many companies (including Amazon) are banking on it as a disruptive force in a myriad of markets. At the same time, the tech is highly controversial — with privacy as a point of concern.
From creating checkout-free retail stores to eliminating concert tickets, here are 16 industries that are starting to transform with facial recognition technology.
1. Law Enforcement
As the government adds a layer of artificial intelligence to its surveillance, startups are playing a key role in providing it with the underlying facial recognition technology. Chinese unicorns like SenseTime, Face++, and CloudWalk, for example, are already working with the Chinese government here.
But even in the United States, interest in the tech is surging, according to the CB Insights patent analysis tool.
Amazon, for example, is selling its tech to law enforcement agencies.
Academic institutions like Carnegie Mellon University are also working on technology to help enhance video surveillance. The university was granted a patent around “hallucinating facial features” — a method to help law enforcement agencies identify masked suspects by reconstructing a full face when only the periocular region of the face is captured. Facial recognition may then be used to compare the “hallucinated face” to images of actual faces to find ones with a strong correlation.
Because this technology is still relatively young, the algorithms haven’t learned enough about the nuances between faces and skin tones, among other things, to provide the total accuracy that law enforcement teams need.
Broadly, facial recognition tech is not free of flaws. Amazon, for example, was in the news for reportedly misidentifying some Congressmen as criminals.
But the tools have the ability to improve. For instance, Amazon was granted a patent that explores additional layers of security, including asking users to perform certain actions like “smile, blink, or tilt his or her head.” These actions can then be combined with “infrared image information, thermal imaging data, or other such information” for more robust authentication.
2. Healthcare
In its most straightforward application, facial recognition has the potential to make it easier for patients to sign in at the doctor or hospital without waiting in line or juggling forms. But there are a number of other ways that the technology can be used to improve services.
Since 2015, Apple has launched two open-source frameworks — ResearchKit and CareKit — to help clinical trials recruit patients and monitor their health remotely. Researchers at Duke University, for example, developed an Autism & Beyond app that uses the iPhone’s front camera and facial recognition algorithms to screen children for autism.
A more futuristic use case is passive monitoring of healthcare biometrics, or extracting changes to facial features over time.
In December 2017, a Google patent was published with an ambitious vision for analyzing cardiovascular function from a person’s skin color or skin displacement.
Amazon applied for a similar patent for passive monitoring in 2014, which was later granted in 2017. It combines recognition of facial features (using neural nets or other algorithmic approaches) with heart rate analysis.
For example, algorithms can track color changes in two areas of the face, like regions near the eyes and cheek, using that data to calculate heart rate detection. Tracking these changes could better position hospital staff to more effectively monitor and treat patients.
3. Retail
Linking facial recognition to personalization for shoppers presents a big opportunity in retail.
Facial recognition, for example, could capture what a shopper is looking at, and enable retailers to later serve up related promotions to the shopper via email or online ads. In another example, Walmart has a patent on tech that would capture and analyze facial expressions of people waiting in lines to gauge their satisfaction.
In China, users scan a QR code while entering JD stores. A camera runs facial recognition algorithms to identify the shopper during entry. Every item in JD’s physical store carries an RFID tag. While exiting, customers stand at a “stand here” sign marked on the floor, where all the RFID tags are scanned at once, and cameras run facial recognition algorithms again to charge your account.
Facial recognition, along with AR, is also making brands data rich — especially beauty brands.
AR beauty company Modiface’s tech collects a variety of data points around facial characteristics, including face shape, skin tone, wrinkles, and more. This can help retailers determine how people with specific facial characteristics may be more likely to purchase certain types of products, thereby potentially predicting inventory with greater accuracy.
Estée Lauder-owned Smashbox partnered with Modiface to use customer eye tracking insights to heat map the areas on a screen receiving more attention by users. This helps Smashbox understand which features are the most interesting and iterate on its website to make the beauty shopping experience more relevant for consumers.
4. Hospitality
Facial recognition is linked to better customer service in the hospitality industry. Tying the technology to a guest’s account allows employees to offer them a better, more personalized experience.
Starting with a photo on one’s account, for example, this technology could allow a guest to simply walk into a hotel to activate her check-in. The person could use her image to enter her room as well.
This could potentially free up the concierge staff to better serve their guests with suggestions, perks, and services tailored to them.
Currently, technology like this is being used in at least two Marriott hotels in China. Guests check in at a kiosk using their face. The process takes about a minute, and once they check in, their payment is automatically charged to their account.
5. Marketing & Advertising
Ad targeting has become more and more precise. Facial recognition could take that precision bounds forward.
It could be used to gather demographic information about people who stop and look at ads or to enhance the immersive ad experience for viewers. It has surfaced in a number of campaigns to date: Virgin Mobile used it in 2013 in an interactive ad, where a viewer’s blink would advance the ad’s story. Italian company Almax implemented it in mannequins that would scan the faces of passersby who stopped to look at products.
Going forward, with facial recognition built into most phones, tablets, and computers, it could be used to analyze facial expressions and serve up different ads based on mood. It could also put actual weight behind an ad view or impression, which is currently a shaky metric.
6. Banking
Banks are beginning to use facial recognition as secure login tools for customers using their online banking sites and apps. HSBC and Singapore’s OCBC have both added facial biometrics as options for logging into mobile banking accounts. The technology is said to offer “less than a one-in-a-million chance of mistaken identity,” potentially making online banking more secure.
The tech isn’t limited to protecting online data: CaixaBank in Spain is using facial recognition at its ATMs to provide bank customers with an extra layer of security. Face scanning helps to prevent fraud by eliminating the need for a PIN or even a card swipe, both of which are susceptible to being stolen.
As the technology develops and becomes more accessible, it could also make brick-and-mortar banking more secure and personalized. Tellers would know customers’ names and account information, offering a better and less clunky customer experience.
7. Events
Last year, a company called Blink Identity received $1.5M from investors including Live Nation (which owns Ticketmaster) and Techstars Ventures. The company uses facial recognition to identify people entering an event without a ticket, reportedly in less than a second.
Other recognition technologies are also digging into this space, offering ways to recognize VIPs like all-access or season ticket holders at events.
Of course, this requires stadiums, arenas, concert halls, and other venues to be equipped with facial recognition cameras or kiosks, and would still require some crowd control, including ways to remove those who aren’t authorized to be there.
8. Social Media/Entertainment
Fighting boredom is yet another use case for facial recognition. Apps like CelebsLike.Me, which uses a person’s photo to find celebrities that look like them, and Google’s Art Selfie, which analyzes a selfie to find art pieces that match facial structures, are making the rounds on social networks.
Snapchat was a pioneer of facial recognition as a personal entertainment tool. Filters and Lenses take a scan of the subject’s face, then apply things like makeup or rabbit ears. These tools also know when the user’s mouth opens, for example, and can animate dog tongues or rainbows.
Snapchat also gave rise to Bitmoji (which doesn’t yet use facial recognition, but could in the future), which inspired Apple’s Animoji and Memoji. Within iMessage, users can create 3D emojis or animations of themselves that mirror their facial expressions, thanks to the iPhone’s Face ID technology.
9. Air Travel
Similar to the advances possible in hotel hospitality and entertainment, travel could see big changes by implementing facial recognition technology. TSA lines could be expedited for everyone and VIP passengers could have an even more tailored experience.
JetBlue is testing facial recognition instead of boarding passes to streamline the process. Kiosks use passport or ID photos from the US customs database to verify passengers and let them aboard. Last year, Delta also tested face-scanning kiosks to help passengers check their own luggage.
Biometric technology in airports is also a win for security. Last fall, facial recognition stopped a man trying to use a fake passport to enter the US. Customs officials say that these cameras have a 99% accuracy rate, as they compare photos of people’s faces on file from official government documents.
10. Automobiles
While self-driving cars are the focus of the facial recognition research in the auto industry currently, there is still potential for other use cases. Face scanning could unlock the car when the owner approaches or function as the key to starting and stopping the car (much like button-start cars function now).
Cars that are have programmable settings for different drivers could make these even simpler with face recognition. Seat distance, car temperature, preferred radio stations could switch instantly when one driver’s face is scanned versus another.
Hyundai’s self-driving concept car, on display at this year’s CES, used facial recognition and AI to “asses the driver’s emotional state” and change the lighting to help avoid accidents caused by distracted driving.
11. Gambling & Casinos
In casinos, face scanning technology is most associated with security. By scanning and recognizing those who come in and gamble, they can spot regulars who might get VIP treatment, cut down on people known for cheating the house, and reduce incidents resulting from upsetting losses.
Recently, it has also emerged as a data intelligence solution going beyond basic security. It helps provide insight on table games so that companies can optimize their casinos for maximum profit.
12. Voting
Biometrics would make voter fraud, complicated voter registration, and other limiting factors that keep people from the polls things of the past.
During last year’s midterm elections, West Virginia allowed citizens to use the Voatz app to vote. First, voters had to take a photo of their government ID and a selfie-style video of their face, which was then uploaded to the app to verify. When a person was approved, they could cast their vote through the mobile app.
The app also uses blockchain to safely store information, creating a secure and digital-first solution to an international issue.
13. Education
Face scanning is in use at a few schools to make the campuses safer from potential threats. Carnegie Mellon University, for example, is working on technology to help enhance video surveillance.
Additionally, new technology called SAFR from RealNetworks uses facial recognition cameras to help improve safety in schools and streamline guest sign-in & campus security. The technology is being tested in two schools in Seattle, where visitors smile at a camera to enter the school. It analyzes their face, identifies them, and allows the visitor onto the campus.
In Australia, facial recognition is serving a different purpose: taking attendance. A company called LoopLearn is helping select schools to keep track of students and cut down on the amount of time spent marking attendance reports.
14. Ride-Hailing
Facial recognition technology adds another layer of verification across the sharing economy.
Singapore-based ride-hailing company Grab has partnered with Microsoft to incorporate facial recognition into accurately identifying drivers and passengers. Uber has also used Microsoft’s technology to identify drivers and reduce safety and security incidents.
In India, Uber drivers use a selfie or face-scanning app on their phone to access their account, verifying that they are who they say they are. This information can then be passed on to the rider, assuring them that they are driving with a verified professional.
Passengers could do the same, using facial identification to open their account and hail a car. When the driver picks them up, they are shown as verified.
In the future, as cars are implemented with face scanning technology, this can go a step further, recognizing the passenger’s face as they enter the car to verify that they are the person who called the car.
15. Food & Beverage
A number of restaurants are using face scanning tech to improve customer experience, especially for repeat visitors.
At a California restaurant CaliBurger, facial recognition connects a patron to their loyalty program account. They can see what they’ve ordered before and receive special discounts, all by approaching a kiosk that scans their face.
Something similar was tested by a KFC in China. Using facial detection and a mobile verification, customers could smile to pay for a meal.
As a person orders, these different kiosks scan their face and take a photo, which is stored with the order data. The next time that person comes in, the kiosk and staff can greet them by name and suggest orders they might enjoy.
16. Consumer Electronics
Already, face scanning is used by companies like Microsoft, Samsung, and Apple to unlock devices like phones, tablets, and computers. In fact, 54% of Americans either already use a device with facial recognition built in or plan to use one to protect their personal data.
From cars to kitchen appliances, other consumer electronics aren’t far off. For example, the HiMirror, originally launched in 2016, offers a voice-controlled smart mirror facial recognition system that provides beauty advice for users. The HiMirror integrates AI to offer a daily skin analysis and personalized recommendations as well as augmented reality to allow users to virtually try on makeup.
Use cases could also be tied to consumer wearables. OrCam, for example, uses facial recognition software to help people who are blind or partially sighted. The device uses a wireless smart camera to read text and identify faces, among other things.
By implementing facial scans into the smart, connected devices that people already use regularly, personalization and privacy can be woven more deeply into the day-to-day.