New Amazon Patent Points To Wireless Heart Rate Detection

Amazon’s latest patent is poised to push facial recognition forward by incorporating heart rate detection into the list of metrics used to identify a single person with a mobile phone camera. On September 5th, 2017 the US Patent & Trademark Office approved Amazon’s December 2014 filing for “Facial Feature Selection For Heart Rate Detection.”

In the past, the e-commerce giant has filed for or been granted patents for everything from talking drones to augmented reality. But facial recognition is relatively new to Amazon, launching a digital image recognition-as-a-service platform called Amazon Rekognition in late 2016. This patent points towards the logical next step for the e-commerce giant — 3D facial recognition. And adding passive heart rate monitoring to the system has potentially far-reaching implications in the broader healthcare sector.

While Amazon previously received a grant for a patent in November 2016 involving user identification via a physiological condition such as a heart rate, this patent presents a more focused technique to the detection of a user’s heart rate with smartphone/tablet cameras.

This patent seeks to improve the accuracy of current mobile facial recognition techniques which, “are imperfect due to the wide range of objects that may be contained in an image and due to the limited time frame that is usually available for detecting the presence of a face.” Amazon points to the increasing prevalence of mobile biometric monitoring capabilities as a possible solution.

PART 1: FACIAL RECOGNITION

The patent notes that the camera will initially identify key facial features, such as the nose, mouth, and eyes, in order to determine head proportions and general position. Once the facial area to be analyzed is determined, the system will then identify unique facial features such as geometric shapes or markings.

Ultimately, the system will be able to use a form of template matching to match the user’s facial characteristics to other images of the user’s face. It will, “analyze features of…various images to attempt to identify features that can accurately predict or identify corresponding features of faces in other images.”

PART 2: HEART RATE ANALYSIS

During the course of facial analysis, a specific region of the face will be chosen where a heart rate signal can be best detected based on factors such as current lighting, glare, facial hair, or other obstructions. Alternatively, a region can be selected based on the computer’s knowledge of human anatomy, as Amazon details below:

“…a region of interest may be selected using knowledge of human anatomy such as locations of facial foramen and/or corresponding locations of arteries or veins on the human face.”

Once 1 or 2 regions of the face are chosen, heart rate analysis is conducted by determining select measurements that occur over time. For example, the graph below would track measurements such as skin color brightness or intensity over time. A Fourier transform calculation is then conducted which transforms this change over time into a frequency, i.e. rate of heart beats over time.

While the patent does not directly hint at a medical application, Amazon appears to be making a number of moves to enter the healthcare space. The company is both hiring and investing in the space, as we have previously detailed in our Amazon Strategy Teardown.