The video would be blurred and voices muffled but the idea is for people to be able to guess whether friends and family are available in order to hop on spontaneous video calls.
In a patent granted to Amazon today, the tech giant proposes connecting households via video screens that would monitor the users’ rooms continuously, and let their friends or family tune in to access blurred visuals and muffled audio to determine when might be a good time for a video chat.
For example, when you arrive home from work your mother (if you granted her access) could start to hear muffled audio coming from your kitchen and would then know you’re home and could request an immediate video chat. If you verbally accept, the screen will automatically clear to enable the chat.
Amazon says it hopes this technology can help in “replicating the spontaneous face-to-face conversations that arise from living in the same place.” It also, of course, would capture huge amounts of data on consumers’ living spaces, social connections, communication habits, and even facial recognition.
Using this sort of always-on visual and audio tracking, the patent lays out a long list of data sets Amazon could capture, including facial expressions, “smile detection,” body movement and gestures, biometric analysis such as pulse detection, eye movement, and more.
The patent also says that the technology could identify furniture and other objects in your home, and compare them to “reference databases” that could help it figure out which brands you’ve bought and how much you’ve paid for them.
To further ease ordering, the device could even potentially visually identify specific products and add them to your cart. You could hold up an empty box of cereal, for example, and the device could note the brand and size and automatically order a refill.
More generally, visual monitoring could help Amazon better understand users’ brand preferences, disposable income, daily habits, and more.
For example, Amazon was previously granted a patent for using drones to observe consumers’ houses from the outside, and offer product suggestions and home improvement tips based on its visual data capture.
Further down the line, we know Amazon may be interested in healthcare services — technology to track users’ pulses and other biometric data in the home through countertop devices, as mentioned in this patent, would open up new possibilities.
Amazon’s Echo Show, which debuted in May 2017, could provide a prototype for the technology in this patent.
The Show, a video touchscreen with Alexa voice control, gained attention for its “Drop In” feature, which would let designated users launch a video call with you at any time. The capabilities described in today’s patent seem like the next step in the development of an Echo Show-type device.
By adding always-on, continual visual and auditory monitoring, however, it would deepen users’ connections to each other and also to Amazon.
Beyond the blurring and unblurring video chat capabilities, the device in this patent could display “active” content (requested by the user) or “passive” content (content it would pull up automatically, making assumptions about what’s most relevant for the user it “sees” in the room).
Amazon says the device could display calendar information, weather, reminders, email, recipes, videos, home automation information, delivery tracking information, and more. Users could interact with voice, touch, and even gestures — for example, swiping through calendar appointments with a wave of the hand, as shown below.
Gathering new offline data streams — especially in shoppers’ homes — could have huge implications for Amazon. Supply chain and logistics, including drone delivery tools, are another high-momentum area for Amazon patents, which we’ve previously discussed here.