Facebook Patents VR Audio Tech That Sends Sound Straight To Your Skull

For a virtual reality environment to feel immersive and “real,” VR users must gain a life-like awareness of the objects and events in their surroundings.

Audio position is key to achieving that: VR must deliver sounds as if it comes from specific virtual locations – such as an avalanche on a mountain many miles away from you.

As of now, there are “spatial audio” technologies for delivering realistic 3D sound, but most rely entirely on headphones. Since they send sound through the ears, headphones alone can limit users’ perception of the position of sounds in VR – making it hard to distinguish if a noise is coming from in front or behind them, for example.

To address this problem in the Oculus and improve audio for users, Facebook may integrate a form of audio tech known as bone conduction that bypasses the ear canal. A new Facebook patent invents a method for using “bone transducers” to deliver sound directly through the vibration of bones in a VR user’s head.

Bone conduction isn’t new, but has mainly been used outside the VR environment – such as in hearing aids, military-grade headsets, and underwater communication devices.

Bone conduction achieves sound by applying vibrations directly to the head. In doing so, it breaks the idea of sound down to its essential parts.

All sounds begin as waves traveling through the air. Once sound waves reach the ear, they pass through the ear canal to the eardrum – which emits a vibration in response.

Three tiny bones called the ossicles then transmit that vibration to the cochlea, which converts it into electrical signals that are sent to the brain and processed as sound.

Yet sound doesn’t need to pass through the ear canal to reach the brain: If the bones in your head are made to vibrate, that vibration can reach the cochlea directly (and be transformed into electrical signals in the same manner).

Bone conduction doesn’t inherently require technology. Beethoven famously discovered it as a hearing method after going deaf: By attaching a rod to his piano and clenching it in his teeth while playing, he could still hear his music.

In audio devices, a “bone transducer” replaces Beethoven’s metal rod – conducting the vibrations that ultimately communicate themselves as sounds.

Bone conduction has increasingly been making its way into wireless headphones and wearables (such as Google Glass). Fans of the technology say that by delivering a physical vibration into the head, bone conduction can help create a hyper-personal relationship between a listener and the “experience” of sound.

With its patent, Facebook suggests that bone conduction can make it easier for users to understand the position of a sound as they hear it.

Calibrating the vibrations in your head

The patented methodology for Using bone transducers to imply positioning of audio data relative to a user notes that when using only headphones or speakers, VR users “have difficulty distinguishing audio data… as originating from certain positions relative to the user’s head.”

The bone transducers would be incorporated into the Audio System [160] of the overall “VR System Environment” modeled below.

Notably, the bone transducers themselves would not be attached to the VR headset (depicted below, at left).

The bone transducers themselves would be attached directly to the VR user’s head, although they could also work in tandem with headphones or speakers, not necessarily replace them.

The transducers themselves would likely be “piezoelectric devices.”

Piezoelectric devices can take many forms, but they all convert mechanical energy into electrical energy (in this case, vibrations). The microphone in your PC, for example, is a piezoelectric device that transforms the sound energy in your voice into electrical signals your computer can interpret.

Each bone transducer device would produce a “vibrational motion in response to a control signal” from the VR console or VR headset.

The bone transducers would be affixed on each side of the VR user’s head behind the outer ear “using any suitable mechanism,” likely a light adhesive.

Once attached to the user’s head, the transducers could be calibrated to the unique specifications of the individual VR user. To do this, the user’s head would be positioned in front of an external speaker.

Using the calibration process outlined below, the external speaker would play audio data to cause vibrations in the user’s head while the transducers take in data about how the bones in the user’s head vibrate (noted as 420 in the below diagram).

Throughout this calibration process, the audio system (using headphones or speakers) may prompt the user to move his or her head closer or further to the speaker or other points.

Once the system has enough information about how the vibration of bones in the user’s head corresponds to different frequencies and different amplitudes of audio data, that information can be used to create one or more machine-learning models.

The models would be used to train the bone transducers to vibrate in ways that will most  effectively mimic the natural vibrations “caused by acoustic waves.” Different machine learning models might be used to train different transducers on the user’s head.

The patent also hints at further ways the performance of the bone transducers could be assessed and improved.

For example, the user may be prompted by the system to “point to” (or otherwise identify) where he or she perceives a sound is coming from inside the VR environment.

Information on the user’s responses could then be incorporated into the machine-learning model, or be used to inform modifications to the amplitude, frequency, or phase specifics of the individual vibrations.

The Oculus sound experience

By laying out how transducers could work in collaboration with headphones and speakers, the patent provides a framework for potentially offering bone conduction as an add-on element of traditional VR audio.

Hypothetically, bone transducers be could sold as a product that enhances the realness of your VR experience: Perhaps you would buy a set of transducers in addition to your Oculus headset, and perform the calibration process using an app that stores and manages your data.

If the transducers are shown to truly enhance the user’s awareness of position in VR, they would likely become a product of keen interest to serious gamers or eSports competitors (who could use them for a competitive edge as they try to outrun an avalanche, or fire a weapon at an enemy).

Based on current use cases for bone conduction technology, it’s highly likely their benefits would be useful in a broad variety of VR simulation environments for police, military, and other applications: Bone conduction headsets, or “bone phones,” are known to be worn by Navy SEALs in combat scenarios.