Recording, Producing, and Rendering Binaural 3D Audio for Virtual Reality

Spatial audio creates an immersive audio experience for listeners. Today, we can hear spatial audio in movies, streaming music, 360/3D videos, VR games, and VR apps. The most direct way to enjoy spatial audio is not through a Dolby Atmos or DTS:X multi-channel sound system, such as a soundbar, but through a pair of headphones. Binaural audio, which delivers the two-channel sound directly to the listener’s two ears with headphones, can recreate the acoustic signals a listener would hear under natural conditions. It can also preserve the interaural time difference (ITD) and interaural level difference (ILD) cues, which are crucial for sound localization.

Since Fall 2017, our 3D audio research team at the Department of Electrical and Computer Engineering, University of Rochester, has recorded over 70 live concerts at the Eastman School of Music and several other concert venues with various binaural and Ambisonic microphones, as well as 360/3D VR cameras. We have built a spatial audio recording database for research and published many concert videos on YouTube. This online lecture will introduce different methods of recording, producing, and rendering head-locked or head-tracking binaural audio for VR videos, games, and apps. It will also demonstrate how to measure binaural impulse responses in a hall and use them to generate binaural sounds for audiovisual VR testing. Please wear headphones to hear binaural 3D audio demos.

Co-sponsored by: Rochester Section Life Member Affinity Group

Speaker(s): Ming-Lun Lee,

Agenda:
7:00 PM – 8:00 PM (Eastern Time): Lecture and Q&A

Virtual: https://events.vtools.ieee.org/m/337204