Parametric Time-Frequency Domain Spatial Audio

14 Spatial Sound Scene Synthesis and Manipulation for Virtual Reality and Audio Effects

Ville Pulkki,¹ Archontis Politis,¹ Tapani Pihlajamäki,² and Mikko-Ville Laitinen²

¹Department of Signal Processing and Acoustics, Aalto University, Finland

²Nokia Technologies, Finland

14.1 Introduction

When a spatial sound scene mimicking an existing or imaginary acoustical condition is synthesized for a subject, the perception of physical presence in locations elsewhere in the real world or in imaginary worlds is created, and the resulting perception is called a virtual reality (Sherman and Craig, 2003). The simulated virtual environment can be similar to the real world in order to create a lifelike experience – for example, for training purposes or for virtual tourism. Alternatively, it can differ significantly from reality, as in virtual reality games. The virtual reality often includes sound sources, and the sound emanated by them should be made audible to the listener in the same manner as a virtual representative of the listener, an avatar, would perceive it if the sound scene was real. In addition to the virtual sound sources, the virtual worlds may also consist of enclosed spaces producing reflections and reverberation, and acoustically reflective objects, whose effect on the sound should be taken into account in the sound synthesis.

When synthesizing a virtual sound scene plausibly for a listener, a number of tasks has to be performed, as discussed by Pulkki and Karjalainen (2015). ...