Audio Source Separation and Speech Enhancement

16Applying Source Separation to Music

Bryan Pardo Antoine Liutkus Zhiyao Duan and Gaël Richard

Separation of existing audio into remixable elements is useful in many contexts, especially in the realm of music and video remixing. Much musical audio content, including audio tracks for video, is available only in mono (e.g., 1940s movies and records) or stereo (YouTube videos, commercially released music where the source tracks are not available). Separated sources from such tracks would be useful to repurpose this audio content. Applications include upmixing video soundtracks to surround sound (e.g., home theater 5.1 systems), facilitating music transcription by separating into individual instrumental tracks, allowing better mashups and remixes for disk jockeys, and rebalancing sound levels after multiple instruments or voices were recorded simultaneously to a single track (e.g., turning up only the dialog in the movie, not the music). Effective separation would also let producers edit out individual musician's note errors in a live recording without the need for an individual microphone on each musician, or apply audio effects (equalization, reverberation) to individual instruments recorded on the same track. Given the large number of potential applications and their impact, it is no surprise that many researchers have focused on the application areas of music recordings and movie soundtracks. In this chapter we provide an overview of the algorithms and approaches designed specifically ...