The Cocktail Fork Problem: Three-Stem Audio Separation for Real-World Soundtracks

by   Darius Petermann, et al.

The cocktail party problem aims at isolating any source of interest within a complex acoustic scene, and has long inspired audio source separation research. Recent efforts have mainly focused on separating speech from noise, speech from speech, musical instruments from each other, or sound events from each other. However, separating an audio mixture (e.g., movie soundtrack) into the three broad categories of speech, music, and sound effects (here understood to include ambient noise and natural sound events) has been left largely unexplored, despite a wide range of potential applications. This paper formalizes this task as the cocktail fork problem, and presents the Divide and Remaster (DnR) dataset to foster research on this topic. DnR is built from three well-established audio datasets (LibriVox, FMA, FSD50k), taking care to reproduce conditions similar to professionally produced content in terms of source overlap and relative loudness, and made available at CD quality. We benchmark standard source separation algorithms on DnR, and further introduce a new mixed-STFT-resolution model to better address the variety of acoustic characteristics of the three source types. Our best model produces SI-SDR improvements over the mixture of 11.3 dB for music, 11.8 dB for speech, and 10.9 dB for sound effects.


page 1

page 2

page 3

page 4


Tackling the Cocktail Fork Problem for Separation and Transcription of Real-World Soundtracks

Emulating the human ability to solve the cocktail party problem, i.e., f...

Breaking Speech Recognizers to Imagine Lyrics

We introduce a new method for generating text, and in particular song ly...

Measurement uncertainty and unicity of single number quantities describing the spatial decay of speech level in open-plan offices

The ISO 3382-3 standard (2012) defines single number quantities (SNQs) w...

Visual Scene Graphs for Audio Source Separation

State-of-the-art approaches for visually-guided audio source separation ...

Neural Sound Field Decomposition with Super-resolution of Sound Direction

Sound field decomposition predicts waveforms in arbitrary directions usi...

Close Miking Empirical Practice Verification: A Source Separation Approach

Close miking represents a widely employed practice of placing a micropho...

Text-Driven Separation of Arbitrary Sounds

We propose a method of separating a desired sound source from a single-c...

Please sign up or login with your details

Forgot password? Click here to reset