Information Distinguished Lecture Series - A Complex Spectral Mapping Approach to Spectrospatial Filtering

We present a complex spectral mapping approach to multi-channel speech separation, which trains a deep neural network (DNN) to directly estimate the real and imaginary spectrograms of the target signal from those of the multi-channel noisy mixture. In this approach, the trained DNN itself becomes a nonlinear, time-varying spectrospatial filter. How does this conceptually simple approach perform relative to commonly-used beamforming techniques on different array configurations and in different acoustic environments? We examine this issue systematically on speech dereverberation, speech enhancement, and speaker separation tasks. Comprehensive evaluations show that multi-channel complex spectral mapping achieves speech separation performance superior to beamforming-based spatial filtering for different array geometries, and reduces to monaural speech separation with single-channel recordings, demonstrating its versatility for multi-channel and single-channel speech separation. In addition, such an approach is computationally more efficient than mask-based beamforming. Recent developments in DNN architecture employ global self-attention, and leverage cross-band and narrow-band correlations. These developments elevate speech separation performance by a considerable margin.