Master's Thesis : Audio frame reconstruction from incomplete observations using Deep Learning techniques

Abstract

In this thesis, we tackle the problem of restoring an audio frame given the preceding and subsequent one, e.g. audio inpainting, and extend our proposed solution to the prediction of an audio frame given the last one. We consider frames of 64 and 128 milliseconds. The proposed solution combines a signal processing pipeline with a Generative adversarial network (GAN). Using as input the absolute value of the STFT of the surrounding frames, the network is able to retrieve the STFT magnitude corresponding to the gap frame. By applying the Griffin-Lim Algorithm, we are then able to estimate also the STFT phase and finally through the inverse STFT to reconstruct the missing audio frame. We compare our method, considering as baseline a Linear predictive coefficient (LPC) technique. The proposed solution shows encouraging results with respect to the baseline both for inpainting and prediction. It outperforms the baseline in term of Signal to noise ratio (SNR) on the magnitude spectrum and performs equally well or better in term of the Objective difference grade (ODG) which is a measure used tu assess the perceived audio quality. Since the phase of the STFT can be only approximately reconstructed through the Griffin-Lim Algorithm, the baseline shows better performances in terms of audio SNR. We further show the model generalization ability, by training and testing on two different types of music datasets.