Samsung patents audio recovery that rebuilds sound from visual data

1. A method for managing an audio based on a spectrogram, the method comprising: receiving, by a transmitter device, the audio to send to a receiver device; generating, by the transmitter device, the spectrogram of the audio; identifying, by the transmitter device, a first spectrogram corresponding to vocals in the audio and a second spectrogram corresponding to music in the audio from the spectrogram of the audio using a neural network model; extracting, by the transmitter device, a music feature from the second spectrogram; and transmitting, by the transmitter device, a signal comprising the first spectrogram, the second spectrogram, the music feature and the audio to the receiver device, receiving, by a receiver device, a signal comprising a first spectrogram, a second spectrogram, a music feature and the audio from a transmitter device, wherein the first spectrogram corresponds to vocals in the audio and the second spectrogram corresponds to a music in the audio; determining, by the receiver device, whether an audio drop is occurring or predicted to occur in the received signal based on a parameter associated with the received signal; and generating, by the receiver device, the audio including the vocals and the music using the first spectrogram, the second spectrogram, the music feature, in response to determining that the audio drop is occurring or is predicted to occur in the received signal, wherein the music feature comprises at least one of texture, dynamics, octaves, pitch, beat rate, and key of the music in the audio, wherein generating, by the receiver device, the audio using the first spectrogram, the second spectrogram, the music feature, comprises: generating, by the receiver device, encoded image vectors of the first spectrogram and the second spectrogram; generating, by the receiver device, a latent space vector by sampling the encoded image vectors; generating, by the receiver device, the audio based on the latent space vector and the music feature.