This is my internship project at NVIDIA Research from May 2022 to Feb 2023.

[Paper]    [Demo page]

​

This paper takes score, lyrics, style label, and singer information as input and generates expressive and realistic singing. It involves a cascade of diffusion models. The pipeline involves (1)  performance control models, including timing, F0 curves, and loudness curves; (2) an acoustic model that generates the mel-spectrograms conditioning on performance control signals; (3) a DiffWave vocoder to generate the waveform from mel-spectrograms and F0 curves. The following figure shows a high-level architecture.