Synchronizing Lyrics to Audio with Syllable Nuclei Detection

This project aims to develop a novel method of automatically synchronizing the lyrics of a song to its corresponding audio, i.e. a line from the lyrics is displayed at the corresponding moment in the song in which that line is sung. The primary method through which this synchronization will be done is through syllabic analysis. Applications exist where the number of syllables spoken in an audio file can be extracted without processing the words themselves, resulting in much faster runtimes. By similarly analyzing the number of syllables in a chunk of text, a process for which several very good algorithms exist, we can roughly match the timestamp of each syllable in the audio file to the corresponding syllable in the lyrical text. This process would take place in a separate module than the actual SwagLyric application so as to facilitate speed of retrieval when the user is actually streaming. Several obstacles, which will be discussed in more detail, exist: these include stripping away background music and getting specific audio information from Spotify’s stream.