MusicLM: Google's New AI Tool That Can Turn Text into Music

Google recently released a new artificial intelligence (AI) tool that can create music of any genre from text prompts and even transpose a whistled or hummed melody into other instruments.

Github's research indicates that the technology known as MusicLM is a text-to-music creation system. It operates by examining the written language and figuring out the size and intricacy of the composition.

In addition to converting hummed tunes into various instruments, this AI algorithm can convert text input into seconds or even minutes of music.

“We present MusicLM, a model that uses text descriptions like "a calming violin melody backed by a distorted guitar riff" to create high-fidelity music. According to the company, "MusicLM generates music at 24 kHz that remains consistent over several minutes" and "casts the process of conditional music generation as a hierarchical sequence-to-sequence modeling task."

Google's artificial intelligence creates 5-minute melodies

Examples include 30-second clips and 5-minute long pieces that sound like songs. They are the result of paragraph-long descriptions, and the music is better when the instructions are clearer. In addition, genre, vibe, and even specific instruments are included in the examples.

“A series of text prompts are used to generate the audio, the researchers stated while saying that this influence how the model continues the semantic tokens derived from the previous caption."

Story Mode

There is also a "story mode" demo in which the model is basically given a number of text inputs and a time limit for each kind of music that needs to be made.

According to the researchers, experiments show that MusicLM outperforms previous systems both in audio quality and adherence to the text description.

Additionally, the researchers demonstrated that MusicLM can be conditioned on both a melody and text by transforming whistled and hummed melodies in accordance with a text caption's style.