Generative Deep Learning, 2nd Edition

Chapter 11. Music Generation

Chapter Goals

In this chapter you will:

• Understand how we can treat music generation as a sequence prediction problem, so we can apply autoregressive models such as Transformers.

• See how to parse and tokenize MIDI files using the music21 package to create a training set.

• Learn how to use sine positional encoding.

• Train a music-generating Transformer, with multiple inputs and outputs to handle note and duration.

• Understand how to handle polyphonic music, including grid tokenization and event-based tokenization.

• Train a MuseGAN model to generate multitrack music.

• Use the MuseGAN to adjust different properties of the generated bars.

Musical composition is a complex and creative process that involves combining different musical elements such as melody, harmony, rhythm, and timbre. While this is traditionally seen as a uniquely human activity, recent advancements have made it possible to generate music that both is pleasing to the ear and has long-term structure.

One of the most popular techniques for music generation is the Transformer, as music can be thought of as a sequence prediction problem. These models have been adapted to generate music by treating musical notes as a sequence of tokens, similar to words in a sentence. The Transformer model learns to predict the next note in the sequence based on the previous notes, resulting in a generated piece of music.

MuseGAN takes a totally different approach to generating music. Unlike Transformers, ...