O'Reilly logo

Music Emotion Recognition by Homer H. Chen, Yi-Hsuan Yang

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Saunder January 24, 2011 10:39 book
13
Music Retrieval in
the Emotion Plane
With MER, many novel emotion-based music retrieval and organization methods
can be developed. One can search music by emotion, find music pieces that are
similar to a given music piece in terms of perceived emotion, or browse a music
collection in the emotion space. This chapter describes a music retrieval platform
that organizes music pieces in the 2D valence-arousal emotion plane. Such a simple
2D user interface is useful for mobile devices that have a small display space and a
limited input capability.
13.1 Emotion-Based Music Retrieval
The most appealing function of music is that it can convey emotion and modulate
a listener’s mood [90]. Past years have witnessed an increase of interests in emotion-
based music organization and retrieval. In academia, more and more multimedia
systems that involve emotion analysis of music signals have been developed, such
as Mood Cloud [185, 187], Moody [141], MusicSense [45], LyQ [136], Mood-
track [328], and i.MTV [370–372], to name a few. In the industry, many music
companies, such as AMG [1], Gracenote [5], MoodLogic [12], Musicovery [13],
Syntonetic [14], and Sourcetone [145], use emotion as a cue for music retrieval.
Some of these systems view emotions from the categorical perspective, while others
from the dimensional perspective. We have introduced two music retrieval systems
that adopt the categorical conceptualization of emotion in Chapter 1.
This chapter describes an emotion-based music retrieval platform, called Mr.
Emo, that views emotion from the dimensional perspective [357, 362]. Mr. Emo
207
Saunder January 24, 2011 10:39 book
208 Music Emotion Recognition
represents a good example showing how the dimensional conceptualization of
emotion and the techniques developed for dimensional MER can be applied to
music retrieval and management in the 2D emotion space.
13.2 2D Visualization of Music
Visualizing music in 2D space has been studied in the past. As conventional ap-
proaches that manage music pieces based on catalog metadata such as artist name,
album name, and song name cannot provide enough information on music simi-
larity, which the users highly expect from a music search tool [188], systems that
help users to retrieve and browse music pieces in a content-based fashion have been
developed. For example, an approach to visualizing a music collection based on
two perceptual attributes, rhythm and timbre, has been developed in [373]. Other
systems that present music pieces in 2D space include the PocketSOM [95] and
the Islands of Music [247], which utilize the self-organizing map [172] to map the
high-dimensional music features to a 2D map grid while preserving similarity re-
lationships in the feature space. However, there is no semantic meaning associated
with the resulting two dimensions.
As some sort of emotional experience is probably the main reason behind most
people’s engagement with music, a plausible way of visualizing music pieces is pre-
senting them in the emotion plane. With Mr. Emo, one can easily retrieve music
pieces of a certain emotion without knowing the titles or browse a personal collection
in the emotion plane. One can also couple emotion-based retrieval with traditional
keyword- or artist-based ones, to retrieve songs similar (in the sense of perceived
emotion) to a favorite piece or to select the songs of an artist according to emotion.
In addition, it is also possible to play back music that matches a user’s current emo-
tion state, which can be estimated from facial or prosodic cues [22,192,253]. Such
a simple 2D user interface for content-based retrieval of music is particularly useful
for tiny mobile devices such as MP3 players or cell phones that have a small display
space and a limited input capability.
13.3 Retrieval Methods
In Mr. Emo, the critical task of predicting the VA values is accomplished by the
regression techniques introduced in Chapter 4. Given the regression models, the
valence and arousal (VA) values of an input song can be automatically computed
without manual labeling. Associated with the VA values, eachmusic piece is visualized
as a point in the emotion plane, and the similarity between music pieces is measured
by Euclidean distance. Many novel retrieval methods can be realized in the emotion
plane, making music information access much easier and more effective. Below we
describe four example music retrieval and organization methods that are performed
in the emotion plane.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required